Coding using intra prediction

Abstract

To provide an apparatus for decoding a prescribed block of an image using intra-prediction.SOLUTION: An apparatus (3000) is configured to derive a set-selective syntax element (522) from a data stream (12) which indicates whether a predetermined block (18) is to be predicted using one of a first set (508) of intra-prediction modes, derive a MPM list index (534) from the data stream (12) when the predetermined block is to be predicted, perform intra-prediction using a predetermined intra-prediction mode (3100), derive a further index (540) from the data stream (12) which indicates a predetermined matrix-based intra-prediction mode out of a second set (520) of matrix-based intra-prediction modes (510) if the set-selective syntax element indicates that the predetermined block is not to be predicted, and predict samples of the predetermined block (18) on the basis of the prediction vector (518).SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present application relates to the field of intra prediction.Embodiments relate to an advantageous method for generating a most probable mode list.

[0002] Today, various methods exist for generating the most probable mode list, however, there are still likely situations where the intra-prediction mode ultimately used is not in this list, and therefore additional syntax elements need to be transmitted. Therefore, the problem is faced to optimize the generation of the most possible mode list and / or to improve the coding efficiency. This is achieved by the subject matter of the independent claims of the present application. Further embodiments according to the invention are defined by the subject matter of the dependent claims of the present application. Summary of the Invention

[0003] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem encountered when forming a list of most likely intra-prediction modes is that unlikely prediction modes take up valuable list positions that negatively affect coding efficiency and increase the likelihood of a situation in which the intra-prediction mode that will ultimately be used to predict a given block is not in this list. According to a first aspect of the present application, this difficulty is overcome by forming a list of most likely intra-prediction modes based on already predicted neighboring blocks that are adjacent to the given block. Thus, unlikely intra-prediction modes can be omitted. A high probability of an intra-prediction mode of a given block similar to that of the neighboring blocks can be expected. In particular, there is no DC intra-prediction mode in the list if at least one of the neighboring blocks is predicted by any angular intra-prediction mode. This allows a list of most likely intra-prediction modes with a diversity of angular intra-prediction modes that increases the likelihood that an intra-prediction mode will be used for a given block that is in the list. Further, matrix-based intra-prediction modes form a separate second set of intra-prediction modes that, for example, are not considered for the list of most likely intra-prediction modes and therefore do not compete with the intra-prediction modes of the first set of intra-prediction modes for a position in the list of most likely intra-prediction modes.

[0004] Therefore, according to a first aspect of the present application, an apparatus for decoding a given block of an image using intra prediction is configured to derive from a data stream a set-selective syntax element indicating whether the given block should be predicted using one of a first set of intra prediction modes including a DC intra prediction mode and an angular prediction mode. Optionally, the first set of intra prediction modes may include a planar intra prediction mode in addition to or instead of a DC intra prediction mode. If the set-selective syntax element indicates that the given block should be predicted using one of the first set of intra prediction modes, the apparatus is configured to form a list of most probable intra prediction modes based on the intra prediction mode using when a neighboring block adjacent to the given block is predicted, derive from the data stream an MPM (i.e., most probable mode) list index that directs the list of most probable intra prediction modes onto the given intra prediction mode, and intra predict the given block using the given intra prediction mode. In other words, in this case, the apparatus is configured to form a list of most probable intra prediction modes based on the intra prediction mode used to predict a neighboring block adjacent to the given block. If the set-selective syntax element indicates that the given block should not be predicted using one of the first set of intra-prediction modes, the apparatus is configured to derive from the data stream a further index indicating a second set of matrix-based intra-prediction modes, i.e., a given matrix-based intra-prediction mode of the second set of intra-prediction modes including the matrix-based intra-prediction mode, by calculating a matrix-vector product between a vector derived from a reference sample in a neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra-prediction mode to obtain a prediction vector, and predicting a sample of the given block based on the prediction vector. In this case, the prediction is similar or equal to the ALWIP prediction described with respect to the embodiment of Figures 5 to 11, for example.The device is configured to perform the formation of a list of most likely intra prediction modes based on intra prediction modes using the time when adjacent blocks adjacent to a given block are predicted, such that the list of most likely intra prediction modes does not include the DC intra prediction mode if the adjacent blocks are predicted by any of the angular intra prediction modes. In other words, the device is configured to perform the formation of a list of most likely intra prediction modes based on intra prediction modes using the time when adjacent blocks adjacent to a given block are predicted, such that the list of most likely intra prediction modes does not include the DC intra prediction mode if the adjacent blocks are predicted exclusively by any of the angular intra prediction modes. Thus, the DC intra prediction mode does not occupy a position in the list of most likely intra prediction modes if the DC intra prediction mode is less likely to be selected for the given block.

[0005] This device introduces an advantageous and efficient method for determining an intra-prediction mode for a given block. In particular, an advantageous analysis of the predictions of neighboring blocks adjacent to the given block, where the neighboring blocks have already been predicted, is presented to form a list of most probable intra-prediction modes.

[0006] According to one embodiment, the device is configured to perform the formation of the list of most likely intra prediction modes based on the intra prediction modes used when neighboring blocks neighboring a given block are predicted, such that the list of most likely intra prediction modes is filled with DC intra prediction modes only if each of the neighboring blocks is predicted using one of at least one non-angular intra prediction modes having a first set including a DC intra prediction mode, or is predicted using one of the block-based intra prediction modes that are mapped to one of the at least one non-angular intra prediction modes by a mapping from a second set of block-based intra prediction modes to an intra prediction mode in the first set used for the formation of the list of most likely intra prediction modes. In other words, the list of most likely intra prediction modes includes DC intra prediction modes in case of prediction of all neighboring blocks, e.g. both neighboring blocks, using one of at least one non-angular intra prediction modes of the first set of intra prediction modes. Alternatively, the list of most probable intra-prediction modes includes the DC intra-prediction mode in case of prediction of all neighboring blocks, e.g. both neighboring blocks, using any of the block-based intra-prediction modes of the second set of intra-prediction modes, and the block-based intra-prediction modes are mapped from the second set of block-based intra-prediction modes to the non-angular intra-prediction modes in the first set. According to one embodiment, the device is configured to position the DC intra-prediction mode before any angular intra-prediction mode in the list of most probable intra-prediction modes. This is based on the idea that in the above case, the DC intra-prediction mode is the most probable mode for a given block, and this positioning thereby increases coding efficiency.

[0007] According to one embodiment, the apparatus is configured to derive MPM syntax elements from the data stream and form the list of most probable intra-prediction modes only if the MPM syntax elements indicate that a given intra-prediction mode of the first set of intra-prediction modes is in the list of most probable intra-prediction modes. This feature improves coding efficiency, since the list of most probable intra-prediction modes is only formed when necessary or advantageous.

[0008] If the given block should be predicted using one of the second set of intra-prediction modes, the device is configured to form a list of most probable block-based intra-prediction modes according to one embodiment. In this case, the device is configured to derive a further MPM list index from the data stream, for example, that indicates the list of most probable block-based intra-prediction modes to a given matrix-based intra-prediction mode, i.e., the given block-based intra-prediction mode. Optionally, this list of most probable block-based intra-prediction modes is formed only if a further MPM syntax element derived from the data stream indicates that the given block-based intra-prediction mode is in the list of most probable block-based intra-prediction modes.

[0009] Thus, the device is configured to form different MPM lists for, for example, the first set of intra-prediction modes and the second set of intra-prediction modes. The list of most probable intra-prediction modes comprises, for example, the intra-prediction modes of the first set of intra-prediction modes, and the list of most probable block-based intra-prediction modes comprises, for example, the intra-prediction modes of the second set of intra-prediction modes, i.e., the intra-prediction modes of the second set of block-based intra-prediction modes. This allows the block-based intra-prediction modes not to have to compete with the intra-prediction modes of the first set of intra-prediction modes, e.g., DC intra-prediction modes and angular prediction modes, for a position in the overall MPM list. This separation increases the likelihood that the intra-prediction mode of a given block is actually in the respective MPM list.

[0010] One embodiment relates to an apparatus for encoding a given block of an image using intra prediction, the apparatus being configured to signal a set-selective syntax element in a data stream indicating whether the given block should be predicted using one of a first set of intra prediction modes including a DC intra prediction mode and an angular prediction mode. Optionally, the first set of intra prediction modes may include a planar intra prediction mode in addition to or instead of a DC intra prediction mode. If the set-selective syntax element indicates that the given block should be predicted using one of the first set of intra prediction modes, the apparatus is configured to form a list of most probable intra prediction modes based on the intra prediction mode used when a neighboring block adjacent to the given block is predicted, signal an MPM list index in the data stream pointing the list of most probable intra prediction modes onto the given intra prediction mode, and intra predict the given block using the given intra prediction mode. In other words, in this case, the apparatus is configured to form a list of most probable intra prediction modes based on the intra prediction mode used to predict a neighboring block adjacent to the given block. If the set-selective syntax element indicates that the given block should not be predicted using one of the first set of intra-prediction modes, the apparatus is configured to signal a further index in the data stream indicating a second set of matrix-based intra-prediction modes, i.e., a given matrix-based intra-prediction mode of the second set of intra-prediction modes including the matrix-based intra-prediction mode, by calculating a matrix-vector product between a vector derived from a reference sample in a neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra-prediction mode to obtain a prediction vector, and predicting samples of the given block based on the prediction vector. In this case, the prediction is similar or equal to the ALWIP prediction described, for example, in relation to the embodiment of Figures 5 to 11.The list of most probable intra-prediction modes is formed based on the intra-prediction modes used when the neighboring blocks adjacent to the given block are predicted, such that if the neighboring blocks are predicted by any of the angular intra-prediction modes, the list of most probable intra-prediction modes does not include the DC intra-prediction mode. In other words, the list of most probable intra-prediction modes is formed based on the intra-prediction modes used when the neighboring blocks adjacent to the given block are predicted, such that if the neighboring blocks are predicted exclusively by any of the angular intra-prediction modes, the list of most probable intra-prediction modes does not include the DC intra-prediction mode.

[0011] One embodiment relates to a method for decoding a given block of an image using intra prediction, the method comprising: deriving a set-selective syntax element from a data stream, the set-selective syntax element indicating whether the given block should be predicted using one of a first set of intra prediction modes, including a DC intra prediction mode and an angular prediction mode. If the set-selective syntax element indicates that the given block should be predicted using one of the first set of intra prediction modes, the method comprises: forming a list of most probable intra prediction modes based on the intra prediction mode when a neighboring block adjacent to the given block is predicted; deriving an MPM list index from the data stream, the MPM list index pointing to the list of most probable intra prediction modes on the given intra prediction mode; and intra predicting the given block using the given intra prediction mode. If the set-selective syntax element indicates that the given block should not be predicted using one of the first set of intra-prediction modes, the method includes deriving from the data stream a further index indicating a given matrix-based intra-prediction mode of the second set of matrix-based intra-prediction modes by calculating a matrix-vector product between a vector derived from a reference sample of a neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra-prediction mode to obtain a prediction vector, and predicting a sample of the given block based on the prediction vector. The list of most probable intra-prediction modes is formed based on the intra-prediction mode using when a neighboring block neighboring the given block is predicted, such that if the neighboring block is predicted by any of the angular intra-prediction modes, the list of most probable intra-prediction modes does not include the DC intra-prediction mode.In other words, the list of most likely intra prediction modes is formed based on the intra prediction modes used when neighboring blocks neighboring a given block are predicted, such that if the neighboring blocks are predicted exclusively by any of the angular intra prediction modes, the list of most likely intra prediction modes does not include the DC intra prediction mode.

[0012] One embodiment relates to a method for encoding a given block of an image using intra prediction, the method comprising: signaling a set-selective syntax element in a data stream indicating whether the given block should be predicted using one of a first set of intra prediction modes, including a DC intra prediction mode and an angular prediction mode. If the set-selective syntax element indicates that the given block should be predicted using one of the first set of intra prediction modes, the method comprises forming a list of most probable intra prediction modes based on the intra prediction mode using when neighboring blocks adjacent to the given block are predicted, signaling an MPM list index in the data stream that directs the list of most probable intra prediction modes onto the given intra prediction mode, and intra predicting the given block using the given intra prediction mode. If the set-selective syntax element indicates that the given block should not be predicted using one of the first set of intra-prediction modes, the method includes signaling a further index in the data stream indicating the given matrix-based intra-prediction mode of the second set of matrix-based intra-prediction modes by calculating a matrix-vector product between a vector derived from reference samples of a neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra-prediction mode to obtain a prediction vector, and predicting samples of the given block based on the prediction vector. The list of most probable intra-prediction modes is formed based on the intra-prediction mode using when neighboring blocks neighboring the given block are predicted, such that if the neighboring blocks are predicted by any of the angular intra-prediction modes, the list of most probable intra-prediction modes does not include the DC intra-prediction mode.In other words, the list of most likely intra prediction modes is formed based on the intra prediction modes used when neighboring blocks neighboring a given block are predicted, such that if the neighboring blocks are predicted exclusively by any of the angular intra prediction modes, the list of most likely intra prediction modes does not include the DC intra prediction mode. One embodiment relates to a data stream having images encoded using the encoding method described herein. An embodiment relates to a computer program having a program code for performing the methods described herein, when the computer program runs on a computer. [Brief description of the drawings]

[0013] The drawings are not necessarily to scale, emphasis instead being placed upon illustrating generally the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Figure 1] FIG. 2 illustrates one embodiment of encoding into a data stream. [Diagram 2] FIG. 2 illustrates an embodiment of an encoder. [Diagram 3] FIG. 1 illustrates one embodiment of image reconstruction. [Figure 4] FIG. 2 illustrates an embodiment of a decoder. [Diagram 5] FIG. 2 is a schematic diagram of prediction of a block for encoding and / or decoding according to one embodiment. [Figure 6] FIG. 1 illustrates a matrix operation for prediction of a block for encoding and / or decoding according to one embodiment. [Figure 7.1] FIG. 1 illustrates a prediction of a block with a reduced sample value vector according to one embodiment. [Figure 7.2] FIG. 2 illustrates prediction of a block using sample interpolation according to one embodiment. [Figure 7.3]FIG. 1 illustrates a prediction of a block with reduced sample value vectors, where only some boundary samples are averaged, according to one embodiment. [Figure 7.4] 1 illustrates prediction of a block with a reduced sample value vector, where groups of four boundary samples are averaged, according to one embodiment. [Figure 8] FIG. 4 illustrates a matrix operation performed by an apparatus according to an embodiment. [Figure 9a] FIG. 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 9b-9c] FIG. 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 10] FIG. 13 illustrates detailed matrix operations performed by the device using offset and scaling parameters according to one embodiment. [Figure 11] 5A-5C illustrate detailed matrix operations performed by the device using offset and scaling parameters according to different embodiments; [Figure 12] 2 is a schematic diagram of an apparatus for decoding a given block according to one embodiment; [Figure 13] FIG. 2 is a schematic diagram illustrating details of decoding and encoding a given block according to one embodiment. [Figure 14] 1 is a schematic diagram of an apparatus for encoding a predetermined block according to an embodiment; [Figure 15] FIG. 2 is a block diagram of a method for decoding a given block according to one embodiment. [Figure 16] 2 shows a block diagram of a method for encoding a given block according to one embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] The same or equivalent elements, or elements having the same or equivalent functionality, are designated in the following description with the same or equivalent reference numbers even if they occur in different figures.

[0015] In the following description, a number of details are set forth to provide a more thorough description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention. Furthermore, features of different embodiments described later in this specification may be combined with each other unless otherwise specified. 1 Introduction

[0016] In the following, different examples, embodiments and aspects of the present invention are described, at least some of which relate to methods and / or apparatuses for, inter alia, video coding and / or for performing intra prediction using, for example, linear or affine transformation with adjacent sample reduction and / or for optimizing video distribution (e.g., broadcasting, streaming, file playback, etc.), for, for example, video applications and / or virtual reality applications.

[0017] Additionally, examples, embodiments, and aspects may refer to High Efficiency Video Coding (HEVC) or successors, and further embodiments, examples, and aspects are defined by the accompanying claims.

[0018] It should be noted that any embodiment, example and aspect defined by the claims can be supplemented by any of the details (features and functions) described in the following sections.

[0019] Also, the embodiments, examples and aspects described in the following sections can be used individually or can be supplemented by any of the features of the other sections or by any features included in the claims.

[0020] It should also be noted that the individual examples, embodiments, and aspects described herein can be used individually or in combination, and thus details can be added to each of the individual aspects described above without adding details to another one of the examples, embodiments, and aspects. It should also be noted that this disclosure explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0021] Moreover, features and functions disclosed herein in relation to the method can also be used in the apparatus. Moreover, any features and functions disclosed herein in relation to the apparatus can also be used in the corresponding method. In other words, the method disclosed herein can be complemented by any of the features and functions described in relation to the apparatus.

[0022] Additionally, any of the features and functionality described herein can be implemented in hardware or software, or using a combination of hardware and software, as described in the "Implementation Options" section.

[0023] Additionally, any of the features listed within brackets ("(...)" or "[...]") may be considered optional in some examples, embodiments, or aspects. 2 Encoder and Decoder

[0024] In the following, various examples are described that can help achieve more effective compression when using block-based prediction. Some examples achieve high compression efficiency by using a set of intra prediction modes. The latter may be added to other intra prediction modes, for example heuristically designed, or may be provided exclusively. Still other examples make use of both of the aforementioned specialties. However, as a permutation of these embodiments, intra prediction may be converted to inter prediction by using reference samples in another image instead.

[0025] To facilitate understanding of the following examples of the present application, the description begins with a presentation of possible encoders and matching decoders that can be constructed for the examples of the present application subsequently outlined. Figure 1 shows an apparatus for block-wise encoding of an image 10 into a data stream 12. The apparatus is indicated using the reference number 14 and may be a still image encoder or a video encoder. In other words, if the encoder 14 is configured to encode a video 16 that includes an image 10 into the data stream 12, the image 10 may be the current image of the video 16 or the encoder 14 may exclusively encode the image 10 into the data stream 12.

[0026] As mentioned above, the encoder 14 performs the encoding on a block-by-block basis. For this, the encoder 14 subdivides the image 10 into blocks, which the encoder 14 encodes into the data stream 12. Examples of possible subdivisions of the image 10 into blocks 18 are described in more detail below. In general, the subdivision can end up into blocks 18 of a fixed size, such as an array of blocks arranged in rows and columns, or blocks 18 of different block sizes, such as by using a hierarchical multi-tree subdivision starting from the entire image area of ​​the image 10, or from a pre-partitioning of the image 10 into an array of tree blocks, and these examples are not treated as excluding other possible ways of subdividing the image 10 into blocks 18.

[0027] Furthermore, the encoder 14 is a predictive encoder configured to predictively encode the image 10 into the data stream 12. For a particular block 18, this means that the encoder 14 determines a prediction signal for the block 18 and encodes into the data stream 12 the prediction residual, i.e. the prediction error where the prediction signal deviates from the actual image content in the block 18.

[0028] The encoder 14 may support different prediction modes to derive a prediction signal for a certain block 18. The prediction mode of interest in the following example is the intra prediction mode, according to which the inside of the block 18 is spatially predicted from neighboring already coded samples of the image 10. The coding of the image 10 into the data stream 12, and therefore the corresponding decoding procedure, may be based on a specific coding order 20 defined between the blocks 18. For example, the coding order 20 may traverse the blocks 18 in a raster scan order, such as row-by-row from top to bottom, while traversing each row from left to right. In case of a hierarchical multi-tree based subdivision, a raster scan ordering may be applied within each hierarchical level, or a depth-first traversal order may be applied, i.e. leaf nodes in a block of a certain hierarchical level may precede blocks of the same hierarchical level with the same parent block according to the coding order 20. Depending on the coding order 20, the neighboring already coded samples of the block 18 may be typically located on one or more sides of the block 18. In the example presented here, for example, the neighboring already coded samples of block 18 are located at the top and to the left of block 18 .

[0029] Intra-prediction modes may not be the only modes supported by encoder 14. For example, if encoder 14 is a video encoder, encoder 14 may also support inter-prediction modes in which blocks 18 are temporally predicted from previously encoded images of video 16. Such inter-prediction modes may be motion-compensated prediction modes according to which motion vectors are signaled for such blocks 18 indicating the relative spatial offset of the portion of which the prediction signal of block 18 is derived as a copy. Additionally or alternatively, other non-intra-prediction modes may be available, such as inter-prediction modes in the case of encoder 14 being a multiview encoder, or non-prediction modes in which the interior of block 18 is coded as is, i.e., without prediction.

[0030] Before starting the description of the present application by focusing on intra-prediction modes, we will give a more specific example of a possible implementation of a possible block-based encoder, i.e., encoder 14, as described with respect to FIG. 2, and then give two corresponding examples of decoders conforming to FIGS. 1 and 2, respectively.

[0031] Fig. 2 shows a possible implementation of the encoder 14 of Fig. 1, i.e. the encoder is configured to use transform coding to code the prediction residual, but this is mostly by way of example and the application is not limited to that kind of prediction residual coding. According to Fig. 2, the encoder 14 comprises a subtractor 22 configured to subtract a corresponding prediction signal 24 from an inbound signal, i.e. the image 10, or on a block basis, a current block 18, to obtain a prediction residual signal 26, which is coded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 comprises a lossy coding stage 28a and a lossless coding stage 28b. The lossy stage 28a comprises a quantizer 30 which receives the prediction residual signal 26 and quantizes the samples of the prediction residual signal 26. As already mentioned above, the present example uses a transform coding of the prediction residual signal 26, therefore the lossy coding stage 28a comprises a transform stage 32 connected between the subtractor 22 and a quantizer 30 to transform such a spectrally decomposed prediction residual 26, the quantization of which is performed on the transform coefficients representing the residual signal 26. The transformation may be a DCT, a DST, an FFT, a Hadamard transform, etc. The transformed and quantized prediction residual signal 34 is then losslessly coded into the data stream 12 by a lossless coding stage 28b, which is an entropy coder entropy coding quantized prediction residual signal 34. The encoder 14 further comprises a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner that is also usable by the decoder, i.e. the coding losses are taken into account in the quantizer 30. For this purpose, the prediction residual reconstruction stage 36 comprises an inverse quantizer 38 performing the inverse of the quantization of the quantizer 30, followed by an inverse transformer 40 performing an inverse transformation to the transformation performed by the transformer 32, such as the inverse of the spectral decomposition, such as the inverse for any of the specific transformation examples mentioned above. The encoder 14 comprises an adder 42 which adds the reconstructed prediction residual signal output by the inverse transformer 40 and the prediction signal 24 to output a reconstructed signal, i.e. reconstructed samples. This output is supplied to a predictor 44 of the encoder 14, which determines the prediction signal 24 on the basis thereof.This is a predictor 44 that supports all prediction modes already described above with respect to Figure 1. Figure 2 also shows that, if the encoder 14 is a video encoder, it may comprise an in-loop filter 46 that filters the fully reconstructed image, which, after being filtered, forms a reference image for the predictor 44 for inter-prediction blocks.

[0032] As already mentioned above, the encoder 14 operates on a block basis. For the purposes of the following description, the block basis of interest is the subdivision image 10 into blocks for which an intra prediction mode is selected from a set or a number of intra prediction modes supported by the predictor 44 or the encoder 14, respectively, and for which the selected intra prediction mode is implemented individually. However, there may also be other types of blocks into which the image 10 is subdivided. For example, the above-mentioned decision whether the image 10 is inter-coded or intra-coded may be made with a granularity or on a block-by-block basis that deviates from the block 18. For example, the inter / intra mode decision may be performed at the level of the coding blocks, where the image 10 is subdivided and each coding block is subdivided into prediction blocks. The prediction blocks having coding blocks for which it is determined that intra prediction is used are subdivided into respective intra prediction mode decisions. For this, for each of these prediction blocks, it is determined which supported intra prediction mode should be used for the respective prediction block. These prediction blocks form the block 18 of interest here. Predictive blocks in a coding block associated with inter prediction are treated differently by the predictor 44. They are inter-predicted from a reference image by determining a motion vector and copying the prediction signal of this block from the location in the reference image pointed to by the motion vector. Another block subdivision concerns the subdivision into transform blocks, the unit at which the transformation by the transformer 32 and the inverse transformer 40 is performed. The transformed blocks may be, for example, the result of further subdivision of the coding block. Of course, the embodiments described herein should not be treated as limiting, and other embodiments exist. For the sake of completeness only, it is noted that the subdivision into coding blocks may, for example, use multi-tree subdivision, and the predictive blocks and / or transform blocks may be obtained by further subdividing the coding block using multi-tree subdivision.

[0033] A decoder 54 or device for block-wise decoding, which is adapted to the encoder 14 of FIG. 1, is shown in FIG. 3. This decoder 54 does the opposite to the encoder 14, i.e. it decodes block-wise from the data stream 12 images 10, and for this purpose supports several intra prediction modes. The decoder 54 may, for example, comprise a residual provider 156. All other possibilities mentioned above with respect to FIG. 1 are also valid for the decoder 54. In contrast, the decoder 54 may be a still image decoder or a video decoder, with all prediction modes and predictability also being supported by the decoder 54. The difference between the encoder 14 and the decoder 54 mainly consists in the fact that the encoder 14 selects or chooses the coding decision according to some optimization, for example to minimize some cost function, which may depend on the coding rate and / or the coding distortion. One of these coding options or coding parameters may include the selection of the intra prediction mode to be used for the current block 18 from among the available or supported intra prediction modes. The selected intra-prediction mode may then be signaled by the encoder 14 of the current block 18 in the data stream 12, with the decoder 54 re-performing the selection using this signaling in the data stream 12 of the block 18. Similarly, the subdivision of the image 10 into blocks 18 may be the subject of optimization in the encoder 14, and the corresponding subdivision information may be conveyed in the data stream 12, with the decoder 54 restoring the subdivision of the image 10 into blocks 18 based on the subdivision information. To summarise the above, the decoder 54 may be a predictive decoder operating on a block basis, and in addition to the intra-prediction mode, the decoder 54 may support other prediction modes, such as inter-prediction modes, for example if the decoder 54 is a video decoder. In decoding, the decoder 54 may also use the coding order 20 described with reference to FIG. 1, which is respected in both the encoder 14 and the decoder 54, so that the same neighbouring samples are available for the current block 18 in both the encoder 14 and the decoder 54.Therefore, in order to avoid unnecessary repetitions, the description of the operating modes of the encoder 14 shall also apply to the decoder 54 as far as the subdivision of the image 10 into blocks is concerned, e.g. as far as the prediction is concerned and as far as the coding of the prediction residuals is concerned. The difference consists in the fact that the encoder 14 selects, by optimization, some coding options or coding parameters and signals and inserts in the data stream 12 the coding parameters which are subsequently derived from the data stream 12 by the decoder 54 in order to re-perform the prediction, subdivision, etc.

[0034] FIG. 4 shows a possible implementation of the decoder 54 of FIG. 3, i.e. one fitting to the implementation of the encoder 14 of FIG. 1 shown in FIG. 2. Many elements of the encoder 54 of FIG. 4 are the same as those occurring in the corresponding encoder of FIG. 2, so the same references with an apostrophe are used in FIG. 4 to indicate these elements. In particular, the adder 42', the optional in-loop filter 46' and the predictor 44' are connected in the prediction loop in the same way as in the encoder of FIG. 2. The reconstructed, i.e. inverse quantized and retransformed prediction residual signal applied to the adder 42' is derived by a sequence of entropy decoders 56 that reverse the entropy coding of the entropy coder 28b, followed by a residual signal reconstruction stage 36' consisting of an inverse quantizer 38' and an inverse transformer 40', as in the case of the encoding side. The output of the decoder is a reconstruction of the image 10. The reconstruction of image 10 may be available directly at the output of adder 42' or it may be available at the output of in-loop filter 46'. Some postfilters may be placed at the output of the decoder to subject the reconstruction of image 10 to some post-filtering to improve the image quality, although this option is not shown in FIG.

[0035] Again, with respect to Figure 4, the explanations given above with respect to Figure 2 are equally valid for Figure 4, except that only the encoder performs optimization tasks and related decisions regarding coding options, however, all the explanations regarding block subdivision, prediction, inverse quantization, and retransformation are also valid for the decoder 54 of Figure 4. 3. ALWIP (Affine Linear Weighted Intra Predictor)

[0036] Some non-limiting examples of ALWIP are described herein, even though ALWIP is not necessary to implement the techniques described herein.

[0037] The present application relates, inter alia, to an improved block-based prediction mode concept for block-based image coding, such as can be used in HEVC or any successor video codec of HEVC. The prediction mode may be an intra-prediction mode, but theoretically the concept described herein can also be transferred to an inter-prediction mode, where the reference sample is part of another image. There is a need for a block-based prediction concept that allows for efficient implementation, including hardware-friendly implementation. This object is achieved by the subject matter of the independent claims of the present application.

[0038] Intra prediction modes are widely used in image and video coding. In video coding, intra prediction modes compete with other prediction modes, such as inter prediction modes, such as motion-compensated prediction modes. In intra prediction modes, the current block is predicted based on neighboring samples, i.e. samples that have already been coded as far as the encoder side is concerned and already decoded as far as the decoder side is concerned. The neighboring sample values ​​are extrapolated to the current block to form a prediction signal for the current block, and the prediction residual is transmitted in the data stream for the current block. The better the prediction signal, the smaller the prediction residual, and therefore the fewer bits are needed to code the prediction residual.

[0039] To be effective, several aspects should be taken into account in order to form an effective framework for intra prediction in a block-wise image coding environment. For example, the more intra prediction modes supported by the codec, the higher the side information rate consumption for signaling the selection to the decoder. On the other hand, the set of supported intra prediction modes must be able to provide a good prediction signal, i.e., a prediction signal that results in a low prediction residual.

[0040] In the following, as a comparative embodiment or basic example, an apparatus (encoder or decoder) for block-wise decoding of an image from a data stream is disclosed, which supports at least one intra prediction mode in which an intra prediction signal for a block of a given size of the image is determined by applying a first template of samples neighboring the current block to an affine linear predictor, eventually called an Affine Linear Weighted Intra Predictor (ALWIP).

[0041] The apparatus may have at least one of the following characteristics (the same may apply to a method or another technique implemented, for example, in a non-transitory storage unit that stores instructions that, when executed by a processor, cause the processor to perform a method and / or operate as an apparatus): 3.1 Predictors can be complementary to other predictors.

[0042] The intra prediction modes, which may form the subject of implementation improvements described further below, may be complementary to other intra prediction modes of the codec. They may thus complement the DC, planar or angular prediction modes defined in the HEVC codec and the JEM reference software. The latter three types of intra prediction modes are hereafter referred to as conventional intra prediction modes. Thus, for a given block of an intra mode, a flag needs to be parsed by the decoder indicating whether one of the intra prediction modes supported by the device should be used or not. 3.2 Two or more proposed prediction modes

[0043] A device may contain more than one ALWIP mode, so if the decoder knows that one of the ALWIP modes supported by the device should be used, the decoder needs to parse additional information that indicates which of the ALWIP modes supported by the device should be used.

[0044] The signaling of supported modes may have the property that encoding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes require fewer bins and which require more bins may depend on information that may be extracted from the already decoded bitstream or may be fixed in advance. 4. Some Aspects

[0045] 2 illustrates a decoder 54 for decoding an image from the data stream 12. The decoder 54 may be configured to decode a given block 18 of the image. In particular, the predictor 44 may be configured to map a set of P neighboring samples of the given block 18 to a set of Q predictions of the samples of the given block using a linear or affine-linear transformation [e.g., ALWIP].

[0046] As shown in FIG. 5, a given block 18 contains Q values ​​to be predicted (which will be the "predicted values" at the end of the operation). If the block 18 has M rows and N columns, then Q=M·N. The Q values ​​of the block 18 can be in the spatial domain (e.g., pixels) or in the transform domain (e.g., DCT, discrete wavelet transform, etc.). The Q values ​​of the block 18 can be predicted based on P values ​​obtained from neighboring blocks 17a-17c, which are generally adjacent to the block 18. The P values ​​of the neighboring blocks 17a-17c may be located closest (e.g., adjacent) to the block 18. The P values ​​of the neighboring blocks 17a-17c have already been processed and predicted. The P values ​​are shown as values ​​in parts 17'a-17'c to distinguish them from the block of which they are a part (in some examples, 17'b is not used).

[0047] As shown in FIG. 6, to perform the prediction, it is possible to operate on a first vector 17P with P entries (each entry is associated with a specific position of the neighboring portions 17'a to 17'c), a second vector 18Q with Q entries (each entry is associated with a specific position of the block 18), and a mapping matrix 17M (each row is associated with a specific position in the block 18 and each column is associated with a specific position in the neighboring portions 17'a to 17'c). The mapping matrix 17M thus performs a prediction of P values ​​of the neighboring portions 17'a to 17'c to values ​​of the block 18 according to a predefined mode. The entries in the mapping matrix 17M can thus be understood as weighting coefficients. In the following sections, the symbols 17a to 17c are used instead of 17'a to 17'c to refer to the neighboring portions of the boundary.

[0048] Several conventional modes are known in the art, such as DC mode, planar mode, and 65-directional prediction mode, for example, 67 modes are known.

[0049] However, it should be noted that it is also possible to utilize a different mode, referred to here as a linear or affine-linear transformation, which includes P·Q weighting coefficients, of which at least ¼ P·Q weighting coefficients are non-zero, for each of the Q predictors, a sequence of P weighting coefficients for the respective predictor, when the sequence is arranged one above the other in raster scan order between the samples of a given block, forming an envelope that is omnidirectionally non-linear.

[0050] It is possible to map P positions of adjacent values ​​17'a to 17'c (templates), Q positions of adjacent samples 17'a to 17'c, and values ​​of P*Q weighting coefficients of matrix 17M. The plane is an example of an envelope of a sequence for DC transformation (is a plane for DC transformation). The envelope is obviously a plane and is therefore excluded by the definition of linear or affine linear transformation (ALWIP). Another example is a matrix resulting in the emulation of an angular mode, whose envelope is excluded from the ALWIP definition and, in plain terms, looks like a hill going diagonally from top to bottom along a direction in the P / Q plane. Planar mode and 65-directional prediction mode have different envelopes, which are linear in at least one direction, i.e., in all directions, for example, for the exemplified DC, and in the hill direction, for example, for the angular mode.

[0051] Conversely, the envelope of a linear or affine transformation is not omnidirectionally linear, it being understood that such types of transformations may, in some circumstances, be optimal for performing the prediction of block 18. Note that it is preferable that at least ¼ of the weighting coefficients are different from 0 (i.e., at least 25% of the P*Q weighting coefficients are different from 0).

[0052] The weighting coefficients may be unrelated to one another according to any regular mapping rule. Thus, the matrix 17M may be such that the values ​​of its entries have no obvious discernible relationship. For example, the weighting coefficients cannot be described by any analytical or differential function.

[0053] In an example, the ALWIP transformation may be performed by averaging the maximum value of the cross-correlation between the weighting coefficients of a first series associated with each predictor and the weighting coefficients of a second series associated with predictors other than the respective predictor, or an inverted version of the latter series, below a predefined threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold in the range between 0.05 and 0.035), even though the maximum value is higher. For example, the combination of the rows of the ALWIP matrix 17M (i 1 , i 2 ) for each i 1 The P value in the row is i 2 The cross-correlation may be calculated by multiplying the P values ​​of the first and second rows. For each cross-correlation obtained, the maximum value may be obtained. Thus, the average (mean) may be obtained for the entire matrix 17M (i.e., the maximum values ​​of the cross-correlation in all combinations are averaged). The threshold may then be, for example, 0.2 or 0.3 or 0.35 or 0.1, for example, a threshold in the range between 0.05 and 0.035.

[0054] The P adjacent samples of blocks 17a-17c may be arranged along a one-dimensional path that extends along the boundaries (e.g., 18c, 18a) of a given block 18. For each of the Q predicted values ​​of a given block 18, the set of P weighting factors associated with each predicted value may be ordered to traverse the one-dimensional path in a predetermined direction (e.g., left to right, top to bottom, etc.). In an example, the ALWIP matrix 17M may be off-diagonal or non-block diagonal. An example of an ALWIP matrix 17M for predicting a 4×4 block 18 from four already predicted neighboring samples may be: { { 37, 59, 77, 28}, { 32, 92, 85, 25}, { 31,69,100,24}, { 33,36,106,29}, { 24,49,104,48}, { 24,21,94,59}, { 29,0,80,72}, { 35,2,66,84}, { 32,13,35,99}, { 39,11,34,103}, { 45,21,34,106}, { 51,24,40,105}, { 50,28,43,101}, { 56,32,49,101}, { 61,31,53,102}, { 61,32,54,100} }.

[0055] (where {37, 59, 77, 28} is the first row of matrix 17M, {32, 92, 85, 25} is the second row, and {61, 32, 54, 100} is the 16th row). Matrix 17M has dimensions 16x4 and contains 64 weighting coefficients (as a result of 16*4=64). This is because matrix 17M has dimensions QxP, where Q=M*N is the number of samples of block 18 to be predicted (block 18 is a 4x4 block) and P is the number of samples of already predicted samples. Here, M=4, N=4, Q=16 (as a result of M*N=4*4=16), P=4. The matrix is ​​non-diagonal and non-block diagonal and is not described by any particular rule.

[0056] As can be seen, less than a quarter of the weighting factors are 0 (in the above matrix, 1 out of 64 weighting factors is 0). The envelope formed by these values, when placed one down in raster scan order, forms an envelope that is omnidirectionally nonlinear. Even though the above description is primarily described with reference to a decoder (eg, decoder 54), the same may be performed in an encoder (eg, encoder 14).

[0057] In some examples, for each block size (in the set of block sizes), the ALWIP transforms of the intra prediction modes in the second set of intra prediction modes for the respective block size are mutually distinct. Additionally or alternatively, the cardinalities of the second set of intra prediction modes for block sizes in the set of block sizes may match, but the associated linear or affine linear transforms of the intra prediction modes in the second set of intra prediction modes for different block sizes may not be transferable to each other by scaling.

[0058] In some examples, an ALWIP transformation may be defined to have "nothing in common" with a conventional transformation (e.g., an ALWIP transformation may have "nothing" to share with a corresponding conventional transformation, even though it is mapped via one of the mappings above).

[0059] In examples, the ALWIP mode is used for both the luma and chroma components, while in other examples, the ALWIP mode is used for the luma component but not for the chroma component. 5 Affine linear weighted intra prediction mode with coder acceleration (e.g. test CE3-1.2.1) 5.1 Description of the method or apparatus

[0060] The Affine Linear Weighted Intra Prediction (ALWIP) mode tested in CE3-1.2.1 may be the same as that proposed in JVET-L0199 under test CE3-2.2.2, except for the following changes:

[0061] · Multiple Reference Line (MRL) intra prediction, especially alignment with the encoder estimation and signaling, i.e. MRL is not combined with ALWIP and transmission of MRL indices is restricted to non-ALWIP blocks.

[0062] Subsampling is mandatory for all blocks W×H ≥ 32×32 (previously it was optional for 32×32). Therefore the additional test in the encoder and the sending of the subsampling flag have been removed.

[0063] ALWIP for 64×N and N×64 blocks (N≦32) has been added by downsampling to 32×N and N×32 respectively and applying the corresponding ALWIP mode. In addition, test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0064] · Joint mode estimation: Conventional and ALWIP modes use a shared Hadamard candidate list for full RD estimation, i.e., ALWIP mode candidates are added to the same list as conventional (and MRL) mode candidates based on their Hadamard cost.

[0065] EMT intrafast and PB intrafast are supported for binding mode lists, with additional optimizations to reduce the number of full RD checks. Only the MPMs of the available left and top blocks are added to the list for the ALWIP full RD estimation, following the same approach as in the conventional mode. 5.2 Complexity Assessment

[0066] Excluding calculations involving the discrete cosine transform, test CE3-1.2.1 required up to 12 multiplications per sample to generate the predicted signal. In addition, 492 parameters of 16 bits each were required for a total of 136. This corresponds to 0.273 megabytes of memory. 5.3 Experimental results

[0067] The test evaluation was performed according to the common test conditions JVET-J1010[2] for Intra-only (AI) and Random Access (RA) configurations using VTM software version 3.0.1. The corresponding simulations were performed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, Turbo Boost off) with Linux OS and GCC 7.2.1 compiler.

[0068] [Table 1]

[0069] [Table 2] 5.4 Affine Linear Weighted Intra Prediction with Reduced Complexity (e.g., test CE3-1.2.2)

[0070] The technique tested in CE2 is related to the "affine linear intra prediction" described in JVET-L0199[1], but simplifies it in terms of memory requirements and computational complexity.

[0071] A prediction matrix that covers all block shapes (e.g., S 0 , S 1 , S 2 , see also below) and bias vectors (e.g., to provide offset values). As a result, the number of parameters is reduced to 14400 10-bit values, which is

number

[0072] The input and output sizes of the predictor are further reduced. Furthermore, instead of transforming the boundaries through a DCT, averaging or downsampling can be performed on the boundary samples, and the generation of the prediction signal can use linear interpolation instead of an inverse DCT. As a result, up to four multiplications per sample may be required to generate the prediction signal. 6. Working Example Here we describe how to perform several predictions (e.g., as shown in Figure 6) using ALWIP prediction.

[0073] In principle, referring to Figure 6, multiplications should be performed of Q*P samples of the QxP ALWIP prediction matrix 17M with P samples of the Px1 neighboring vector 17P to obtain Q=M*N values ​​of the predicted MxN block 18. Thus, in general, at least P=M+N value multiplications are required to obtain each of the Q=M*N values ​​of the MxN block 18 to be predicted.

[0074] These multiplications have a highly undesirable effect. The dimension P of the boundary vector 17P generally depends on the number M+N of boundary samples (bins or pixels) 17a, 17c that are adjacent (for example, adjacent) to the M×N block 18 to be predicted. This means that the larger the size of the block 18 to be predicted, the larger the number of boundary pixels M+N (17a, 17c) will be, so the dimension P=M+N of the P×1 boundary vector 17P, the length of each row of the Q×P ALWIP prediction matrix 17M, and thus the number of multiplications required (generally speaking, Q=M*N=W*H, where W (Width) is another symbol for N, H (Height) is another symbol for M, and P=M+N=H+W if the boundary vector is formed by only one row and / or one column of samples).

[0075] This problem is generally exacerbated by the fact that in microprocessor-based systems (or other digital processing systems), multiplication is generally a power-consuming operation. It can be assumed that a large number of multiplications carried out on a large number of samples of a large number of blocks would generally cause an undesirable waste of computing power. It is therefore preferable to reduce the number of multiplications Q*P required to predict the M×N block 18 .

[0076] It is understood that by intelligently selecting easier to process operations to replace multiplications, it is possible to reduce in some way the computational power required for each intra prediction of each predicted block 18.

[0077] In particular, with reference to Figures 7.1 to 7.4, it has been understood that the encoder or decoder may predict a given block (e.g., 18) of an image using a number of adjacent samples (e.g., 17a, 17c), by reducing (e.g., in step 811) the number of adjacent samples (e.g., 17a, 17c) (e.g., by averaging or downsampling) to obtain a reduced set of sample values ​​having a lower number of samples compared to the number of adjacent samples, and by subjecting (e.g., in step 812) the reduced set of sample values ​​to a linear or affine-linear transformation to obtain a predicted value of a given sample of the given block.

[0078] In some cases, the decoder or encoder may also derive, for example by interpolation, a prediction value for a further sample of a given block based on the prediction value of the given sample and a number of adjacent samples, thus resulting in an upsampling strategy.

[0079] In an example, it is possible to perform several averages on the samples of boundary 17 (e.g., in step 811) so as to reach a reduced set 102 of samples with a reduced number of samples (Figures 7.1 to 7.4) (at least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). For example, if the original boundary has P = M + N samples, the reduced set of samples can have M red <M and N red <at least one of N, and P red <such that P red = M red + N red Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries, but has P red <P with P red × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have a Q × P dimension, but has at least P red <P for (M red <M and N red <at least one of N) a Q × P with the number of matrix elements reduced ( red (or Q red x P red , see below) for at least P.

[0080] In some examples (e.g., Figures 7.2, 7.3), the block obtained by ALWIP (in step 812) is

Number

Number

Number

Number

Number

Number

[0081] These techniques can be advantageous because while the matrix multiplication involves a reduced number of multiplications (Q red *P red Or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (or avoiding) multiplications. For example, downsampling, averaging, and / or interpolation can be performed (e.g., in steps 811 and / or 813) by employing binary operations that require non-computation power such as addition and shift. Also, addition is a very simple operation that can be easily performed without much computational effort.

[0082] This shifting operation can be used, for example, to average two boundary samples and / or to interpolate two samples (support values) of (or obtained from) the reduced prediction block to obtain the final prediction block. (For the interpolation, two sample values ​​are needed: within a block we always have two predefined values, but to interpolate samples along the left and top boundaries of a block, as in Figure 7.2, we only have one predefined value and therefore use the boundary sample as the support value for the interpolation.) below, First, add up the values ​​of the two samples; then halving the sum value (e.g., by right shifting); A two-step procedure such as: Or, first halving each of the samples (e.g., by left shifting); then summing the values ​​of the two halved samples; is possible.

[0083] An even easier operation can be performed when downsampling (eg, in step 811) since only one sample amount and a group of samples (eg, samples adjacent to each other) need to be selected.

[0084] It is therefore now possible to define a technique(s) for reducing the number of multiplications to be performed. Some of these techniques may be based, inter alia, on at least one of the following principles:

[0085] Even if the size of the actual predicted block 18 is M × N, it is possible to shrink the block (in at least one of the two dimensions) and obtain the reduced size Q red xP red (where:

number

Number

Number

[0086] In addition, or alternatively, instead of predicting all Q = M*N values of the block to be predicted 18 by multiplication, a reduced block with reduced dimensions (e.g.,

Number

Number

Number

[0087] According to the example shown in Fig. 7.1, a 4x4 block 18 (M=4, N=4, Q=M*N=16) is to be predicted, and the neighbourhoods 17 (vertical matrix with 4 already predicted samples) and 17c (horizontal row with 4 already predicted samples) of sample 17a have already been predicted in the previous iteration (neighborhoods 17a and 17c can be collectively denoted by 17). In advance, by using the formulas shown in Fig. 6, the prediction matrix 17M should be a QxP=16x8 matrix (due to Q=M*N=4*4 and P=M+N=4+4=8) and the boundary vector 17P should have an 8x1 dimension (due to P=8). However, this leads to the need to perform 8 multiplications for each of the 16 samples of the 4x4 block 18 to be predicted, and therefore to the need to perform 16*8=128 multiplications in total. (Note that the average number of multiplications per sample is a good estimate of computational complexity. Traditional intra prediction requires 4 multiplications per sample, which increases the computational effort involved. Therefore, it is possible to use this as an upper bound for ALWIP, ensuring that the complexity is reasonable and does not exceed that of traditional intra prediction.)

[0088] Nevertheless, by using this technique, in step 811, the number of samples 17a and 17c adjacent to the predicted block 18 can be reduced from P to P redIt is understood that it is possible to reduce it to . In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that operating as block 18 was a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus two samples are selected for row 17c and two samples are selected for column 17a. Thus, the horizontal row 17c is processed as having two samples (for example, averaged samples) instead of having four original samples, and the vertical column 17a, which originally had four samples, is processed as having two samples (for example, averaged samples). After subdividing row 17c and column 17a into groups 110 of two samples each, it can also be understood that a single sample is maintained (for example, the average of the samples in group 110 or a simple selection between the samples in group 110). Thus, by having only a set 102 of four samples (M red =2, N red =2, P red =M red +N red =4, P red <P), a so-called reduced set 102 of sample values is obtained.

[0089] It is understood that it is possible to perform operations (such as averaging or downsampling 100) without performing an excessive amount of multiplication at the processor level. That is, the averaging or downsampling 100 performed in step 811 can be easily obtained by simple and computationally non-power-consuming operations such as addition and shift.

[0090] At this point, it is understood that the reduced set of sample values ​​102 can be subjected to a linear or affine-linear (ALWIP) transform 19 (e.g., using a predictor matrix such as matrix 17M of FIG. 6 ). In this case, the ALWIP transform 19 directly maps the four samples 102 to sample values ​​104 of block 18. In this case, no interpolation is required.

[0091] In this case, the ALWIP matrix 17M has dimensions Q × P red = 16 × 4. This follows from the fact that all Q = 16 samples of the block 18 to be predicted are obtained directly by ALWIP multiplication (no interpolation is required).

[0092] Therefore, in step 812a, the dimensions Q×P red An appropriate ALWIP matrix 17M having A is selected. The selection may be based, for example, at least in part, on signaling from the data stream 12. The selected ALWIP matrix 17M is also k where k is the number of data streams 12 (possibly the matrix

number

[0093] In step 812b, the selected Q×P red ALWIP matrix 17M(A k(also denoted as P red A multiplication with the x1 boundary vector 17P is performed.

[0094] In step 812c, an offset value (e.g., b k ) can be added to all the acquired values ​​104 of the vector 18Q acquired by ALWIP, for example. k , or possibly even more

number

[0095] As can be appreciated, it is possible to obtain a suitable value in step 812 by relying on straightforward and computationally low power operations such as averaging (possibly adding and / or shifting and / or downsampling).

[0096] Referring to Figure 7.2, the predicted block 18 is here an 8x8 block (M=8, N=8) of 64 samples. Now, a priori, the prediction matrix 17M should have size QxP=64x16 (Q=M*N=8*8=64, where Q=64 with M=8 and N=8, and P=M+N=8+8=16). Thus, a priori, P=16 multiplications would be required for each of the Q=64 samples of the predicted 8x8 block 18, amounting to 64*16=1024 multiplications for the whole 8x8 block 18!

[0097] However, as seen in Figure 7.2, instead of using all 16 samples of the boundary, a method 820 can be provided in which only eight values ​​are used (e.g., the 4 in the horizontal boundary row 17c and the 4 in the vertical boundary column 17a between the original samples of the boundary). From the boundary column 17c, four samples may be used instead of eight (e.g., they may be an average of 2x2 and / or a selection of one sample out of two). Thus, the boundary vector is a Px1 = 16x1 vector, rather than a Px1 = 16x1 vector. red ×1=8×1 vectors only (P red =M red +N red =4+4). Instead of the original P=16 samples, P red It is understood that it is possible to select or average (for example 2×2) the samples of the horizontal rows 17c and the samples of the vertical columns 17a so as to have only Q=M*N=8*8=64 boundary values, forming a reduced set 102 of sample values. This reduced set 102 makes it possible to obtain a reduced version of the block 18, which has Q=M*N=8*8=64 (instead of Q=M*N=8*8=64). red =M red*N red = 4*4 = 16 samples. Size M red xN red = 4 × 4 blocks. The reduced version of block 18 includes the samples shown in grey in scheme 106 of Figure 7.2, and the samples shown in grey boxes (including samples 118' and 118'') are the Q obtained in step 812 of interest. red = 16 values, the 4x4 reduced block is obtained by applying the linear transformation 19 in object step 812. After obtaining the values ​​of the 4x4 reduced block, it is possible to obtain the values ​​of the remaining samples (shown as white samples in scheme 106) for example by interpolation.

[0098] As with method 810 in FIG. 7.1, this method 820 estimates the remaining QQ of the predicted M×N=8×8 block 18. red It may further include a step 813 of deriving predicted values ​​for the remaining QQ = 64 to 16 = 48 samples (white squares), for example by interpolation. red Q = 64 ~ 16 = 48 samples are obtained directly by interpolation (interpolation can also use the values ​​of the boundary samples, for example) red = 16 samples. As can be seen in Figure 7.2, samples 118' and 118'' are obtained in step 812 (as shown by grey boxes), while sample 108' (which is intermediate between samples 118' and 118'' and shown by white boxes) is obtained by interpolation between samples 118' and 118'' in step 813. It is understood that the interpolation can also be obtained by operations similar to those for averaging, such as shifting and adding. Thus, in Figure 7.2, value 108' can generally be determined as a value intermediate between the value of sample 118' and the value of sample 118'' (can be the average).

[0099] It is also possible to perform an interpolation to arrive at the final version of the M×N=8×8 block 18 in step 813 based on the multiple sample values ​​shown in 104 . Therefore, when comparing using this technology with not using it, Without this technology: A prediction target block 18 with dimensions M=8, N=8, and Q=M*N=8*8=64 samples in the prediction target block 18; P=M+N=8+8=16 samples within boundary 17, P = 16 multiplications for each of the predicted Q = 64 values; Total number of P*Q = 16*64 = 1028 multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=16. In this technology, Block 18 to be predicted, with dimensions M=8, N=8 Q=M*N=8*8=64 final predicted values,

[0100] However, the Q used red xP red The ALWIP matrix is ​​P red =M red +N red , Q red =M red *N red , M red =4, N red = 4, P red =M red +N red = 4 + 4 = 8 samples within the boundary, P red <P P red = Q of predicted 4x4 reduced blocks red = 8 multiplications for each of the 16 values ​​(formed by grey squares in scheme 106), P red *Q red Total number of times = 8 * 16 = 128 times (far less than 1024!) The ratio of the number of multiplications to the number of final values ​​to be obtained is P red *Q red / Q = 128 / 64 = 2 (much less than 16 obtained without using this technology!) Therefore, the technology presented in this specification requires 8 times less power than the previous technology.

[0101] Figure 7.3 shows another example (obtainable based on method 820), where the predicted block 18 is a rectangular 4×8 block (M = 8, N = 4) with Q = 4 * 8 = 32 predicted samples. The boundary 17 is formed by a row 17c of N = 8 samples and a column 17a of M = 4 samples. Therefore, beforehand, the boundary vector 17P has dimension P×1 = 12×1, but the predicted ALWIP matrix should be a Q×P = 32×12 matrix, and thus Q * P = 32 * 12 = 384 multiplications are required.

[0102] However, for example, it is possible to average or downsample at least 8 samples of the horizontal row 17c to obtain a reduced horizontal row of only 4 samples (e.g., averaged samples). In some examples, the vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimension P red = 8, and P red < P. Therefore, the boundary vector 17P has dimension P red ×1 = 8×1. The ALWIP prediction matrix 17M becomes a matrix with dimension M * N red * P red = 4 * 4 * 8 = 64. The 4×4 reduced block (formed by the gray columns of schema 107) directly obtained in the target step 812 has size Q red = M * N red = 4 * 4 = 16 samples (instead of Q = 4 * 8 = 32 of the original 4×8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, the offset value b k(step 812c) and it is possible to perform the interpolation in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18 and the value 108′ not obtained in step 812 is obtained in step 813 by interpolating the values ​​118′ and 118″ (grey squares) obtained in step 812. Therefore, when comparing using this technology with not using it, Without this technology: Prediction target block 18 with dimensions M=4 and N=8 Q=M*N=4*8=32 predicted values, P = M + N = 4 + 8 = 12 samples at the boundary, P = 12 multiplications for each of the predicted Q = 32 values, Total number of P*Q = 12*32 = 384 multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=12. In this technology, Prediction target block 18 with dimensions M=4 and N=8 Q=M*N=4*8=final 32 predicted values,

[0103] However, Q red xP red = 16 × 8 ALWIP matrices can be used, M = 4, N red =4, Q red =M*N red =16, P red =M+N red =4+4=8, P red =M+N red = 4 + 4 = 8 samples within the boundary, P red <P P red = Q of the reduced block to be predicted red = 8 multiplications for each of the 16 values Q red *P red Total number of times = 16 * 8 = 128 times (less than 384!) The ratio of the number of multiplications to the number of final values ​​to be obtained is P red*Q red / Q=128 / 32=4 (much less than the 12 obtained without this technique!). Therefore, the computational effort is reduced by a factor of 3 with this technique.

[0104] Figure 7.4 shows the example of block 18 to be predicted with dimensions M×N=16×16, the final predicted value is Q=M*N=16*16=256 and P=M+N=16+16=32 boundary samples. This results in a prediction matrix with dimensions Q×P=256×32, which means 256*32=8192 multiplications!

[0105] However, by applying method 820, in step 811 it is possible to reduce (e.g., by averaging or downsampling) the number of boundary samples, for example from 32 to 8, leaving, for example, a single sample (e.g., selected from among the four samples or an average of the samples) for each group 120 of four consecutive samples in row 17a, and one sample (e.g., selected from among the four samples or an average of the samples) for each group of four consecutive samples in column 17c.

[0106] Here, the ALWIP matrix 17M is Q red xP red = 64 × 8 matrix. This means that it is possible to reduce the P red = 8, and the fact that the reduced block predicted in step 812 is an 8x8 block (in scheme 109 the grey squares are 64).

[0107] Thus, once the 64 samples of the reduced 8×8 block are obtained in step 812, the remaining QQ of the predicted block 18 are red It is possible to derive 256-64=192 values ​​104.

[0108] In this case, it has been chosen to use all samples in boundary columns 17a and only the substitute samples in boundary rows 17c to perform the interpolation. Other choices may be made.

[0109] In this method, the ratio between the number of multiplications and the number of values ​​finally obtained is Q red *P red / Q=8*64 / 256=2, which is much less than the 32 multiplications of each value without this technique! The comparison between using this technology and not using it is as follows: Without this technology: Prediction target block 18 with dimensions M=16, N=16 Q=M*N=16*16=256 predicted values, P=M+N=16*16=32 samples within the boundary, P = 32 multiplications for each of the predicted Q = 256 values, The total number of P*Q multiplications is 32*256=8192. The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=32. In this technology, Prediction target block 18 with dimensions M=16, N=16 Q=M*N=16*16=final 256 predicted values,

[0110] However, the Q used red xP red =64×8 ALWIP matrix is ​​M red =4, N red =4, Q red = 8*8 = 64 samples predicted by ALWIP, P red =M red +N red =4+4=8. P red =M red +N red = 4 + 4 = 8 samples within the boundary, P red <P P red = Q of the reduced block to be predicted red = 8 multiplications for each of the 64 values Q red *P red Total number of times = 64 * 4 = 256 times (less than 8192!) The ratio between the number of multiplications and the number of final values ​​obtained is P red *Q red / Q=8*64 / 256=2 (much less than the 32 obtained without this technique!). Therefore, the computational power required by the present technique is 16 times less than conventional techniques. therefore,

[0111] reducing (100, 813) a number of adjacent samples to obtain a set of reduced sample values ​​(102) having a smaller number of samples compared to the number of adjacent samples (17);

[0112] subjecting (812) the reduced set of sample values ​​(102) to a linear or affine linear transformation (19, 17M) to obtain a predicted value of a given sample (104, 118', 188'') of a given block (18), whereby A number of adjacent samples (17) can be used to predict a given block (18) of an image.

[0113] In particular, the reduction (100, 813) can be performed by downsampling a number of adjacent samples to obtain a reduced set (102) of sample values ​​having a smaller number of samples compared to the number of adjacent samples (17).

[0114] Alternatively, the reduction (100, 813) can be performed by averaging multiple adjacent samples to obtain a reduced set of sample values ​​(102) that has a smaller number of samples compared to the multiple adjacent samples (17).

[0115] Further, it is possible to derive (813) a predicted value for a further sample (108, 108') of a given block (18) by interpolation based on the predicted values ​​of the given sample (104, 118', 118'') and a number of adjacent samples (17).

[0116] The plurality of adjacent samples (17a, 17c) may extend in one dimension along two sides (e.g., toward the right and bottom in FIGS. 7.1-7.4) of the given block (18). The given samples (e.g., those obtained by ALWIP in step 812) may also be arranged in rows and columns, and may be arranged at every nth position from the given sample 112 samples (112) adjacent to both sides of the given block 18 along at least one of the rows and columns.

[0117] A support value (118) of one of a plurality of adjacent positions (118) aligned with each of the rows and / or columns can be determined for each of the rows and / or columns based on the plurality of adjacent samples (17). A predicted value 118 of a further sample (104, 118', 118'') of the given block (18) can also be derived by interpolation based on the predicted value of the given sample (108, 108') and the support values ​​of the adjacent samples (118) aligned with each of the rows and / or columns.

[0118] A given sample (104) may be located at every n-th position from the samples (112) adjacent to the two sides of a given block 18 along a row, and a given sample is located at every m-th position from the samples (112) of a given sample adjacent to the two sides of a given block (18) along a column, where n, m>1. In some cases, n=m (e.g., in Figures 7.2 and 7.3, samples 104, 118', 118'' obtained directly by ALWIP in 812 and shown in grey boxes are alternated along rows and columns with samples 108, 108' subsequently obtained in step 813).

[0119] It may be possible to perform the step of determining the support value by downsampling or averaging (122) a group (120) of adjacent samples in a plurality of adjacent samples including the adjacent sample (118) for which the respective support value is determined along at least one of the rows (17c) and columns (17a). Thus, in FIG. 7.4, in step 813 it is possible to obtain the value of sample 119 by using the value of a given sample 118''' (previously obtained in step 812) and the values ​​of the adjacent samples 118 as support values.

[0120] The multiple adjacent samples may extend in one dimension along two sides of a given block 18. It may be possible to perform the reduction 811 by grouping the multiple adjacent samples 17 into one or more groups 110 of consecutive adjacent samples and performing downsampling or averaging on each of the one or more groups 110 of adjacent samples having two or more adjacent samples.

[0121] In the example, a linear or affine linear transformation is red *Q red or P red *Q weighting factor may be included, P red is the number of sample values ​​in the reduced set of sample values ​​(102), and Q red or Q is the number of given samples in a given block (18). red *Q red or 1 / 4 P red *Q weighting coefficients are non-zero weighting values. P red *Q red or P red *Q weighting factors are Q or Q red For each of the given samples, a set of P red A set of P redThe weighting coefficients, when arranged up and down in raster scan order between a given sample of a given block (18), form an envelope that is nonlinear in all directions. red *Q or P red *Q red The weighting factors may be independent of each other through a regular mapping rule. The maximum value of the cross-correlation between the weighting factors of the first sequence associated with each given sample and the weighting factors of the second sequence associated with given samples other than each given sample, or the inverted version of the latter sequence, is lower than a predetermined threshold, even though the maximum value is higher. The predetermined threshold may be 0.3 [or possibly 0.2 or 0.1]. P red The adjacent samples (17) may be arranged along a one-dimensional path extending along either side of a given block (18), and may be Q or Q red For each of the given samples, a set of P red The weighting factors are ordered to traverse a one-dimensional path in a given direction. 6.1 Method and Apparatus Description

[0122] width

number

number

number

[0123] 1. Of the boundary samples 17, samples 102 (e.g., 4 samples when W=H=4, and / or 8 samples in other cases) can be extracted by averaging or downsampling (e.g., step 811).

[0124] 2. A matrix-vector multiplication followed by the addition of an offset may be performed with the averaged samples (or samples remaining from the downsampling) as input. The result may be a reduced prediction signal (e.g., step 812) on the set of subsampled samples in the original block.

[0125] 3. Prediction signals for the remaining positions may be generated, for example, by upsampling, for example, from prediction signals on the subsampled set, for example by linear interpolation (eg, step 813).

[0126] Thanks to steps 1. (811) and / or 3. (813), the total number of multiplications required to compute the matrix-vector product is always

number

[0127] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b k ) may be, for example, a set (e.g., three sets) of matrices that may be stored in the storage units of the decoder and the encoder, e.g.

number

number

[0128] In some cases, the set

number

number

number

number

number

number

number

[0129] In some instances,

number

number

number

number

number

number

number

number

number

number

number

[0130] Additionally or instead:

number

number

number

number

number

number

[0131] As described above, the boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0132] In the first step, the input boundary

Number

Number

Number

Number

Number

Number

Number

[0133] Similarly

Number

Number

number

number

number

[0134] In all other cases (e.g., for blocks with a wither width or height different from 4), the block width W

number

number

number

number

[0135] In still other cases, the boundary may be downsampled (e.g., by selecting one particular boundary sample from a group of boundary samples) to reduce the number of samples. For example,

number

number

number

number

number

number

number

[0136] Two reduced boundaries

number

number

number

number

number

number

number

number

number

number

[0137] Therefore, in a particular state (

number

number

[0138] Other strategies may be implemented. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, the three sets S 0 , S 1 , S 2 Each of these consists of 18 matrices (hence,

number

number

number

number

[0139] The mode and transpose information are not necessarily stored and / or transmitted as one combined mode index, "mode." That is, in some instances, the transpose flag and the matrix index (S 0 0 to 15 for S 1 0~7 for S 2 In some cases, it may be signaled explicitly as 0–5).

[0140] In some cases, a combination of a transpose flag and a matrix index may be interpreted as a set index. For example, there may be one bit that acts as a transpose flag and several bits that indicate a matrix index, collectively denoted as a "set index." 6.3 Generating reduced prediction signals by matrix-vector multiplication Characteristics are provided here with respect to step 812.

[0141] The reduced input vector

number

number

number

number

number

number

number

number

number

number

number

[0142] Where:

number

number

number

number

number

number

number

number

[0143] Matrix A and vector

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0144] Other strategies may be implemented. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, the three sets S 0 , S 1 , S 2 Each of these consists of 18 matrices (hence,

number

number

number

number

[0145] Interpolation of the subsampled prediction signal, for large blocks, a second version of the averaged boundary may be needed, i.e.

number

number

number

number

number

number

number

number

[0146] Additionally or alternatively, it is possible to have a "hard downsampling"

number

number

[0147] Also,

number

number

number

number

[0148] The linear interpolation may be given as follows (other examples are possible):

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0149] Where:

number

number

number

number

number

number

number

number

[0150] This is an example of an interpolation that uses reduced boundary samples for the first interpolation (horizontal or vertical) and the original boundary samples for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation or no interpolation is needed. If both horizontal and vertical interpolation are needed, the order depends on the width and height of the block. However, different techniques may be implemented, for example the original boundary samples may be used for both the first and second interpolation, and the order may be fixed, for example first horizontal, then vertical (or in other cases first vertical, then horizontal). Thus, the interpolation order (horizontal / vertical) and the use of shrink / original boundary samples can be changed.

[0151] 6.5 Explaining the Overall ALWIP Process Example The whole process of averaging, matrix vector multiplication, and linear interpolation is illustrated for different shapes in Figure 7.1 to Figure 7.4. Note that the remaining shapes are treated like one of the illustrated cases.

[0152] 1.

number

number

number

[0153] 2.

number

number

number

[0154] 3.

number

number

number

[0155] 4.

number

number

number

number

number

number

[0156] 6.6 Evaluating the Number and Complexity of Required Parameters The necessary parameters for all possible proposed intra prediction modes are given in the set

number

number

number

[0157] 6.7 Proposed Signaling of Intra Prediction Modes For luma blocks, for example, 35 ALWIP modes are proposed (other numbers of modes may be used). For each coding unit (CU) in intra mode, a flag is transmitted in the bitstream indicating whether an ALWIP mode should be applied to the corresponding prediction unit (PU) or not. The signaling of the latter indicator can be harmonized with the MRL in the same way as for the first CE test. If an ALWIP mode is applied, the index of the ALWIP mode

number

[0158] Here, the derivation of the MPM may be done using the intra modes of the above and left PUs as follows:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0159] In all other cases:

number

number

number

number

number

number

number

[0160] The embodiments described herein are not limited by the above-mentioned signaling of the proposed intra-prediction modes. According to alternative embodiments, MPM and / or mapping tables are not used for MIP (ALWIP).

[0161] 6.8 Adaptive MPM List Derivation for Conventional Luma Intra Prediction Mode and Chroma Intra Prediction Mode The proposed ALWIP mode can be harmonized with MPM-based coding of conventional intra prediction modes as follows: The luma and chroma MPM list derivation process of conventional intra prediction modes is based on a fixed table

number

number

number

number

number

number

[0162] 7. Efficient implementation The above examples are briefly summarized as they may form the basis for further extensions of the embodiments described below. To predict a given block 18 of an image 10, the use of multiple adjacent samples 17a, 17c is used. The reduction 100 by averaging of multiple adjacent samples is performed to obtain a reduced set of sample values ​​102 with a smaller number of samples compared to the multiple adjacent samples. This reduction is optional in the embodiments herein and results in a so-called sample value vector, which will be mentioned below. The reduced set of sample values ​​undergoes a linear or affine linear transformation 19 to obtain a predicted value of a given sample 104 of a given block. This transformation is later shown using a matrix A and an offset vector b, which can be obtained by machine learning (ML) and whose implementation should be performed efficiently.

[0163] 2. By interpolation, a predicted value of a further sample 108 of a given block is derived based on the predicted values ​​of the given sample and a number of adjacent samples. Theoretically, it can be said that the result of the affine / linear transformation can be associated with a non-full-pel sample position of the block 18, such that all samples of the block 18 can be obtained by interpolation according to an alternative embodiment. Interpolation may not even be necessary at all.

[0164] The plurality of adjacent samples may extend in one dimension along two sides of the predetermined block, and the predetermined samples may be arranged in rows and columns and along at least one of the rows and columns, and the predetermined samples may be arranged at every n-th position from the samples (112) of the predetermined samples adjacent to the two sides of the predetermined block. Based on the plurality of adjacent samples, for each of the rows and at least one of the columns, a support value of one of the plurality of adjacent positions (118) may be determined, which is aligned to each of the rows and at least one of the columns, and by interpolation, a predicted value of a further sample 108 of the predetermined block may be derived based on the predicted value of the predetermined sample and the support value of the adjacent sample aligned to at least one of the rows and at least one of the columns. The predetermined sample may be arranged at every n-th position from the samples 112 of the predetermined samples adjacent to both sides of the predetermined block along the rows, and the predetermined sample may be arranged at every m-th position from the samples 112 of the predetermined samples adjacent to both sides of the predetermined block along the columns, where n, m>1. n may be m. Along at least one of the rows and columns, the determination of the support values ​​may be made by averaging (122) for each support value a group 120 of adjacent samples within the plurality of adjacent samples that includes the adjacent sample 118 for which the respective support value is determined. The plurality of adjacent samples may extend in one dimension along two sides of a given block, and the reduction may be made by grouping the plurality of adjacent samples into one or more groups 110 of contiguous adjacent samples and performing the averaging for each of the one or more groups of adjacent samples having three or more adjacent samples.

[0165] For a given block, a prediction residual may be transmitted in the data stream, from which it may be derived at the decoder, where the given block is reconstructed using the prediction residual and the predicted values ​​of the given samples, and at the encoder, where the prediction residual is coded into the data stream. The image may be subdivided into a number of blocks of different block sizes. These number of blocks comprise the given block. Then, a linear or affine linear transformation of the block 18 is selected according to the width W and height H of the given block, the linear or affine linear transformation selected for the given block is selected from the first set of linear or affine linear transformations as long as the width W and height H of the given block are within a first set of width / height pairs, and the second set of linear or affine linear transformations is selected as long as the width W and height H of the given block are within a second set of width / height pairs different from the first set of width / height pairs. Again, it will become clear later that the affine / linear transformation is represented by other parameters, namely the weights C, as well as optionally offset and scale parameters.

[0166] The decoder and encoder subdivide the image into a number of blocks of different block sizes, including a predetermined block, and the linear or affine linear transformation selected for the predetermined block is: from a first set of linear or affine-linear transformations as long as the width W and height H of a given block are within a first set of width / height pairs; a second set of linear or affine-linear transformations are selected from the second set of linear or affine-linear transformations so long as the width W and the height H of a given block are within a second set of width / height pairs that are not common to the first set of width / height pairs; and selecting a linear or affine-linear transformation depending on the width W and height H of the given block such that the linear or affine-linear transformation is selected from a third set of linear or affine-linear transformations as long as the width W and height H of the given block are within a third set of one or more width / height pairs, independent of the first and second sets of width / height pairs. The third set of one or more width / height pairs includes simply one width / height pair W', H', where each linear or affine linear transformation in the first set of linear or affine linear transformations is for transforming an N' sample value to a W'*H' predicted value for a W'xH' array of sample locations.

[0167] Each of the first and second sets of width / height pairs is W p H p First width / height vs. W not equal p , H p And, H q =W p and Wq = H p The second width / height pair W q , H q and Each of the first and second sets of width / height pairs is a third width / height pair W p , H p and W p H p is equal to H p >H q It is. For a given block, a set index may be transmitted in the data stream that indicates which linear or affine-linear transform of a given set of linear or affine-linear transforms should be selected for the block 18.

[0168] The multiple adjacent samples may extend in one dimension along two sides of the predetermined block, and the reduction may be performed by, for a first subset of the multiple adjacent samples adjacent to a first side of the predetermined block, grouping the first subset into a first group 110 of one or more consecutive adjacent samples, and for a second subset of the multiple adjacent samples adjacent to a second side of the predetermined block, grouping the second subset into a second group 110 of one or more consecutive adjacent samples, and performing averaging on each of the first and second groups of one or more adjacent samples having two or more adjacent samples to obtain a first sample value from the first group and a second sample value of the second group. A linear or affine linear transformation may then be selected in response to a set index from a predefined set of linear or affine linear transformations, such that two distinct states of the set index result in a selection of one of the linear or affine linear transformations of the predefined set of linear or affine linear transformations, the reduced set of sample values ​​being subjected to the predefined linear or affine linear transformation to generate an output vector of predicted values ​​for the set index assuming a first of the two distinct states in the form of a first vector, the predicted values ​​of the output vector being distributed along the first scan order to the predefined samples of the predefined block, and ... second of the two distinct states in the form of a second vector. For a set index assumed in the form of a vector of , the first and second vectors are populated by one of the second sample values ​​in the first vector and one of the sample values ​​of the first sample value in the second vector, such that a component populated by one of the second sample values ​​in the first vector is populated by one of the first sample values ​​in the second vector, generating an output vector of predicted values ​​and distributing the predicted values ​​of the output vector along the second scan order over a given sample of a given block transposed with respect to the first scan order.

[0169] Each linear or affine linear transformation in the first set of linear or affine linear transformations is 1 The sample value is taken as the w 1 ×h 1 w about arrays 1 *h 1 and each linear or affine linear transformation in the second set of linear or affine linear transformations may be for transforming the predicted value into a 2 The sample value is taken as the w 2 ×h 2 w about arrays 2 *h 2 For a first given one of the width / height pairs, w 1 can exceed the width of the first predetermined width / height pair, or h 1 can exceed the height of the first predetermined width / height pair, and for one of the first second predetermined ones of the width / height pairs, w 1 The step of reducing (100) a plurality of adjacent samples to obtain a reduced set of sample values ​​(102) by averaging may be performed such that, when a given block is of the first predetermined width / height pair, and when a given block is of the second predetermined width / height pair, the reduced set of sample values ​​102 is N 1 10 sample values, and subjecting the reduced set of sample values ​​to a selected linear or affine-linear transformation may be performed to have w 1 ×h 1 may be performed using only a first sub-part of a selected linear or affine linear transformation associated with the sub-sampling of the array, w 1 along the width dimension, or, if h exceeds the width of one width / height pair, 1 Along the height dimension if exceeds the height of one width / height pair, the selected linear or affine-linear transformation is performed completely if the given block is a first given width / height pair and if the given block is a second given width / height pair.

[0170] Each linear or affine linear transformation in the first set of linear or affine linear transformations is 1 =h 1 At sample position w 1 ×h 1 About Array 1 Sample value w 1 *h 1 and each linear or affine linear transformation in the second set of linear or affine linear transformations may be for transforming w 2 =h 2 At sample position w 2 For an N × h2 array 2 Sample value w 2 *h 2 This is to convert it into a predicted value. All the above embodiments are merely exemplary in that they may form the basis of the embodiments described herein below. That is, the above concepts and details are useful for understanding the following embodiments and serve as a reservoir for possible extensions and modifications of the embodiments described later in this specification. In particular, many of the above details are optional, such as the averaging of adjacent samples, the fact that adjacent samples are used as reference samples, etc. More generally, the embodiments described herein assume that a prediction signal on a rectangular block is generated from already reconstructed samples, such as an intra prediction signal on a rectangular block is generated from already reconstructed samples of the left and top neighbors of the block. The generation of the prediction signal is based on the following steps:

[0171] 1. However, without excluding the possibility of transferring the description to a reference sample located elsewhere, from a reference sample, called the boundary sample, samples can be extracted by averaging. Here, averaging is performed on both the left and top boundary samples of the block, or only on the boundary samples of one of the two sides. If no averaging is performed on one side, the samples on that side remain unchanged. 2. A matrix-vector multiplication is performed, optionally followed by an offset, where the input vector of the matrix-vector multiplication is either the concatenation of the left of the block's averaged boundary sample with the original boundary sample above the block, if averaging was applied to the left side only, or the concatenation of the left of the block's original boundary sample with the averaged boundary sample above the block, if averaging was applied to the top side only, or the concatenation of the left of the block's averaged boundary sample with the averaged boundary sample above the block, if averaging was applied to both sides of the block. Again, alternatives exist, such as no averaging at all.

[0172] 3. The result of the matrix-vector multiplication and optional offset addition may optionally be a reduced prediction signal on a set of subsampled samples in the original block. Predictions at the remaining positions may be generated from the prediction signals on the subsampled sets by linear interpolation. The matrix-vector product calculation in step 2 should preferably be performed in integer arithmetic. Thus,

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0173] FIG. 8 illustrates an improved ALWIP prediction. Samples of a given block may be predicted based on a first matrix-vector product between a matrix A 1100 derived by some machine learning based training algorithm and a sample value vector 400. Optionally, an offset b 1110 may be added. To achieve an integer or fixed point approximation of this first matrix-vector product, the sample value vector may undergo a reversible linear transformation 403 to determine a further vector 402. A second matrix-vector product between a further matrix B 1200 and the further vector 402 may be equal to the result of the first matrix-vector product. Due to the characteristics of the further vector 402, the second matrix-vector product can be integer-approximated by a matrix-vector product 404 between the predefined prediction matrix C 405, the further vector 402 and the further offset 408. The further vector 402 and the further offset 408 can consist of integer or fixed-point values. All components of the further offset are, for example, the same. The predefined prediction matrix 405 can be a quantized matrix or a matrix to be quantized. The result of the matrix-vector product 404 between the predefined prediction matrix 405 and the further vector 402 can be understood as a prediction vector 406.

[0174] Further details regarding this integer approximation are provided below. Possible solutions according to example I: Subtracting and adding average values Usable expressions for the above scenarios

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0175] The specified value 1400 is not necessarily the average value.

number

number

number

number

number

number

number

number

number

number

number

number

[0176] According to one embodiment, the device 1000 is configured to include a number of reversible linear transforms 403, each of which is associated with a component of the further vector 402. The device is also configured to, for example, select a given component 1500 of the components of the sample value vector 400 and use a reversible linear transform 403 of the plurality of reversible linear transforms associated with the given component 1500 as the given reversible linear transform. This can be done, for example, by selecting i 0 th row, i.e. due to different positions of the rows of the invertible linear transform 403 corresponding to the given component depending on the position of the given component in the further vector. For example, if the first component of the further vector 402, i.e. y1, is the given component, then i 0 The th row replaces the first row of the reversible linear transform.

[0177] As shown in FIG. 9b, the column 412 of the given prediction matrix 405 corresponding to the given component 1500 of the further vector 402, i.e., 0 The matrix elements 414 of the given prediction matrix 405 in the th column are, for example, all zeros. In this case, the device is configured to calculate the matrix-vector product 404 by performing a multiplication by calculating a matrix-vector product 407 between the reduced prediction matrix C' 405 resulting from the given prediction matrix C 405 by leaving the column 412 and the yet further vector 410 resulting from the further vector 402 by leaving the given elements 1500, as shown in, for example, Fig. 9c. Thus, the prediction vector 406 can be calculated with fewer multiplications.

[0178] As shown in Fig. 8, Fig. 9b and Fig. 9c, the device 1000 may be configured to calculate, for each component of the prediction vector 406, a sum of the respective component and a, i.e., a predetermined value 1400, when predicting samples of a given block based on the prediction vector 406. This sum may be represented by a sum of the prediction vector 406 and the vector 409, where all components of the vector 409 are equal to the predetermined value 1400, as shown in Fig. 8 and Fig. 9c. Alternatively, the addition may be represented by a sum of the prediction vector 406 and a matrix-vector product 1310 between the integer matrix M 1300 and the further vector 402, as shown in Fig. 9b, where the matrix components of the integer matrix 1300 are the columns of the integer matrix 1300 that correspond to the predetermined components 1500 of the further vector 402, i.e., i 0 in the column is a 1 and all other components are, for example, 0.

[0179] The result of the sum of the given prediction matrix 405 and the integer matrix 1300 is, for example, equal to or close to the further matrix 1200 shown in FIG. In other words, the column 412 of the given prediction matrix 405 corresponding to the given element 1500 of the further vector 402, i.e. 0 The matrix obtained by summing each matrix element of a given prediction matrix C 405 in the th column with the reversible linear transform 403 (i.e., matrix B) times the matrix B 1200 corresponds to a quantized version of the machine learning prediction matrix A 1100, for example, as shown in Figures 8, 9a and 9b. As shown in Figure 9b, 0 Summing each matrix element of the predefined prediction matrix C 405 in the th column 412 may correspond to the sum of the predefined prediction matrix 405 and the integer matrix 1300. As shown in FIG. 8, the machine learning prediction matrix A 1100 may be equal to 1200 times the result of the further matrix of the reversible linear transform 403. This is

number

[0180] Matrix multiplication using integer arithmetic only For a low complexity implementation (in terms of the complexity of adding and multiplying scalar values, as well as the storage required for the entries of the part-taking matrices), it is desirable to perform the matrix multiplication 404 using integer arithmetic only.

number

number

[0181] According to one embodiment, real values ​​are converted using arithmetic on integers only.

number

number

number

[0182] (1) final_offset=1 <<(right_shift_result-1); for i in 0…m-1 { accumulator=0 for j in 0…n-1 { accumulator:=accumulator+y[j]*C[i,j] } z[i]=(accumulator+final_offset)>> right_shift_result; } Here, array C, i.e., the predefined prediction matrix 405, stores fixed point numbers, e.g. as integers. The final addition of final_offset and right shift operation by right_shift_result reduces precision by rounding to obtain the required fixed point format at the output.

[0183] To allow for an increase in the range of real values ​​representable by the integers in C, two additional matrices are added, as shown in the embodiment of Figs. 10 and 11.

number

number

number

number

number

number

number

number

number

number

number

number

[0184] According to one embodiment, the prediction parameters include weights, each associated with a corresponding matrix element of the prediction matrix, i.e., a given prediction matrix is, for example, replaced or represented by the prediction parameters, the weights being, for example, integer and / or fixed point values. According to one embodiment, the prediction parameters are scaled by one or more scaling factors, e.g., the value scale i , jeach of which may be a weight, e.g., an integer value, associated with one or more corresponding matrix elements of the predetermined prediction matrix 405.

number

number

number

number

[0185] For example, in one preferred embodiment,

number

number

number

number

number

number

number

number

[0186] Offset Expression

number

number

[0187] A wide range of embodiments resulting from the solution The above solution implies the following embodiments. 1. A prediction method as described in section I, in which in step 2 of section I the following is performed for integer approximations of the matrix-vector products involved: (averaged) boundary samples

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0188] 2. A prediction method as described in section I, in which in step 2 of section I the following is done for integer approximations of the matrix-vector products involved: (averaged) boundary samples

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0189] 4. The prediction method according to section I, wherein step 2 uses one of K matrices, each of which is a different matrix with k=0, so that multiple prediction modes can be calculated.

number

number

number

number

[0190] To make the prediction, a sample value vector 400 is formed from reference samples, such as reference samples 17a and 17c. Possible formations have been described above. The formation may include averaging, thereby reducing the number of samples 102 or the number of components of the vector 400 compared to the reference samples 17 that contribute to the formation. The formation may also depend in some way on the dimensions or size of the blocks, such as the width and height of the blocks 18, as described above. It is this vector 400 that should undergo an affine or linear transformation to obtain the prediction of block 18. Different nomenclatures have been used above. The objective is to perform the prediction by applying vector 400 to matrix A by a matrix-vector product, within the scope of performing an addition, using the latest, with an offset vector b. The offset vector b is arbitrary. The affine or linear transformation determined by A or A and b can be determined by the encoder and decoder, or more precisely, for the prediction based on the size and dimensions of block 18, as already mentioned above.

[0191] However, in order to achieve the computational efficiency improvements outlined above or to make the prediction more effective in terms of implementation, the affine or linear transformation is quantized and the encoder and decoder, or its predictor, use C and T applied in the manner described above to represent and perform the linear or affine transformation, with C and T applied in the manner described above, representing a quantized version of the affine transformation. In particular, instead of directly applying the vector 400 to the matrix A, the predictor of the encoder and decoder applies the vector 402 obtained from the sample value vector 400 by subjecting it to a mapping through a predetermined invertible linear transformation T. The transformation T used here is the same as long as the vector 400 has the same size, i.e. does not depend on the dimensions of the block, i.e. width and height, or at least the same for different affine / linear transformations. Above, the vector 402 is denoted by y. The exact matrix for performing the affine / linear transformation determined by machine learning was B. However, instead of performing B exactly, the prediction in the encoder and decoder is performed by an approximation or quantized version of it. In particular, the representation is done by appropriately representing C in the manner outlined above, and C+M representing a quantized version of B.

[0192] Therefore, the prediction in the encoder and decoder is further accomplished by calculating in the manner described above a matrix-vector product 404 between the vector 402 and a predefined prediction matrix C, appropriately represented and stored in the encoder and decoder. A vector 406 resulting from this matrix-vector product is then used to predict the samples 104 of the block 18. As described above, for the prediction, each component of the vector 406 may undergo a summation with a parameter a, as indicated at 408, to compensate for the corresponding definition of C. An optional summation of the vector 406 with an offset vector b may also be included in the derivation of the prediction of the block 18 based on the vector 406. As described above, each component of the vector 406, and therefore each component of the sum of the vector 406, the vector of all a, as indicated at 408, and the optional vector b, may directly correspond to a sample 104 of the block 18 and thus indicate a predicted value of the sample. It is also possible that only a subset of the samples 104 of the block is predicted in this way, with the remaining samples of the block 18, such as 108, being derived by interpolation.

[0193] As mentioned above, there are different embodiments for setting a. For example, it may be the arithmetic mean of the components of the vector 400. In that case, see FIG. 9. The reversible linear transform T may be as shown in FIG. 9. 0 are the sample-valued vectors and the given components of vector 402, respectively, and are replaced by a. However, as also indicated above, there are other possibilities. However, as far as the representation of C is concerned, it has also been indicated above that it may be embodied differently. For example, matrix-vector product 404 may, in its actual computation, be an actual computation of a smaller matrix-vector product having a lower dimension. In particular, as stated above, by the definition of C, its i-th 0 Since the entire column 412 is zero, the actual calculation of product 404 is:

number

[0194] According to one embodiment, the apparatus described herein for predicting a given block of an image may be configured to use matrix-based intra-sample prediction including the following features: The apparatus is configured to form a sample value vector pTemp[x] 400 from a number of reference samples 17. Assuming that pTemp[x] is 2*boundarySize, pTemp[x] may be populated, for example by direct copying or by subsampling or pooling, with the neighboring sample redT[x] located at the top of a given block at x=0..boundarySize-1, followed by the neighboring sample redL[x] located to the left of the given block at x=0..boundarySize-1 (e.g., if isTransposed=0), or vice versa in case of a transposed operation (e.g., if isTransposed=1).

[0195] Input values ​​p[x] for x=0..inSize-1 are derived, i.e. the apparatus is configured to derive from the sample value vector pTemp[x] a further vector p[x] to which the sample value vector pTemp[x] is mapped by a predetermined invertible linear transformation, or more specifically a predetermined invertible affine linear transformation, as follows: If -mipSizeId is equal to 2, the following applies: p[x]=pTemp[x+1]-pTemp[0] - Otherwise (mipSizeId is less than 2), the following applies: p[0]=(1<<(BitDepth-1))-pTemp[0] p[x] = pTemp[x] - pTemp[0] (for x = 1) inSize - 1 Here, the variable mipSizeId indicates the size of the given block, i.e., according to the present embodiment, the reversible transformation for deriving the further vector from the sample value vector depends on the size of the given block. The dependency may be given according to the following:

[0196] [Table 3]

[0197] predSize indicates the number of predicted samples in a given block, 2*boundarySize indicates the size of the sample value vector and is related to inSize, i.e. the S size of the farter vector, according to inSize=(2*boundarySize)-(mipSizeId==2)?1:0. More precisely, inSize indicates the number of components of the further vector that are actually involved in the calculation. inSize is as large as the size of the sample value vector for small block sizes and one component smaller for large block sizes. In the former case, one component, i.e. the component corresponding to a given component of the further vector, may be decomposed, as in the matrix-vector product calculated later, the contribution of the corresponding vector component will be zero in any case and therefore does not actually need to be calculated. The dependency on the block size can be excluded in the case of alternative embodiments in which only one of the two alternatives is necessarily used, i.e. regardless of the block size (the option corresponding to mipSizeId is less than 2 or the option corresponding to mipSizeId is equal to 2).

[0198] In other words, a given reversible linear transformation is defined such that, for example, a given component of the further vector p is a and all other components correspond to the sample value vector minus a, for example a=pTemp[0]. In the case of the first option corresponding to mipSizeId equal to 2, this is easily seen and only separately formed components of the further vector are further taken into account, i.e. in the case of the first option, the further vector is actually {p[0...inSize], pTemp[0]}, where pTemp[0] is and the actually calculated part of the matrix-vector multiplication, i.e. the result of the multiplication, is limited only to the inSize components of the further vector and the corresponding columns of the matrix, since the matrix has a 0 column that does not require calculation. Otherwise, corresponding to a mipSizeId smaller than 2, all components of the further vector except p[0], i.e. each of the other components p[x] (for x=1..inSize-1) of the further vector p except for the given component p[0], are equal to the corresponding component -a of the sample value vector pTemp[x], but a=pTemp[0] is chosen such that p[0] is a constant -a. The matrix-vector product is then calculated. The constant is the average of the representable values, i.e. 2 x-1 (i.e., 1<<(BitDepth-1)), where x indicates the bit depth of the computation representation used. Note that if p[0] is instead selected to be pTemp[0], the calculated product will deviate from the product calculated using p[0] as above (p[0]=(1<<(BitDepth-1))-pTemp[0]) by a constant vector that can be taken into account when predicting the block interior based on the product, i.e., the prediction vector. Thus, the value a is a predetermined value, for example pTemp[0]. The predetermined value pTemp[0] is in this case, for example, the component of the sample value vector pTemp that corresponds to the predetermined component p[0]. It may be the sample closest to the top left corner of the predetermined block, adjacent above the predetermined block or to the left of the predetermined block.

[0199] In case of an intra-sample prediction process with predModeIntra, e.g. specifying an intra prediction mode, the apparatus is configured, for example, to apply the following steps, e.g. performing at least the first step. 1. The matrix-based intra prediction samples predMip[x][y], where x=0..predSize-1, y=0..predSize-1, are derived as follows: - The variable modeId is set equal to predModeIntra. The weight matrix mWeight[x][y], for x=0..inSize-1, y=0..predSize*predSize-1, is derived by invoking the MIP weight matrix derivation process with mipSizeId and modeId as input. The matrix-based intra prediction samples predMip[x][y], with x=0..predSize-1, y=0..predSize-1, are derived as follows: oW=32-32*(

number

number

[0200] In other words, the device performs a matrix-vector product between a further vector p[i], or, if mipSizeId is equal to 2, {p[i], pTemp[0]} and a given prediction matrix mWeight, or, if mipSizeId is less than 2, the prediction matrix mWeight, now having an additional zero weight line corresponding to the omitted component of p, to obtain a prediction vector already assigned to the array of block positions {x, y} distributed inside the given block to result in the array predMip[x][y], which corresponds to the concatenation of the rows of predMip[x][y] or the columns of predMip[x][y], respectively. According to one embodiment, or according to a different interpretation, the component ((

number

number

[0201] 2. A matrix-based intra prediction sample predMip[x][y], with x=0..predSize-1, y=0..predSize-1, is clipped, for example, as follows: predMip[x][y]=Clip1(predMip[x][y]) 3. If isTransposed is equal to TRUE, then the predSize×predSize array predMip[x][y], for x=0..predSize−1, y=0..predSize−1, is transposed, for example, as follows: predTemp[y][x]=predMip[x][y] predMip=predTemp 4. Prediction samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1, are derived, for example, as follows: - If nTbW, which specifies the transform block width, is greater than predSize or nTbH, which specifies the transform block height, is greater than predSize, the MIP prediction upsampling process is called with inputs: input block size predSize, matrix-based intra prediction samples predMip[x][y] having x=0..predSize-1, y=0..predSize-1, transform block width nTbW, transform block height nTbH, upper reference samples refT[x] having x=0..nTbW-1, and left reference samples refL[y] having y=0..nTbH-1, and the output is the prediction sample array predSamples. - Otherwise, predSamples[x][y], for x=0..nTbW-1, y=0..nTbH-1, is set equal to predMip[x][y]. In other words, the apparatus is configured to predict the samples predSamples of a given block based on the prediction vector predMip.

[0202] 8. EMBODIMENTS USING BLOCK-BASED INTRA PREDICTION MODES WITH OTHER INTRA PREDICTION MODES All the above descriptions shall be considered as optional implementation details of the embodiments described herein. Note that in the following, the term block-based intra prediction is used to indicate an intra prediction mode that may be embodied or equivalent to that indicated by ALWIP above.

[0203] 12 illustrates an embodiment of an apparatus 3000 for decoding a given block 18 of an image 10 using intra prediction. The apparatus 3000 is configured to derive a set-selective syntax element 522 from a data stream 12 indicating whether the given block 18 should be predicted using one of a first set 508 of intra prediction modes including a DC intra prediction mode 506 and an angular prediction mode 500. The data stream 12 may include different syntax elements and / or indices that indicate the capabilities of the apparatus 3000.

[0204] If the set-selective syntax element 522 indicates that the given block 18 should be predicted using one of the first set of intra-prediction modes 508, the apparatus 3000 is configured to form a list 528 of most probable intra-prediction modes based on the intra-prediction modes 3050 using when the neighboring blocks 524, 526 neighboring the given block 18 are predicted. In other words, the intra-prediction modes 506, 500 of the first set of intra-prediction modes 508 are positioned / arranged in the list 528 of most probable intra-prediction modes based on the intra-prediction modes 3050 used to predict the neighboring blocks 524 and 526. The apparatus 3000 is configured, for example, to store the prediction modes used for already predicted blocks and obtain the prediction modes 3050 used for the neighboring blocks 524 and 526 from the stored prediction modes, or to analyze the neighboring blocks 524 and 526 to obtain the prediction modes 3050 used for the neighboring blocks 524 and 526. According to one embodiment, the device 3000 is configured to search for intra prediction modes that are equal or similar to the prediction mode 3050 used for the neighboring blocks 524 and 526 in the first set of intra prediction modes 508, and form a list 528 of the most likely intra prediction modes among these equal or similar intra prediction modes.

[0205] The list of most likely intra prediction modes 528 is formed such that the list of most likely intra prediction modes 528 does not include the DC intra prediction mode 500 if the neighboring blocks 524 and 526 are predicted exclusively by any of the angular intra prediction modes 506. Thus, the availability of the DC intra prediction mode 506 depends only on the neighboring blocks 524 and 526 of a given block 18, and not on other blocks of the image 10. The DC intra prediction mode 506 is / is not placed in the list of most likely intra prediction modes 528 if, for example, at least one neighboring block 524 or 526 is predicted using the angular intra prediction mode 500. The list of most likely intra prediction modes 528 may also not include the DC intra prediction mode 506 if both neighboring blocks 524 and 526 are predicted using the angular intra prediction mode 500.

[0206] Further, the apparatus is configured to derive an MPM list index 534 from the data stream if the set-selective syntax element 522 indicates that the given block 18 should be predicted using one of the first set of intra-prediction modes 508. The MPM list index 534 points the list of most probable intra-prediction modes 528 to the given intra-prediction mode. The apparatus 3000 is configured to intra-predict the given block 18 using the given intra-prediction mode 3100.

[0207] If the set selective syntax element 522 indicates that the given block 18 should not be predicted using one of the first set of intra-prediction modes 508, the apparatus 3000 is configured to derive from the data stream 12 a further index 540 indicating a given matrix-based intra-prediction mode of the second set of matrix-based intra-prediction modes 520, i.e. a given block-based intra-prediction mode 3200 of the block-based intra-prediction modes 510. In other words, the given block-based intra-prediction mode 3200 of the second set of block-based intra-prediction modes 520 is selected for prediction of the given block 18 based on the further index 540. The apparatus 3000 is configured to calculate a matrix-vector product 512 between a vector 514 derived from reference samples 17 in the vicinity of the given block 18 and a given prediction matrix 516 associated with a given matrix-based intra-prediction mode 3200 to obtain a prediction vector 518, and to predict samples of the given block 18 based on the prediction vector 518 if the setting selection syntax element 522 indicates that the given block 18 is not predicted using one of the first set of intra-prediction modes 508. The data stream 12 includes either the MPM list index 534 or the further index 540 based on the set selective syntax element 522 . The device 3000 may include features and / or functionality as described with respect to FIG.

[0208] Thereby, the embodiment described below with respect to FIG. 13 relates to a decoder and an encoder supporting intra prediction for decoding / encoding a given block 18, in which different intra prediction modes are supported. There are angular intra prediction modes 500 in which reference samples 17 adjacent to the given block 18 are used to fill the given block 18 in order to obtain an intra prediction signal of the given block 18. In particular, the reference samples 17 located along the boundaries of the given block 18, such as along the top and left edges of the given block 18, represent image content that is extrapolated or copied to the inside of the given block 18 along a predetermined direction 502. Before extrapolation or copying, the image content represented by the adjacent samples 17 may undergo interpolation filtering, or, stated differently, may be derived from the adjacent samples 17 in the case of interpolation filtering. The angular intra prediction modes 500 differ from each other in intra prediction direction 502. Each angular intra-prediction mode 500 may have an associated index, and the association of the indices to the angular intra-prediction modes 500 may be such that the direction 500 rotates monotonically clockwise or counterclockwise when ordering the angular intra-prediction modes 500 according to the associated mode index.

[0209] Non-angular intra-prediction modes are also possible. For example, FIG. 13 shows a planar intra-prediction mode 504, optionally included in the set 508, according to which a two-dimensional linear function defined by a horizontal gradient, a vertical gradient and an offset is derived based on neighboring samples 17, which linear function defines a predicted sample value for a given block 18. A horizontal slope, a vertical slope and an offset are derived based on neighboring samples 17. According to one embodiment, the first set of intra-prediction modes 508 includes a planar intra-prediction mode 504. A particular non-angular intra-prediction mode, DC mode, included in the set 508 is shown at 506, where a single value, which is a pseudo DC value, is derived based on neighboring samples 17, and this single DC value is attributed to all samples of a given block 18 to obtain an intra-prediction signal. Two examples of non-intra-prediction modes are shown, although there may be just one or more than two.

[0210] The intra-prediction modes 500, 504 and 506 form a set 508 of intra-prediction modes supported by the encoder and the decoder, which compete with block-based intra-prediction modes generally indicated using the reference sign 510, in the sense of rate / distortion optimization, and in the example of which, described above using the abbreviation ALWIP. As described above, according to these block-based intra-prediction modes 510, a matrix-vector product 520 is performed between a vector 514 derived from neighboring samples 17, on the one hand, and a given prediction matrix 516, on the other hand. The result of the multiplication 512 is a prediction vector 518, which is used to predict the samples of a given block 18. The block-based intra-prediction modes 510 differ from each other in the prediction matrix 516 associated to the respective mode. Thus, briefly summarized, the encoder and decoder according to the embodiments described herein comprise a set of intra prediction modes 508, i.e. a first set of intra prediction modes, and a set of block-based intra prediction modes 520, i.e. a second set of matrix-based intra prediction modes, which compete with each other.

[0211] According to an embodiment of the present application, a given block 18 is coded / decoded using intra prediction in the following manner. In particular, first, a set selection syntax element 522 indicates whether the given block 18 is predicted using any of the set of intra prediction modes 508 or any of the set of block-based intra prediction modes 520. If the set selection syntax element indicates that the given block 18 should be predicted using any mode of the set 508, i.e., the first set of intra prediction modes, a list 528 of most likely candidates from the set 508 is interpreted / formed in the decoder and encoder based on which intra prediction modes the neighboring block 18, and the neighboring blocks exemplarily shown at 524 and 526, were predicted using. The neighboring blocks 524 and 526 can be determined in a predetermined manner with respect to the position of the given block 18, such as by determining the neighboring block that overlays a particular neighboring sample of the block 18, such as a sample, on the top left sample of the block 18, and the block 526 that includes a sample to the left of the corner sample mentioned above. Of course, this is only one example. The same applies to the number of neighboring blocks used for mode prediction, which is not limited to two for all embodiments. It may be more than two or only one. If any of these blocks 524 and 526 are missing, a default intra-prediction mode may be used by default as a replacement for the intra-prediction mode of the missing neighboring block. The same may be true if any of the blocks 524 and 526 are encoded / decoded using an inter-prediction mode, such as by motion-compensated prediction.

[0212] The construction of the list of modes from the set 508, i.e. the list 528 of the most probable intra-prediction modes, is as follows: The list length of the list 528, i.e. the number of the most probable modes therein, may be fixed by default. The length may be 4, as shown in FIG. 13, or it may be different, such as 5 or 6. The latter case applies to the concrete example described below. An index in the data stream described below may indicate one mode of the list 528 to be used for a given block 18. The indexing is performed along a list order or ranking 530, the list index being of variable length, for example, coded such that the length of the index increases monotonically along the order 530. It is therefore worthwhile to initially add only the most probable modes from the set 508 to the list 528, and to place the more probable modes upstream along the order 530, compared to the less probable modes suitable for the block 18. The modes of the list 528 are derived based on the modes used for the blocks 524 and 526, i.e. the neighboring blocks adjacent to the given block 18. If any of blocks 524 and 526 are intra predicted using a block-based mode 510 from set 520, then the above-mentioned mapping of such "ALWIP" or block-based modes 510 to modes in set 508, such as non-ALWIP modes, is used. The latter mapping may, for example, map a majority (i.e., more than half) of the block-based modes 510 to DC mode 506 (or either DC 506 or planar mode 504).

[0213] According to one embodiment, the list 528 of most likely intra-prediction modes is filled with the planar intra-prediction modes 504 in a manner independent of the intra-prediction mode using when the neighboring blocks are predicted. Thus, for example, only the DC intra-prediction mode 506 and the angular intra-prediction mode 500 are populated in the list 528 depending on the intra-prediction mode used to predict the neighboring blocks 524 and 526. The planar intra-prediction mode 504 is placed in the first position of the list 528 of most likely intra-prediction modes independent of the intra-prediction mode using when the neighboring blocks 524 and 526 are predicted.

[0214] As illustrated in more detail below, the list of most likely intra-prediction modes 528 is constructed such that the list 528 does not include the DC intra-prediction mode 500 if the neighboring blocks 524 and 526 are predicted exclusively by the angular intra-prediction mode 506. The DC intra-prediction mode 506 is not in the list of most likely intra-prediction modes 528 if one neighboring block 524 or 526 is predicted by any angular intra-prediction mode 500 and / or if both neighboring blocks 524 and 526 are predicted by any angular intra-prediction mode 500. According to the embodiments presented herein below, for example, the list 528 is populated with the DC mode 506 only if the following conditions apply for all neighboring blocks 524 and 526: either one is coded using one of the non-angular intra-prediction modes 504 and 506, or one is intra-predicted using the block-based intra-prediction mode 510, which is mapped to one of the non-angular intra-prediction modes 504 and 506 by the aforementioned mapping of the block-based intra-prediction modes 510 to the modes in the set 508. Only then is the DC intra-prediction mode 506 positioned in the list 528. In that case, it may be placed before any angular intra-prediction mode 500 in the order 530, as can be seen from the following examples. In other words, the list of most likely intra prediction modes 528 is filled with the DC intra prediction mode 508 only if, for each of the neighboring blocks 524 and 526, the neighboring blocks are predicted using any of the at least one non-angular intra prediction mode 504 and 506 having a first set 508 that includes the DC intra prediction mode 506, or are predicted using any of the block-based intra prediction modes 510 that are mapped to any of the at least one non-angular intra prediction mode 500 by mapping from the second set 520 of block-based intra prediction modes 510 to an intra prediction mode in the first set 506 used to form the list of most likely intra prediction modes 528.

[0215] Thus, resuming the description of how a given block 18 is encoded into data stream 12, if set-selective syntax element 522 indicates that the given block 18 should be encoded by any mode from first set 508, data stream 12 optionally includes an MPM syntax element 532 indicating whether the intra-prediction mode to be used for the given block 18 is in list 528, and if so, data stream 12 includes an MPM list index 534 pointing into list 528 indicating the mode to be used for the given block 18 from list 528, i.e., the pre-determined intra-prediction mode, by indexing same along order 530. However, if no mode from set 508 is in list 528 as indicated by MPM syntax element 532, data stream 12 includes a further syntax element 536 indicating for block 18 which mode, i.e., the pre-determined intra-prediction mode, should be used for block 18 from set 508. Further syntax element 536 may indicate the mode by simply distinguishing a mode from set 508 that is not included in list 528 . In other words, the apparatus 3000 is configured to derive from the data stream an MPM syntax element 532 indicating whether a given intra-prediction mode of the first set of intra-prediction modes 508 is in the list of most probable intra-prediction modes 528, for example, where the set selection syntax element 522 indicates that a given block 18 is predicted using one of the first set of intra-prediction modes 508. If the MPM syntax element 532 indicates that a given intra-prediction mode of the first set of intra-prediction modes 508 is in the list of most probable intra-prediction modes 528, the apparatus 3000 is configured to perform formation of the list of most probable intra-prediction modes 528 based on the intra-prediction modes using when neighboring blocks 524, 526 neighboring the given block 100 are predicted, for example, and to perform derivation of an MPM list index 534 from the data stream 12 pointing to the given intra-prediction mode of the list of most probable intra-prediction modes 528. If the MPM syntax element 532 from the data stream 12 indicates that the given intra-prediction mode of the first set of intra-prediction modes 508 is not in the list of most probable intra-prediction modes 528, the apparatus 3000 is configured to derive a further list index 536 from the data stream indicating the given intra-prediction mode of the first set of intra-prediction modes. Thus, based on the MPM syntax element 532, the data stream 12 includes either an MPM list index 534 or a further list index 536 for prediction of the given block 18.

[0216] By eliminating the situation where the list 528 includes a DC intra-prediction mode 506, the following advantages are achieved: In particular, the inventors of the present application have found that "consuming" a valuable list position of the list 528 with a DC intra-prediction mode 506 from the set 508 to encode / decode a given block 18 that should use one of the intra-prediction modes of the set 508 indicated by the syntax element 522, i.e., the set-selective syntax element, would have a negative impact on coding efficiency, since such a DC intra-prediction mode 506 from the set 508 would conflict with the block-based intra-prediction mode 510 anyway. Thus, "consuming" a list position of the list 528 with such a DC intra-prediction mode 506 from the set 508 would increase the likelihood of a situation where the intra-prediction mode that should ultimately be used for the given block 18, i.e., the given intra-prediction mode, is not in the list 528, and as a result, a syntax element 536, i.e., an additional list index, needs to be transmitted in the data stream 12. In particular, since syntax element 522 already indicates whether block 18 should be predicted using any of the modes in set 508 or any of the block-based modes 510 of set 520, if syntax element 522 indicates that a mode in set 508 is preferred for block 18 and therefore block-based mode 510 is not used for block 18, it is very unlikely that DC prediction mode 506 of set 508 is suitable for block 18, and therefore its occurrence in list 528 should be restricted to a very limited set of constellations of modes used for neighboring blocks 524 and 526, i.e., the constellations set out above.

[0217] In the other case, i.e., if the set selective syntax element 522 indicates that a given block 18 is predicted using one of the block-based intra-prediction modes 510, then the encoding of the block 18 into and decoding from the data stream 12 may be performed in the manner described above. To this end, indexing may be used to index or indicate with respect to a block-based intra-prediction mode 510 out of the set 520, i.e., a selected one of the second set of block-based intra-prediction modes. The further MPM syntax element 538 can indicate whether the indexing is done by index 540, i.e. by a further MPM list index indicating the block-based intra-prediction mode 510 to be used for block 18 out of a list 542 of the most probable block-based intra-prediction modes 510, i.e. by indexing along the list order 544, or whether the block-based intra-prediction mode 510 to be used for block 18 is indicated by a further syntax element 546, i.e. by a still further list index indicating the block-based intra-prediction mode 510 out of the set 520, the latter syntax element 546 for example being able to distinguish only between those modes 510 in the set 520 that are not already included in the list 542. The list construction of the list 542 can be done based on the modes in which the blocks 524 and 526 were predicted. If any of the blocks 524 and 526 are not available because they are out of the picture or are inter-predicted, a default intra-prediction mode, such as one of the set 508, can be used instead.For each block 524 and 526 that is intra predicted using a mode outside of set 508 but not set 520, the above mapping from a mode in set 508 to a mode outside of set 520 is used to obtain the intra prediction mode 510, i.e., the predetermined block-based intra prediction mode, for the respective block, i.e., the predetermined block 18, and list 542 is interpreted based on the resulting block-based intra prediction modes of blocks 524 and 526.

[0218] According to one embodiment, the device 3000 is configured to derive a further MPM syntax element 538 from the data stream 12 indicating whether a given block 18 of the second set 520 of block-based intra-prediction modes 510 is in the list 542 of most probable block-based intra-prediction modes if the set selective syntax element 522 indicates that a given block 18 is not predicted using one of the first set 508 of intra-prediction modes. If the further MPM syntax element 538 indicates that the given block-based intra-prediction mode of the second set 520 of block-based intra-prediction modes 510 is in the list of most probable block-based intra-prediction modes 542, the apparatus 3000 is configured to form the list of most probable block-based intra-prediction modes 542 based on the intra-prediction mode, for example using when the neighboring blocks 524, 526 neighboring the given block 18 are predicted, and to derive from the data stream 12 a further MPM list index 540 indicating the list of most probable block-based intra-prediction modes 542 on the given block-based intra-prediction mode. If the further MPM syntax element 538 indicates that the given block-based intra-prediction mode of the second set 520 of block-based intra-prediction modes is not in the list of most probable block-based intra-prediction modes 542, the apparatus 3000 is configured to derive from the data stream 12 a yet further list index 546 indicating the given block-based intra-prediction mode of the second set 520 of block-based intra-prediction modes. Thus, based on the further MPM syntax element 538 , the data stream 12 includes either a further MPM list index 540 or a yet further list index 546 for the prediction of a given block 18 .

[0219] 13 as being in parallel with MPM syntax element 532, MPM list index 534, and further list index 536, it will be apparent that further MPM syntax element 538, and either an index associated with further MPM syntax element 538, e.g., further MPM list index 540 or still further list index 546, are constructed by data stream 12 or MPM syntax element 532, and an index associated with MPM syntax element 532, e.g., MPM list index 534 or further list index 536. Which of the syntax elements and indexes are included in data stream 12 depends, for example, on set-selective syntax element 522. An example of a syntax element portion of data stream 12 written as pseudocode may appear as follows, with reference numbers indicating which syntax elements correspond to previously described syntax elements:

[0220] [Table 4-1] [Table 4-2] [Table 4-3] [Table 4-4] [Table 4-5]

[0221] The list construction of the list 528 may be defined as follows, where candIntraPredModeA / B indicates the intra prediction mode in which either of the blocks 524 and 526 was predicted, such as A for block 524 and B for block 526, if the corresponding block 524 or 526 was intra predicted using one of the block-based intra prediction modes 510, or the mode to which the intra prediction mode from the set 508 is mapped. INTRA_DC is used to indicate the mode 506, and the angular mode 500 is indicated by INTRA_ANGULAR#, which orders and numbers the angular modes as exemplarily shown above, i.e., the angular directions 502 monotonically decrease or increase with increasing number. The order between the modes in the set 508 may be as defined in the subsequent table, where INTRA_PLANAR indicates the mode 504.

[0222] It should be noted that in the above example, index 534 is in fact distributed over syntax elements 534' and 534''. That is, the former 534' is specific to the first, order 530 position of list 528, where, according to this example, INTRA_PLANAR mode 504 is necessarily located. The latter 534'' points to any of the subsequent positions of list 528, where, as explained, DC mode 506 is included only in the special circumstances described.

[0223] Furthermore, in the above example, in the case of syntax element 522 indicating that any of the modes in set 508 is used, further syntax elements are included in data stream 12, which further syntax elements parameterize the intra-prediction modes in set 508 in some way. For example, syntax element 600 parameterizes or varies the region in which reference sample 17 is located, based on which the modes in set 508 intra-predict the interior of block 18, such as with respect to the distance towards the periphery of block 18. Additionally or alternatively, syntax element 602 parameterizes or varies whether reference sample 17 is used by the modes in set 508 to intra-predict the interior of block 18 globally or within the block, or whether intra-prediction is performed in sub-parts or parts of block 18 to be sequentially intra-predicted, so that the prediction residuals coded in data stream 12 for one part can serve to adopt new reference samples to intra-predict a subsequent part. The latter coding option controlled by a syntax element may only be available (and a corresponding syntax element may only be present in the data stream) if the syntax element 600 has a predefined state corresponding to, for example, a region in which there is a reference sample 17 adjacent to the block 18. The portions may be defined by subdividing the block along a predefined direction, such as either horizontally, thereby resulting in a portion of the same height as the block 18, or vertically, thereby resulting in a portion of the same width as the block 18. If the split is signaled as active, a syntax element 604 may be present in the data stream that controls as to which split direction is used.As can be seen, the positions in list 528 reserved for INTRA_PLANAR mode are only available in the case of a particular parameterization of the mode by the parameterization syntax elements described above, such as when syntax element 600 has a predefined state corresponding to an area in which reference samples 17 adjacent to block 18 are present and / or when partial-wise intra prediction mode is not active as signaled by syntax element 602.

[0224] All syntax elements shown in the table and not specifically mentioned above are optional and will not be described further herein. - If candIntraPredModeB is equal to candIntraPredModeA and candIntraPredModeA is greater than INTRA_DC, then candModeList[x], for x=0..4, is derived as follows: candModeList[0]=candIntraPredModeA candModeList[1]=2+((candIntraPredModeA+61)% 64) candModeList[2]=2+((candIntraPredModeA-1)% 64) candModeList[3]=2+((candIntraPredModeA+60)% 64) candModeList[4]=2+(candIntraPredModeA % 64) - Else, if candIntraPredModeB is not equal to candIntraPredModeA and candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC, the following applies: The variables minAB and maxAB are derived as follows:

[0225] minAB=Min(candIntraPredModeA,candIntraPredModeB) maxAB=Max(candIntraPredModeA, candIntraPredModeB) - If both candIntraPredModeA and candIntraPredModeB are greater than INTRA_DC, candModeList[x], for x=0..4, is derived as follows: candModeList[0]=candIntraPredModeA candModeList[1]=candIntraPredModeB If -maxAB-minAB is equal to 1, the following applies: candModeList[2]=2+((minAB+61)% 64) candModeList[3]=2+((maxAB-1)% 64) candModeList[4]=2+((minAB+60)% 64) - Otherwise, if maxAB-minAB is ≥ 62 then the following applies: candModeList[2]=2+((minAB-1)% 64) candModeList[3]=2+((maxAB+61)% 64) candModeList[4]=2+(minAB % 64) - Otherwise, if maxAB-minAB is equal to 2, the following applies: candModeList[2]=2+((minAB-1)% 64) candModeList[3]=2+((minAB+61)% 64) candModeList[4]=2+((maxAB-1)% 64) - Otherwise the following applies: candModeList[2]=2+((minAB+61)% 64) candModeList[3]=2+((minAB-1)% 64)(8-36) candModeList[4]=2+(((maxAB+61))% 64) - Otherwise (candIntraPredModeA or candIntraPredModeB is greater than INTRA_DC), candModeList[x], for x=0..4, is derived as follows: candModeList[0]=maxAB candModeList[1]=2+((maxAB+61)% 64) candModeList[2]=2+((maxAB-1)% 64)(8-41) candModeList[3]=2+((maxAB+60)% 64) candModeList[4]=2+(maxAB % 64) - Otherwise the following applies: candModeList[0]=INTRA_DC candModeList[1]=INTRA_ANGULAR50( candModeList[2]=INTRA_ANGULAR18 candModeList[3]=INTRA_ANGULAR46 candModeList[4]=INTRA_ANGULAR54

[0226] [Table 5]

[0227] In the case of a set selection syntax element 522 indicating that a given block 18 should not be predicted using one of the first set of intra prediction modes 508, an apparatus for decoding the given block 18 and / or an apparatus for encoding the given block 18 may include one or more of the following features. According to one embodiment, the device is configured to form a sample value vector from a plurality of reference samples 17, for example the sample value vector 400 as described with respect to one of the embodiments of figures 6 to 9, and to derive a vector 514 from the sample value vector, such that the sample value vector is mapped to the vector 514 by a predefined reversible linear transformation. In this case, the vector 514 can be understood as a further vector. The vector 514 is determined and / or defined, for example as described for the further vector 402 with respect to one of the embodiments of figures 8 to 11.

[0228] According to one embodiment, the apparatus is configured to form a sample value vector from the plurality of reference samples 17 by, for each component of the sample value vector, taking one reference sample of the plurality of reference samples as the respective component of the sample value vector and / or by averaging two or more components of the sample value vector to obtain the respective component of the sample value vector. A number of reference samples 17 are placed in the image, for example along the outer edge of a given block 18 . The reversible linear transformation is defined, for example, such that a predetermined component of vector 514, e.g., a predetermined component of a further vector, is a, and each of the other components of vector 514 except for the predetermined component is equal to the corresponding component of the sample value vector minus a. The value a is, for example, a predetermined value.

[0229] According to one embodiment, the predetermined value is one of an average, such as an arithmetic or weighted average, of the components of the sample value vector, a default value, a value signaled in the data stream in which the image is encoded, and a component of the sample value vector that corresponds to the predetermined component. The reversible linear transform is defined, for example, such that a predetermined component of vector 514, e.g., a predetermined component of a further vector, is a, and each of the other components of vector 514 except for the predetermined component is equal to the corresponding component of the sampled vector minus a, where a is the arithmetic mean of the components of the sampled vector. The reversible linear transform is defined, for example, such that a predetermined component of the vector 514, e.g. a predetermined component of a further vector, is a and each of the other components of the vector 514 except for the predetermined component is equal to a corresponding component of the sample-valued vector minus a, where a is the component of the sample-valued vector that corresponds to the predetermined component. The apparatus is configured, for example, to comprise a plurality of reversible linear transforms, each associated with a component of the vector 514, to select the predetermined component from the components of the sample-valued vector, and to use a reversible linear transform from the plurality of reversible linear transforms associated with the predetermined component as the predetermined reversible linear transform.

[0230] According to one embodiment, the matrix elements of the prediction matrix 516, e.g. in a column of the prediction matrix 516 corresponding to a predetermined element of the vector 514 of the further vector, are all zero. The device is configured to calculate the matrix-vector product 512 by performing a multiplication by calculating a matrix-vector product 512 between a reduced prediction matrix resulting from the prediction matrix 516 by excluding a column and a further vector resulting from the vector 514 by excluding a predetermined element. According to one embodiment, the device is configured, when predicting samples of a given block 18 based on the prediction vector 518, to calculate, for each component of the prediction vector 518, the sum of the respective component and a. A matrix obtained by summing each matrix element of the prediction matrix 516 in a column of the prediction matrix 516 corresponding to a given element of the vector 514, e.g., a further vector, times the reversible linear transform (i.e., matrix B in FIG. 8 ), corresponds, for example, to a quantized version of the machine learning prediction matrix.

[0231] According to one embodiment, the apparatus is configured to calculate the matrix vector product 512 using fixed-point arithmetic. According to one embodiment, the apparatus is configured to calculate the matrix vector product 512 without floating point operations. According to one embodiment, the apparatus is configured to store a fixed-point representation of the prediction matrix 516 .

[0232] According to one embodiment, the device is configured to use the prediction parameters to represent a prediction matrix 516 and to calculate a matrix-vector product 512, for example by performing multiplications and additions on the components of the vector 514 of further vectors and the prediction parameters and intermediate results resulting therefrom, the absolute values ​​of the prediction parameters being representable by an n-bit fixed-point representation, n being less than or equal to 14, or alternatively 10, or alternatively 8. This can be performed similarly or as described in FIG. 10 or FIG. 11. The prediction parameters include, for example, weights, each associated with a corresponding matrix element of the prediction matrix 516 . The prediction parameters may further include, for example, one or more scaling coefficients associated with one or more corresponding matrix elements of the prediction matrix 516, each for scaling weights associated with the one or more corresponding matrix elements of the prediction matrix 516, and / or one or more offsets associated with one or more corresponding matrix elements of the prediction matrix 516, each for offsetting weights associated with the one or more corresponding matrix elements of the prediction matrix 516.

[0233] According to one embodiment, when predicting samples of a given block 18 based on a prediction vector 518, the device is configured to use interpolation to calculate at least one sample position of the given block 18 based on the prediction vector 518, each component of which is associated with a corresponding position within the given block 18.

[0234] FIG. 14 illustrates an apparatus 6000 for encoding a given block 18 of an image 10 using intra prediction, the apparatus being configured to signal a set-selective syntax element 522 in the data stream 12 indicating whether the given block 18 should be predicted using one of a first set of intra prediction modes 508 including a DC intra prediction mode 506 and an angular prediction mode 500. If the set-selective syntax element 522 indicates that the given block 18 should be predicted using one of the first set of intra prediction modes 508, the apparatus 6000 is configured to form a list 528 of most probable intra prediction modes based on the intra prediction modes in which the neighboring blocks 524, 526 adjacent to the given block 18 are predicted, signal an MPM list index 534 in the data stream 12 pointing the list 528 of most probable intra prediction modes onto the given intra prediction mode 3100, and intra predict the given block 18 using the given intra prediction mode 3100. If the set selective syntax element 522 indicates that the given block 18 should not be predicted using one of the first set of intra-prediction modes 508, the device 6000 is configured to signal a further index 540 in the data stream 12 indicating a given matrix-based intra-prediction mode 3200 of the second set 520 of matrix-based intra-prediction modes 510 by calculating a matrix-vector product 512 between a vector 514 derived from reference samples 17 in the vicinity of the given block 18 and a given prediction matrix 516 associated with the given matrix-based intra-prediction mode 3200 to obtain a prediction vector 518, and predicting samples of the given block 18 based on the prediction vector 518.

[0235] The list of most likely intra-prediction modes 528 is formed based on the intra-prediction modes used by neighboring blocks 524, 526 adjacent to a given block 18, such that if at least one of the neighboring blocks 524, 526 is predicted by any of the angular intra-prediction modes 500, the list of most likely intra-prediction modes does not include the DC intra-prediction mode 506. The first set of intra-prediction modes 508 further includes, for example, a planar intra-prediction mode 504 .

[0236] According to one embodiment, the device 6000 may include similar features and / or functionality as described with respect to the device 3000 of Figure 12 and / or Figure 13. Similarly, the device 3000 may include similar features and / or functionality as described with respect to the device 6000 of Figure 14. If the set selective syntax element 522 indicates that the given block 18 should be predicted using one of the first set of intra-prediction modes 508, the apparatus 6000 is configured to, for example, signal an MPM syntax element 532 in the data stream 12 indicating whether the given intra-prediction mode 3100 of the first set of intra-prediction modes 508 is in the list of most probable intra-prediction modes 528. If the MPM syntax element 532 indicates that the given intra-prediction mode of the first set of intra-prediction modes 508 is in the list of most probable intra-prediction modes 528, the apparatus 6000 is configured to, for example, perform formation of the list of most probable intra-prediction modes 528 based on the intra-prediction modes in which the neighboring blocks 524, 526 neighboring the given block 18 are predicted, and to perform signaling of an MPM list index 534 in the data stream 12 pointing to the given intra-prediction mode 3100 of the list of most probable intra-prediction modes 528. If the MPM syntax element 532 in the data stream 12 indicates that a particular intra-prediction mode 3100 of the first set of intra-prediction modes 508 is not in the list of most probable intra-prediction modes 528, the device 6000 is configured to signal, for example, a further list index 536 in the data stream 12 indicating the particular intra-prediction mode 3100 of the first set of intra-prediction modes 508.

[0237] If the set selective syntax element 522 indicates that a given block 18 should not be predicted using one of the first set of intra prediction modes 508, the device 6000 is configured to signal a further MPM syntax element 538 in the data stream 12 indicating, for example, whether the given block-based intra prediction mode 3200 of the second set of block-based intra prediction modes 520 is in the list of most probable block-based intra prediction modes 542, i.e., in the second list of most probable block-based intra prediction modes, for example, in the list of most probable block-based intra prediction modes 542 of the second set of block-based intra prediction modes 520. If the further MPM syntax element 538 indicates that a given block-based intra-prediction mode 3200 of the second set of block-based intra-prediction modes 520 is in the list of most probable block-based intra-prediction modes 542, the device 6000 is configured to form the list of most probable block-based intra-prediction modes 542 based on the intra-prediction mode 3050 in which the neighboring blocks 524, 526 adjacent to the given block 18 are predicted, for example, and to signal a further MPM list index 540 in the data stream 12 that points the list of most probable block-based intra-prediction modes 542 onto the given block-based intra-prediction mode 3200. If the further MPM syntax element 538 indicates that the given block-based intra prediction mode 3200 of the second set of block-based intra prediction modes 520 is not in the list of most probable block-based intra prediction modes 542, the device 6000 is configured to signal, for example, a further list index 546 in the data stream 12 indicating the given block-based intra prediction mode 3200 of the second set of block-based intra prediction modes 520.

[0238] According to one embodiment, the apparatus 6000 is configured to perform the formation of the list 528 of most probable intra prediction modes of the first set 508 of intra prediction modes using which neighboring blocks 524, 526 neighbor a given block 18 such that the list 528 is populated with the DC intra prediction mode 506 only if the respective neighboring blocks 524 and 526 are predicted using either the at least one non-angular intra prediction mode 504 and / or 506 having a first set 508 including the DC intra prediction mode 506, or using either the block-based intra prediction mode 510 predicted by mapping from the second set 520 of block-based intra prediction modes to an intra prediction mode in the first set 508 used to form the list 528 of most probable intra prediction modes, which is mapped to either the at least one non-angular intra prediction mode 504 and / or 506.

[0239] According to one embodiment, the device 6000 is configured to perform the formation of the list of most probable intra-prediction modes 528 based on the intra-prediction modes with which the neighboring blocks 524, 526 adjacent to the given block 18 are predicted, for each of the neighboring blocks 524 and 526, which are predicted using any of the at least one non-angular intra-prediction modes 504 and / or 506 having a first set 508 including the DC intra-prediction mode 506, or which are predicted using any of the block-based intra-prediction modes 510 that are mapped to any of the at least one non-angular intra-prediction modes 504 and / or 506 by a mapping from the second set 520 of block-based intra-prediction modes to the intra-prediction modes in the first set 508 used to form the list of most probable intra-prediction modes 528. The DC intra-prediction mode 506 is placed before any angular intra-prediction modes 500 in the list of most probable intra-prediction modes 528.

[0240] According to one embodiment, the device 6000 is configured to perform the formation of a list 528 of most likely intra-prediction modes based on the intra-prediction modes in which the neighboring blocks 524, 526 adjacent to a given block 18 are predicted, such that the list 528 is populated with planar intra-prediction modes 504 in a manner independent of the intra-prediction modes in which the neighboring blocks 524, 526 are predicted. According to one embodiment, the device 6000 is configured to perform the formation of a list 528 of most probable intra-prediction modes based on the intra-prediction modes in which the neighboring blocks 524, 526 adjacent to a given block 18 are predicted, whereby the planar intra-prediction mode 504 is placed in the first position of the list 528 of most probable intra-prediction modes independent of the intra-prediction modes in which the neighboring blocks 524, 526 are predicted.

[0241] 15 shows a block diagram of a method 4000 for decoding a given block 18 of an image using intra prediction, including a step 4100 of deriving a set-selective syntax element from a data stream indicating whether the given block should be predicted using one of a first set of intra prediction modes including a DC intra prediction mode and an angular prediction mode. If the set-selective syntax element indicates 4150 that the given block should be predicted using one of the first set of intra prediction modes, the method 4000 includes a step 4200 of forming a list of most probable intra prediction modes based on the intra prediction modes in which neighboring blocks adjacent to the given block are predicted, a step 4300 of deriving an MPM list index from the data stream that directs the list of most probable intra prediction modes to the given intra prediction mode, and a step 4400 of intra predicting the given block using the given intra prediction mode. If the set-selective syntax element indicates that the given block should not be predicted using one of the first set of intra-prediction modes 4155, the method 4000 includes deriving 4250 from the data stream a further index indicating the given matrix-based intra-prediction mode of the second set of matrix-based intra-prediction modes by calculating 4350 a matrix-vector product between a vector derived from reference samples in the neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra-prediction mode to obtain a prediction vector, and predicting 4450 samples of the given block based on the prediction vector. The list of most probable intra-prediction modes is formed 4200 based on the intra-prediction modes in which neighboring blocks neighboring the given block are predicted, such that the list of most probable intra-prediction modes does not include the DC intra-prediction mode if at least one of the neighboring blocks is predicted by any of the angular intra-prediction modes.

[0242] Figure 16 shows a block diagram of a method 5000 for encoding a given block of an image using intra prediction, including signaling 5100 a set-selective syntax element in a data stream indicating whether the given block should be predicted using one of a first set of intra prediction modes including a DC intra prediction mode and an angular prediction mode. If the set-selective syntax element indicates 5150 that the given block should be predicted using one of the first set of intra prediction modes, the method 5000 includes steps 5200 of forming a list of most probable intra prediction modes based on the intra prediction modes in which neighboring blocks adjacent to the given block are predicted, 5300 of signaling an MPM list index in the data stream that directs the list of most probable intra prediction modes to the given intra prediction mode, and 5400 of intra predicting the given block using the given intra prediction mode. If the set selective syntax element indicates 5155 that the given block should not be predicted using one of the first set of intra prediction modes, the method 5000 includes a step 5250 of signaling a further index in the data stream indicating a given matrix-based intra prediction mode of the second set of matrix-based intra prediction modes by a step 5350 of calculating a matrix-vector product between a vector derived from reference samples of a neighborhood of the given block and a given prediction matrix associated with the given matrix-based intra prediction mode to obtain a prediction vector, and a step 5450 of predicting samples of the given block based on the prediction vector. The list of most probable intra prediction modes is formed 5200 based on the intra prediction modes in which neighboring blocks neighboring the given block are predicted, such that the list of most probable intra prediction modes does not include DC intra prediction mode if at least one of the neighboring blocks is predicted by any of the angular intra prediction modes.

[0243] References [1] P. Helle et al., "Non-linear weighted intra prediction," JVET-L0199, Macau, China, October 2018. [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, “JVET common test conditions and software reference configurations for SDR video”, JVET-K 1010, Ljubljana, SI, July 2018.

[0244] Further embodiments and examples In general, the examples may be implemented as a computer program product having program instructions that operate to perform one of the methods when the computer program product is executed on a computer. The program instructions may be stored on a machine-readable medium, for example. Other examples comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier. In other words, therefore, an example of a method is a computer program having program instructions for performing one of the methods described herein, when the computer program runs on a computer.

[0245] A further example of a method is therefore a data carrier medium (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier medium, digital storage medium, or recording medium is tangible and / or non-transitory rather than an intangible, transitory signal. A further example of a method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be transferred via a data communication connection, for example via the Internet.

[0246] Further examples include a processing means, for example a computer, or a programmable logic device for performing one of the methods described herein. A further example comprises a computer having installed thereon the computer program for performing one of the methods described herein. Further examples include an apparatus or system for transferring (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0247] In some examples, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some examples, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods may be performed by any suitable hardware apparatus.

[0248] The above examples are merely illustrative of the principles described above. It is understood that modifications and variations of the arrangements and details described herein are obvious. It is therefore the intention to be limited by the appended claims and not by the specific details presented by way of illustration and description of the embodiments herein. The same or equivalent elements, or elements having the same or equivalent functionality, are designated in the following description with the same or equivalent reference numbers even if they occur in different figures.

Claims

1. 1. A decoder for decoding an image from a data stream, comprising: For a current block of the image, determining whether (i) a first block adjacent to the current block is predicted using an intra prediction mode of an angular mode, and (ii) a second block adjacent to the current block is predicted using an angular intra prediction mode; forming a list of most probable intra prediction modes (MPMs) based on the intra prediction modes used to predict the first block and the second block, and excluding a DC intra prediction mode from the list of MPMs based in part on a determination that (i) the first block is predicted using an angular mode intra prediction mode and (ii) the second block is predicted using a non-angular intra prediction mode; Decoding from the data stream an index indicating an intra-prediction mode from the list of MPMs; predicting the current block using the intra-prediction mode indicated by the index; The decoder is configured to:

2. To form the list of MPMs, the decoder configured to not include the DC intra prediction mode in the list of MPMs in response to determining that the first block is predicted using an angular intra prediction mode. The decoder of claim 1.

3. the index is a first index, The decoder comprises: decode, for the current block of the image, from the data stream a syntax element that indicates whether the current block is predicted using an intra-prediction mode from a first set of intra-prediction modes or an intra-prediction mode from a second set of intra-prediction modes; After determining based on the syntax element that the current block is predicted using an intra-prediction mode from the first set of intra-prediction modes, decode from the data stream the first index indicating the intra-prediction mode from the list of MPMs; decoding, from the data stream, a second index indicating a matrix-based intra-prediction mode after determining based on the syntax element that the current block is predicted using an intra-prediction mode from the second set of intra-prediction modes; The decoder of claim 1 further configured to:

4. the first set of intra prediction modes comprises at least one angular prediction mode, a planar intra prediction mode, and a DC intra prediction mode; the second set of intra-prediction modes comprises at least one matrix-based intra-prediction mode. A decoder according to claim 3.

5. 2. The decoder of claim 1, wherein the list of MPMs includes a DC intra-prediction mode if both the first block and the second block are not predicted using an intra-prediction mode of an angular mode.

6. 1. A method for decoding an image from a data stream, comprising: determining, for a current block of the image, (i) whether a first block adjacent to the current block is predicted using an intra prediction mode of an angular mode, and (ii) whether a second block adjacent to the current block is predicted using an angular intra prediction mode; forming a list of most probable intra prediction modes (MPMs) based on intra prediction modes used to predict the first block and the second block, wherein a DC intra prediction mode is excluded from the list of MPMs based in part on a determination that (i) the first block is predicted using an angular mode intra prediction mode and (ii) the second block is predicted using a non-angular intra prediction mode; decoding from the data stream an index indicating an intra-prediction mode from the list of MPMs; predicting the current block using the intra-prediction mode indicated by the index; The method is configured to:

7. Forming the list of MPMs comprises:

7. The method of claim 6, comprising: in response to determining that the first block is predicted using an angular intra-prediction mode, not including the DC intra-prediction mode in the list of MPMs.

8. the index is a first index, decoding, for a current block of the image, from the data stream, a syntax element that indicates whether the current block is predicted using an intra-prediction mode from a first set of intra-prediction modes or an intra-prediction mode from a second set of intra-prediction modes; After determining that the current block is predicted using an intra-prediction mode from the first set of intra-prediction modes based on the syntax element, decoding the first index from the data stream indicating the intra-prediction mode from the list of MPMs; decoding, from the data stream, a second index indicating a matrix-based intra-prediction mode after determining based on the syntax element that the current block is predicted using an intra-prediction mode from the second set of intra-prediction modes; The method of claim 6 further comprising:

9. the first set of intra prediction modes comprises at least one angular prediction mode, a planar intra prediction mode, and a DC intra prediction mode; the second set of intra-prediction modes comprises at least one matrix-based intra-prediction mode. The method according to claim 8.

10. 7. The method of claim 6, wherein the list of MPMs includes a DC intra-prediction mode if both the first block and the second block are not predicted using an intra-prediction mode of an angular mode.

11. 1. An encoder for encoding an image into a data stream, comprising: For a current block of the image, determining whether (i) a first block adjacent to the current block is predicted using an intra prediction mode of an angular mode, and (ii) a second block adjacent to the current block is predicted using an angular intra prediction mode; forming a list of most probable intra prediction modes (MPMs) based on the intra prediction modes used to predict the first block and the second block, and excluding a DC intra prediction mode from the list of MPMs based in part on a determination that (i) the first block is predicted using an angular mode intra prediction mode and (ii) the second block is predicted using a non-angular intra prediction mode; encoding an index indicating an intra prediction mode from the list of MPMs for predicting the current block into the data stream; The encoder is configured as follows:

12. To form the list of MPMs, the encoder configured to not include the DC intra prediction mode in the list of MPMs in response to determining that the first block is predicted using an angular intra prediction mode. The encoder of claim 11.

13. the index is a first index, The encoder comprises: encoding, for a current block of the image, a syntax element into the data stream that indicates whether the current block is predicted using an intra-prediction mode from a first set of intra-prediction modes or an intra-prediction mode from a second set of intra-prediction modes; After determining based on the syntax element that the current block is predicted using an intra-prediction mode from the first set of intra-prediction modes, encoding the first index indicating the intra-prediction mode from the list of MPMs into the data stream; encoding a second index into the data stream indicating a matrix-based intra-prediction mode after determining based on the syntax element that the current block is predicted using an intra-prediction mode from the second set of intra-prediction modes. The encoder of claim 11 further configured to:

14. the first set of intra prediction modes comprises at least one angular prediction mode, a planar intra prediction mode, and a DC intra prediction mode; the second set of intra-prediction modes comprises at least one matrix-based intra-prediction mode. The encoder of claim 13.

15. The encoder of claim 11 , wherein the list of MPMs includes a DC intra-prediction mode if both the first block and the second block are not predicted using an intra-prediction mode of an angular mode.

16. 1. A method for encoding an image into a data stream, comprising the steps of: determining, for a current block of the image, (i) whether a first block adjacent to the current block is predicted using an intra prediction mode of an angular mode, and (ii) whether a second block adjacent to the current block is predicted using an angular intra prediction mode; forming a list of most probable intra prediction modes (MPMs) based on intra prediction modes used to predict the first block and the second block, wherein a DC intra prediction mode is excluded from the list of MPMs based in part on a determination that (i) the first block is predicted using an angular mode intra prediction mode and (ii) the second block is predicted using a non-angular intra prediction mode; encoding an index indicating an intra prediction mode from the list of MPMs for predicting the current block into the data stream; A method comprising:

17. Forming the list of MPMs comprises:

17. The method of claim 16, comprising: in response to determining that the first block is predicted using an angular intra-prediction mode, not including the DC intra-prediction mode in the list of MPMs.

18. the index is a first index, encoding, for a current block of the image, a syntax element into the data stream that indicates whether the current block is predicted using an intra-prediction mode from a first set of intra-prediction modes or an intra-prediction mode from a second set of intra-prediction modes; After determining that the current block is predicted using an intra-prediction mode from the first set of intra-prediction modes based on the syntax element, encoding the first index indicating the intra-prediction mode from the list of MPMs into the data stream; encoding a second index into the data stream indicating a matrix-based intra-prediction mode after determining based on the syntax element that the current block is predicted using an intra-prediction mode from the second set of intra-prediction modes; 20. The method of claim 16, further comprising:

19. the first set of intra prediction modes comprises at least one angular prediction mode, a planar intra prediction mode, and a DC intra prediction mode; the second set of intra-prediction modes comprises at least one matrix-based intra-prediction mode.

20. The method of claim 18.

20. 17. The method of claim 16, wherein the list of MPMs includes a DC intra-prediction mode if both the first block and the second block are not predicted using an intra-prediction mode of an angular mode.

21. A computer program which, when executed on a computer or signal processor, causes said computer or signal processor to carry out a method according to claim 6 or 16.

Images