Block-Based Prediction

Abstract

To provide a method and apparatus advantageous for determining a prediction vector.SOLUTION: An apparatus 1000 for predicting a predetermined block 18 of a picture 10 using a plurality of reference samples 171, 172 forms 100 a sample value vector 400 from a plurality of reference samples, derives from the sample value vector 400 a further vector 402 to which the sample value vector 400 is mapped by a predetermined invertible linear transformation 403, and for obtaining a prediction vector 406 calculates a matrix-vector product 404 between the further vector 402 and a predetermined prediction matrix 405, and predicts samples of the predetermined block 18 based on the prediction vector 406.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] FIELD OF THE DISCLOSURE The present application relates to the field of block-based prediction.Embodiments relate to an advantageous method for determining a prediction vector. [Background technology]

[0002] Today, there are different block-based intra- and inter-prediction modes: samples adjacent to the block to be predicted or samples taken from other pictures can form a sample vector that can undergo matrix multiplication to determine the prediction signal of the block to be predicted.

[0003] The matrix multiplication should preferably be performed in integer arithmetic, and matrices derived by some machine learning based training algorithm should be used for the matrix multiplication. Summary of the Invention [Problem to be solved by the invention]

[0004] However, such training algorithms usually only yield matrices given with floating-point precision, and therefore the challenge is to specify integer arithmetic such that matrix multiplication can be sufficiently approximated using these integer arithmetic, and / or to achieve improved computational efficiency and / or to make predictions more effective with respect to implementation. [Means for solving the problem]

[0005] This is achieved by the subject matter of the independent claims of the present application. Further embodiments of the invention are defined by the subject matter of the dependent claims of this application.

[0006] According to a first aspect of the present invention, the inventors of the present application have recognized that one problem encountered when trying to determine a prediction vector by an encoder or decoder is the possibility of not using integer arithmetic to calculate a prediction vector of a given block. According to a first aspect of the present application, this difficulty is overcome by deriving from the sample value vector a further vector to which the sample value vector is mapped by a predefined reversible linear transformation, so that the sample value vector is not directly applied to the matrix-vector product that calculates the prediction vector. Instead, a matrix-vector product is calculated between the further vector and a predefined prediction matrix to calculate the prediction vector. The further vector is derived, for example, such that the samples of the given block can be predicted by a device that uses integer arithmetic and / or fixed-point arithmetic. This is based on the idea that an advantageous predefined reversible linear transformation can be used to obtain a further vector with mainly small entries, whereby the components of the sample value vector are correlated, thereby allowing, for example, integer matrices and / or matrices with fixed-point values ​​and / or matrices with small prediction quantization errors to be used as the predefined prediction matrix.

[0007] Thus, according to a first aspect of the present application, an apparatus for predicting a given block of a picture using a plurality of reference samples is configured to form a sample value vector from a plurality of reference samples. The reference samples are, for example, samples adjacent to the given block in intra prediction or samples of another picture in inter prediction. According to an embodiment, the reference samples can be reduced, for example by averaging to obtain a sample value vector having a reduced number of values. Furthermore, the apparatus is configured to derive from the sample value vector a further vector to which the sample value vector is mapped by a predetermined reversible linear transformation, calculate a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector, and predict a sample of the given block based on the prediction vector. Based on the further vector, the prediction of the sample of the given block can represent an integer approximation of a direct matrix-vector product between the sample value vector and a matrix to obtain a prediction sample of the given block.

[0008] The direct matrix-vector product between the sample value vector and the matrix can be equal to a second matrix-vector product between the further vector and the second matrix. The second matrix and / or the matrix is, for example, a machine learning prediction matrix. According to an embodiment, the second matrix can be based on a predefined prediction matrix and an integer matrix. The second matrix is, for example, equal to the sum of the predefined prediction matrix and the integer matrix. In other words, the second matrix-vector product between the further vector and the second matrix can be represented by a matrix-vector product between the further vector and the predefined prediction matrix and a further matrix-vector product between the integer matrix and the further vector. The integer matrix is, for example, a matrix with a predefined column i0 consisting of 1s and columns i≠i0 being 0s. Thus, a good integer approximation and / or a good fixed-point value approximation of the first and / or second matrix-vector product can be achieved by the device. This is based on the idea that the given prediction matrix can be quantized or is already a quantized matrix since the further vector contains mainly small values, resulting in a negligible effect of possible quantization errors in the approximation of the first and / or second matrix-vector product.

[0009] According to an embodiment, a reversible linear transform that multiplies a given prediction vector by the sum of an integer matrix can correspond to a quantized version of the machine learning prediction matrix, for example a matrix where a given column i0 is made up of ones and columns i≠i0 are zeros.

[0010] According to an embodiment, the reversible linear transformation is defined such that a predetermined component of the further vector is a and each other component of the further vector, except for the predetermined component, is equal to the corresponding component of the sample value vector minus a, where a is a predetermined value. Thus, a small value of the further vector can be realized, allowing the quantization of the predetermined prediction matrix, resulting in a small effect of quantization error on the predicted samples of the predetermined block. This further vector allows the samples of the predetermined block to be predicted by integer arithmetic and / or fixed-point arithmetic.

[0011] According to an embodiment, the predefined value is one of the following: an average, such as an arithmetic or weighted average, of the components of the sample value vector, a default value, a value signaled in the data stream in which the picture is encoded, and a component of the sample value vector corresponding to the predefined component. The sample value vector is for example constituted by a plurality of reference samples or by an average of a group of reference samples from a plurality of reference samples. The group of reference samples includes for example at least two reference samples, preferably adjacent reference samples.

[0012] The predetermined value is, for example, an arithmetic or weighted average of some components (e.g., of at least two components) of the sample-valued vector or of all components of the sample-valued vector. This is based on the idea that the components of the sample-valued vector may be correlated, i.e. the values ​​of the components may be similar and / or at least some of the components may have equal values, whereby the components of the further vector are not equal to the predetermined components of the further vector, i.e. components i, i≠i0, where i0 represents the predetermined component, likely have smaller absolute values ​​than the corresponding components of the sample-valued vector. Thus, further vectors of smaller values ​​can be realized.

[0013] The predefined value may be a default value, which is for example selected from a list of default values ​​or is the same for all block sizes, prediction modes, etc. The elements of the list of default values ​​may be associated with different block sizes, prediction modes, sample value vector sizes, averages of values ​​of sample value vectors, etc. Thus, for example, depending on a given block, i.e. depending on the decoding or encoding settings associated with the given block, an optimized default value is selected by the device from the list of default values.

[0014] Alternatively, the predefined value may be a value signaled in the data stream in which the picture is encoded, in which case, for example, the encoding device determines the predefined value, which may be based on the same considerations as discussed above in the context of the default values.

[0015] The components of the further vector are not equal to the predetermined component of the further vector, i.e., component i, where i≠i0, where i0 represents the predetermined component, has a smaller absolute value than the corresponding component of the sample value vector, e.g. using a default value or a value signaled in the data stream as the predetermined value.

[0016] According to an embodiment, the predetermined value can be the component of the sample-valued vector corresponding to the predetermined component. In other words, the value of the component of the sample-valued vector corresponding to the predetermined component does not change by applying a reversible linear transformation. Thus, the value of the component of the sample-valued vector corresponding to the predetermined component is, for example, equal to the value of the predetermined component of a further vector.

[0017] The predetermined components are selected by default, for example as described above with respect to the predetermined values ​​as default values. It is clear that the predetermined components can be selected by alternative procedures. The predetermined components are selected, for example, similar to the predetermined values. According to an embodiment, the predetermined components are selected such that the value of the corresponding component of the sample value vector is equal to the average of the values ​​of the sample value vector or has only marginal deviations from the average of the values ​​of the sample value vector.

[0018] According to an embodiment, the matrix elements of the predetermined prediction matrix in the columns of the predetermined prediction matrix corresponding to the predetermined components of the further vector are all zero. The device is configured to calculate the matrix-vector product, i.e. the matrix-vector product between the further vector and the predetermined prediction matrix, by performing a multiplication by calculating a matrix-vector product between a reduced prediction matrix resulting from the predetermined prediction matrix by leaving a column, i.e. a column consisting of zeros, and a further vector resulting from the further vector by leaving a predetermined component. This is based on the idea that the predetermined components of the further vector are set to a predefined value, and that this predefined value is exact or close to the sample values ​​in the predicted signal of the predetermined block if the values ​​of the sample value vector are correlated. Thus, the prediction of the samples of the predetermined block is optionally based on the predetermined prediction matrix multiplied by the further vector, or rather the reduced prediction matrix multiplied by the further vector, and an integer matrix, of which the column i0 corresponds to the predetermined component and all other columns i≠i0 are zero, multiplied by the further vector. In other words, the machine learning prediction matrix, for example transformed by the inverse transformation of the predetermined reversible linear transformation, can be divided into a predetermined prediction matrix, or rather a reduced prediction matrix and an integer matrix, based on the further vector. Thus, only the prediction matrix should be quantized to obtain an integer approximation of the machine learning prediction matrix and / or the transformed machine learning prediction matrix, which is advantageous because the further vector does not contain the predetermined component and all other components have a much smaller absolute value than the corresponding components of the sample value vector, which allows for a smaller impact of quantization errors in the resulting quantization of the machine learning prediction matrix and / or the transformed machine learning prediction matrix. Furthermore, with the reduced prediction matrix and the further vector, fewer multiplications have to be performed to obtain the prediction vector, reducing the complexity and resulting in higher computational efficiency. Optionally, when predicting the samples of the predetermined block, a vector whose all components are the predetermined value a can be added to the prediction vector. This vector can be obtained by a matrix-vector product between the integer matrix and the further matrix, as described above.

[0019] According to an embodiment, the matrix resulting from summing each matrix element of the predetermined prediction matrix in a column of the predetermined prediction matrix corresponding to a predetermined component of the further vector with 1 times the predetermined reversible linear transformation corresponds to a quantized version of the machine learning prediction matrix. Summing each matrix element of the predetermined prediction matrix in a column of the predetermined prediction matrix corresponding to a predetermined component of the further vector with 1 represents, for example, a transformed machine learning prediction matrix. A transformed machine learning prediction matrix represents, for example, a machine learning prediction matrix transformed by the inverse transformation of the predetermined reversible linear transformation. The summation can correspond to the summation of the predetermined prediction matrix with an integer matrix, where the column i0, whose column corresponds to the predetermined component, consists of 1 and all other columns i≠i0 are 0.

[0020] According to an embodiment, the device is configured to use the prediction parameters to represent a predefined prediction matrix and to calculate a matrix-vector product by performing multiplication and addition on the components of the further vector and the prediction parameters and intermediate results resulting therefrom. The absolute value of the prediction parameters can be represented by an n-bit fixed-point representation, where n is 14 or less, alternatively 10 or less, alternatively 8 or less. In other words, the prediction parameters are multiplied and / or added to the elements of the matrix-vector product, such as the further vector, the predefined prediction matrix and / or the prediction vector. By the multiplication and addition operations, for example, a fixed-point format of the predefined prediction matrix, the prediction vector, and / or the prediction samples of the predefined block can be obtained.

[0021] An embodiment according to the present invention relates to an apparatus for coding a picture, comprising an apparatus for predicting a given block of a picture using a plurality of reference samples according to any of the embodiments described herein to obtain a prediction signal. Furthermore, the apparatus comprises an entropy encoder configured to encode a prediction residual of the given block to correct the prediction signal. For prediction of the given block to obtain the prediction signal, the apparatus is configured for example to form a sample value vector from the plurality of reference samples, to derive from the sample value vector a further vector to which the sample value vector is mapped by a predefined reversible linear transformation, to calculate a matrix-vector product between the further vector and a predefined prediction matrix to obtain a prediction vector, and to predict samples of the given block based on the prediction vector.

[0022] An embodiment according to the present invention relates to an apparatus for decoding a picture, comprising an apparatus for predicting a given block of the picture using a plurality of reference samples according to any of the embodiments described herein to obtain a prediction signal. Furthermore, the apparatus comprises an entropy decoder configured to decode a prediction residual of the given block, and a prediction corrector configured to correct the prediction signal using the prediction residual. For prediction of the given block to obtain the prediction signal, the apparatus is configured for example to form a sample value vector from the plurality of reference samples, to derive from the sample value vector a further vector to which the sample value vector is mapped by a predefined reversible linear transformation, to calculate a matrix-vector product between the further vector and a predefined prediction matrix to obtain a prediction vector, and to predict samples of the given block based on the prediction vector.

[0023] An embodiment according to the present invention relates to a method for predicting a given block of a picture using a plurality of reference samples, the method comprising: forming a sample value vector from the plurality of reference samples; deriving from the sample value vector a further vector to which the sample value vector is mapped by a predetermined reversible linear transformation; computing a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector; and predicting samples of the given block based on the prediction vector.

[0024] An embodiment according to the present invention relates to a method for coding a picture, comprising predicting a given block of the picture using a number of reference samples according to the method described above to obtain a prediction signal, and entropy coding the prediction residual of the given block to correct the prediction signal.

[0025] An embodiment according to the present invention relates to a method for decoding a picture, comprising predicting a given block of the picture using a plurality of reference samples according to one of the methods described above to obtain a prediction signal, entropy decoding the prediction residual of the given block, and correcting the prediction signal using the prediction residual. An embodiment according to the present invention relates to a data stream having pictures encoded using the method described herein for encoding pictures.

[0026] An embodiment according to the present invention also relates to a computer program having a program code for performing any of the methods of the embodiments described herein, when the computer program runs on a computer.

[0027] The drawings are not necessarily to scale, emphasis instead being placed upon illustrating generally the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which: [Brief description of the drawings]

[0028] [Figure 1] 1 illustrates an embodiment of encoding into a data stream. [Diagram 2] 2 shows an embodiment of an encoder. [Diagram 3] 1 illustrates an embodiment of a picture reconstruction. [Figure 4] 2 shows an embodiment of a decoder. [Diagram 5] 1 shows a schematic diagram of prediction of a block for encoding and / or decoding according to an embodiment; [Figure 6] 1 illustrates a matrix operation for prediction of a block for encoding and / or decoding according to an embodiment. [Figure 7.1] 1 illustrates a prediction of a block with a reduced sample value vector according to an embodiment. [Figure 7.2] 1 illustrates block prediction using sample interpolation according to an embodiment. [Figure 7.3] 13 illustrates prediction of a block with a reduced sample value vector in which only some boundary samples are averaged, according to an embodiment. [Figure 7.4] 13 illustrates prediction of a block with a reduced sample value vector in which groups of four boundary samples are averaged, according to an embodiment. [Figure 8] 1 shows a schematic diagram of an apparatus for predicting a block according to an embodiment; [Figure 9] 4 illustrates a matrix operation performed by an apparatus according to an embodiment. [Figure 10a] 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 10b-10c] 4 illustrates detailed matrix operations performed by an apparatus according to an embodiment. [Figure 11] 13 illustrates detailed matrix operations performed by an apparatus using offset and scaling parameters according to an embodiment. [Figure 12]4 illustrates detailed matrix operations performed by an apparatus using offset and scaling parameters according to different embodiments; [Figure 13] 1 shows a block diagram of a method for predicting a given block according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] Equal or equivalent elements or elements having equal or equivalent functions are indicated in the following description by equal or equivalent reference signs even if they occur in different figures.

[0030] In the following description, a number of details are set forth to provide a more complete description of the embodiments of the present invention. However, it will be apparent to those skilled in the art that the embodiments of the present invention can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring the embodiments of the present invention. Furthermore, the features of different embodiments described below can be combined with each other unless otherwise specified. 1 Introduction

[0031] In the following, different examples, embodiments and aspects of the present invention are described, at least some of which refer, inter alia, to methods and / or apparatuses for video coding and / or for performing block-based prediction using, for example, linear or affine transformation with adjacent sample reduction, and / or for optimizing video distribution (e.g., broadcasting, streaming, file playback, etc.), for, for example, video applications and / or virtual reality applications.

[0032] Additionally, examples, embodiments, and aspects may refer to High Efficiency Video Coding (HEVC) or successors. Further embodiments, examples, and aspects are defined by the appended claims.

[0033] It should be noted that any embodiment, example and aspect defined by the claims can be supplemented by any of the details (features and functions) described in the following sections.

[0034] Also, the embodiments, examples and aspects described in the following sections can be used individually or can be supplemented by any of the features of the other sections or by any features included in the claims.

[0035] It should also be noted that the individual examples, embodiments, and aspects described herein can be used individually or in combination, and thus details can be added to each of the individual aspects without adding details to another one of the examples, embodiments, and aspects. It should also be noted that this disclosure explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0036] Moreover, features and functions disclosed herein in relation to a method can also be used in an apparatus. Furthermore, any features and functions disclosed herein in relation to an apparatus can also be used in the corresponding method. In other words, the method disclosed herein can be supplemented by any of the features and functions described in relation to the apparatus.

[0037] Additionally, any of the features and functionality described herein can be implemented in hardware or software, or using a combination of hardware and software, as described in the "Implementation Alternatives" section.

[0038] Additionally, any features listed within parentheses ("(...)" or "[...]") may be considered as optional in some examples, embodiments, or aspects.

[0039] 2 Encoder, Decoder In the following, various examples are described that can help to achieve more effective compression when using block-based prediction. Some examples achieve high compression efficiency by using a set of intra-prediction modes. The latter can be added to other intra-prediction modes, for example heuristically designed, or provided exclusively. And even other examples take advantage of both of the specialties described here. However, as a variation of these embodiments, intra-prediction can be converted to inter-prediction by using reference samples in another picture instead.

[0040] To facilitate understanding of the following examples of this application, the description begins with a presentation of possible encoders and decoders to which the examples subsequently outlined in this application can be built. Figure 1 shows an apparatus for block-wise encoding of a picture 10 into a data stream 12. The apparatus is indicated using reference sign 14 and can be a still picture encoder or a video encoder. In other words, the picture 10 can be a current picture from a video 16 if the encoder 14 is configured to encode a video 16 containing the picture 10 into the data stream 12, or if the encoder 14 can exclusively encode the picture 10 into the data stream 12.

[0041] As mentioned above, the encoder 14 performs the encoding in a block-wise manner or on a block basis. To this end, the encoder 14 subdivides the picture 10 into blocks, which units of the encoder 14 encode the picture 10 into the data stream 12. Examples of possible subdivisions of the picture 10 into blocks 18 are given in more detail below. In general, the subdivision may result in blocks 18 of a fixed size, such as an array of blocks arranged in rows and columns, or blocks 18 of different block sizes, such as by using a hierarchical multi-tree subdivision starting from the entire picture area of ​​the picture 10 or from a pre-partition of the picture 10 into an array of tree blocks, and these examples should not be treated as excluding other possible ways of subdividing the picture 10 into blocks 18.

[0042] Moreover, the encoder 14 is a predictive encoder configured to predictively encode the picture 10 into the data stream 12. For a particular block 18, this means that the encoder 14 determines a prediction signal for the block 18 and encodes into the data stream 12 the prediction residual, i.e. the prediction error where the prediction signal deviates from the actual picture content in the block 18.

[0043] The encoder 14 may support different prediction modes to derive a prediction signal for a particular block 18. The prediction mode of interest in the following example is an intra prediction mode, where the interior of the block 18 is spatially predicted from neighboring, already coded samples of the picture 10. The coding of the picture 10 into the data stream 12, and therefore the corresponding decoding procedure, may be based on a particular coding order 20 defined among the blocks 18. For example, the coding order 20 may traverse the blocks 18 in a raster scan order, such as row-by-row from top to bottom, while traversing each row from left to right. In case of a hierarchical multi-tree based subdivision, the raster scan order may be applied within each hierarchical level, and a depth-first traversal order may be applied. That is, leaf nodes in a block at a particular hierarchical level may precede blocks at the same hierarchical level that have the same parent block according to the coding order 20. Depending on the coding order 20, neighboring, already coded samples of the block 18 may be typically located on one or more sides of the block 18. In the example presented here, for example, the neighboring, already coded samples of block 18 are located above and to the left of block 18 .

[0044] Intra-prediction modes may not be the only modes supported by the encoder 14. For example, if the encoder 14 is a video encoder, the encoder 14 may also support inter-prediction modes in which the blocks 18 are temporally predicted from previously encoded pictures of the video 16. Such inter-prediction modes may be motion-compensated prediction modes in which a motion vector is signaled for such blocks 18 indicating the relative spatial offset of the portion of which the prediction signal of the block 18 is derived as a copy. Additionally or alternatively, other non-intra-prediction modes may also be available, such as inter-prediction modes in the case of the encoder 14 being a multiview encoder, or non-prediction modes in which the interior of the block 18 is coded as is, i.e., without prediction.

[0045] Before beginning the description of the present application by focusing on intra-prediction modes, we will describe a more specific example of a possible block-based encoder, i.e., a possible implementation of encoder 14 as described with respect to FIG. 2 and then present two corresponding examples of decoders compatible with FIGS. 1 and 2, respectively.

[0046] Fig. 2 shows a possible implementation of the encoder 14 of Fig. 1, i.e. the encoder is configured to use transform coding to code the prediction residual, but this is mostly by way of example and the application is not limited to that kind of prediction residual coding. According to Fig. 2, the encoder 14 comprises a subtractor 22 configured to subtract a corresponding prediction signal 24 from an inbound signal, i.e. the picture 10, or on a block basis, a current block 18, to obtain a prediction residual signal 26 which is subsequently coded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 comprises a lossy coding stage 28a and a lossless coding stage 28b. The lossy stage 28a comprises a quantizer 30 which receives the prediction residual signal 26 and quantizes the samples of the prediction residual signal 26. As already mentioned above, the present example uses a transform coding of the prediction residual signal 26, therefore the lossy coding stage 28a comprises a transformation stage 32 connected between the subtractor 22 and the quantizer 30 so as to transform such a prediction residual 26, spectrally decomposed, by a quantization of the quantizer 30 performed on the transformed coefficients presenting the residual signal 26. The transformation can be a DCT, a DST, an FFT, a Hadamard transform, etc. The transformed and quantized prediction residual signal 34 then undergoes lossless coding by a lossless coding stage 28b, which is an entropy coder that entropy codes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 further comprises a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 so as to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a way that is also exploitable by the decoder. That is to say, it is the quantizer 30 that takes into account the coding losses. To this end, the prediction residual reconstruction stage 36 comprises an inverse quantizer 38 which performs the inverse of the quantization of the quantizer 30, followed by an inverse transformer 40 which performs an inverse transformation to the transformation performed by the transformer 32, such as the inverse of the spectral decomposition, such as the inverse of any of the specific transformation examples mentioned above. The encoder 14 comprises an adder 42 which adds the reconstructed prediction residual signal output by the inverse transformer 40 with the prediction signal 24 to output a reconstructed signal, i.e. reconstructed samples.This output is supplied to a predictor 44 of the encoder 14, which determines thereon the predicted signal 24. It is the predictor 44 that supports all prediction modes already mentioned above with reference to figure 1. Figure 2 also shows that, if the encoder 14 is a video encoder, it may also comprise an in-loop filter 46 which filters the fully reconstructed picture which, after being filtered, forms the reference picture of the predictor 44 with respect to the inter-prediction blocks.

[0047] As already mentioned above, the encoder 14 operates on a block basis. In the following description, the block basis of interest is a subdivision of the picture 10 into blocks for which an intra prediction mode is selected from a set or a plurality of intra prediction modes supported by the predictor 44 or the encoder 14, respectively, and the selected intra prediction mode is executed individually. However, there may be other types of blocks into which the picture 10 is sub-divided as well. For example, the above determination regarding whether the picture 10 is inter-coded or intra-coded can be made with granularity or on a block-by-block basis deviating from the block 18. For example, the inter / intra mode decision can be performed at the level of the coding blocks into which the picture 10 is sub-divided and each coding block is sub-divided into predictive blocks. The predictive blocks having coding blocks for which it is determined that intra prediction is used are each subdivided into an intra prediction mode decision. Thus, for each of these predictive blocks, it is determined which supported intra prediction mode should be used for the respective predictive block. These predictive blocks form the block 18 of interest here. Prediction blocks in a coding block related to inter-prediction will be treated differently by the predictor 44. They will be inter-predicted from a reference picture by determining a motion vector and copying the prediction signal of this block from the location in the reference picture pointed to by the motion vector. Another block subdivision concerns the subdivision into transform blocks in the units where the transformation by the transformer 32 and the inverse transformer 40 is performed. The transformed blocks can be, for example, the result of further subdivision of the coding block. Naturally, the examples described here should not be treated as limiting and other examples exist. Just for the sake of completeness, it is noted that the subdivision into coding blocks can, for example, use multi-tree subdivision, and similarly, the prediction blocks and / or transform blocks can be obtained by further subdividing the coding block using multi-tree subdivision.

[0048] A decoder 54 or device for block-wise decoding, which is compatible with the encoder 14 of FIG. 1, is shown in FIG. 3. This decoder 54 does the opposite to the encoder 14, i.e. it decodes the picture 10 from the data stream 12 block-wise and for this purpose supports several intra prediction modes. The decoder 54 may, for example, comprise a residual provider 156. All other possibilities discussed above with respect to FIG. 1 are also valid for the decoder 54. Thus, the decoder 54 may be a still picture decoder or a video decoder, and all prediction modes and predictability are also supported by the decoder 54. The difference between the encoder 14 and the decoder 54 mainly consists in the fact that the encoder 14 selects or chooses the coding decision according to some optimization, for example to minimize some cost function that may depend on the coding rate and / or the coding distortion. One of these coding options or coding parameters may include the selection of the intra prediction mode to be used for the current block 18 from among the available or supported intra prediction modes. The selected intra-prediction mode is then signaled by the encoder 14 of the current block 18 in the data stream 12, and the decoder 54 uses this signaling of the data stream 12 of the block 18 to redo the selection. Similarly, the subdivision of the picture 10 into blocks 18 may be subject to optimization in the encoder 14, and corresponding subdivision information may be conveyed in the data stream 12, and the decoder 54 restores the subdivision of the picture 10 into blocks 18 based on the subdivision information. To summarize the above, the decoder 54 may be a predictive decoder operating on a block basis, and in addition to the intra-prediction mode, the decoder 54 may support other prediction modes, such as, for example, inter-prediction modes if the decoder 54 is a video decoder. In decoding, the decoder 54 may also use the coding order 20 described with reference to FIG. 1, which is followed in both the encoder 14 and the decoder 54, so that the same neighboring samples are available for the current block 18 in both the encoder 14 and the decoder 54.Therefore, in order to avoid unnecessary repetitions, the description of the operating modes of the encoder 14 must also apply to the decoder 54 as far as the subdivision of the picture 10 into blocks is concerned, for example as far as the prediction and as far as the coding of the prediction residuals is concerned. The difference lies in the fact that the encoder 14 selects in or inserts into the data stream 12, by optimization, several coding options or coding parameters and signals, which are derived from the data stream 12 by the decoder 54 in order to redo the prediction, such as the subdivision.

[0049] FIG. 4 shows a possible implementation of the decoder 54 of FIG. 3, i.e. one that matches the implementation of the encoder 14 of FIG. 1, as shown in FIG. 2. Many elements of the encoder 54 of FIG. 4 are the same as those occurring in the corresponding encoder of FIG. 2, so the same reference numbers with an apostrophe are used in FIG. 4 to indicate these elements. In particular, the adder 42', the optional in-loop filter 46' and the predictor 44' are connected in the prediction loop in the same way as they are in the encoder of FIG. 2. The reconstructed, i.e. inverse quantized and retransformed prediction residual signal applied to the adder 42' is derived by a sequence of an entropy decoder 56 that reverses the entropy coding of the entropy coder 28b, followed by a residual signal reconstruction stage 36' consisting of an inverse quantizer 38' and an inverse transformer 40' as in the encoding side. The output of the decoder is a reconstruction of the picture 10. The reconstruction of the picture 10 can be made available directly at the output of the adder 42' or alternatively at the output of the in-loop filter 46'. In order to improve the picture quality, some postfilters can be placed at the output of the decoder to subject the reconstruction of picture 10 to some post-filtering, although this option is not shown in FIG.

[0050] Again, with respect to Figure 4, the explanations given above with respect to Figure 2 are valid for Figure 4, except that the encoder only performs optimization tasks and related decisions regarding coding options. However, all the explanations regarding block subdivision, prediction, inverse quantization, and retransformation are also valid for the decoder 54 of Figure 4.

[0051] 3. ALWIP (Affine Linear Weighted Intra Predictor) Some non-limiting examples of ALWIP are described herein, even though ALWIP is not necessarily required to implement the techniques described herein.

[0052] This application relates, inter alia, to the concept of improved block-based prediction modes for block-based picture coding, as may be used in video codecs such as HEVC or successors of HEVC. The prediction modes may be intra-prediction modes, but theoretically the concept described herein can also be translated to inter-prediction modes, where the reference samples are part of another picture. There is a need for a block-based prediction concept that allows for efficient implementation, including hardware-friendly implementation. This object is achieved by the subject matter of the independent claims of the present application.

[0053] Intra prediction modes are widely used in picture and video coding. In video coding, intra prediction modes compete with other prediction modes, such as inter-prediction modes, such as motion-compensated prediction modes. In intra prediction modes, the current block is predicted based on neighboring samples, i.e., samples that have already been coded as far as the encoder side is concerned, and already decoded as far as the decoder side is concerned. The neighboring sample values ​​are extrapolated to the current block to form a prediction signal for the current block, and the prediction residual is transmitted in the data stream for the current block. The better the prediction signal, the smaller the prediction residual, and therefore the fewer bits required to code the prediction residual.

[0054] To be effective, several aspects need to be taken into account in order to form an effective framework for intra prediction in a block-based picture coding environment. For example, the more intra prediction modes supported by a codec, the more side information rate is consumed to inform the decoder of the selection. On the other hand, the set of supported intra prediction modes needs to be able to provide a good prediction signal, i.e. a prediction signal that results in a low prediction residual.

[0055] In the following, as a comparative embodiment or basic example, an apparatus (encoder or decoder) for block-wise decoding of a picture from a data stream is disclosed, which supports at least one intra prediction mode in which an intra prediction signal for a block of a predetermined size of the picture is determined by applying a first template of samples adjacent to the current block to an affine linear predictor, which shall ultimately be called an affine linear weighted predictor (ALWIP).

[0056] The apparatus may have at least one of the following characteristics (the same may apply to a method or another technique implemented, for example, in a non-transitory storage unit that stores instructions that, when executed by a processor, cause the processor to perform a method and / or operate as an apparatus):

[0057] 3.1 A predictor can be complementary to other predictors. The intra prediction modes, which may form the subject of implementation improvements described further below, may be complementary to other intra prediction modes of the codec. They may thus be complementary to the DC, planar or angular prediction modes defined in the HEVC codec and the JEM reference software. The latter three types of intra prediction modes shall henceforth be referred to as conventional intra prediction modes. Thus, for a given block of an intra mode, a flag must be analyzed by the decoder indicating whether one of the intra prediction modes supported by the device should be used or not.

[0058] 3.2 Two or more proposed prediction modes A device may contain more than one ALWIP mode, so if the decoder knows that one of the ALWIP modes supported by the device should be used, the decoder needs to parse additional information that indicates which of the ALWIP modes supported by the device should be used.

[0059] The signaling of supported modes can have the property that the encoding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes require fewer bins and which require more bins can depend on information that can be extracted from an already decoded bitstream or that can be fixed in advance.

[0060] 4. Some Aspects 2 illustrates a decoder 54 for decoding a picture from data stream 12. Decoder 54 may be configured to decode a given block 18 of the picture. In particular, predictor 44 may be configured to map a set of P neighboring samples of the given block 18 to a set of Q predictions of the samples of the given block using a linear or affine-linear transformation [e.g., ALWIP].

[0061] As shown in FIG. 5, a given block 18 contains Q values ​​to be predicted (which at the end of the operation will be the "predicted values"). If the block 18 has M rows and N columns, then Q=M·N. The Q values ​​of the block 18 can be in the spatial domain (e.g. pixels) or in the transform domain (e.g. DCT, discrete wavelet transform, etc.). The Q values ​​of the block 18 can be predicted based on P values ​​obtained from neighboring blocks 17a-17c, which are generally adjacent to the block 18. The P values ​​of the neighboring blocks 17a-17c may be located closest (e.g. adjacent) to the block 18. The P values ​​of the neighboring blocks 17a-17c have already been processed and predicted. The P values ​​are shown as values ​​in parts 17'a-17'c to distinguish them from the block of which they are a part (in some examples, 17'b is not used).

[0062] As shown in FIG. 6, to perform the prediction, it is possible to operate on a first vector 17P with P entries (each entry is associated with a specific position of the neighboring portions 17'a to 17'c), a second vector 18Q with Q entries (each entry is associated with a specific position of the block 18), and a mapping matrix 17M (each row is associated with a specific position in the block 18 and each column is associated with a specific position in the neighboring portions 17'a to 17'c). The mapping matrix 17M thus performs a prediction of P values ​​of the neighboring portions 17'a to 17'c to values ​​of the block 18 according to a predefined mode. The entries in the mapping matrix 17M can therefore be understood as weighting coefficients. In the following description, the symbols 17a to 17c are used instead of 17'a to 17'c to refer to the neighboring portions of the boundary.

[0063] Several conventional modes are known in the art, such as DC mode, planar mode, and 65-directional prediction mode, for example, 67 modes are known.

[0064] However, it is noted that it is also possible to utilize a different mode, referred to herein as a linear or affine-linear transformation, which includes P·Q weighting coefficients, of which at least ¼P·Q are non-zero weight values, for each of the Q predictors, a sequence of P weighting coefficients for each predictor. When the sequence is arranged one above the other in raster scan order between the samples of a given block, it forms an envelope that is omnidirectionally non-linear.

[0065] It is possible to map P positions of the neighboring values ​​17'a to 17'c (templates), Q positions of the neighboring samples 17'a to 17'c, and values ​​of P*Q weighting coefficients of the matrix 17M. The plane is an example of the envelope of the sequence for the DC transformation (is a plane for the DC transformation). The envelope is obviously a plane and is therefore excluded by the definition of linear or affine linear transformation (ALWIP). Another example is a matrix that results in the emulation of the following angular mode: the envelope is excluded from the ALWIP definition and, in plain terms, looks like a hill going diagonally from top to bottom along a direction in the P / Q plane. The planar mode and the 65-directional prediction mode have different envelopes, which are linear in at least one direction, i.e., in all directions for example the DC illustrated, and in the hill direction for example the angular mode.

[0066] Conversely, the envelope of a linear or affine transformation is not omnidirectionally linear, and it is understood that such types of transformations may be optimal in some circumstances for performing the prediction of block 18. Note that it is preferable that at least ¼ of the weighting coefficients are different from 0 (i.e., at least 25% of the P*Q weighting coefficients are different from 0).

[0067] The weighting coefficients may be unrelated to one another according to any regular mapping rule. Thus, the matrix 17M may be such that the values ​​of its entries have no obvious discernible relationship. For example, the weighting coefficients cannot be described by any analytical or differential function.

[0068] In an example, the ALWIP transformation is such that the average of the maximum value of the cross-correlation between the weighting coefficients of a first series associated with the respective predictor and the weighting coefficients of a second series associated with the predictor other than the respective predictor, or an inverted version of the latter series, may be lower than a predefined threshold (for example, 0.2 or 0.3 or 0.35 or 0.1, for example, a threshold in the range between 0.05 and 0.035), even though it leads to a higher maximum value. For example, for each combination (i1, i2) of rows of the ALWIP matrix 17M, the cross-correlation may be calculated by multiplying the P value of the i1th row by the P value of the i2th row. For each cross-correlation obtained, a maximum value may be obtained. Thus, an average (mean) may be obtained for the entire matrix 17M (i.e., the maximum values ​​of the cross-correlation in all combinations are averaged). The threshold may then be, for example, 0.2 or 0.3 or 0.35 or 0.1, for example, a threshold in the range between 0.05 and 0.035.

[0069] The P adjacent samples of blocks 17a-17c may be arranged along a one-dimensional path that extends along the boundaries (e.g., 18c, 18a) of a given block 18. For each of the Q predicted values ​​of a given block 18, the set of P weighting factors associated with each predicted value may be ordered to traverse the one-dimensional path in a predetermined direction (e.g., left to right, top to bottom, etc.). In an example, the ALWIP matrix 17M may be off-diagonal or non-block diagonal. An example of an ALWIP matrix 17M for predicting a 4×4 block 18 from four already predicted neighboring samples may be: { { 37, 59, 77, 28}, { 32, 92, 85, 25}, { 31,69,100,24}, { 33,36,106,29}, { 24,49,104,48}, { 24,21,94,59}, { 29,0,80,72}, { 35,2,66,84}, { 32,13,35,99}, { 39,11,34,103}, { 45,21,34,106}, { 51,24,40,105}, { 50,28,43,101}, { 56,32,49,101}, { 61,31,53,102}, { 61,32,54,100} }.

[0070] (Here, {37,59,77,28} is the first row, {32,92,85,25} is the second row, {61,32,54,100} is the 16th row of matrix 17M.) Matrix 17M has dimensions 16x4 and contains 64 weighting coefficients (as a result of 16*4=64). This is because matrix 17M has dimensions QxP, where Q=M*N is the number of samples of block 18 to be predicted (block 18 is a 4x4 block) and P is the number of samples of already predicted samples. Here, M=4, N=4, Q=16 (as a result of M*N=4*4=16) and P=4. The matrix is ​​non-diagonal and non-block diagonal and is not described by any particular rule.

[0071] As can be seen, less than a quarter of the weighting factors are 0 (in the above matrix, 1 out of 64 weighting factors is 0). The envelope formed by these values, when placed one step down in raster scan order, forms an envelope that is omnidirectionally nonlinear.

[0072] Even though the above description has been primarily described with reference to a decoder (e.g., decoder 54), the same may be performed in an encoder (e.g., encoder 14).

[0073] In some examples, for each block size (in the set of block sizes), the ALWIP transforms of the intra prediction modes in the second set of intra prediction modes for each block size are different from each other. Additionally or alternatively, the cardinality of the second set of intra prediction modes for block sizes in the set of block sizes can match, but the associated linear transforms or affine linear transforms of the intra prediction modes in the second set of intra prediction modes for different block sizes can be inconvertible to each other by scaling.

[0074] In some examples, ALWIP transformations may be defined such that they have "nothing in common" with conventional transformations (e.g., an ALWIP transformation may have "nothing in common" with a corresponding conventional transformation, even if it is mapped via one of the mappings above).

[0075] In some examples, the ALWIP mode is used for both the luma and chroma components, while in other examples, the ALWIP mode is used for the luma component but not for the chroma components.

[0076] 5 Affine linear weighted intra prediction mode with coder acceleration (e.g. test CE3-1.2.1) 5.1 Description of the method or apparatus The Affine Linear Weighted Intra Prediction (ALWIP) mode tested in CE3-1.2.1 may be the same as that proposed in JVET-L0199 under test CE3-2.2.2, except for the following changes:

[0077] · Multiple Reference Line (MRL) intra prediction, especially alignment with the encoder estimation and signaling, i.e. MRL is not combined with ALWIP and transmission of MRL indices is restricted to non-ALWIP blocks.

[0078] Subsampling is now mandatory for all blocks W×H ≥ 32×32 (previously it was optional for 32×32). Therefore the additional test in the encoder and the transmission of the subsampling flag have been removed.

[0079] ALWIP for 64×N and N×64 blocks (N≦32) has been added by downsampling to 32×N and N×32 respectively and applying the corresponding ALWIP mode. In addition, test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0080] · Joint mode estimation: Conventional and ALWIP modes use a shared Hadamard candidate list for full RD estimation, i.e., ALWIP mode candidates are added to the same list as conventional (and MRL) mode candidates based on their Hadamard cost. EMT Intra Fast and PB Intra Fast are supported for combined mode lists and have additional optimizations to reduce the number of full RD checks. Only the MPMs of the available left and top blocks are added to the list for the ALWIP full RD estimation, following the same approach as in the conventional mode.

[0081] 5.2 Complexity Assessment Excluding calculations involving discrete cosine transforms, test CE3-1.2.1 required up to 12 multiplications per sample to generate the predicted signal. In addition, a total of 136492 parameters of 16 bits each were required, which corresponds to 0.273 megabytes of memory.

[0082] 5.3 Experimental results The test evaluation was performed according to the common test conditions JVET-J1010[2] for Intra-only (AI) and Random Access (RA) configurations using VTM software version 3.0.1. The corresponding simulations were performed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, Turbo Boost off) with Linux OS and GCC 7.2.1 compiler.

[0083] [Table 1]

[0084] [Table 2]

[0085] 5.4 Affine Linear Weighted Intra Prediction with Reduced Complexity (e.g., test CE3-1.2.2) The technique tested in CE2 is related to the "Affine Linear Intra Prediction" described in JVET-L0199[1], but simplifies it in terms of memory requirements and computational complexity as follows:

[0086] There can be only three different sets of prediction matrices (e.g., S0, S1, S2, see also below) and bias vectors (e.g., to provide offset values) that cover all block shapes. As a result, the number of parameters is reduced to 14400 10-bit values, which is

number

[0087] The input and output sizes of the predictor are further reduced. Furthermore, instead of transforming the boundaries through a DCT, averaging or downsampling can be performed on the boundary samples, and the generation of the predicted signal can use linear interpolation instead of an inverse DCT. As a result, up to four multiplications per sample may be required to generate the predicted signal.

[0088] 6. Example Here we describe how to perform several predictions (e.g., as shown in Figure 6) using ALWIP prediction. In principle, referring to Figure 6, multiplications of Q*P samples of the QxP ALWIP prediction matrix 17M with P samples of the Px1 neighboring vector 17P should be performed to obtain Q=M*N values ​​of the MxN block 18 to be predicted. Thus, in general, at least P=M+N value multiplications are required to obtain each of the Q=M*N values ​​of the MxN block 18 to be predicted.

[0089] These multiplications have a highly undesirable effect. The dimension P of the boundary vector 17P generally depends on the number M+N of boundary samples (bins or pixels) 17a, 17c that are adjacent (for example, adjacent) to the M×N block 18 to be predicted. This means that the larger the size of the block 18 to be predicted, the larger the number M+N of boundary pixels (17a, 17c) will be, so that the dimension P=M+N of the P×1 boundary vector 17P and the length of each row of the Q×P ALWIP prediction matrix 17M, and thus the number of multiplications required, will be large (generally speaking, Q=M*N=W*H, where W (width) is another symbol for N, H (height) is another symbol for M, and P=M+N=H+W if the boundary vector is formed by only one row and / or one column of samples).

[0090] This problem is generally exacerbated by the fact that in microprocessor-based systems (or other digital processing systems), multiplication is generally a power-consuming operation. It can be inferred that a large number of multiplications performed on a large number of samples of a large number of blocks will generally cause undesirable waste of computational power. It is therefore preferable to reduce the number of multiplications Q*P required to predict the M×N block 18 .

[0091] It is understood that by intelligently selecting easier to process operations to replace multiplications, it is possible to reduce in some way the computational power required for each intra prediction of each block 18 to be predicted. In particular, referring to Figures 7.1 to 7.4, the encoder or decoder:

[0092] shrinking (e.g., in step 811) a plurality of adjacent samples (e.g., 17a, 17c) to obtain a set of reduced sample values ​​having a smaller number of samples compared to the plurality of adjacent samples (e.g., by averaging or downsampling);

[0093] subjecting the reduced set of sample values ​​to a linear or affine-linear transformation (e.g., in step 812) to obtain a predicted value for a given sample of a given block;

[0094] It is understood that a given block of a picture (eg, 18) can be predicted using multiple neighboring samples (eg, 17a, 17c).

[0095] In some cases, the decoder or encoder can also derive, for example by interpolation, a prediction value of a further sample of a given block based on the prediction value of the given sample and a number of neighboring samples. Thus, an upsampling strategy can be obtained.

[0096] In an example, it is possible to perform several averages on the samples of boundary 17 (e.g., in step 811) so as to reach a reduced set 102 of samples with a reduced number of samples (Figures 7.1 to 7.4) (at least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples, or a selection of the original boundary samples). For example, if the original boundary has P = M + N samples, the reduced set of samples can have P red <P such that M red <M and N red <N is at least one of, P red = M red + N red Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries, but P red <P that is, P red × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have dimensions Q × P, but (M red <M and N red <N is at least one of) at least P red <P being such that the number of elements of the matrix is reduced to Q × P red (or Q red × P red , see below).

[0097] In some examples (e.g., Figures 7.2, 7.3), the block obtained by ALWIP (in step 812) is

Number

Number

Number

Number

Number

Number

[0098] These techniques can be advantageous because while the matrix multiplication involves a reduced number of multiplications (Q red *P red or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (and even avoiding) multiplications. For example, downsampling, averaging, and / or interpolation can be performed (e.g., in steps 811 and / or 813) by employing binary operations that require non-computation power such as addition and shift. Also, addition is a very simple operation that can be easily performed without much computational effort.

[0099] This shifting operation can be used, for example, to average two boundary samples and / or to interpolate two samples (support values) of (or taken from) the downscaled prediction block to obtain the final prediction block. (For the interpolation, two sample values ​​are needed: within a block we always have two predefined values, but to interpolate samples along the left and top boundaries of a block, we have only one predefined value, as in Figure 7.2, and therefore use the boundary sample as the support value for the interpolation.)

[0100] A two-step procedure can be used as follows: First add the values ​​of the two samples, The sum value is then halved (eg, by a right shift). Alternatively, you can: Each of the samples is first halved (e.g., by left shifting),

[0101] The values ​​of the two halved samples are then added. Easier operations can be performed during downsampling (eg, in step 811) since only one sample amount and a group of samples (eg, samples adjacent to each other) need to be selected. It is now therefore possible to define techniques for reducing the number of multiplications to be performed. Some of these techniques may be based, inter alia, on at least one of the following principles:

[0102] Even if the actual predicted size of the block 18 is M × N, the block is reduced (in at least one of the two dimensions) and the reduced size Q red ×P red (

number

[0103] Additionally or alternatively, instead of predicting all Q = M * N values of the block 18 to be predicted by multiplication, a reduced block (e.g., [Number] which is [Number] and / or [Number] which is) can be predicted. The remaining samples of the block 18 to be predicted are obtained, for example, by interpolation using Q red samples as support values for the remaining Q - Q red values.

[0104] According to the example shown in Fig. 7.1, a 4x4 block 18 (M=4, N=4, Q=M*N=16) is to be predicted, and the neighbourhoods 17 (vertical matrix with 4 already predicted samples) and 17c (horizontal row with 4 already predicted samples) of sample 17a have already been predicted in the previous iteration (neighborhoods 17a and 17c can be collectively denoted by 17). A priori, by using the formulas shown in Fig. 5, the prediction matrix 17M should be a QxP=16x8 matrix (because Q=M*N=4*4 and P=M+N=4+4=8) and the boundary vector 17P should have 8x1 dimensions (because P=8). However, this leads to the need to perform 8 multiplications for each of the 16 samples of the 4x4 block 18 to be predicted, and thus to the need to perform 16*8=128 multiplications in total. (Note that the average number of multiplications per sample is a good estimate of computational complexity. Traditional intra prediction requires 4 multiplications per sample, which increases the computational effort involved. Therefore, it is possible to use this as an upper bound for ALWIP, ensuring that the complexity is reasonable and does not exceed that of traditional intra prediction.)

[0105] Nevertheless, by using this technique, in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted is reduced from P redIt is understood that it is possible to reduce to . In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus two samples are selected for row 17c and two samples are selected for column 17a. Thus, the horizontal row 17c is processed as having two samples (for example, averaged samples) instead of having four original samples, and the vertical column 17a, which originally had four samples, is processed as having two samples (for example, averaged samples). After subdividing rows 17c and column 17a into groups 110 of two samples each, it can also be understood that a single sample is maintained (for example, the average of the samples in group 110 or a simple selection between the samples in group 110). Thus, a so-called reduced set 102 of sample values is obtained by a set 102 having only four samples (M red =2, N red =2, P red =M red +N red =4, P red <P).

[0106] It is understood that it is possible to perform operations (such as averaging or downsampling 100) without performing an excessive number of multiplications at the processor level: the averaging or downsampling 100 performed in step 811 can be easily obtained by simple and computationally non-power-consuming operations such as addition and shift.

[0107] At this point, it is understood that the reduced set of sample values ​​102 can be subjected to a linear or affine-linear (ALWIP) transformation 19 (e.g., using a predictor matrix such as matrix 17M of FIG. 5), in which case the ALWIP transformation 19 directly maps the four samples 102 to sample values ​​104 of block 18. In this case, no interpolation is required.

[0108] In this case, the ALWIP matrix 17M has dimensions Q × P red = 16 × 4: this follows from the fact that all Q = 16 samples of the block 18 to be predicted are obtained directly by ALWIP multiplication (no interpolation is required).

[0109] Therefore, in step 812a, the dimensions Q×P red An appropriate ALWIP matrix 17M having A is selected. The selection may be based, for example, at least in part, on signaling from the data stream 12. The selected ALWIP matrix 17M is also k where k can be understood as an index that can be signaled in the data stream 12 (in some cases, the matrix can also be represented as

number

[0110] In step 812b, the selected Q×P red ALWIP matrix 17M(A k (also denoted as P red A multiplication with the x1 boundary vector 17P is performed.

[0111] In step 812c, an offset value (e.g., b k ) can be added, for example, to all the values ​​104 obtained in the vector 18Q obtained by ALWIP. k , or in some cases

number

[0112] As can be appreciated, it is possible to obtain a suitable value in step 812 by relying on easy and computationally low power operations such as averaging (possibly adding and / or shifting and / or downsampling).

[0113] With reference to Figure 7.2, the block 18 to be predicted is here an 8x8 block (M=8, N=8) of 64 samples. Now, a priori, the prediction matrix 17M should have size QxP=64x16 (Q=M*N=8*8=64, Q=64 because M=8 and N=8 and because P=M+N=8+8=16). Thus, a priori, P=16 multiplications would be required for each of the Q=64 samples of the 8x8 block 18 to be predicted, arriving at 64*16=1024 multiplications for the entire 8x8 block 18!

[0114] However, as can be seen from FIG. 7.2, instead of using all 16 samples of the boundary, a method 820 can be provided in which only eight values ​​are used (e.g., four in the horizontal boundary row 17c and four in the vertical boundary column 17a between the original samples of the boundary). From the boundary row 17c, four samples may be used instead of eight (e.g., they may be an average of 2×2 and / or a selection of one sample out of two). Thus, the boundary vector is a P×1=16×1 vector, rather than a P×1=16×1 vector. red ×1=8×1 vectors only (P red =M red +N red =4+4). Instead of the original P=16 samples, P red It is understood that it is possible to select or average (for example 2×2) the samples of the horizontal rows 17c and the samples of the vertical columns 17a so as to have only Q=M*N=8*8=64 boundary values, forming a reduced set 102 of sample values. This reduced set 102 makes it possible to obtain a reduced version of the block 18, which has Q=M*N=8*8=64 (instead of Q=M*N=8*8=64). red=M red *N red = 4*4 = 16 samples. Size M red ×N red It is possible to apply the ALWIP matrix to predict a block of Q = 4 × 4. The reduced version of block 18 includes the samples shown in grey in scheme 106 of Figure 7.2: the samples shown in grey squares (including samples 118' and 118'') are the Q obtained in step 812 of interest. red = 16 values, which are obtained by applying the linear transformation 19 in the step 812 in question. After obtaining the values ​​of the 4x4 reduced block, it is possible to obtain the values ​​of the remaining samples (shown as white samples in the scheme 106) for example by interpolation.

[0115] As with method 810 in FIG. 7.1, method 820 may be implemented, for example, by using the remaining QQ of the M×N=8×8 block 18 to be predicted. red 8. The method may further include a step 813 of deriving predicted values ​​for the remaining QQ red = 64 - 16 = 48 samples are obtained directly by interpolation (interpolation can also use the values ​​of the boundary samples, for example) red 7.2, the value 108′ can be obtained from 118′=16 samples. As can be seen from FIG. 7.2, samples 118′ and 118″ are obtained in step 812 (as shown by the grey squares), while sample 108′ (which is intermediate between samples 118′ and 118″ and shown by the white square) is obtained by interpolation between samples 118′ and 118″ in step 813. It is understood that the interpolation can also be obtained by operations similar to those for averaging, such as shifting and adding. Thus, in FIG. 7.2, the value 108′ can generally be determined as a value intermediate between the values ​​of samples 118′ and 118″ (which can be the average).

[0116] By performing interpolation, it is also possible to reach the final version of the M×N = 8×8 block 18 based on the plurality of sample values shown in 104 in step 813. Therefore, comparing using this technology with not using it, it is as follows: When not using this technology: The block 18 to be predicted, which is a block with dimensions M = 8 and N = 8, and Q = M*N = 8*8 = 64 samples of the block 18 to be predicted, P = M + N = 8 + 8 = 16 samples of the boundary 17, P = 16 multiplications for each of the Q = 64 values to be predicted, The total number of multiplications of P*Q = 16*64 = 1028 The ratio of the number of multiplications to the number of final values obtained is P*Q / Q = 16 When using this technology: The block 18 to be predicted, having dimensions M = 8 and N = 8 The last Q = M*N = 8*8 = 64 values to be predicted,

[0117] However, P red = M red + N red 、Q red = M red * N red 、M red = 4、N red = 4, for Q red × P red The ALWIP matrix is used P red < P, the boundary P red = M red + N red = 4 + 4 = 8 samples Q of the 4×4 reduced block to be predicted red = 8 multiplications for each of the 16 values of P (formed by the gray squares in scheme 106), red For each of the 16 values of the 4×4 reduced block to be predicted P red * Q red = 8*16 = 128 multiplications in total (much less than 1024!)

[0118] The ratio of the number of multiplication times to the number of final values to be obtained is P red *Q red / Q = 128 / 64 = 2 (much less than 16 obtained without using this technology!). Therefore, the technology presented in this specification requires 8 times less power than the previous technology.

[0119] Figure 7.3 shows another example (which can be based on method 820), where the block 18 to be predicted is a rectangular 4×8 block (M = 8, N = 4) with Q = 4 * 8 = 32 samples to be predicted. The boundary 17 is formed by a horizontal row 17c of N = 8 samples and a vertical column 17a of M = 4 samples. Therefore, a priori, the boundary vector 17P has dimensions P×1 = 12×1, but the prediction ALWIP matrix should be a Q×P = 32×12 matrix, and thus Q * P = 32 * 12 = 384 multiplications are required.

[0120] However, for example, it is possible to average or downsample at least 8 samples of the horizontal row 17c to obtain a reduced horizontal row of only 4 samples (e.g., averaged samples). In some examples, the vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimensions P red = 8, and P red < P. Therefore, the boundary vector 17P has dimensions P red ×1 = 8×1. The ALWIP prediction matrix 17M is a matrix with dimensions M * N red *P red = 4 * 4 * 8 = 64. The 4×4 reduced block directly obtained in the target step 812 (formed by the gray columns of schema 107) has a size Q red = M * N red = 4 * 4 = 16 samples (instead of Q = 4 * 8 = 32 of the original 4×8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, the offset value b kAdd (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108' not obtained in step 812 is obtained in step 813 by interpolating the values 118' and 118'' (gray squares) obtained in step 812. Therefore, comparing using this technique with not using it, it is as follows: When not using this technique: The block 18 to be predicted, which is a block having dimensions M = 4 and N = 8 Q = M * N = 4 * 8 = 32 values to be predicted, P = M + N = 4 + 8 = 12 samples at the boundary, P = 12 multiplications for each of the Q = 32 values to be predicted, Total number of multiplications of P * Q = 12 * 32 = 384 The ratio of the number of multiplications to the number of final values obtained is P * Q / Q = 12 When using this technique: The block 18 to be predicted, which is a block having dimensions M = 4 and N = 8 The last Q = M * N = 4 * 8 = 32 values to be predicted,

[0121] However, M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8, for Q red ×P red = 16 × 8 An ALWIP matrix can be used P red < P, the P at the boundary red = M + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = For each of the 16 values, P red = 8 multiplications, Q red *P red=16*8=128 total multiplications (less than 384!)

[0122] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red *Q red / Q=128 / 32=4 (much less than the 12 obtained without using this technique!). Therefore, the computational effort is reduced by a factor of 3 with this technique.

[0123] Figure 7.4 shows the example of a block 18 to predict with dimensions MxN=16x16, with Q=M*N=16*16=256 values ​​to predict at the end, and P=M+N=16+16=32 boundary samples. This results in a prediction matrix with dimensions QxP=256x32, which means 256*32=8192 multiplications!

[0124] However, by applying method 820, it is possible to reduce (e.g., by averaging or downsampling) the number of boundary samples in step 811, for example, from 32 to 8, so that, for example, for each group 120 of four consecutive samples in row 17a, a single sample (e.g., selected from among the four samples, or the average of the samples) remains, and for each group of four consecutive samples in column 17c, a single sample (e.g., selected from among the four samples, or the average of the samples) remains.

[0125] Here, the ALWIP matrix 17M is Q red ×P red = 64 × 8 matrix: this means that it is P (by using 8 averaged samples or samples selected from the 32 boundaries) red This is due to the fact that we chose =8 and that the reduced block to be predicted in step 812 is an 8x8 block (in scheme 109 the grey squares are 64).

[0126] Thus, once the 64 samples of the reduced 8×8 block are obtained in step 812, the remaining QQ of the block 18 to be predicted are red It is possible to derive 256-64=192 values ​​104.

[0127] In this case, it has been chosen to use all samples in boundary columns 17a and only the substitute samples in boundary rows 17c to perform the interpolation, although other choices may be made.

[0128] In this method, the ratio between the number of multiplications and the number of values ​​finally obtained is Q red *P red / Q=8*64 / 256=2, which is much less than the 32 multiplications of each value without using this technique! So the comparison with using this technology versus not using it is as follows: Without this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 Q = M*N = 16*16 = 256 values ​​to predict, P=M+N=16*16=32 samples on the boundary, P=32 multiplications for each of the Q=256 values ​​to be predicted, P*Q=32*256=8192 total multiplications The ratio between the number of multiplications and the number of final values ​​obtained is P*Q / Q=32. When using this technology: Block 18 to be predicted, which is a block with dimensions M=16, N=16 The final Q=M*N=16*16=256 values ​​to predict,

[0129] However, M red =4, N red = 4, Q of the target to be predicted by ALWIP red = 8*8 = 64 samples, P red =M red +N red =4+4=8, Qred ×P red = 64 × 8 ALWIP matrix is used P red <P, the boundary P red = M red + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = P for each of the 64 values red = 8 multiplications, Q red * P red = 64 * 4 = 256 total multiplications (less than 8192!)

[0130] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red / Q = 8 * 64 / 256 = 2 (much less than 32 obtained without using this technology!) Therefore, the computational power required by this technology is 16 times less than that of the conventional technology! Therefore,

[0131] Reduce a plurality of adjacent samples (100, 813), compare with a plurality of adjacent samples (17), and obtain a set of reduced sample values (102) with fewer samples,

[0132] Subject the set of reduced sample values (102) to a linear or affine linear transformation (19, 17M) to obtain predicted values of predetermined samples (104, 118’, 188’’) of a predetermined block (18) (812) By doing so, it is possible to predict a predetermined block (18) of a picture using a plurality of adjacent samples (17).

[0133] In particular, it is possible to perform reduction (100, 813) by downsampling a plurality of adjacent samples and obtaining a set of reduced sample values (102) with fewer samples compared to the plurality of adjacent samples (17).

[0134] Alternatively, the reduction (100, 813) can be performed by averaging multiple adjacent samples to obtain a reduced set of sample values ​​(102) that has a smaller number of samples compared to the multiple adjacent samples (17).

[0135] Further, it is possible to derive (813) a predicted value for a further sample (108, 108') of a given block (18) by interpolation based on the predicted values ​​of the given sample (104, 118', 118'') and a number of adjacent samples (17).

[0136] The plurality of adjacent samples (17a, 17c) may extend in one dimension along both sides (e.g., toward the right and bottom in FIGS. 7.1-7.4) of the given block (18). The given samples (e.g., those obtained by ALWIP in step 812) may also be arranged in rows and columns, and along at least one of the rows and columns, the given samples may be arranged at every nth position from the given samples 112 adjacent to both sides of the given block 18 (112).

[0137] A support value (118) of one of a plurality of adjacent positions (118) aligned with each of the rows and / or columns can be determined for each of the rows and / or columns based on the plurality of adjacent samples (17). A predicted value 118 of a further sample (108, 108') of the given block (18) can also be derived by interpolation based on the predicted value of the given sample (104, 118', 118'') and the support value of the adjacent sample (118) aligned with each of the rows and / or columns.

[0138] A given sample (104) may be located at every nth position from adjacent samples (112) on either side of a given block 18 along a row, and a given sample is located at every mth position from adjacent samples (112) on either side of a given block (18) along a column, where n, m>1. In some cases, n=m (e.g., in Figures 7.2 and 7.3, samples 104, 118', 118'' obtained directly by ALWIP in 812 and shown in grey boxes are alternated along rows and columns with samples 108, 108' subsequently obtained in step 813).

[0139] It may be possible to perform determining the support values ​​along at least one of the rows (17c) and columns (17a), for example by downsampling or averaging (122) a group (120) of adjacent samples within a plurality of adjacent samples including the adjacent sample (118) for which the respective support value is determined, for each support value. Thus, in FIG. 7.4, in step 813, it is possible to obtain a value for sample 119 by using the values ​​of a given sample 118''' (previously obtained in step 812) and the adjacent samples 118 as support values.

[0140] The multiple adjacent samples may extend in one dimension along both sides of the given block 18. It may be possible to perform the reduction 811 by grouping the multiple adjacent samples 17 into one or more groups of contiguous adjacent samples 110 and performing downsampling or averaging on each of the one or more groups of adjacent samples 110 having two or more adjacent samples.

[0141] In the example, a linear or affine linear transformation is red *Q red or P red *Q weighting factor may be included, P red is the number of sample values ​​(102) in the reduced set of sample values, and Q redOr Q is the number of given samples in a given block (18). red *Q red or 1 / 4 P red *Q weighting coefficients are non-zero weighting values. P red *Q red or P red *Q weighting factors are Q or Q red For each of the given samples, a set of P red A set of weighting factors may be included that, when placed one above the other in a raster scan order between given samples of a given block (18), form an envelope that is omnidirectionally nonlinear. red *Q or P red *Q red The weighting factors may be independent of each other through a regular mapping rule. The average of the maximum value of the cross-correlation between the weighting factors of the first sequence associated with each given sample and the weighting factors of the second sequence associated with given samples other than each given sample, or the inverted version of the latter sequence, results in a higher maximum value but is lower than a predetermined threshold. The predetermined threshold may be 0.3 [or possibly 0.2 or 0.1]. P red The adjacent samples (17) may be arranged along a one-dimensional path extending along either side of a given block (18), with Q or Q red For each of the given samples, a set of P red The weighting factors are ordered to traverse a one-dimensional path in a given direction.

[0142] 6.1 Method and Apparatus Description width

number

number

number

[0143] 1. Of the boundary samples 17, samples 102 (e.g., 4 samples when W=H=4, and / or 8 samples in other cases) may be extracted by averaging or downsampling (e.g., step 811).

[0144] 2. A matrix-vector multiplication followed by the addition of an offset can be performed with the averaged samples (or samples remaining from the downsampling) as input. The result can be a reduced prediction signal on the set of subsampled samples in the original block (e.g., step 812).

[0145] 3. Prediction signals for the remaining positions may be generated from prediction signals on the subsampled set, for example by upsampling, for example by linear interpolation (eg, step 813).

[0146] Thanks to steps 1. (811) and / or 3. (813), the total number of multiplications required to compute the matrix-vector product is always

number

[0147] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b k ) may be, for example, a set (e.g., three sets) of matrices that may be stored in the storage units of the decoder and the encoder, e.g.,

number

number

[0148] In some cases, the set

number

number

number

number

number

number

number

[0149] In some cases, the set

number

number

number

number

number

number

number

number

number

number

number

[0150] Additionally or alternatively, a set

number

number

number

number

number

Number

[0151] 6.2 Averaging or downsampling of boundaries Here, features are provided regarding step 811. As described above, the boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0152] In the first step, the input boundary

Number

Number

Number

Number

Number

Number

number

number

[0153] It is possible to define

number

number

number

number

number

[0154] In all other cases (e.g., for blocks with a wither width or height different from 4), the block width W is W

number

number

number

number

[0155] In still other cases, it is possible to downsample the boundary (e.g., by selecting one particular boundary sample from a group of boundary samples) to arrive at a reduced number of samples. For example,

number

number

number

number

number

number

number

[0156] Two reduced boundaries

number

number

number

number

number

number

number

number

number

number

[0157] Therefore, in a particular state (

number

number

[0158] Other strategies may be implemented. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, it is not necessary for each of the three sets S0, S1, S2 to have 18 matrices (thus,

number

number

number

number

[0159] The mode and transposition information is not necessarily stored and / or transmitted as one combined mode index "mode" - in some examples it may be explicitly signaled as a transposition flag and a matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2).

[0160] In some cases, the combination of the transpose flag and the matrix index may be interpreted as a set index. For example, there may be one bit that acts as a transpose flag and several bits that indicate the matrix index, collectively denoted as a "set index."

[0161] 6.3 Generating reduced prediction signals by matrix-vector multiplication Characteristics are provided here with respect to step 812. The reduced input vector

number

number

number

number

number

number

number

number

number

[0162] Reduced predicted signal

number

number

[0163] Where:

number

number

number

number

number

number

number

number

[0164] Matrix A and vector

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0165] Other strategies may be implemented. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, it is not necessary for each of the three sets S0, S1, S2 to have 18 matrices (thus,

number

number

number

number

[0166] 6.4 Linear Interpolation to Generate the Final Prediction Characteristics are provided here with respect to step 812. For the interpolation of the subsampled prediction signal in large blocks, a second version of the averaged boundary may be required, namely:

number

number

number

number

number

number

number

number

[0167] Additionally or alternatively, it is possible to have a "hard downsampling", where:

number

number

[0168] Also,

number

number

number

number

[0169] The linear interpolation may be given as follows (other examples are possible):

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0170] This is an example of an interpolation that uses the scaled down border samples for the first interpolation (horizontal or vertical) and the original border samples for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation or no interpolation is needed. If both horizontal and vertical interpolation are needed, the order depends on the width and height of the block. However, different techniques may be implemented, for example the original boundary samples may be used for both the first and second interpolation, and the order may be fixed, for example first horizontal, then vertical (or in other cases first vertical, then horizontal). Thus, the interpolation order (horizontal / vertical) and the use of shrink / original boundary samples can be changed.

[0171] 6.5 Explaining the Overall ALWIP Process Example The whole process of averaging, matrix vector multiplication, and linear interpolation is illustrated for different shapes in Figure 7.1 to Figure 7.4. Note that the remaining shapes are treated like one of the illustrated cases.

[0172] 1.

number

number

number

[0173] 2.

number

number

number

[0174] 3.

number

number

number

[0175] 4.

number

number

number

number

[0176] Finally, for W×4 blocks where W>8,

number

number

[0177] 6.6 Evaluating the Number and Complexity of Required Parameters The necessary parameters for all possible proposed intra prediction modes are given in the set

number

number

number

[0178] 6.7 Proposed Signaling of Intra Prediction Modes For luma blocks, for example, 35 ALWIP modes are proposed (other numbers of modes may be used). For each coding unit (CU) in intra mode, a flag is transmitted in the bitstream indicating whether the ALWIP mode should be applied to the corresponding prediction unit (PU). The signaling of the latter index can be coordinated with the MRL in the same way as for the first CE test. If the ALWIP mode is to be applied, the index of the ALWIP mode

number

[0179] Here, the derivation of the MPM may be performed using the intra modes of the top and left PUs as follows:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0180] The embodiments described herein are not limited by the above-mentioned signaling of the proposed intra-prediction modes. According to alternative embodiments, MPM and / or mapping tables are not used for MIP (ALWIP).

[0181] 6.8 Adaptive MPM List Derivation for Conventional Luma Intra Prediction Mode and Chroma Intra Prediction Mode The proposed ALWIP mode can be harmonized with MPM-based coding of conventional intra prediction modes as follows: The luma and chroma MPM list derivation process of conventional intra prediction modes is based on a fixed table

number

number

number

number

[0182] For Luma MPM list derivation, ALWIP mode

number

number

[0183] 7. Efficient implementation The above examples are briefly summarized as they can form the basis for further extensions of the embodiments described below. To predict a given block 18 of a picture 10, the use of multiple adjacent samples 17a,c is used.

[0184] A reduction 100 by averaging of multiple adjacent samples is performed to obtain a reduced set of sample values ​​102 with a smaller number of samples compared to the multiple adjacent samples. This reduction is optional in the embodiments herein and results in a so-called sample value vector, which is mentioned below. The reduced set of sample values ​​undergoes a linear or affine linear transformation 19 to obtain a predicted value of a given sample 104 of a given block. This transformation is later shown using a matrix A and an offset vector b, which can be obtained by machine learning (ML) and whose implementation should be performed efficiently.

[0185] Through the interpolation, a predicted value of another sample 108 of a given block is derived based on the predicted values ​​of the given sample and a number of adjacent samples. Theoretically, it should be said that the result of the affine / linear transformation can be related to a non-full-pel sample position of the block 18, so that all samples of the block 18 can be obtained by the interpolation according to an alternative embodiment. Interpolation may not even be necessary at all.

[0186] The plurality of adjacent samples may extend in one dimension along both sides of the given block, the predetermined samples being arranged in rows and columns, and the predetermined samples may be arranged along at least one of the rows and columns, and the predetermined samples may be arranged at every n-th position from the samples (112) of the predetermined samples adjacent to both sides of the given block. Based on the plurality of adjacent samples, a support value for one of the plurality of adjacent positions (118) for each of the rows and / or columns may be determined, which is aligned with each of the rows and / or columns, and by interpolation, a predicted value for a further sample 108 of the given block may be derived based on the predicted value for the predetermined sample and the support value for the adjacent sample aligned with the row and / or column. The predetermined sample may be arranged at every n-th position from the samples 112 of the predetermined samples adjacent to both sides of the given block along the rows, and the predetermined sample may be arranged at every m-th position from the samples 112 of the predetermined samples adjacent to both sides of the given block along the columns, where n, m>1. n=m may be. Along at least one of the rows and columns, the determination of the support values ​​may be performed by averaging (122) for each support value a group 120 of adjacent samples within the plurality of adjacent samples including the adjacent sample 118 for which the respective support value is determined. The plurality of adjacent samples may extend in one dimension along both sides of the given block, and the reduction may be performed by grouping the plurality of adjacent samples into one or more groups 110 of contiguous adjacent samples and performing the averaging for each of the one or more groups of adjacent samples having three or more adjacent samples.

[0187] For a given block, a prediction residual can be transmitted in the data stream, from which it can be derived in the decoder and the given block can be reconstructed using the prediction residual and the predicted values ​​of the given samples, and in the encoder, the prediction residual is coded into the data stream in the encoder. The picture can be subdivided into a number of blocks of different block sizes, including the given block. Then, the linear or affine linear transformation of the block 18 is selected according to the width W and height H of the given block, so that the linear or affine linear transformation selected for the given block is selected from the first set of linear or affine linear transformations as long as the width W and height H of the given block are within a first set of width / height pairs, and the second set of linear or affine linear transformations is selected as long as the width W and height H of the given block are within a second set of width / height pairs different from the first set of width / height pairs. Similarly, it will be clear later that the affine / linear transformation is represented by other parameters, namely the weight of C, and optionally offset and scale parameters.

[0188] The decoder and the encoder may be configured to subdivide a picture into a plurality of blocks of different block sizes, including a predetermined block, and select a linear or affine linear transformation according to a width W and a height H of the predetermined block, such that the selected linear or affine linear transformation for the predetermined block is: a first set of linear or affine-linear transformations, as long as the width W and height H of a given block are within a first set of width / height pairs; a second set of linear or affine-linear transformations, so long as the width W and height H of a given block are within a second set of width / height pairs that are distinct from the first set of width / height pairs; and a third set of linear or affine-linear transformations, so long as the width W and height H of a given block are within a third set of one or more width / height pairs that are different from the first and second sets of width / height pairs.

[0189] The third set of one or more width / height pairs includes simply one width / height pair W', H', where each linear or affine linear transformation in the first set of linear or affine linear transformations is for transforming the N' sample values ​​to a W'*H' predicted value for a W'×H' array of sample locations. Each of the first and second sets of width / height pairs is W p H p First width / height vs. W not equal p , H p And, H q =W p and W q =H p The second width / height pair W q , H q and

[0190] Each of the first and second sets of width / height pairs is a third width / height pair W p , H p and W p H p is equal to H p >H q It is. For a given block, a set index may be transmitted in the data stream that indicates which linear or affine-linear transform of a given set of linear or affine-linear transforms should be selected for the block 18.

[0191] The multiple adjacent samples may extend in one dimension along both sides of the predetermined block, and the downscaling may be performed by, for a first subset of the multiple adjacent samples adjacent to a first side of the predetermined block, grouping the first subset into a first group 110 of one or more consecutive adjacent samples, and for a second subset of the multiple adjacent samples adjacent to a second side of the predetermined block, grouping the second subset into a second group 110 of one or more consecutive adjacent samples, and performing averaging for each of the first and second groups of one or more adjacent samples having three or more adjacent samples to obtain a first sample value from the first group and a second sample value of the second group. Then, the linear or affine linear transformation can be selected according to a set index from a predetermined set of linear or affine linear transformations, so that two different states of the set index result in the selection of one of the linear or affine linear transformations from the predetermined set of linear or affine linear transformations; in the case of a set index assuming a first of two different states in the form of a first vector to result in an output vector of predicted values, the reduced set of sample values ​​can be subjected to a predetermined linear or affine linear transformation; the predicted values ​​of the output vector can be distributed to predetermined samples of a predetermined block along a first scanning order; in the case of a set index assuming a second of two different states in the form of a second vector, the first and second vectors differ such that a component in which one of the first sample values ​​of the first vector is input is input to one of the second samples of the second vector and a component in which one of the second sample values ​​of the first vector is input to one of the first samples of the second vector to generate an output vector of predicted values, and the predicted values ​​of the output vector can be distributed along a second scanning order to predetermined samples of a predetermined block that is transposed with respect to the first scanning order.

[0192] Each linear or affine linear transform in the first set of linear or affine linear transforms may be for transforming the N1 sample values ​​into w1*h1 predicted values ​​for a w1×h1 array of sample locations, and each linear or affine linear transform in the first set of linear or affine linear transforms may be for transforming the N2 sample values ​​into w2*h2 predicted values ​​for a w2×h2 array of sample locations, where for a first predetermined one of the first width / height pairs, w1 may exceed the width of the first predetermined width / height pair or h1 may exceed the height of the first predetermined width / height pair, and for a second predetermined one of the first width / height pairs, w1 may not exceed the width of the second predetermined width / height pair and h1 may not exceed the height of the second predetermined width / height pair. Then, obtaining (100) a plurality of adjacent samples to obtain a reduced set of sample values ​​(102) by averaging may be performed such that the reduced set of sample values ​​102 has N1 sample values ​​when the given block is of a first predetermined width / height pair and when the given block is of a second predetermined width / height pair, and subjecting the reduced set of sample values ​​to a selected linear or affine-linear transformation may be performed by using only a first sub-portion of the selected linear or affine-linear transformation associated with a sub-sampling of the w1×h1 array of sample locations along the width dimension if w1 exceeds the width of one width / height pair, or along the height dimension if h1 exceeds the height of one width / height pair, when the given block is of the first predetermined width / height pair, or performing the selected linear or affine-linear transformation completely when the given block is of the second predetermined width / height pair.

[0193] Each linear or affine linear transformation in the first set of linear or affine linear transformations can be for transforming N1 sample values ​​for a w1×h1 array at sample locations where w1=h1 to w1*h1 predicted values, and each linear or affine linear transformation in the second set of linear or affine linear transformations is for transforming N2 sample values ​​for a w2×h2 array at sample locations where w2=h2 to w2*h2 predicted values.

[0194] All the above-mentioned embodiments are merely exemplary in that they may form the basis of the embodiments described herein below. That is, the above concepts and details are useful for understanding the following embodiments and serve as a reservoir for possible extensions and corrections of the embodiments described later in this specification. In particular, many of the details mentioned above are optional, such as the averaging of adjacent samples, the fact that adjacent samples are used as reference samples, etc.

[0195] More generally, the embodiments described herein assume that a prediction signal on a rectangular block is generated from already reconstructed samples, such as an intra prediction signal on a rectangular block is generated from already reconstructed samples of the left and top neighbors of the block. The generation of the prediction signal is based on the following steps:

[0196] 1. However, without excluding the possibility of transferring the description to a reference sample located elsewhere, from a reference sample called the boundary sample, samples can be extracted by averaging. Here, averaging is performed on both the left and top boundary samples of the block, or only on one of the boundary samples on both sides. If no averaging is performed on one side, the sample on that side remains unchanged.

[0197] 2. A matrix-vector multiplication is performed, optionally followed by the addition of an offset, where the input vector of the matrix-vector multiplication is either the concatenation of the left of the block's averaged boundary sample with the original boundary sample above the block, if averaging is applied to the left side only, or the concatenation of the left of the block's original boundary sample with the averaged boundary sample above the block, if averaging is applied to the top side only, or the concatenation of the left of the block's averaged boundary sample with the averaged boundary sample above the block, if averaging is applied to both sides of the block. Again, alternatives exist, such as no averaging at all.

[0198] 3. The result of the matrix-vector multiplication and optional offset addition can optionally be a downscaled prediction signal on the set of subsampled samples in the original block. The prediction signals at the remaining positions can be generated from the prediction signals on the subsampled set by linear interpolation.

[0199] The matrix-vector product calculation in step 2 should preferably be performed in integer arithmetic. Thus,

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0200] FIG. 8 shows an embodiment of an apparatus 1000 for predicting a given block 18 of a picture 10 using multiple reference samples 17. The multiple reference samples 17 may depend on a prediction mode used by the apparatus 1000 to predict the given block 18. If the prediction mode is, for example, intra-prediction, a reference sample 171 adjacent to the given block may be used. In other words, the multiple reference samples 17 are arranged in the picture 10, for example, along the outer edge of the given block 18. For example, if the prediction mode is inter-prediction, a reference sample 172 of another picture 10' may be used.

[0201] The device 1000 is configured to form 100 a sample value vector 400 from a plurality of reference samples 17. The sample value vector can be obtained by different techniques. The sample value vector can, for example, include all reference samples 17. Optionally, the reference samples can be weighted. According to another example, the sample value vector 400 can be formed as described with respect to any of the figures 7.1 to 7.4 for the sample value vector 102. In other words, the sample value vector 400 can be formed by averaging or downsampling. Thus, for example, a group of reference samples can be averaged to obtain a sample value vector 400 with a reduced set of values. In other words, the apparatus is configured to form 100 a sample value vector 102 from a plurality of reference samples 17, for example, by taking, for each component of the sample value vector 400, one reference sample of the plurality of reference samples 17 as the respective component of the sample value vector, and / or by averaging two or more components of the sample value vector 400, i.e., by averaging two or more reference samples of the plurality of reference samples 17 to obtain each component of the sample value vector 400.

[0202] The device 1000 is configured to derive 401 from the sample value vector 400 a further vector 402 to which the sample value vector 400 is mapped by a predefined reversible linear transformation 403. The further vector 402 for example contains only integer and / or fixed point values. The reversible linear transformation 403 is for example chosen such that the prediction of the samples of a given block 18 is performed by integer or fixed point arithmetic.

[0203] Furthermore, the device 1000 is configured to calculate a matrix-vector product 404 between the further vector 402 and a predefined prediction matrix 405 to obtain a prediction vector 406, and to predict samples of the predefined block 18 based on the prediction vector 406. Based on the advantageous further vector 402, the predefined prediction matrix can be quantized to enable integer and / or fixed-point operations with only negligible impact of quantization errors on the predicted samples of the predefined block 18.

[0204] According to an embodiment, the apparatus 1000 is configured to calculate the matrix vector product 404 using fixed point arithmetic. Alternatively, integer arithmetic can be used. According to an embodiment, the apparatus 1000 is configured to calculate the matrix vector product 404 without floating point operations.

[0205] According to an embodiment, the apparatus 1000 is configured to store a fixed-point representation of the predefined prediction matrix 405. Additionally or alternatively, an integer representation of the predefined prediction matrix 405 can be stored. According to an embodiment, the device 1000 is configured, in predicting samples of the given block 18 based on the prediction vector 406, to use interpolation to calculate at least one sample position of the given block 18 based on the prediction vector 406, each component of which is associated with a corresponding position in the given block 18. The interpolation can be performed as described with respect to any of the embodiments shown in Figures 7.1 to 7.4.

[0206] Figure 9 illustrates the inventive concept described herein. Samples of a given block can be predicted based on a first matrix-vector product between a matrix A 1100 derived by some machine learning based training algorithm and a sample value vector 400. Optionally, an offset b 1110 can be added. To achieve an integer or fixed point approximation of this first matrix-vector product, the sample value vector can undergo a reversible linear transformation 403 to determine a further vector 402. A second matrix-vector product between a further matrix B 1200 and the further vector 402 can be equal to the result of the first matrix-vector product.

[0207] Due to the characteristics of the further vector 402, the second matrix-vector product can be integer-approximated by a matrix-vector product 404 between a predefined prediction matrix C 405, the further vector 402 and a further offset 408. The further vector 402 and the further offset 408 can consist of integer or fixed-point values. All components of the further offset are, for example, the same. The predefined prediction matrix 405 can be a quantized matrix or a matrix to be quantized. The result of the matrix-vector product 404 between the predefined prediction matrix 405 and the further vector 402 can be understood as a prediction vector 406.

[0208] Further details regarding this integer approximation are provided below. Possible Solutions for Embodiment I: Subtracting and Adding Average Values Usable expressions for the above scenarios

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0209] The specified value 1400 is not necessarily the average value.

number

number

number

number

number

number

number

number

number

number

[0210] Alternatively, the predetermined value 1400 is a default value or a value that is signaled in the data stream in which the picture is encoded. The predetermined value 1400 is, for example, 2 bitdepth-1 In this case, the further vector 402 is equal to y0=2 for i>0. bitdepth-1 and y i =x i It can be defined by -x0.

[0211] Alternatively, the predetermined component 1500 is a constant obtained by subtracting the predetermined value 1400. The constant may be, for example, 2 bitdepth-1 According to an embodiment, a predetermined component of the further vector y 402 is

number

number

[0212] According to an embodiment, the device 1000 is configured to include a number of reversible linear transforms 403, each of which is associated to one component of the further vector 402. Furthermore, the device is configured to, for example, select a given component 1500 of the components of the sample value vector 400 and use a reversible linear transform 403 of the plurality of reversible linear transforms associated with the given component 1500 as the given reversible linear transform. This is due, for example, to different positions of the i0th row, i.e. the row of the reversible linear transform 403 corresponding to the given component, depending on the position of the given component in the further vector 402. For example, if the first component of the further vector 402, i.e. y1, is the given component, then the i0th row, i.e. the o The row of replaces the first row of the reversible linear transform.

[0213] As shown in Fig. 10b, the column 412 of the predetermined prediction matrix 405, i.e. the matrix element 414 of the predetermined prediction matrix C 405 in the i0th column, which corresponds to the predetermined element 1500 of the further vector 402, is, for example, all zero. In this case, the device is configured to calculate the matrix-vector product 404 by performing a multiplication by calculating a matrix-vector product 407 between the reduced prediction matrix C' 405 resulting from the predetermined prediction matrix C 405 by leaving the column 412 and the yet further vector 410 resulting from the further vector 402 by leaving the predetermined element 1500, as shown in Fig. 10c. Thus, the prediction vector 406 can be calculated with fewer multiplications.

[0214] As shown in Fig. 9, Fig. 10b and Fig. 10c, the device 1000 may be configured to, when predicting samples of a given block based on the prediction vector 406, calculate for each component of the prediction vector 406 the sum of the respective component and a, i.e. a predefined value 1400. This addition may be represented by the sum of the prediction vector 406 and the vector 409, as shown in Fig. 9 and Fig. 10c, where all components of the vector 409 are equal to the predefined value 1400. Alternatively, the addition may be represented by the sum of the prediction vector 406 and a matrix-vector product 1310 between the integer matrix M 1300 and the further vector 402, as shown in Fig. 10b, where the matrix components of the integer matrix 1300 are 1 in the column of the integer matrix 1300 corresponding to the predefined component 1500 of the further vector 402, i.e. the i0th column, and all other components are, for example, 0.

[0215] The result of the sum of the given prediction matrix 405 and the integer matrix 1300 is, for example, equal to or close to the further matrix 1200 shown in FIG. In other words, the matrix obtained by summing each matrix element of the predetermined prediction matrix C 405 in the i0th column 412 of the predetermined prediction matrix 405 corresponding to the predetermined element 1500 of the further vector 402 with 1 (i.e., matrix B) times the reversible linear transform 403, i.e., the further matrix B 1200, corresponds to a quantized version of the machine learning prediction matrix A 1100, as shown in, for example, Fig. 9, Fig. 10a and Fig. 10b. Summing each matrix element of the predetermined prediction matrix C 405 in the i0th column 412 with 1 can correspond to the sum of the predetermined prediction matrix 405 and the integer matrix 1300, as shown in Fig. 10b. As shown in Fig. 9, the machine learning prediction matrix A 1100 can be equal to the result of the reversible linear transform 403 times the further matrix 1200. This means that

number

[0216] Matrix multiplication using integer arithmetic only For a low complexity implementation (in terms of the complexity of adding and multiplying scalar values, as well as the storage required for the entries of the part-taking matrices), it is desirable to perform the matrix multiplication 404 using integer arithmetic only.

number

number

number

number

number

[0217] Matrix vector product 404 with a matrix of size m×n, i.e., a given prediction matrix 405, can be performed as shown in this pseudocode, where <<,>> are arithmetic binary left and right shift operations, and +, - and * operate on integer values ​​only.

[0218] (1) final_offset=1<<(right_shift_result-1); For i in 0...m-1 { Accumulator = 0 For j in 0...n-1 { Accumulator:=Accumulator+y[j]*C[i,j] } z[i] = (accumulator + final_offset) >> right_shift_result; }

[0219] Here, array C, i.e., the predefined prediction matrix 405, stores fixed point numbers, for example as integers. The final addition of final_offset and right shift operation by right_shift_result reduces precision by rounding to obtain the required fixed point format at the output. To allow for an increase in the range of real values ​​representable by the integers in C, two additional matrices are added, as shown in the embodiment of Figs. 11 and 12.

number

number

number

number

number

number

number

number

number

[0220] In other words, the apparatus 1000 may estimate a prediction parameter, e.g., an integer value

number

number

number

[0221] According to an embodiment, the prediction parameters include weights, each of which is associated with a corresponding matrix element of the prediction matrix. In other words, a given prediction matrix is, for example, replaced or represented by the prediction parameters. The weights are, for example, integer and / or fixed point values.

[0222] According to an embodiment, the prediction parameters are scaled by one or more scaling factors, e.g., values ​​scale i,jeach of which may be a weight, e.g., an integer value, associated with one or more corresponding matrix elements of the predetermined prediction matrix 405.

number

number

number

number

[0223] For example, in one preferred embodiment,

number

number

number

number

[0224] According to an embodiment,

number

number

number

number

[0225] Offset Expression

number

number

[0226] A wide range of embodiments resulting from the solution The above solution implies the following embodiments: 1. A prediction method as described in Section I, wherein in step 2 of Section I, the following is performed for integer approximations of the matrix-vector products involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0227] 2. A prediction method as described in Section I, wherein in step 2 of Section I, the following is performed for integer approximations of the matrix-vector products involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0228] 3. A method of prediction as described in Section I, comprising the steps of:

number

number

number

[0229] 4. A prediction method as described in section I, wherein step 2 uses one of K matrices so that multiple prediction modes can be calculated, each with k=0...K-1, a different matrix.

number

number

number

number

[0230] That is, according to an embodiment of the present application, the encoder and decoder function as follows to predict a given block 18 of a picture 10, with reference to Fig. 9: For the prediction, a number of reference samples are used. As outlined above, the embodiment of the present application is not limited to intra-coding, and thus the reference samples are not limited to adjacent samples, i.e. samples of adjacent blocks 18 of the picture 10. In particular, the reference samples are not limited to those located along the outer edge of the block 18, such as samples abutting the outer edge of the block. However, this situation is certainly an embodiment of the present application.

[0231] To make the prediction, a sample value vector 400 is formed from reference samples, such as reference samples 17a and 17c. Possible formations have been described above. The formation may include averaging, thereby reducing the number of samples 102 or the number of components of the vector 400 compared to the reference samples 17 that contribute to the formation. The formation may also depend in some way on the dimensions or size of the block 18, such as the width and height of the block 18, as described above.

[0232] It is this vector 400 that should undergo an affine or linear transformation to obtain the prediction of the block 18. Different nomenclatures have been used above. The objective is to perform the prediction by applying the vector 400 to the matrix A by a matrix-vector product, within the scope of performing an addition, using the latest one, with an offset vector b. The offset vector b is arbitrary. The affine or linear transformation determined by A or A and b can be determined by the encoder and decoder, or more precisely, for the prediction based on the size and dimensions of the block 18, as already mentioned above.

[0233] However, in order to achieve the computational efficiency improvements outlined above or to make the prediction more effective in terms of implementation, the affine or linear transformation is quantized and the encoder and decoder, or its predictor, use C and T applied in the manner described above to represent and perform the linear or affine transformation, with C and T applied in the manner described above, representing a quantized version of the affine transformation. In particular, the predictor in the encoder and decoder, instead of directly applying the vector 400 to the matrix A, applies the vector 402 obtained from the sample value vector 400 by subjecting it to a mapping via a predetermined invertible linear transformation T. The transformation T used here is the same as long as the vector 400 has the same size, i.e. it does not depend on the dimensions of the block, i.e. width and height, or at least it is the same for different affine / linear transformations. Above, the vector 402 is denoted by y. The exact matrix for performing the affine / linear transformation determined by machine learning was B. However, instead of performing B exactly, the prediction in the encoder and decoder is performed by an approximation or quantized version of it. In particular, the representation is done by appropriately representing C in the manner outlined above, and C+M representing a quantized version of B.

[0234] Therefore, the prediction in the encoder and decoder is further accomplished by calculating in the manner described above a matrix-vector product 404 between the vector 402 and a predefined prediction matrix C, appropriately represented and stored in the encoder and decoder. A vector 406 resulting from this matrix-vector product is then used to predict the samples 104 of the block 18. As described above, for the prediction, each component of the vector 406 may undergo an addition with a parameter a, as shown at 408, to compensate for the corresponding definition of C. An optional addition of the vector 406 with an offset vector b may also be included in the derivation of the prediction of the block 18 based on the vector 406. As described above, each component of the vector 406, and therefore each component of the sum of the vector 406, the vector of all a, as shown at 408, and the optional vector b, may directly correspond to a sample 104 of the block 18 and thus indicate a predicted value of the sample. It is also possible that only a subset of the samples 104 of the block is predicted in this way, and the remaining samples of the block 18, such as 108, are derived by interpolation.

[0235] As mentioned above, there are different embodiments for setting a. For example, it may be the arithmetic mean of the components of the vector 400. In that case, please refer to FIG. 10. The reversible linear transform T can be as shown in FIG. 10. i0 is a given component of the sample value vector and the vector 402, respectively, which is replaced by a. However, as also shown above, there are other possibilities. However, as far as the representation of C is concerned, it has also been shown above that it can be embodied differently. For example, the matrix-vector product 404 can, in its actual calculation, be an actual calculation of a smaller matrix-vector product having a lower dimension. In particular, as mentioned above, by the definition of C, the entire i0th column 412 of it is zero, so that the actual calculation of the product 404 is a calculation of the component i0.

number

[0236] The weights of C or C', i.e. the elements of this matrix, can be represented and stored in a fixed-point representation. However, these weights 414 can also be stored in a manner associated with different scales and / or offsets, as described above. The scale and offset can be defined for the entire matrix C, i.e. equal for all weights 414 of matrix C or matrix C', or defined to be constant or equal for all weights 414 of the same row or all weights 414 of the same column of matrices C and C', respectively. In this regard, FIG. 11 shows that the calculation of the matrix-vector product, i.e. the result of the product, can in fact be performed slightly differently, i.e. for example by shifting the multiplication with the scale towards the vector 402 or 404, thereby reducing the number of multiplications that must be performed further. FIG. 12 shows the case of using one scale and one offset for all weights 414 of C or C', as was done in calculation (2) above.

[0237] According to an embodiment, the device described herein for predicting a given block of a picture may be configured to use a matrix-based intra-sample prediction including the following features: The apparatus is configured to form a sample value vector pTemp[x] 400 from a number of reference samples 17. Assuming pTemp[x] is 2*boundarySize, pTemp[x] can be populated, for example, by direct copying or by subsampling or pooling, with the neighboring sample located at the beginning of the given block, redT[x], with x=0..boundarySize-1, followed by the neighboring sample located to the left of the given block, redL[x], with x=0..boundarySize-1 (for example, when transpose=0), or vice versa in case of a transposition operation (for example, when transpose=1).

[0238] Input values ​​p[x] for x=0..inSize-1 are derived, i.e. the apparatus is configured to derive from the sample value vector pTemp[x] a further vector p[x] to which the sample value vector pTemp[x] is mapped by a predetermined invertible linear transformation, or more specifically a predetermined invertible affine linear transformation, as follows: If -mipSizeId is equal to 2, the following applies: p[x]=pTemp[x+1]-pTemp[0] - Otherwise (mipSizeId is less than 2), the following applies: p[0]=(1<<(BitDepth-1))-pTemp[0] p[x]=pTemp[x]-pTemp[0] for x=1..inSize-1

[0239] Here, the variable mipSizeId indicates the size of the given block, i.e., according to the present embodiment, the reversible transformation by which the further vector is derived from the sample value vector depends on the size of the given block. The dependency can be given according to the following:

[0240] [Table 3]

[0241] predSize indicates the number of prediction samples in a given block, 2*boundarySize indicates the size of the sample value vector and is related to the size of the further vector according to inSize, i.e. inSize=(2*boundarySize)-(mipSizeId==2)?1:0. 1:0. More precisely, inSize indicates the number of components of the further vector that are actually involved in the calculation. inSize is the same size as the size of the sample value vector for small block sizes and one component smaller for large block sizes. In the former case, one component, i.e. the component corresponding to a given component of the further vector, as in the matrix-vector product calculated later, may be ignored, and the contribution of the corresponding vector component will be zero in any case and therefore does not actually need to be calculated. The dependency on the block size can be left in the case of an alternative embodiment in which only one of the two alternatives is necessarily used, i.e. regardless of the block size (the option corresponding to mipSizeId is less than 2 or the option corresponding to mipSizeId is equal to 2).

[0242] In other words, a given reversible linear transformation is defined such that, for example, a given component of the further vector p is a and all other components correspond to the components of the sample value vector minus a, for example a=pTemp[0]. In the case of the first option corresponding to a mipSizeId equal to 2, this is easily seen and only separately formed components of the further vector are further taken into account, i.e. in the case of the first option, the further vector is actually {p[0...inSize];pTemp[0]}, where pTemp[0] is a, and the actually calculated part of the matrix-vector multiplication to result in the matrix-vector product, i.e. the result of the multiplication, is limited only to the inSize components of the further vector and the corresponding columns of the matrix, since the matrix has 0 columns that do not require calculation. Otherwise, corresponding to a mipSizeId less than 2, a=pTemp[0] is selected for all components of the further vector except p[0], i.e., each of the other components p[x] (for x=1..inSize-1) of the further vector p except for a given component p[0] is equal to the corresponding component of the sample value vector pTemp[x] minus a, whereas p[0] is selected to be a constant minus a. The matrix-vector product is then calculated. The constant is the average of the representable values, i.e. 2 x-1 (i.e., 1<<(BitDepth-1)), and x indicates the bit depth of the computation representation used. Note that if p[0] is instead selected to be pTemp[0], the calculated product will deviate from the product calculated using p[0] as above (p[0]=(1<<(BitDepth-1))-pTemp[0]) by a constant vector that can be taken into account when predicting the block interior based on the product, i.e., the prediction vector. Thus, the value a is a predetermined value, for example, pTemp[0]. The predetermined value pTemp[0] is in this case, for example, the component of the sample value vector pTemp that corresponds to the predetermined component p[0]. It can be the sample closest to the top left corner of the predetermined block, adjacent above the predetermined block or to the left of the predetermined block.

[0243] In case of an intra-sample prediction process according to predModeIntra, e.g. specifying an intra prediction mode, the apparatus is configured for example to apply the following steps, e.g. performing at least the first step: 1. A matrix-based intra prediction sample predMip[x][y], where x=0..predSize-1, y=0..predSize-1, is derived as follows: - The variable modeId is set equal to predModeIntra. The weight matrix mWeight[x][y], for x=0..inSize-1, y=0..predSize*predSize-1, is derived by invoking the MIP weight matrix derivation process with mipSizeId and modeId as input. The matrix-based intra prediction samples predMip[x][y], where x=0..predSize-1, y=0..predSize-1, are derived as follows: oW=32-32*(

number

number

[0244] In other words, the apparatus is now configured to calculate a matrix-vector product between a further vector p[i], or {p[i]; pTemp[0]} and a given prediction matrix mWeight if mipSizeId is equal to 2, or a prediction matrix mWeight with additional zero weight lines corresponding to omitted components of p if mipSizeId is smaller than 2, to obtain prediction vectors already assigned to an array of block positions {x,y} distributed inside a given block to result in an array predMip[x][y]. The prediction vector corresponds to the concatenation of the rows of predMip[x][y] or the columns of predMip[x][y], respectively.

[0245] According to an embodiment, or according to a different interpretation, the component (((

number

[0246] The apparatus optionally performs the following steps in predicting samples of a given block based on a prediction vector:

number

[0247] 2. A matrix-based intra prediction sample predMip[x][y], with x=0..predSize-1, y=0..predSize-1, is clipped, for example, as follows: predMip[x][y]=Clip1(predMip[x][y])

[0248] 3. If transpose is equal to true, then a predSize×predSize array predMip[x][y], with x=0..predSize-1 and y=0..predSize-1, is transposed, for example, as follows: predTemp[y][x]=predMip[x][y] predMip=predTemp

[0249] 4. The predicted samples predSamples[x][y], where x=0..nTbW-1 and y=0..nTbH-1, are derived, for example, as follows: - If nTbW, which specifies the transform block width, is greater than predSize or nTbH, which specifies the transform block height, is greater than predSize, the MIP prediction upsampling process is called with inputs: input block size predSize, matrix-based intra prediction samples predMip[x][y] having x=0..predSize-1, y=0..predSize-1, transform block width nTbW, transform block height nTbH, upper reference samples refT[x] having x=0..nTbW-1, and left reference samples refL[y] having y=0..nTbH-1, and the output is the prediction sample array predSamples. - Otherwise, predSamples[x][y], for x=0..nTbW-1, y=0..nTbH-1, is set equal to predMip[x][y]. In other words, the apparatus is configured to predict the samples predSamples of a given block based on the prediction vector predMip.

[0250] FIG. 13 shows a method 2000 for predicting a given block of a picture using a plurality of reference samples, comprising forming a sample value vector from the plurality of reference samples 2100, deriving from the sample value vector a further vector to which the sample value vector is mapped by a predetermined reversible linear transformation 2200, calculating a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector 2300, and predicting samples of the given block based on the prediction vector 2400.

[0251] References [1] P. Helle et al., “Non-linear weighted intra prediction,” JVET-L0199, Macau, China, October 2018. [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, "JVET common test conditions and software reference configurations for SDR video", JVET-K1010, Ljubljana, Slovenia, July 2018.

[0252] Further embodiments and examples In general, the examples can be implemented as a computer program product comprising program instructions that function to perform one of the methods when the computer program product is executed on a computer. The program instructions may be stored on, for example, a machine-readable medium. Other examples comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0253] An example of a method is therefore a computer program having program instructions for performing one of the methods described herein, when the computer program runs on a computer.

[0254] A further example of a method is therefore a data carrier medium (or digital storage medium, or computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier medium, digital storage medium or recording medium is tangible and / or non-transitory rather than an intangible, transitory signal. A further example of a method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can be transferred via a data communication connection, for example the Internet.

[0255] A further example comprises a processing means for performing one of the methods described herein, for example a computer, or a programmable logic device. A further example comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0256] Further examples include an apparatus or system for transferring (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may comprise, for example, a file server for transferring the computer program to the receiver.

[0257] In some examples, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some examples, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods may be performed by any suitable hardware apparatus.

[0258] The above examples are merely illustrative of the principles described above. It is understood that modifications and variations of the arrangements and details described herein will be apparent. It is therefore the intention to be limited by the scope of the appended claims, and not by the specific details presented by way of description and illustration of the examples herein. Equal or equivalent elements or elements having equal or equivalent functions are indicated in the following description by equal or equivalent reference signs even if they occur in different figures.

Claims

1. An apparatus (1000) for predicting a given block (18) of a picture (10) using a plurality of reference samples (17a, c), comprising: forming (100) a sample value vector (102, 400), pTemp[x], from the plurality of reference samples (17a, c); Obtain a variable mipSizeId indicating the size of the predetermined block; Let inSize be the number of components of a further vector, p[x], involved in the derivation of said further vector, p[x], for x=1, 2, ..., inSize, if said variable mipSizeId is equal to 2, p[x]=pTemp[x+1]−pTemp[0] If the variable mipSizeId is less than 2, then: p[x]=pTemp[x]-pTemp[0] deriving a further vector (402), p[x], to which the sample value vector (102, 400), pTemp[x], is mapped by a predetermined reversible linear transformation (403) by [0010] calculating a matrix-vector product (404) between said further vector (402), p[x], and a predefined prediction matrix (405), mWeight[x][y], to obtain a prediction array (406), predMip[x][y], according to An apparatus (1000) configured to predict samples of the given block (18) based on the prediction array (406), predMip[x][y].

2. The predetermined reversible linear transformation (403) is The predetermined component (1500) of the further vector (402) is a, each of the other components of the further vector (402), except for the predetermined component (1500), is equal to the corresponding component of the sample value vector (102,400) minus a; The apparatus (1000) of claim 1, wherein a is defined as a predetermined value (1400).

3. The predetermined value (1400) is an average, such as an arithmetic or weighted average, of the components of said sample value vector (102, 400); Default value, a value signalled in a data stream in which said picture (10) is encoded, and 3. The apparatus (1000) of claim 2, wherein the predetermined component (1500) is one of the components of the sample value vector (102, 400).

4. The predetermined reversible linear transformation (403) is The predetermined component (1500) of the further vector (402) is a, each of the other components of the further vector (402), except for the predetermined component (1500), is equal to the corresponding component of the sample value vector (102,400) minus a; 2. The apparatus (1000) of claim 1, wherein a is defined as the arithmetic mean of the components of the sample value vector (102, 400).

5. The predetermined reversible linear transformation (403) is a predetermined component (1500) of said further vector (402) is a or a constant minus a; each of the other components of the further vector (402), except for the predetermined component (1500), is equal to the corresponding component of the sample value vector (102, 400) minus a; a is the component of the sampled vector (102, 400) that corresponds to the given component (1500), The device (1000), a plurality of reversible linear transformations, each associated with one component of said further vector (402); Selecting the predetermined component (1500) from the components of the sample value vector (102, 400); The apparatus (1000) of claim 1 , configured to use the reversible linear transform of the plurality of reversible linear transforms associated with the given component (1500) as the given reversible linear transform (403).

6. all matrix elements of the predetermined prediction matrix (405) in a column of the predetermined prediction matrix (405) corresponding to the predetermined element (1500) of the further vector (402) are zero, and the device (1000) 3. The apparatus (1000) of claim 2, configured to calculate the matrix-vector product (404) by performing a multiplication by calculating a matrix-vector product (407) between a reduced prediction matrix (405) resulting from the predetermined prediction matrix (405) by excluding the column and a yet further vector (410) resulting from the further vector (402) by excluding the predetermined component (1500).

7. In predicting the samples of the given block (18) based on the prediction array (406), The apparatus (1000) of claim 2 configured to calculate, for each element of the prediction array (406), the sum of the respective element and a.

8. 3. The apparatus (1000) of claim 2, wherein a matrix obtained by summing each matrix element of the predetermined prediction matrix (405) in a column of the predetermined prediction matrix (405) corresponding to the predetermined element (1500) of the further vector (402) with 1 times the predetermined reversible linear transform (403) corresponds to a quantized version of the machine learning prediction matrix (1100).

9. forming (100) said sample value vector (102, 400) from said plurality of reference samples (17a, c) for each component of said sample value vector (102, 400); taking one reference sample of said plurality of reference samples (17a, c) as said respective component of said sample value vector (102, 400); and / or averaging two or more components of the sampled vector (102, 400) to obtain the respective components of the sampled vector (102, 400); The apparatus (1000) of claim 1, configured to:

10. The apparatus (1000) of claim 1, wherein the plurality of reference samples (17a,c) are arranged in the picture (10) along an outer edge of the given block (18).

11. The apparatus (1000) of claim 1 , configured to compute the matrix-vector product (404) using fixed-point arithmetic.

12. The apparatus (1000) of claim 1 , configured to compute the matrix-vector product (404) without using floating-point arithmetic.

13. The apparatus (1000) of claim 1 configured to store a fixed-point representation of the predetermined prediction matrix (405).

14. 2. The apparatus of claim 1, configured to represent the predetermined prediction matrix using prediction parameters and to calculate the matrix-vector product by performing multiplications and additions on the components of the further vector and the prediction parameters and intermediate results resulting therefrom, wherein absolute values ​​of the prediction parameters can be represented by an n-bit fixed-point representation, n being 14 or less, alternatively 10 or less, alternatively 8 or less.

15. The prediction parameters are: The apparatus (1000) of claim 14, each including a weight associated with a corresponding matrix element of the predetermined prediction matrix (405).

16. The prediction parameters are: one or more scaling factors associated with one or more corresponding matrix elements of the predetermined prediction matrix (405), each of which is used to scale the weights associated with the one or more corresponding matrix elements of the predetermined prediction matrix (405); and / or 16. The apparatus (1000) of claim 15, further comprising one or more offsets associated with one or more corresponding matrix elements of the predetermined prediction matrix (405), each offsetting the weights associated with the one or more corresponding matrix elements of the predetermined prediction matrix (405).

17. In predicting the samples of the given block (18) based on the prediction array (406), 2. The apparatus (1000) of claim 1, configured to use interpolation to calculate at least one sample position of the given block (18) based on the prediction array (406), each component being associated with a corresponding position within the given block (18).

18. 1. An apparatus for encoding a picture, comprising: A device for predicting a given block (18) of the picture using a plurality of reference samples (17a, c) according to any one of claims 1 to 17 to obtain a prediction signal, an entropy encoder configured to encode a prediction residual of the given block to correct the prediction signal.

19. 1. An apparatus for decoding a picture, comprising: A device for predicting a given block (18) of the picture using a plurality of reference samples (17a, c) according to any one of claims 1 to 17 to obtain a prediction signal, an entropy decoder configured to decode a prediction residual for the given block; a prediction corrector for correcting the prediction signal using the prediction residual.

20. A method (2000) for predicting a given block (18) of a picture using a plurality of reference samples (17a, c), comprising: forming a sample value vector (102,400), pTemp[x], from the plurality of reference samples (2100,100); Obtaining a variable mipSizeId indicating a size of the predetermined block; Let inSize be the number of components of a further vector, p[x], involved in the derivation of said further vector, p[x], for x=1, 2, ..., inSize, if said variable mipSizeId is equal to 2, p[x]=pTemp[x+1]−pTemp[0] If the variable mipSizeId is less than 2, then: p[x]=pTemp[x]-pTemp[0] deriving (2200) a further vector (402), p[x], to which said sample value vector, pTemp[x], is mapped by a predetermined reversible linear transformation (403) by [0025] calculating (2300) a matrix-vector product (404) between said further vector (402), p[x], and a predefined prediction matrix (405), mWeight[x][y], to obtain a prediction array (406), predMip[x][y], according to predicting (2400) samples of the given block based on the prediction array (406), predMip[x][y].

21. 1. A method for encoding a picture, comprising: - predicting a given block (18) of said picture using a plurality of reference samples (17a, c) according to the method (2000) of claim 20 to obtain a prediction signal; and entropy coding the prediction residual of the given block to correct the prediction signal.

22. 1. A method for decoding a picture, comprising: - predicting a given block (18) of said picture using a plurality of reference samples (17a, c) for a method (2000) according to claim 20 to obtain a prediction signal; entropy decoding a prediction residual for the given block; and correcting the prediction signal using the prediction residual.

23. A computer program having a program code for performing the method according to any one of claims 20 to 22, when the computer program runs on a computer.

Images