Block-based prediction

Abstract

This invention provides an advantageous method and apparatus for determining prediction vectors in block-based prediction. [Solution] The device for predicting blocks is a device for predicting a given block (18) of a picture using a plurality of reference samples (17a, c). The device forms a sample value vector (102, 400) from the plurality of reference samples (100), derives a further vector to which the sample value vector is mapped by a predetermined inverse linear transformation from the sample value vector, calculates a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector, and predicts the sample of the given block based on the prediction vector.

Description

[Technical Field]

[0001] This application relates to the field of block-based prediction. Embodiments relate to an advantageous method for determining a prediction vector. [Background technology]

[0002] Today, different block-based intra-prediction modes and inter-prediction modes exist. Samples acquired from samples adjacent to the block to be predicted or from other pictures can form a sample vector that can be matrix-multiplied to determine the predicted signal for the block to be predicted.

[0003] Matrix multiplication should preferably be performed using integer arithmetic, and the matrix used for matrix multiplication should be derived by some machine learning-based training algorithm. [Overview of the project] [Problems that the invention aims to solve]

[0004] However, such training algorithms typically only yield matrices given with floating-point precision. Therefore, we face the challenge of specifying integer operations so that matrix multiplication is sufficiently approximated using these integer operations, and / or achieving improved computational efficiency, and / or making predictions more effective with respect to implementation. [Means for solving the problem]

[0005] This is achieved by the subject matter of the independent claims of this application. Further embodiments of the present invention are defined by the subject matter of the dependent claims of this application.

[0006] According to a first aspect of the present invention, the inventors of this application have recognized that one problem encountered when attempting to determine a prediction vector by an encoder or decoder is the possibility of not using integer arithmetic to compute the prediction vector for a given block. According to a first aspect of this application, this difficulty is resolved by deriving a further vector from the sample value vector, to which the sample value vector is mapped by a predetermined reversible linear transformation, such that the sample value vector is not directly applied to the matrix-vector product used to compute the prediction vector. Instead, a matrix-vector product is computed between the further vector and a predetermined prediction matrix to compute the prediction vector. The further vector is derived such that, for example, a device using integer arithmetic and / or fixed-point arithmetic can predict the samples for a given block. This is based on the idea that the components of the sample value vector are correlated, and thereby a favorable predetermined reversible linear transformation can be used to obtain a further vector having mainly small entries, which allows, for example, an integer matrix and / or a matrix with fixed-point values ​​and / or a matrix with small prediction quantization errors to be used as the given prediction matrix.

[0007] Accordingly, according to a first aspect of this application, an apparatus for predicting a given block of a picture using a plurality of reference samples is configured to form a sample value vector from the plurality of reference samples. The reference samples are, for example, samples adjacent to a given block in intra-prediction, or samples from another picture in inter-prediction. According to the embodiment, the reference samples can be reduced, for example, by averaging to obtain a sample value vector having a reduced number of values. Furthermore, the apparatus is configured to derive from the sample value vector a further vector to which the sample value vector is mapped by a predetermined inverse linear transformation, calculate the matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector, and predict the samples of a given block based on the prediction vector. Based on the further vector, the prediction of the samples of a given block can represent an integer approximation of the direct matrix-vector product between the sample value vector and the matrix to obtain the predicted samples of the given block.

[0008] The direct matrix-vector product between a sample value vector and a matrix can be equal to the second matrix-vector product between a further vector and a second matrix. The second matrix and / or matrix is, for example, a machine learning prediction matrix. According to the embodiment, the second matrix can be based on a predetermined prediction matrix and an integer matrix. The second matrix is, for example, equal to the sum of a predetermined prediction matrix and an integer matrix. In other words, the second matrix-vector product between a further vector and a second matrix can be expressed by the matrix-vector product between a further vector and a predetermined prediction matrix, and a further matrix-vector product between an integer matrix and a further vector. The integer matrix is, for example, a matrix in which a predetermined column i0 is 1 and column i ≠ i0 is 0. Thus, a good integer approximation and / or a good fixed-point value approximation of the first and / or second matrix-vector products can be achieved by the device. This is based on the idea that a given prediction matrix can be quantized, or is already a quantized matrix, because the further vectors mainly contain small values ​​and have only a slight effect of possible quantization errors in the approximation of the first and / or second matrix-vector product.

[0009] According to one embodiment, a reversible linear transformation that multiplies a given prediction vector by the sum of an integer matrix can correspond to a quantized version of a machine learning prediction matrix. The integer matrix is, for example, a matrix in which a given column i0 consists of 1s and column i ≠ i0 consists of 0s.

[0010] According to one embodiment, the inverse linear transformation is defined such that a predetermined component of the further vector is a, and each of the other components of the further vector, excluding the predetermined component, is equal to the corresponding component of the sample value vector minus a, where a is a predetermined value. Thus, a further vector of small values ​​can be realized, enabling the quantization of a given prediction matrix, resulting in a small influence of quantization error in the predicted samples of a given block. This further vector makes it possible to predict the samples of a given block by integer and / or fixed-point arithmetic.

[0011] According to the embodiment, the predetermined value is one of the following: an average such as the arithmetic mean or weighted average of the components of the sample value vector, a default value, a value signaled in the data stream in which the picture is encoded, and a component of the sample value vector corresponding to the predetermined component. The sample value vector is composed, for example, of a plurality of reference samples, or of the average of a group of reference samples from a plurality of reference samples. A group of reference samples includes, for example, at least two reference samples, preferably adjacent reference samples.

[0012] The predetermined value is, for example, the arithmetic mean or weighted mean of some components (e.g., at least two components) of the sample value vector or all components of the sample value vector. This is based on the idea that the components of the sample value vector are correlated, i.e., the values ​​of the components may be similar, and / or at least some of the components may have equal values, thereby the components of a further vector being not equal to a predetermined component of the further vector, i.e., a component i (where i0 represents the predetermined component) i≠i0 probably has a smaller absolute value than the corresponding component of the sample value vector. Thus, further vectors of smaller values ​​can be realized.

[0013] A predetermined value can be a default value, which may be selected from a list of default values, or it may be the same for all block sizes, prediction modes, etc. The components of the list of default values ​​may be associated with different block sizes, prediction modes, sample value vector sizes, the average of the sample value vector values, etc. Thus, for example, depending on a given block, i.e., depending on the decoding or encoding settings associated with a given block, an optimized default value is selected by the instrument from the list of default values.

[0014] Alternatively, the predetermined value can be a value signaled within the data stream in which the picture is encoded. In this case, for example, the encoding device determines the predetermined value. The determination of the predetermined value can be based on the same considerations described above in the context of the default value.

[0015] The components of a further vector are not equal to a given component of the further vector; that is, a component i (where i0 represents a given component) i ≠ i0 has an absolute value smaller than the corresponding component of the sample value vector, for example, by using a default value or a value signaled within the data stream as the given value.

[0016] According to the embodiment, the predetermined value can be a component of the sample value vector corresponding to the predetermined component. In other words, the value of the component of the sample value vector corresponding to the predetermined component does not change even when a reversible linear transformation is applied. Therefore, the value of the component of the sample value vector corresponding to the predetermined component is equal to, for example, the value of the predetermined component of a further vector.

[0017] A predetermined component is selected by default, for example, with respect to a predetermined value as described above. It is clear that a predetermined component can be selected by an alternative procedure. A predetermined component is selected, for example, in the same way as a predetermined value. According to the embodiment, a predetermined component is selected such that the value of the corresponding component of the sample value vector is equal to the mean of the values ​​of the sample value vector, or has only peripheral deviations from the mean of the values ​​of the sample value vector.

[0018] According to the embodiment, all matrix components of the predetermined prediction matrix in the column of the predetermined prediction matrix corresponding to a predetermined component of the further vector are 0. The device is configured to calculate the matrix-vector product, i.e., the matrix-vector product between the further vector and the predetermined prediction matrix, by performing multiplication by calculating the matrix-vector product between the reduced prediction matrix resulting from the predetermined prediction matrix by leaving a column, i.e., a column consisting of zeros, and a further further vector resulting from the further vector by leaving a predetermined component. This is based on the idea that if a predetermined component of the further vector is set to a predetermined value and the value of the sample value vector correlates, then this predetermined value is exactly or close to the sample value in the predicted signal of a predetermined block. Thus, the prediction of a sample in a predetermined block is optionally based on a predetermined prediction matrix multiplied by the further vector, or rather a reduced prediction matrix multiplied by a further further vector, and an integer matrix, the column i0 of which consists of a predetermined component, and all other columns i≠i0 being 0, multiplied by the further vector. In other words, a machine learning prediction matrix transformed by, for example, the inverse transform of a given invertible linear transformation can be partitioned into a given prediction matrix, or rather, a reduced prediction matrix and an integer matrix, based on further vectors. Therefore, only the prediction matrix should be quantized to obtain an integer approximation of the machine learning prediction matrix and / or the transformed machine learning prediction matrix, which is advantageous because the further vectors do not contain a given component, and all other components have an absolute value much smaller than the corresponding component of the sample value vector, allowing for a small influence of quantization error in the quantization resulting from the machine learning prediction matrix and / or the transformed machine learning prediction matrix. Furthermore, using the reduced prediction matrix and further vectors reduces the number of multiplications that need to be performed to obtain the prediction vector, reducing complexity and resulting in higher computational efficiency. Optionally, when predicting samples in a given block, a vector whose all components are a predetermined value a can be added to the prediction vector. This vector can be obtained by the matrix-vector product between the integer matrix and the further matrix, as described above.

[0019] According to one embodiment, the matrix obtained by summing each matrix component of a predetermined prediction matrix in a column of a predetermined prediction matrix corresponding to a predetermined component of a further vector by a factor of 1 of a predetermined inverse linear transformation corresponds to a quantized version of the machine learning prediction matrix. Summing each matrix component of a predetermined prediction matrix in a column of a predetermined prediction matrix corresponding to a predetermined component of a further vector with 1 represents, for example, a transformed machine learning prediction matrix. The transformed machine learning prediction matrix represents, for example, a machine learning prediction matrix transformed by the inverse transform of a predetermined inverse linear transformation. The sum can correspond to the sum of a predetermined prediction matrix with an integer matrix, where the column i0 corresponding to a predetermined component is 1 and all other columns i≠i0 are 0.

[0020] According to one embodiment, the apparatus is configured to represent a predetermined prediction matrix using prediction parameters and to calculate a matrix-vector product by performing multiplication and addition on the components of further vectors, as well as on the prediction parameters and the resulting intermediate results. The absolute value of the prediction parameters can be represented by an n-bit fixed-point number representation, where n is 14 or less, or 10 or less, or 8 or less. In other words, the prediction parameters are multiplied and / or added to the elements of the matrix-vector product, such as further vectors, a predetermined prediction matrix, and / or prediction vectors. The multiplication and addition operations can yield, for example, a fixed-point format of a predetermined prediction matrix, prediction vectors, and / or prediction samples of a predetermined block.

[0021] Embodiments of the present invention relate to an apparatus for encoding a picture, comprising an apparatus for predicting a given block of a picture using a plurality of reference samples according to any of the embodiments described herein in order to obtain a prediction signal. The apparatus further comprises an entropy encoder configured to encode the prediction residual of a given block in order to correct the prediction signal. For predicting a given block in order to obtain a prediction signal, the apparatus is configured, for example, to form a sample value vector from a plurality of reference samples, derive a further vector from the sample value vector to which the sample value vector is mapped by a predetermined inverse linear transformation, calculate a matrix-vector product between the further vector and a predetermined prediction matrix in order to obtain a prediction vector, and predict the samples of the given block based on the prediction vector.

[0022] Embodiments of the present invention relate to a device for decoding a picture, comprising a device for predicting a given block of a picture using a plurality of reference samples according to any embodiment described herein in order to obtain a prediction signal. The device further comprises an entropy decoder configured to decode the prediction residual of a given block, and a prediction corrector configured to correct the prediction signal using the prediction residual. For predicting a given block in order to obtain a prediction signal, the device is configured, for example, to form a sample value vector from a plurality of reference samples, derive a further vector from the sample value vector to which the sample value vector is mapped by a predetermined inverse linear transformation, calculate a matrix-vector product between the further vector and a predetermined prediction matrix in order to obtain a prediction vector, and predict the samples of the given block based on the prediction vector.

[0023] Embodiments of the present invention relate to a method for predicting a given block of a picture using a plurality of reference samples, the method comprising: forming a sample value vector from the plurality of reference samples; deriving a further vector from the sample value vector to which the sample value vector is mapped by a predetermined inverse linear transformation; calculating a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector; and predicting a sample of the given block based on the prediction vector.

[0024] Embodiments of the present invention relate to a method for encoding a picture, comprising: predicting a predetermined block of a picture using a plurality of reference samples in accordance with the method described above in order to obtain a prediction signal; and entropy coding the prediction residual of the predetermined block in order to correct the prediction signal.

[0025] Embodiments of the present invention relate to a method for decoding a picture, comprising: predicting a predetermined block of the picture using a plurality of reference samples according to one of the above-described methods in order to obtain a prediction signal; entropy decoding the prediction residual of the predetermined block; and correcting the prediction signal using the prediction residual. Embodiments of the present invention relate to a data stream having a picture encoded using the method described herein for encoding a picture.

[0026] Embodiments of the present invention relate to computer programs having program code for performing any of the methods of the embodiments described herein when executed on a computer.

[0027] The drawings are not necessarily to scale, and instead focus on illustrating the general principles of the present invention. Various embodiments of the present invention will be described below with reference to the following drawings. [Brief explanation of the drawing]

[0028] [Figure 1] This illustrates an embodiment of encoding to a data stream. [Figure 2] An embodiment of the encoder is shown. [Figure 3] This shows an embodiment of picture reconstruction. [Figure 4] An embodiment of the decoder is shown. [Figure 5] A schematic diagram of block prediction for encoding and / or decoding according to an embodiment is shown. [Figure 6] This embodiment illustrates a matrix operation for predicting blocks for encoding and / or decoding. [Figure 7.1] This shows the prediction of a block having a reduced sample value vector according to the embodiment. [Figure 7.2] This embodiment demonstrates block prediction using sample interpolation. [Figure 7.3] This embodiment shows a prediction of a reduced block with a sample value vector, where only some boundary samples are averaged. [Figure 7.4] The embodiment shows a prediction of a block having a reduced sample value vector, where four groups of boundary samples are averaged. [Figure 8] This diagram shows a schematic representation of a device for predicting blocks according to an embodiment. [Figure 9] This shows matrix operations performed by the apparatus according to the embodiment. [Figure 10a] This shows the detailed matrix operations performed by the apparatus according to the embodiment. [Figures 10b-10c] This shows the detailed matrix operations performed by the apparatus according to the embodiment. [Figure 11] This shows detailed matrix operations performed by the device using offset and scaling parameters according to the embodiment. [Figure 12]This shows detailed matrix operations performed by the device using offset and scaling parameters according to different embodiments. [Figure 13] This shows a block diagram of a method for predicting a predetermined block according to an embodiment. [Modes for carrying out the invention]

[0029] Elements that are equal or equivalent, or elements that have equal or equivalent functions, are indicated in the following description by equal or equivalent reference numerals, even if they occur in different figures.

[0030] The following description includes several details to provide a more complete description of embodiments of the present invention. However, it will be apparent to those skilled in the art that embodiments of the present invention can be carried out without these specific details. In other examples, well-known structures and apparatus are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present invention. Furthermore, features of the different embodiments described below can be combined with each other unless otherwise specified. 1. Introduction

[0031] Different examples, embodiments, and aspects of the present invention are described below. At least some of these examples, embodiments, and aspects refer, among other things, to methods and / or apparatus for video coding, and / or for performing block-based prediction using linear or affine transforms with, for example, adjacent sample reduction, and / or for optimizing video delivery (e.g., broadcast, streaming, file playback, etc.) for, for example, video applications and / or virtual reality applications.

[0032] Furthermore, examples, embodiments, and aspects may refer to High Efficiency Video Coding (HEVC) or its successors. Further embodiments, examples, and aspects are defined by the appended claims.

[0033] It should be noted that any embodiments, examples, and aspects defined by the claims may be supplemented by any of the details (features and functions) described in the following chapters.

[0034] Furthermore, the embodiments, examples, and aspects described in the following chapters may be used individually and may be supplemented by any of the features in other chapters or any features included in the claims.

[0035] Furthermore, it should be noted that the individual examples, embodiments, and aspects described herein can be used individually or in combination. Therefore, details can be added to each of the individual aspects without adding further detail to another of the aforementioned examples, embodiments, and aspects. Note that this disclosure also explicitly or implicitly describes the features of the decoding and / or encoding systems and / or methods.

[0036] Furthermore, the features and functions disclosed herein in relation to the methods may also be used in the apparatus. Moreover, any features and functions disclosed herein in relation to the apparatus may also be used in the corresponding methods. In other words, the methods disclosed herein may be supplemented by any of the features and functions described in relation to the apparatus.

[0037] Furthermore, any of the features and functions described herein can be implemented in hardware, software, or a combination of hardware and software, as described in the "Implementation Alternatives" section.

[0038] Furthermore, any of the features indicated in parentheses ("(...)" or "[...]") may be considered optional in some examples, embodiments, or aspects.

[0039] 2 Encoders, Decoders The following describes various examples that can help achieve more effective compression when using block-based prediction. Some examples achieve high compression efficiency by expending a set of intra-prediction modes. The latter may be provided exclusively or in addition to other heuristically designed intra-prediction modes, for example. And even other examples utilize both of the specializations described here. However, as an oscillation of these embodiments, intra-prediction can be converted to inter-prediction by instead using reference samples in a separate picture.

[0040] To facilitate understanding of the following examples of this application, the description begins with the presentation of possible encoders and decoders that can be adapted to construct the examples outlined later in this application. Figure 1 shows an apparatus for encoding a picture 10 into a data stream 12 in blocks. The apparatus is indicated by reference numeral 14 and may be a static picture encoder or a video encoder. In other words, the picture 10 may be the current picture from a video 16 if the encoder 14 is configured to encode a video 16 containing the picture 10 into a data stream 12, or if the encoder 14 can exclusively encode the picture 10 into a data stream 12.

[0041] As described above, the encoder 14 performs encoding in a block-by-block manner or on a block basis. For this purpose, the encoder 14 subdivides the picture 10 into blocks, and the units of the encoder 14 encode the picture 10 into a data stream 12. Examples of possible subdivisions of the picture 10 into blocks 18 are shown in more detail below. In general, the subdivision may end up in blocks 18 of a fixed size, such as an array of blocks arranged in rows and columns, or in blocks 18 of different block sizes, such as by using hierarchical multitree subdivision, which starts from the entire picture area of ​​the picture 10 or from a pre-partition of the picture 10 into an array of tree blocks, and these examples should not be treated as excluding other possible ways of subdividing the picture 10 into blocks 18.

[0042] Furthermore, the encoder 14 is a predictive encoder configured to predictively encode the picture 10 into the data stream 12. For a particular block 18, this means that the encoder 14 determines the predictive signal for block 18 and encodes the predictive residual, i.e., the prediction error that causes the predictive signal to deviate from the actual picture content within block 18, into the data stream 12.

[0043] The encoder 14 can support different prediction modes to derive the prediction signal for a particular block 18. The prediction mode important in the following example is the intra-prediction mode, in which the interior of block 18 is spatially predicted from adjacent, already encoded picture 10 samples. The encoding of picture 10 into the data stream 12, and therefore the corresponding decoding procedure, can be based on a specific encoding order 20 defined among the blocks 18. For example, the encoding order 20 can traverse the blocks 18 in a raster scan order such as top to bottom row by row, traversing each row from left to right. In the case of hierarchical multitree-based subdivision, the raster scan order can be applied within each hierarchy level, and a depth-first traversal order can be applied. That is, leaf nodes in a block at a particular hierarchy level can precede blocks at the same hierarchy level that have the same parent block according to the encoding order 20. Depending on the encoding order 20, adjacent, already encoded samples of block 18 can typically be placed on one or more sides of block 18. In the examples presented herein, for example, the already encoded samples adjacent to block 18 are located at the top and left of block 18.

[0044] The encoder 14 may not support any mode other than intra-prediction. For example, if the encoder 14 is a video encoder, it may also support an inter-prediction mode in which block 18 is temporarily predicted from a picture of a previously encoded video 16. Such an inter-prediction mode may be a motion-compensated prediction mode in which a motion vector is signaled to such block 18, indicating the relative spatial offset of the portion from which the predicted signal of block 18 is derived as a copy. Additionally or alternatively, other non-intra-prediction modes may also be available, such as an inter-prediction mode when the encoder 14 is a multi-view encoder, or a non-prediction mode in which the inside of block 18 is encoded as is, i.e., without prediction.

[0045] Before beginning the description of this application with a focus on the intra-predictive mode, we will describe a more specific example of a possible block-based encoder, namely, a possible implementation of encoder 14 such that, as described with respect to Figure 2, we will then present two corresponding examples of decoders that fit Figures 1 and 2, respectively.

[0046] Figure 2 shows a possible implementation of the encoder 14 of Figure 1, i.e., one in which the encoder is configured to use transform coding to encode the predictive residuals, but this is merely an example and the present application is not limited to this type of predictive residual coding. According to Figure 2, the encoder 14 includes a subtractor 22 configured to subtract the corresponding predictive signal 24 from the inbound signal, i.e., picture 10, or block-based from the current block 18, to obtain a predictive residual signal 26 which will later be encoded into a data stream 12 by the predictive residual encoder 28. The predictive residual encoder 28 consists of an irreversible coding stage 28a and a reversible coding stage 28b. The irreversible stage 28a includes a quantizer 30 which receives the predictive residual signal 26 and quantizes samples of the predictive residual signal 26. As already mentioned above, this example uses transform coding of the predicted residual signal 26, and therefore the irreversible coding stage 28a comprises a transform stage 32 connected between the subtractor 22 and the quantizer 30 to transform such predicted residuals 26 spectrally decomposed by the quantizer 30 performing quantization on the transformed coefficients that present the residual signal 26. The transform can be DCT, DST, FFT, Hadamard transform, etc. Next, the transformed and quantized predicted residual signal 34 undergoes reversible coding by a reversible coding stage 28b, which is an entropy coder that entropy codes the quantized predicted residual signal 34 into a data stream 12. The encoder 14 further comprises a predicted residual signal reconstruction stage 36 connected to the output of the quantizer 30 to reconstruct the predicted residual signal from the transformed and quantized predicted residual signal 34 in a manner also available to the decoder. That is, it is the quantizer 30 that considers coding loss. For this purpose, the prediction residual reconstruction stage 36 comprises an inverse quantizer 38 that performs the inverse of the quantization of the quantizer 30, and a subsequent inverse converter 40 that performs the inverse transform of the transform performed by the converter 32, such as the inverse of spectral decomposition, such as the inverse of any of the specific transform examples described above. The encoder 14 comprises an adder 42 that adds the reconstructed prediction residual signal output by the inverse converter 40 and the prediction signal 24 to output a reconstructed signal, i.e., a reconstructed sample.This output is fed to the predictor 44 of the encoder 14, which determines the predicted signal 24 based on it. It is the predictor 44 that supports all the prediction modes already described in relation to Figure 1. Figure 2 also shows that if the encoder 14 is a video encoder, the encoder 14 may also include an in-loop filter 46 that filters out a fully reconstructed picture that, after filtering, forms the reference picture of the predictor 44 with respect to the mutual prediction block.

[0047] As already mentioned above, the encoder 14 operates on a block basis. In the following description, the block base in question is a subdivision of the picture 10 into blocks, for which an intra-prediction mode is selected from a set or multiple intra-prediction modes supported by the predictor 44 or encoder 14, respectively, and the selected intra-prediction modes are executed individually. However, there may also be other types of blocks into which the picture 10 is subdivision. For example, the above determination of whether the picture 10 is intercoded or intracoded can be made at a granular level, or at the unit of blocks that deviate from block 18. For example, the inter-mode / intra-mode determination can be performed at the level of the coded block in which the picture 10 is subdivision and each coded block is subdivided into a prediction block. Each prediction block that has a coded block in which it has been determined that intra-prediction is to be used is subdivided into an intra-prediction mode determination. Thus, for each of these prediction blocks, it is determined which supported intra-prediction mode should be used for that prediction block. These prediction blocks form block 18 of interest here. Prediction blocks within a coded block related to mutual prediction will be handled differently by the predictor 44. They will be mutually predicted from a reference picture by determining the motion vector and copying the prediction signal of this block from the position in the reference picture indicated by the motion vector. Another block subdivision relates to the subdivision into transformed blocks in the unit where the transformation by the converter 32 and inverse converter 40 is performed. The transformed blocks may be, for example, the result of further subdivision of the coded block. Of course, the examples described here should not be treated as limiting, and other examples exist. For completeness only, it should be noted that subdivision into coded blocks can use, for example, multi-tree subdivision, and similarly, prediction blocks and / or transformed blocks can be obtained by further subdivision of the coded block using multi-tree subdivision.

[0048] Figure 3 shows a decoder 54 or device for block-level decoding that conforms to the encoder 14 of Figure 1. This decoder 54 does the opposite of the encoder 14; that is, it decodes the picture 10 from the data stream 12 in blocks, and for this purpose supports multiple intra-prediction modes. Decoder 54 may include, for example, a residual provider 156. All the other possibilities described above with respect to Figure 1 are also valid for decoder 54. For this reason, decoder 54 can be a still picture decoder or a video decoder, and all prediction modes and predictability are also supported by decoder 54. The difference between encoder 14 and decoder 54 is mainly the fact that encoder 14 selects or chooses coding decisions according to some optimization, such as minimizing some cost function which may depend on coding speed and / or coding distortion. One of these coding options or coding parameters may include the selection of an intra-prediction mode to be used for the current block 18 from among the available or supported intra-prediction modes. Next, the selected intra-prediction mode is signaled by the encoder 14 of the current block 18 in the data stream 12, and the decoder 54 uses this signaling of block 18 in the data stream 12 to redo the selection. Similarly, the subdivision of picture 10 into block 18 can be subject to optimization within encoder 14, the corresponding subdivision information can be transmitted within the data stream 12, and the decoder 54 recovers the subdivision of picture 10 into block 18 based on the subdivision information. To summarize the above, decoder 54 can be a block-based predictive decoder, and in addition to the intra-prediction mode, decoder 54 can support other predictive modes, such as mutual predictive mode, if decoder 54 is a video decoder. In decoding, decoder 54 can also use the coding order 20 described with respect to Figure 1, which is followed by both encoder 14 and decoder 54, so that the same neighboring samples are available for the current block 18 in both encoder 14 and decoder 54.Therefore, to avoid unnecessary repetition, the description of the operating modes of the encoder 14 must also apply to the decoder 54, for example, as far as prediction is concerned and as far as coding of prediction residuals is concerned. The difference lies in the fact that the encoder 14, by optimization, selects or inserts several coding options or coding parameters and signals into the data stream 12, which are then derived from the data stream 12 by the decoder 54 for re-predicting, such as through re-partitioning.

[0049] Figure 4 shows a possible implementation of the decoder 54 in Figure 3, i.e., one that fits the implementation of the encoder 14 in Figure 1, as shown in Figure 2. Since many elements of the encoder 54 in Figure 4 are the same as those occurring in the corresponding encoder in Figure 2, the same reference code with an apostrophe is used in Figure 4 to represent these elements. In particular, the adder 42', the optional in-loop filter 46', and the predictor 44' are connected to the predictor loop in the same way as they are in the encoder in Figure 2. The reconstructed, i.e., inversely quantized and retransformed predictor residual signal applied to the adder 42' is derived by a sequence of entropy decoders 56 that reverse the entropy coding of the entropy encoder 28b, followed by a residual signal reconstruction stage 36' consisting of an inverse quantizer 38' and an inverse converter 40', just as in the case of the encoding side. The output of the decoder is the reconstruction of picture 10. The reconstruction of picture 10 can be made available directly at the output of adder 42' or at the output of in-loop filter 46'. To improve picture quality, several post-filters can be placed at the decoder output to apply some post-filtering to the reconstruction of picture 10, but this option is not shown in Figure 4.

[0050] To reiterate, the explanation given above for Figure 2 is also valid for Figure 4, except that the encoder simply performs the optimization task and the relevant decisions regarding the encoding options. However, all explanations regarding block subdivision, prediction, inverse quantization, and retransformation are also valid for the decoder 54 in Figure 4.

[0051] 3. ALWIP (Affine Linear Weighted Intra Predictor) Some non-limiting examples of ALWIP are described herein, even if ALWIP is not necessarily required to embody the technology described herein.

[0052] This application relates, in particular, to the concept of an improved block-based predictive mode for block-by-block picture coding, such as that usable in video codecs such as HEVC or its successors. The predictive mode may be an intra-predictive mode, but theoretically, the concept described herein can also be translated into an inter-predictive mode, where the reference sample is part of another picture. There is a need for block-based predictive concepts that enable efficient implementations, such as hardware-friendly implementations. This objective is achieved by the subject matter of the independent claims of this application.

[0053] Intra-predictive mode is widely used in picture and video encoding. In video encoding, intra-predictive mode competes with other prediction modes such as mutual prediction modes, including motion-compensated prediction mode. In intra-predictive mode, the current block is predicted based on adjacent samples, i.e., samples that have already been encoded as far as the encoder side and already decoded as far as the decoder side. The adjacent sample values ​​are extrapolated to the current block to form the predicted signal for the current block, and the predicted residual is transmitted in the data stream of the current block. The better the predicted signal, the smaller the predicted residual, and therefore the fewer bits are needed to encode the predicted residual.

[0054] To be effective, several aspects need to be considered in order to form an effective framework for intra-prediction in a block-based picture coding environment. For example, the more intra-prediction modes supported by the codec, the greater the consumption of side information rate to notify the decoder of its selection. On the other hand, the set of supported intra-prediction modes must be able to provide good prediction signals, i.e., prediction signals with low prediction residuals.

[0055] Disclosed below, as comparative embodiments or basic examples, is a device (encoder or decoder) for decoding pictures from a data stream in blocks, which supports at least one intra-prediction mode, wherein an intra-prediction signal for a block of a given size of picture is determined by applying a first template of samples adjacent to the current block to an affine linear predictor, which is ultimately referred to as an affine linear weighted predictor (ALWIP).

[0056] The device may have at least one of the following characteristics (the same may apply to a method or other technique, for example, if performed by a processor, implemented in a non-temporary memory unit that stores instructions causing the processor to perform the method and / or to operate as a device):

[0057] 3.1 Predictors can be complementary to other predictors. The intra-prediction modes that can form the subject of implementation improvements described below can be complementary to other intra-prediction modes of the codec. Therefore, they can complement the DC prediction mode, planar prediction mode, or angle prediction mode defined in the HEVC codec and JEM reference software. Hereafter, the latter three types of intra-prediction modes will be referred to as conventional intra-prediction modes. Therefore, for a given block of intra-modes, the decoder needs to analyze a flag indicating whether one of the intra-prediction modes supported by the device should be used.

[0058] 3.2 Two or more proposed prediction modes A device may have two or more ALWIP modes. Therefore, if the decoder knows that one of the ALWIP modes supported by the device should be used, the decoder needs to analyze additional information indicating which of the ALWIP modes supported by the device should be used.

[0059] The signal transmission of supported modes may have the characteristic that encoding some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes requires fewer bins and which require more may depend on information that can be extracted from the already decoded bitstream or that can be fixed in advance.

[0060] 4. Several aspects Figure 2 shows a decoder 54 for decoding a picture from a data stream 12. The decoder 54 can be configured to decode a given block 18 of the picture. In particular, the predictor 44 can be configured to map a set of P adjacent samples adjacent to a given block 18 to a set of Q predicted values ​​of the samples in the given block using a linear or affine linear transformation [e.g., ALWIP].

[0061] As shown in Figure 5, a given block 18 contains Q predicted values ​​(which become the “predicted values” at the end of the operation). If block 18 has M rows and N columns, then Q = M·N. The Q values ​​of block 18 can be in the spatial domain (e.g., pixels) or the transformation domain (e.g., DCT, discrete wavelet transform, etc.). The Q values ​​of block 18 can generally be predicted based on P values ​​obtained from adjacent blocks 17a-17c adjacent to block 18. The P values ​​of adjacent blocks 17a-17c may be in the closest position to block 18 (e.g., adjacent). The P values ​​of adjacent blocks 17a-17c have already been processed and predicted. The P values ​​are shown as values ​​in parts 17'a-17'c to distinguish them from the blocks they are part of (in some examples, 17'b is not used).

[0062] As shown in Figure 6, to perform the prediction, it is possible to operate with a first vector 17P having P entries (each entry associated with a specific position in the adjacent portion 17'a~17'c), a second vector 18Q having Q entries (each entry associated with a specific position in block 18), and a mapping matrix 17M (each row associated with a specific position in block 18, and each column associated with a specific position in the adjacent portion 17'a~17'c). Thus, the mapping matrix 17M performs the prediction of the P values ​​of the adjacent portion 17'a~17'c to the values ​​in block 18 according to a predetermined mode. Thus, the entries in the mapping matrix 17M can be understood as weight coefficients. In the following description, the signs 17a~17c are used instead of 17'a~17'c to refer to the adjacent portions of the boundary.

[0063] In this technical field, several conventional modes are known, such as the DC mode, the planar mode, and the 65-direction prediction mode. For example, 67 modes are known.

[0064] However, it should be noted that it is also possible to use a different mode called linear or affine linear transformation. A linear or affine linear transformation includes P·Q weight coefficients, of which at least 1 / 4P·Q weight coefficients are non-zero weight values, and for each of the Q predicted values, it includes a set of P weight coefficients relating to each predicted value. When the sequence is arranged vertically according to the raster scanning order between samples of a given block, it forms an envelope that is nonlinear in all directions.

[0065] It is possible to map the P positions of adjacent values ​​17'a~17'c (template), the Q positions of adjacent samples 17'a~17'c, and the P*Q weight coefficient values ​​of matrix 17M. The plane is an example of the envelope of a sequence for DC transformation (it is a plane for DC transformation). The envelope is obviously a plane and is therefore excluded by the definition of linear or affine linear transformation (ALWIP). Another example is the matrix that results in the following emulation of angular modes: the envelope is excluded from the ALWIP definition and, simply put, looks like a hill that slopes diagonally from top to bottom along the direction in the P / Q plane. Planar modes and 65 direction predictive modes have different envelopes, which are linear in at least one direction, i.e., all directions of the exemplified DC and, for example, the hill direction of the angular modes.

[0066] Conversely, the envelopes of linear or affine transformations are not linear in all directions. It is understood that such types of transformations may, in some situations, be optimal for performing the predictions in block 18. Note that it is preferable that at least 1 / 4 of the weight coefficients are non-zero (i.e., at least 25% of the P*Q weight coefficients are non-zero).

[0067] The weight coefficients may be independent of each other according to any regular mapping rule. Therefore, matrix 17M may be such that the values ​​of its entries do not have any obvious, recognizable relationships. For example, the weight coefficients cannot be described by any analytic or differential function.

[0068] In the example, the ALWIP transformation may be lower than a predetermined threshold (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold within the range of 0.05 to 0.035), even though the average of the maximum cross-correlations between the weight coefficients of a first series associated with each predicted value and the weight coefficients of a second series associated with the predicted values ​​other than each predicted value, or an inverted version of the latter series, leads to a higher maximum value. For example, for each row join (i1,i2) of the ALWIP matrix 17M, the cross-correlation can be calculated by multiplying the P-value of row i1 by the P-value of row i2. For each resulting cross-correlation, the maximum value can be obtained. Thus, the mean (average) can be obtained over the entire matrix 17M (i.e., the maximum cross-correlations for all combinations are averaged). The threshold may then be, for example, 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold within the range of 0.05 to 0.035.

[0069] The P adjacent samples of blocks 17a to 17c may be arranged along a one-dimensional path extending along the boundary of a given block 18 (e.g., 18c, 18a). For each of the Q predicted values ​​of a given block 18, a set of P weight coefficients associated with each predicted value may be ordered to traverse the one-dimensional path in a predetermined direction (e.g., left to right, top to bottom, etc.). In this example, the ALWIP matrix 17M may be off-diagonal or non-block diagonal. An example of an ALWIP matrix 17M for predicting a 4x4 block 18 from four already predicted adjacent samples may be as follows: { { 37,59,77,28}, { 32,92,85,25}, { 31,69,100,24}, { 33,36,106,29}, { 24,49,104,48}, { 24,21,94,59}, { 29,0,80,72}, { 35,2,66,84}, { 32,13,35,99}, { 39,11,34,103}, { 45,21,34,106}, { 51,24,40,105}, { 50, 28, 43, 101}, { 56,32,49,101}, { 61,31,53,102}, { 61,32,54,100} }.

[0070] (Here, {37, 59, 77, 28} is the first row. {32, 92, 85, 25} is the second row. {61, 32, 54, 100} is the 16th row of matrix 17M.) Matrix 17M has dimensions 16 × 4 and contains 64 weight coefficients (as a result of 16 * 4 = 64). This is because matrix 17M has dimensions Q x P, where Q = M * N is the number of samples in the block 18 to be predicted (block 18 is a 4 × 4 block), and P is the number of samples already predicted. Here, M = 4, N = 4, Q = 16 (as a result of M * N = 4 * 4 = 16), and P = 4. The matrix is ​​off-diagonal and non-block diagonal and is not described by any particular rule.

[0071] As can be seen, less than 1 / 4 of the weight coefficients are 0 (in the case of the matrix above, one of the 64 weight coefficients is 0). The envelope formed by these values, when placed one step down according to the raster scan order, forms an envelope that is nonlinear in all directions.

[0072] Even if the above explanation primarily refers to the decoder (e.g., decoder 54), the same may be performed in the encoder (e.g., encoder 14).

[0073] In some examples, for each block size (in a set of block sizes), the ALWIP transformations of the intra-prediction modes in a second set of intra-prediction modes for each block size are different from one another. Additionally or alternatively, the cardinality of the second set of intra-prediction modes for a block size in a set of block sizes can match, but the associated linear or affine linear transformations of the intra-prediction modes in the second set for different block sizes can be made incompatible with one another by scaling.

[0074] In some examples, ALWIP transformations may be defined as having "nothing in common" with conventional transformations (for example, an ALWIP transformation can have "nothing in common" with its corresponding conventional transformation, even if it is mapped via one of the above mappings).

[0075] In one example, ALWIP mode is used for both the luminous and chroma components, while in another example, ALWIP mode is used for the luminous component but not for the chroma component.

[0076] 5. Affine linear weighted intra-prediction mode achieved by speeding up the encoder (e.g., Test CE3-1.2.1) 5.1 Description of method or apparatus The affine linear weighted intra-prediction (ALWIP) mode tested in CE3-1.2.1 can be the same as the one proposed in JVET-L0199 under test CE3-2.2.2, except for the following changes:

[0077] • Multiple Reference Lines (MRL) intra-prediction, particularly the coding estimation and signaling compatibility, i.e., MRLs are not combined with ALWIP, and the transmission of MRL indices is restricted to non-ALWIP blocks.

[0078] Subsampling is now mandatory for all blocks W×H ≥ 32×32 (it was previously optional for 32×32). Therefore, additional testing and transmission of the subsampling flag in the encoder have been eliminated.

[0079] ALWIP for 64×N and N×64 blocks (N≦32) is added by downsampling to 32×N and N×32 respectively and applying the corresponding ALWIP mode. Furthermore, test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0080] • Combined mode estimation: Conventional and ALWIP modes use a shared Hadamard candidate list for complete RD estimation; i.e., ALWIP mode candidates are added to the same list as conventional (and MRL) mode candidates based on their Hadamard costs. EMT Intra-Fast and PB Intra-Fast are supported for the join mode list and have additional optimizations to reduce the number of full RD checks. Only the MPMs from the available left and top blocks are added to the list for full RD estimation of ALWIP, following the same methodology as in the conventional mode.

[0081] 5.2 Complexity Assessment Aside from the calculations involving the discrete cosine transform, test CE3-1.2.1 required up to 12 multiplications per sample to generate the predicted signal. Furthermore, a total of 136,492 parameters, each 16 bits in size, were required. This corresponds to 0.273 megabytes of memory.

[0082] 5.3 Experimental Results The test evaluation was performed using VTM software version 3.0.1 for intra-only (AI) and random access (RA) configurations, according to common test conditions JVET-J1010[2]. The corresponding simulations were run on an Intel Xeon cluster (E5-2697A v4, AVX2 on, Turbo Boost off) with the Linux® OS and GCC 7.2.1 compiler.

[0083] [Table 1]

[0084] [Table 2]

[0085] 5.4 Affine linear weighted intra-prediction with complexity reduction (e.g., Test CE3-1.2.2) The techniques tested in CE2 are related to the “affine linear intraprediction” described in JVET-L0199[1], but are simplified in terms of memory requirements and computational complexity as follows:

[0086] Only three different sets of prediction matrices (e.g., S0, S1, S2, see also below) and bias vectors (e.g., to provide offset values) can exist to cover all block shapes. As a result, the number of parameters is reduced to 14,400 10-bit values, which is...

number

[0087] The input and output sizes of the predictor are further reduced. Furthermore, instead of transforming the boundary via DCT, averaging or downsampling can be performed on the boundary samples, and linear interpolation can be used instead of inverse DCT for generating the predicted signal. As a result, up to four multiplications per sample may be required to generate the predicted signal.

[0088] 6. Example This section describes how to perform several predictions (for example, as shown in Figure 6) using ALWIP prediction. As a general rule, referring to Figure 6, in order to obtain the Q=M*N values ​​of the M×N block 18 to be predicted, the multiplication of the Q*P samples of the Q×P ALWIP prediction matrix 17M and the P samples of the P×1 neighbor vector 17P should be performed. Therefore, in general, in order to obtain each of the Q=M*N values ​​of the M×N block 18 to be predicted, the multiplication of at least P=M+N values ​​is required.

[0089] These multiplications have highly undesirable effects. The dimension P of the boundary vector 17P generally depends on the number M+N of boundary samples (bins or pixels) 17a, 17c adjacent to the M×N block 18 to be predicted (e.g., adjacent). This means that as the size of the block 18 to be predicted increases, the number of boundary pixels (17a, 17c) M+N increases accordingly, so the dimension P=M+N of the P×1 boundary vector 17P and the length of each row of the Q×P ALWIP prediction matrix 17M, and consequently the number of multiplications required, increases (generally speaking, Q=M*N=W*H, where W (width) is another symbol for N, H (height) is another symbol for M, and P=M+N=H+W if the boundary vector is formed by only one row and / or one column of samples).

[0090] This problem is generally exacerbated by the fact that multiplication is generally a power-consuming operation in microprocessor-based systems (or other digital processing systems). It can be inferred that a large number of multiplications performed on a very large number of samples across a large number of blocks generally result in an undesirable waste of computational power. Therefore, it is preferable to reduce the number of multiplications Q*P required to predict the M×N block 18.

[0091] It is understood that by intelligently selecting a simpler operation as an alternative to multiplication, it is possible to reduce the computational power required for each intra-prediction in each block 18 being predicted. In particular, referring to Figures 7.1 to 7.4, the encoder or decoder is,

[0092] To obtain a reduced set of sample values ​​with fewer samples compared to multiple adjacent samples, multiple adjacent samples (e.g., 17a, 17c) are reduced (e.g., in step 811), (e.g., by averaging or downsampling),

[0093] The reduced set of sample values ​​is subjected to a linear or affine linear transformation (for example, in step 812) to obtain predicted values ​​for a given sample in a given block,

[0094] It is understood that a given block of a picture (e.g., 18) can be predicted using multiple adjacent samples (e.g., 17a, 17c).

[0095] In some cases, the decoder or encoder may also derive, for example, by interpolation, predictions for further samples in a given block based on predictions for a given sample and several adjacent samples. Thus, an upsampling strategy can be obtained.

[0096] In an example, it is possible to perform several averages on the samples of boundary 17 (e.g., in step 811) so as to reach a reduced set 102 of samples with a reduced number of samples (Figures 7.1 to 7.4) (at least one of the samples of the reduced number of samples 102 may be the average of two samples of the original boundary samples or a selection of the original boundary samples). For example, if the original boundary has P = M + N samples, the reduced set of samples can have P red <P such that M red <M and N red <N is at least one of, P red = M red + N red Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) does not have P × 1 entries, but has P red <P which is, P red × 1 entries. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have a Q × P dimension, but has a reduced number of matrix elements of Q × P red <M and N red <N is at least one of) at least P red <P which is, so that the number of matrix elements is reduced to Q × P<007>(or Q red × P red [[ID=Q]] red (see below).

[0097] In some examples (e.g., Figures 7.2, Figure 7.3), the block obtained by ALWIP (in step 812) is

Number

Number

Number

Number

Number

Number

[0098] <Q These techniques can be advantageous because while the matrix multiplication involves a reduced number of multiplications (Q red *P red or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (and even avoiding) multiplications. For example, downsampling, averaging, and / or interpolation can be performed (e.g., in steps 811 and / or 813) by employing binary operations that require non-computation power such as addition and shift. Also, addition is a very simple operation that can be easily performed without much computational effort.

[0099] This shift operation can be used, for example, to average two boundary samples to obtain the final predicted block, and / or to interpolate two samples (support values) of the reduced predicted block (or taken from the boundary). (Two sample values ​​are required for interpolation. Within a block, there are always two predetermined values, but to interpolate samples along the left and top boundaries of the block, there is only one predetermined value, as shown in Figure 7.2, and therefore the boundary sample is used as the support value for interpolation.)

[0100] The following two-step procedure can be used: First, add the values ​​of the two samples. Next, halve the sum (for example, by right-shifting). Alternatively, the following is possible: First, halve each sample (for example, by left-shifting),

[0101] Next, the values ​​of the two halved samples are added together. Since only one sample quantity and group of samples (e.g., adjacent samples) need to be selected, even simpler calculations can be performed during downsampling (e.g., in step 811). Therefore, it is possible to define techniques here for reducing the number of multiplications that need to be performed. Some of these techniques may be based, among other things, on at least one of the following principles:

[0102] Even if the actual predicted size of block 18 is M×N, the block will be reduced (in at least one of the two dimensions) and the reduced size will be Q. red ×P red (

number

Number

Number

[0103] Additionally or alternatively, instead of predicting all Q = M*N values of the block 18 to be predicted by multiplication, a reduced block with reduced dimensions (e.g.,

Number

Number

Number

[0104] As shown in the example in Figure 7.1, a 4x4 block 18 (M=4, N=4, Q=M*N=16) is predicted, and the neighborhoods 17 (a vertical matrix with 4 already predicted samples) and 17c (a horizontal row with 4 already predicted samples) of sample 17a have already been predicted in the previous iteration (neighborhoods 17a and 17c can be collectively represented by 17). Priorily, by using the formula shown in Figure 5, the prediction matrix 17M should be a Q×P=16×8 matrix (Q=M*N=4*4 and P=M+N=4+4=8), and the boundary vector 17P should have dimensions of 8×1 (P=8). However, this leads to the need to perform 8 multiplications for each of the 16 samples in the 4x4 block 18 to be predicted, and thus a total of 16*8=128 multiplications. (Note that the average number of multiplications per sample is a good indicator of computational complexity. Conventional intraprediction requires four multiplications per sample, which increases the computational effort involved. Therefore, it is possible to use this as an upper limit for ALWIP, ensuring that the complexity is reasonable and does not exceed the complexity of conventional intraprediction.)

[0105] Nevertheless, by using this technology, in step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted is changed from P to P redIt is understood that it is possible to reduce to . In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that the operation as block 18 is a 2×2 block (the reduced boundary is formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for row 17c and two samples for column 17a. Thus, the horizontal row 17c is processed as having two samples (for example, averaged samples) instead of having four original samples, and the vertical column 17a, which originally had four samples, is processed as having two samples (for example, averaged samples). After subdividing row 17c and column 17a into groups 110 of two samples each, it can also be understood that a single sample is maintained (for example, the average of the samples in group 110 or a simple selection between the samples in group 110). Thus, a so-called reduced set 102 of sample values is obtained by a set 102 having only four samples (M red =2, N red =2, P red =M red +N red =4, P red <P).

[0106] It is understood that it is possible to perform operations (such as averaging or downsampling 100) without performing an excessive number of multiplications at the processor level: the averaging or downsampling 100 performed in step 811 can be easily obtained by simple and computationally non-power-consuming operations such as addition and shift.

[0107] At this point, it is understood that the reduced set of sample values ​​102 can be subjected to a linear or affine linear (ALWIP) transformation 19 (for example, using a prediction matrix such as matrix 17M in Figure 5). In this case, the ALWIP transformation 19 directly maps the four samples 102 to the sample values ​​104 of block 18. In this case, interpolation is not required.

[0108] In this case, the ALWIP matrix 17M is equal to the dimension Q × P red It has =16×4: This follows the fact that all Q=16 samples of block 18 to be predicted are directly obtained by ALWIP multiplication (interpolation is not required).

[0109] Therefore, in step 812a, dimension Q × P red A suitable ALWIP matrix 17M having A is selected. The selection may be based, for example, at least in part on signaling from data stream 12. The selected ALWIP matrix 17M is also A k This can be shown as follows, where k can be understood as an index that can be signaled in data stream 12 (in some cases, the matrix can also be

number

[0110] In step 812b, the selected Q × P red ALWIP matrix 17M(A k (Also shown as) and P red Multiplication by ×1 with the boundary vector 17P is performed.

[0111] In step 812c, the offset value (for example, b k ) can be added to all the acquired values ​​104 of the vector 18Q obtained by ALWIP, for example. Offset (b k , or in some cases

number

[0112] As can be understood, it is possible to obtain the appropriate value in step 812 by relying on simple and computationally inefficient operations such as averaging (and in some cases, addition and / or shifting and / or downsampling).

[0113] Referring to Figure 7.2, the block 18 to be predicted is an 8x8 block (M=8, N=8) with 64 samples. Here, a priori, the prediction matrix 17M should have a size Q×P=64×16 (Q=M*N=8*8=64, given by M=8 and N=8, and P=M+N=8+8=16, so Q=64). Therefore, a priori, for each of the Q=64 samples in the 8x8 block 18 to be predicted, P=16 multiplications are required, resulting in 64*16=1024 multiplications for the entire 8x8 block 18!

[0114] However, as can be seen from Figure 7.2, instead of using all 16 samples of the boundary, a method 820 can be provided in which only 8 values ​​(e.g., 4 from the horizontal boundary row 17c and 4 from the vertical boundary column 17a between the original samples of the boundary) are used. From the boundary row 17c, 4 samples may be used instead of 8 (e.g., they may be the mean of 2×2 and / or a selection of one of 2 samples). Thus the boundary vector is not a P×1=16×1 vector, but P red ×1 = 8 × 1 vector only (P red =M red +N red =4+4). Instead of the original P=16 samples, P red It is understood that it is possible to select or average (e.g., 2 × 2) samples from horizontal row 17c and vertical column 17a so that there are only 8 boundary values, thereby forming a reduced set 102 of sample values. This reduced set 102 makes it possible to obtain a reduced version of block 18, and the reduced version is Q (instead of Q = M * N = 8 * 8 = 64). red=M red *N red =4*4=16 samples. Size M red ×N red It is possible to apply the ALWIP matrix that predicts a 4x4 block. A reduced version of block 18 includes the sample shown in gray in scheme 106 in Figure 7.2: the sample shown in gray squares (including samples 118' and 118'') is the Q obtained in step 812. red = Forms a 4x4 reduced block with 16 values. The 4x4 reduced block is obtained by applying the linear transformation 19 in the relevant step 812. After obtaining the values ​​of the 4x4 reduced block, it is possible to obtain the values ​​of the remaining samples (samples shown as white samples in scheme 106) by interpolation, for example.

[0115] Regarding method 810 in Figure 7.1, method 820 is, for example, the remaining QQ of the M×N=8×8 block 18 to be predicted. red Step 813 may further include deriving the predicted values ​​for 64-16=48 samples (white squares) by interpolation. The remaining QQ red =64-16=48 samples were obtained directly by interpolation (interpolation can also utilize the values ​​of boundary samples, for example). red =16 samples can be obtained. As can be seen from Figure 7.2, samples 118' and 118'' are obtained in step 812 (as shown by the gray squares), while sample 108' (which is midway between samples 118' and 118'' and is shown by the white square) is obtained in step 813 by interpolation between samples 118' and 118''. It is understood that interpolation can also be obtained by operations similar to those for averaging, such as shift and addition. Therefore, in Figure 7.2, the value 108' can generally be determined as a value midway between the value of sample 118' and the value of sample 118'' (it can be the average).

[0116] By performing interpolation, it is also possible to reach the final version of the M×N = 8×8 block 18 based on the plurality of sample values shown in 104 in step 813. Therefore, comparing the use and non - use of this technology, it is as follows: When not using this technology: The block 18 to be predicted, which is a block with dimensions M = 8 and N = 8, and The Q = M*N = 8*8 = 64 samples of the block 18 to be predicted, The P = M + N = 8 + 8 = 16 samples of the boundary 17, P = 16 multiplications for each of the Q = 64 values to be predicted, The total number of multiplications of P*Q = 16*64 = 1028 The ratio of the number of multiplications to the number of final values obtained is P*Q / Q = 16 When using this technology: The block 18 to be predicted, which has dimensions M = 8 and N = 8 The last Q = M*N = 8*8 = 64 values to be predicted,

[0117] However, P red = M red + N red , Q red = M red * N red , M red = 4, N red = 4, for Q red ×P red The ALWIP matrix is used P red < P, the P of the boundary red = M red + N red = 4 + 4 = 8 samples <> The Q of the 4×4 reduced block to be predicted red = 16 multiplications for each of the P = 8 times for each of the 16 values (formed by the gray squares in scheme 106), red P P red * Q red = 8*16 = 128 multiplications in total (much less than 1024!)

[0118] The ratio of the number of multiplication times to the number of final values to be obtained is P red *Q red / Q = 128 / 64 = 2 (much less than 16 obtained without using this technology!). Therefore, the technology presented in this specification requires 8 times less power than the previous technology.

[0119] Figure 7.3 shows another example (which can be based on method 820), where the block 18 to be predicted is a rectangular 4×8 block (M = 8, N = 4) with Q = 4 * 8 = 32 samples to be predicted. The boundary 17 is formed by a horizontal row 17c of N = 8 samples and a vertical column 17a of M = 4 samples. Therefore, a priori, the boundary vector 17P has dimensions P×1 = 12×1, but the prediction ALWIP matrix should be a Q×P = 32×12 matrix, and thus Q * P = 32 * 12 = 384 multiplications are required.

[0120] However, for example, it is possible to average or downsample at least eight samples of the horizontal row 17c to obtain a reduced horizontal row of only four samples (e.g., averaged samples). In some examples, the vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimensions P red = 8, and P red < P. Therefore, the boundary vector 17P has dimensions P red ×1 = 8×1. The ALWIP prediction matrix 17M is a matrix with dimensions M * N red *P red = 4 * 4 * 8 = 64. The 4×4 reduced block directly obtained in the target step 812 (formed by the gray columns of the schema 107) has a size Q red = M * N red = 4 * 4 = 16 samples (instead of Q = 4 * 8 = 32 of the original 4×8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, the offset value b kAdd (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4×4 block is expanded to a 4×8 block 18, and the value 108' not obtained in step 812 is obtained in step 813 by interpolating the values 118' and 118'' (gray squares) obtained in step 812. Therefore, comparing using this technology with not using it, it is as follows: When not using this technology: The block 18 to be predicted, which is a block having dimensions M = 4 and N = 8 Q = M * N = 4 * 8 = 32 values to be predicted, P = M + N = 4 + 8 = 12 samples at the boundary, P = 12 multiplications for each of the Q = 32 values to be predicted, Total number of multiplications of P * Q = 12 * 32 = 384 The ratio of the number of multiplications to the number of final values obtained is P * Q / Q = 12 When using this technology: The block 18 to be predicted, which is a block having dimensions M = 4 and N = 8 The last Q = M * N = 4 * 8 = 32 values to be predicted,

[0121] However, M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8, for Q red ×P red = 16 × 8 The ALWIP matrix can be used P red < P, the P at the boundary red = M + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = For each of the 16 values, P red = 8 multiplications, Q red *P red= 16 * 8 = 128 total number of multiplications (less than 384!)

[0122] The ratio of the number of multiplications to the number of final values ​​to be obtained is P red *Q red / Q = 128 / 32 = 4 (much less than the 12 obtained without using this technique!). Therefore, this technology reduces computational effort by one-third.

[0123] Figure 7.4 shows an example of block 18 to be predicted, with dimensions M×N=16×16, where the last predicted object has Q=M*N=16*16=256 values ​​and P=M+N=16+16=32 boundary samples. This results in a prediction matrix with dimensions Q×P=256×32, which means 256*32=8192 multiplications!

[0124] However, by applying method 820, in step 811, it is possible to reduce the number of boundary samples from, for example, 32 to 8 (e.g., by averaging or downsampling), so that for every group 120 of four consecutive samples in row 17a, a single sample remains (e.g., selected from the four samples or the average of the samples). Similarly, for every group of four consecutive samples in column 17c, a single sample remains (e.g., selected from the four samples or the average of the samples).

[0125] Here, the ALWIP matrix 17M is Q red ×P red = 64 × 8 matrix: This is because it is P (by using 8 averaged samples or samples selected from 32 boundaries) red This is due to the fact that we selected =8, and that in step 812 the reduced block to be predicted is an 8x8 block (in scheme 109, the gray squares are 64).

[0126] Therefore, once 64 samples of the reduced 8x8 block are obtained in step 812, in step 813, the remaining QQ of the block 18 to be predicted red =256-64=192 values ​​can be derived from 104.

[0127] In this case, it is chosen to use all samples from boundary column 17a and only the alternative samples from boundary row 17c to perform interpolation. Other choices may be made.

[0128] In this method, the ratio between the number of multiplications and the number of values ​​ultimately obtained is Q. red *P red Q = 8 * 64 / 256 = 2, which is far less than 32 multiplications of each value without using this technique! Therefore, comparing the use of this technology with the non-use of it, the results are as follows: If this technology is not used: Block 18, the block to be predicted, has dimensions M=16 and N=16. The number of values ​​to be predicted is Q = M * N = 16 * 16 = 256. The boundary has P = M + N = 16 * 16 = 32 samples. For each of the 256 values ​​of the target Q to be predicted, P is multiplied 32 times. P*Q = 32 * 256 = 8192 total multiplications The ratio of the number of multiplications to the number of final values ​​obtained is P*Q / Q = 32. When using this technology: Block 18, the block to be predicted, has dimensions M=16 and N=16. The last set of prediction targets is Q = M * N = 16 * 16 = 256 values.

[0129] However, M red =4, N red =4, Q of the target of prediction by ALWIP red =8*8=64 samples, P red =M red +N red =4+4=8, Qred ×P red = 64 × 8 ALWIP matrix is used P red P which is the boundary, <P red = M red + N red = 4 + 4 = 8 samples Q of the reduced block to be predicted red = P for each of the 64 values red = 8 multiplications Q red * P red = 64 * 4 = 256 total multiplications (less than 8192!)

[0130] The ratio of the number of multiplications to the number of final values to be obtained is P red * Q red / Q = 8 * 64 / 256 = 2 (much less than 32 obtained without using this technology!). Therefore, the computational power required by this technology is 16 times less than that of the conventional technology! Therefore

[0131] Reduce a plurality of adjacent samples (100, 813), compare with a plurality of adjacent samples (17), and obtain a set of reduced sample values (102) with fewer sample numbers

[0132] Subject the set of reduced sample values (102) to a linear or affine linear transformation (19, 17M) to obtain predicted values of predetermined samples (104, 118’, 188’’) of a predetermined block (18) (812) By doing so, it is possible to predict a predetermined block (18) of a picture using a plurality of adjacent samples (17).

[0133] In particular, it is possible to perform reduction (100, 813) by downsampling a plurality of adjacent samples and obtaining a set of reduced sample values (102) with fewer sample numbers compared to the plurality of adjacent samples (17).

[0134] Alternatively, reduction (100, 813) can be performed by averaging multiple adjacent samples to obtain a reduced set (102) of sample values ​​with fewer samples compared to multiple adjacent samples (17).

[0135] Furthermore, based on the predicted values ​​of a given sample (104, 118', 118'') and multiple adjacent samples (17), it is possible to derive predicted values ​​for further samples (108, 108') of a given block (18) by interpolation (813).

[0136] Multiple adjacent samples (17a, 17c) may extend in one dimension along both sides of a given block (18) (for example, to the right and down in Figures 7.1 to 7.4). A given sample (for example, one acquired by ALWIP in step 812) may also be arranged in rows and columns, and along at least one of the rows and columns, a given sample may be positioned every n samples (112) of a given sample 112 adjacent to both sides of the given block 18.

[0137] Based on multiple adjacent samples (17), it is possible to determine the support value (118) for one of multiple adjacent positions (118) aligned to at least one of the rows and columns, for at least one of the rows and columns. It is also possible to derive the predicted values ​​118 for further samples (108, 108') of a given block (18) by interpolation based on the predicted values ​​of the given samples (104, 118', 118'') and the support values ​​of adjacent samples (118) aligned to at least one of the rows and columns.

[0138] A given sample (104) may be positioned every nth sample (112) adjacent to a given block 18 along the row, and the given sample may be positioned every mth sample (112) adjacent to a given sample (112) adjacent to a given block (18) along the column, where n, m > 1. In some cases, n = m (for example, in Figures 7.2 and 7.3, samples 104, 118', 118'', which are directly acquired by ALWIP at 812 and shown as gray squares, are alternated along the row and column with respect to samples 108, 108', which are subsequently acquired in step 813).

[0139] Along at least one of the rows (17c) and columns (17a), for example, it may be possible to determine the support value by downsampling or averaging (122) a group of adjacent samples (120) within a plurality of adjacent samples, including the adjacent sample (118) for which each support value is determined. Thus, in Figure 7.4, it is possible to obtain the value of sample 119 in step 813 by using the values ​​of a given sample 118''' (previously obtained in step 812) and adjacent sample 118 as support values.

[0140] Multiple adjacent samples may extend one-dimensionally along both sides of a given block (18). It may be possible to perform reduction (811) by grouping multiple adjacent samples (17) into one or more consecutive groups (110) of adjacent samples, and then performing downsampling or averaging for each of the one or more groups (110) of adjacent samples having two or more adjacent samples.

[0141] In the example, a linear or affine linear transformation is P red *Q red or P red *Q can include weighting coefficients, P red Q is the number of sample values ​​(102) in the reduced set of sample values. redAlternatively, Q is the number of a given sample in a given block (18). At least 1 / 4 of P red *Q red or 1 / 4 P red *Q weight coefficients are non-zero weight values. P red *Q red or P red *The Q weighting coefficient is Q or Q red For each of the specified samples, a series of P for each specified sample red Weighting coefficients may be included, and when a series of weighting coefficients are arranged vertically according to the raster scanning order between predetermined samples of a given block (18), they form an envelope that is nonlinear in all directions. red *Q or P red *Q red The weight coefficients may be independent of each other through a regular mapping rule. The average of the maximum cross-correlations between the weight coefficients of a first series associated with each given sample and the weight coefficients of a second series associated with all other given samples, or the inverted version of the latter series, is lower than a predetermined threshold, even though it yields a higher maximum value. The predetermined threshold can be 0.3 [or possibly 0.2 or 0.1]. red The adjacent samples (17) may be arranged along a one-dimensional path extending along both sides of a given block (18), and Q samples or Q red For each of the specified samples, a series of P for each specified sample red The weight coefficients are ordered so as to traverse the one-dimensional path in a predetermined direction.

[0142] 6.1 Description of Method and Apparatus width

number

number

Number

[0143] 1. Among the boundary samples 17, samples 102 (e.g., 4 samples when W = H = 4 and / or 8 samples in other cases) can be extracted by averaging or downsampling (e.g., step 811).

[0144] 2. Matrix-vector multiplication followed by addition of an offset can be performed with the averaged samples (or the samples remaining from downsampling) as input. The result can be a reduced prediction signal on a set of subsampled samples within the original block (e.g., step 812).

[0145] 3. The prediction signals at the remaining positions can be generated from the prediction signal on the subsampled set, for example, by upsampling and, for example, by linear interpolation (e.g., step 813).

[0146] Thanks to step 1. (811) and / or 3. (813), the total number of multiplications required for the calculation of the matrix-vector product is always

Number

[0147] In some examples, the matrix (e.g., 17M) and offset vector (e.g., b k ) required to generate the prediction signal can be a set of matrices (e.g., 3 sets) stored in, for example, the memory units of the decoder and encoder, e.g.,

Number

Number

[0148] In some examples, the set

Number

Number

Number

Number

Number

Number

Number

[0149] In some examples, the set

Number

Number

Number

Number

Number

Number

Number

number

number

number

number

[0150] Additionally or alternatively, set

number

number

number

number

number

Number

[0151] 6.2 Averaging or downsampling of boundaries Here, features are provided regarding step 811. As described above, the boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0152] In the first step, the input boundary

Number

Number

Number

Number

Number

Number

number

number

[0153] It is possible to define and similarly

number

number

number

number

number

[0154] In all other cases (for example, for Wither blocks with widths or heights different from 4), the block width W is W

number

number

number

number

[0155] In other cases, it is possible to downsample the boundary (for example, by selecting a specific boundary sample from a group of boundary samples) to reach a reduced number of samples. For example,

number

number

number

number

number

number

number

[0156] Two reduced boundaries

number

number

number

number

number

number

number

number

number

number

[0157] Therefore, a specific state (

number

number

[0158] Other strategies may be employed. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, each of the three sets S0, S1, and S2 does not necessarily have to have 18 matrices (therefore,

number

number

number

number

[0159] Mode and transpose information are not necessarily stored and / or transmitted as a single combined mode index "mode". In some examples, they may be explicitly signaled as a transpose flag and matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2).

[0160] In some cases, the combination of the transpose flag and the matrix index may be interpreted as a set index. For example, there may be one bit that acts as the transpose flag and several bits that represent the matrix index, which are collectively shown as a "set index".

[0161] 6.3 Generation of a reduced prediction signal using matrix-vector multiplication Features are provided here with respect to step 812. Reduced input vector

number

number

number

number

number

number

number

number

number

[0162] Reduced prediction signal

number

number

[0163] Here,

number

number

number

number

number

number

number

number

[0164] Matrix A and vector

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0165] Other strategies may be employed. In other examples, the mode index "mode" is not necessarily in the range of 0 to 35 (other ranges may be defined). Furthermore, each of the three sets S0, S1, and S2 does not necessarily have to have 18 matrices (therefore,

number

number

number

number

[0166] 6.4 Linear interpolation for generating the final predicted signal Features are provided here with respect to step 812. In interpolating subsampled prediction signals in large blocks, a second version of the averaged boundary may be required. That is,

number

number

number

number

number

number

number

number

[0167] Additionally or alternatively, "hard downsampling" may be used, where

number

number

[0168] Also,

number

number

number

number

[0169] Linear interpolation may be given as follows (other examples are also possible):

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0170] This is an example of interpolation that uses a reduced boundary sample for the first interpolation (horizontal or vertical) and the original boundary sample for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation or no interpolation at all may be required. If both horizontal and vertical interpolation are needed, the order depends on the width and height of the block. However, different techniques may be implemented, for example, the original boundary samples may be used for both the first and second interpolations, and the order may be fixed, for example, horizontal first, then vertical (otherwise, vertical first, then horizontal). Therefore, the interpolation order (horizontal / vertical) and the use of reduced / original boundary samples can be changed.

[0171] 6.5 Explanation of an example of the entire ALWIP process The entire process of averaging, matrix-vector multiplication, and linear interpolation is shown for different shapes in Figures 7.1 to 7.4. Note that the remaining shapes are treated as one of the illustrated examples.

[0172] 1.

number

number

number

[0173] 2.

number

number

number

[0174] 3.

number

number

number

[0175] 4.

number

number

number

number

[0176] Finally, for a W x 4 block where W > 8,

number

number

[0177] 6.6 Evaluation of the number and complexity of required parameters The parameters required for all possible proposed intra-prediction modes are set

number

number

number

[0178] 6.7 Proposed Intra Predictive Mode Signaling For the Luma Block, for example, 35 ALWIP modes have been proposed (other numbers of modes may be used). For each coding unit (CU) in the intra-mode, a flag indicating whether or not the ALWIP mode should be applied to the corresponding prediction unit (PU) is transmitted in the bitstream. The signaling of the latter index can be harmonized with the MRL in the same way as in the initial CE test. If an ALWIP mode is to be applied, the index of the ALWIP mode

number

[0179] Here, the derivation of the MPM may be performed using the intra-modes of the upper and left PUs, as follows: Each conventional intra-predictive mode

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0180] The embodiments described herein are not limited to the above-described signal transmission of the proposed intra-predictive mode. According to alternative embodiments, the MPM and / or mapping table are not used for MIP (ALWIP).

[0181] 6.8 Derivation of Adaptive MPM Lists for Conventional Luminetra Prediction Mode and Chrominetra Prediction Mode The proposed ALWIP mode can be harmonized with the conventional intra-predictive mode's MPM-based coding as follows: The conventional intra-predictive mode's luma and chroma MPM list derivation process is fixed table

number

number

number

number

[0182] In the case of Luma MPM list derivation, ALWIP mode

number

number

[0183] 7. Efficient Implementation Methods The above example is briefly summarized so as it can form the basis for further extending the embodiments described below. To predict a given block 18 of picture 10, multiple adjacent samples 17a, c are used.

[0184] Reduction 100 by averaging multiple adjacent samples is performed to obtain a reduced set of sample values ​​102 with a smaller sample size compared to multiple adjacent samples. This reduction is optional in the embodiments herein and results in a so-called sample value vector, which is referred to below. The reduced set of sample values ​​undergoes a linear or affine linear transformation 19 to obtain predictive values ​​for a given sample 104 of a given block. This transformation is later shown using a matrix A and an offset vector b, which should be obtained by machine learning (ML) and implemented efficiently.

[0185] Through interpolation, the predicted values ​​of another sample 108 in a given block are derived based on the predicted values ​​of the given sample and multiple neighboring samples. Theoretically, it should be said that the results of the affine / linear transformation can be associated with non-fullpel sample locations in block 18, such that all samples in block 18 can be obtained by interpolation according to an alternative embodiment. Interpolation may not even be necessary.

[0186] Multiple adjacent samples may extend one-dimensionally along both sides of a given block, and the given samples may be arranged in rows and columns, and along at least one of the rows and columns, and the given samples may be placed at every nth position from the sample (112) of the given samples adjacent to both sides of the given block. Based on the multiple adjacent samples, a support value for one of the multiple adjacent positions (118) can be determined for each of at least one of the rows and columns, which is aligned to each of at least one of the rows and columns, and by interpolation, a predicted value for a further sample 108 of the given block can be derived based on the predicted value for the given sample and the support value for the adjacent sample aligned to at least one of the rows and columns. The given samples may be placed along the rows at every nth position from the sample 112 of the given samples adjacent to both sides of the given block, and the given samples may be placed along the columns at every mth position from the sample 112 of the given samples adjacent to both sides of the given block, where n, m>1. n=m may also be the case. Along at least one of the rows and columns, the determination of support values ​​can be performed for each support value by averaging (122) a group of neighboring samples 120 within a plurality of neighboring samples, each containing the neighboring sample 118 for which the respective support value is determined. The plurality of neighboring samples may extend one-dimensionally along both sides of a given block, and the reduction may be performed by grouping the plurality of neighboring samples into one or more groups 110 of consecutive neighboring samples and performing averaging for each of the one or more groups of neighboring samples having three or more neighboring samples.

[0187] For a given block, the predicted residual can be transmitted within the data stream. This can be derived in the decoder, and the given block can be reconstructed using the predicted residual and predicted value for a given sample. In the encoder, the predicted residual is encoded into the data stream. A picture can be subdivided into multiple blocks of different block sizes, each containing a given block. Next, a linear or affine linear transformation of block 18 is selected based on the width W and height H of the given block. As a result, the linear or affine linear transformation selected for the given block is chosen from a first set of linear or affine linear transformations, provided that the width W and height H of the given block fall within a first set of width / height pairs. A second set of linear or affine linear transformations is selected, provided that the width W and height H of the given block fall within a second set of width / height pairs that differs from the first set. Similarly, it will later become clear that affine / linear transformations can be represented by other parameters, namely the weights of C, as well as optionally offset and scale parameters.

[0188] The decoder and encoder can be configured to subdivide the picture into multiple blocks of different block sizes, each containing a predetermined block, and to select a linear or affine linear transform depending on the width W and height H of the predetermined block, thereby determining the linear or affine linear transform selected for the predetermined block. As long as the width W and height H of a given block are within a first set of width / height pairs, the first set of linear or affine linear transformations As long as the width W and height H of a given block are within a second set of width / height pairs that is different from the first set of width / height pairs, the second set of linear or affine linear transformations, and A third set of linear or affine linear transformations is selected, provided that the width W and height H of a given block are within one or more third sets of width / height pairs that are different from the first and second sets of width / height pairs.

[0189] A third set of one or more width / height pairs simply contains one width / height pair W', H', and each linear or affine linear transformation in the first set of linear or affine linear transformations is for transforming N' sample values ​​into W'*H' predicted values ​​of a W'×H' array of sample positions. Each of the width / height pairs in the first and second sets is W p H p The first width / height is not equal to W. p H p And, H q =W p and W q =H p The second width / height is W q H q It can include this.

[0190] Each of the first and second sets of width / height pairs corresponds to the third width / height pair W p H p It can further include, W p is H p Equal to H p >H q That is the case. For a given block, a set index indicating which linear or affine linear transformation from a given set of linear or affine linear transformations should be selected for block 18 can be transmitted within the data stream.

[0191] Multiple adjacent samples can extend one-dimensionally along both sides of a given block, and reduction can be performed by grouping a first subset of multiple adjacent samples adjacent to a first side of a given block into a first group 110 of one or more consecutive adjacent samples, and a second subset of multiple adjacent samples adjacent to a second side of a given block into a second group 110 of one or more consecutive adjacent samples, and then performing averaging on each of the first and second groups of one or more adjacent samples having three or more adjacent samples in order to obtain a first sample value from the first group and a second sample value from the second group. Next, a linear or affine linear transformation can be selected according to a set index from a given set of linear or affine linear transformations, so that two different states of the set index result in a selection of one linear or affine linear transformation from a given set of linear or affine linear transformations, and in the case of a set index that assumes a first of two different states in the form of a first vector to yield an output vector of predicted values, the reduced set of sample values ​​can be subjected to a given linear or affine linear transformation, and the predicted values ​​of the output vector can be distributed to a given sample of a given block along a first scan order, and in the case of a set index that assumes a second of two different states in the form of a second vector, the first and second vectors are different such that one of the first sample values ​​of the first vector is input to one of the second sample of the second vector, and one of the second sample values ​​of the first vector is input to one of the first sample of the second vector, and the predicted values ​​of the output vector can be distributed along a second scan order to a given sample of a given block, transposed with respect to the first scan order, in order to generate an output vector of predicted values.

[0192] Each linear or affine linear transformation in a first set of linear or affine linear transformations may be for converting N1 sample values ​​into w1*h1 predicted values ​​for a w1×h1 array of sample positions, each linear or affine linear transformation in a first set of linear or affine linear transformations may be for converting N2 sample values ​​into w2*h2 predicted values ​​for a w2×h2 array of sample positions, wherein for one of the first predetermined width / height pairs, w1 may exceed the width of the first predetermined width / height pair, or h1 may exceed the height of the first predetermined width / height pair, and for one of the second predetermined width / height pairs, w1 may not exceed the width of the second predetermined width / height pair, and h1 may not exceed the height of the second predetermined width / height pair. Then, taking multiple adjacent samples (100) to obtain a reduced set of sample values ​​(102) by averaging may be done such that the reduced set of sample values ​​102 has N1 sample values ​​if the given block is of a first predetermined width / height pair, and if the given block is of a second predetermined width / height pair, subjecting the reduced set of sample values ​​to a selected linear or affine linear transformation may be done by using only the first sub-part of a selected linear or affine linear transformation relating to subsampling of a w1 × h1 array of sample positions along the width dimension if w1 exceeds the width of one width / height pair, or along the height dimension if h1 exceeds the height of one width / height pair, if the given block is of a second predetermined width / height pair, and the selected linear or affine linear transformation may be done in full if the given block is of a second predetermined width / height pair.

[0193] Each linear or affine linear transformation in the first set of linear or affine linear transformations can be used to transform N1 sample values ​​into w1*h1 predicted values ​​for a w1×h1 array of sample locations where w1=h1, and each linear or affine linear transformation in the second set of linear or affine linear transformations can be used to transform N2 sample values ​​into w2*h2 predicted values ​​for a w2×h2 array of sample locations where w2=h2.

[0194] All embodiments described above are merely illustrative in that they can form the basis for the embodiments described below herein. That is, the above concepts and details are helpful in understanding the embodiments below and serve as a reservoir for possible extensions and modifications of the embodiments described below herein. In particular, many of the details described above are optional, such as the averaging of adjacent samples and the fact that adjacent samples are used as reference samples.

[0195] More generally, the embodiments described herein assume that the prediction signal on a rectangular block is generated from already reconstructed samples, such that the intra-prediction signal on the rectangular block is generated from adjacent already reconstructed samples to the left and above the block. The generation of the prediction signal is based on the following steps:

[0196] 1. However, without ruling out the possibility of transferring the description to a reference sample located elsewhere, a sample can be extracted from a reference sample called a boundary sample by averaging. Here, averaging is performed on both the left and top boundary samples of the block, or on only one of the boundary samples on either side. If averaging is not performed on one side, the sample on that side remains unchanged.

[0197] 2. Matrix-vector multiplication is performed, followed optionally by offset addition, and the input vector for the matrix-vector multiplication is either the concatenation of the left-side averaged boundary sample of the block and the original boundary sample on top of the block if averaging is applied only to the left side, or the concatenation of the left-side original boundary sample of the block and the averaged boundary sample on top of the block if averaging is applied only to the top side, or the concatenation of the left-side averaged boundary sample of the block and the averaged boundary sample on top of the block if averaging is applied to both sides of the block. Again, alternative examples exist, such as those where averaging is not used at all.

[0198] 3. The results of matrix-vector multiplication and optional offset addition can optionally be a reduced prediction signal on a subsampled set of samples within the original block. The prediction signal at the remaining positions can be generated from the prediction signal on the subsampled set by linear interpolation.

[0199] The matrix-vector product calculation in step 2 should preferably be performed using integer arithmetic. Therefore,

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0200] Figure 8 shows an embodiment of the apparatus 1000 for predicting a given block 18 of a picture 10 using multiple reference samples 17. The multiple reference samples 17 may depend on the prediction mode used by the apparatus 1000 to predict the given block 18. If the prediction mode is, for example, intra-prediction, a reference sample 171 adjacent to the given block may be used. In other words, the multiple reference samples 17 are arranged within the picture 10, for example, along the outer edge of the given block 18. If the prediction mode is, for example, inter-prediction, a reference sample 172 of another picture 10' may be used.

[0201] The apparatus 1000 is configured to form a sample value vector 400 from a plurality of reference samples 17. The sample value vector can be obtained by different techniques. The sample value vector can, for example, include all of the reference samples 17. Optionally, the reference samples can be weighted. According to another example, the sample value vector 400 can be formed as described with respect to any of Figures 7.1 to 7.4 for the sample value vector 102. In other words, the sample value vector 400 can be formed by averaging or downsampling. Thus, for example, a group of reference samples can be averaged to obtain a sample value vector 400 having a reduced set of values. In other words, the device is configured to form a sample value vector 102 from multiple reference samples 17 by, for example, adopting one of multiple reference samples 17 as each component of the sample value vector 400, and / or averaging two or more components of the sample value vector 400, that is, by averaging two or more reference samples 17 to obtain each component of the sample value vector 400.

[0202] The apparatus 1000 is configured to derive a further vector 402 from a sample value vector 400, to which the sample value vector 400 is mapped by a predetermined reversible linear transformation 403. The further vector 402 contains, for example, only integer values ​​and / or fixed-point values. The reversible linear transformation 403 is selected, for example, so that the prediction of samples in a given block 18 is performed by integer arithmetic or fixed-point arithmetic.

[0203] Furthermore, the device 1000 is configured to calculate a matrix-vector product 404 between an additional vector 402 and a predetermined prediction matrix 405 to obtain a prediction vector 406, and to predict a sample of a predetermined block 18 based on the prediction vector 406. Based on the favorable additional vector 402, the predetermined prediction matrix can be quantized to enable integer and / or fixed-point arithmetic with only a small impact of quantization error on the predicted sample of the predetermined block 18.

[0204] According to one embodiment, the device 1000 is configured to calculate the matrix-vector product 404 using fixed-point arithmetic. Alternatively, integer arithmetic can be used. According to one embodiment, the apparatus 1000 is configured to calculate the matrix-vector product 404 without floating-point arithmetic.

[0205] According to one embodiment, the device 1000 is configured to store a fixed-point representation of a predetermined prediction matrix 405. Additionally or alternatively, an integer representation of the predetermined prediction matrix 405 may be stored. According to one embodiment, when predicting a sample of a given block 18 based on a prediction vector 406, the apparatus 1000 is configured to use interpolation to calculate at least one sample position of the given block 18 based on the prediction vector 406, with each component associated with a corresponding position within the given block 18. The interpolation can be performed as described with respect to any of the embodiments shown in Figures 7.1 to 7.4.

[0206] Figure 9 illustrates the concept of the invention described herein. Samples of a given block can be predicted based on a first matrix-vector product between a matrix A 1100 and a sample value vector 400, derived by some machine learning-based training algorithm. Optionally, an offset b 1110 can be added. To achieve an integer or fixed-point approximation of this first matrix-vector product, the sample value vector can undergo a reversible linear transformation 403 to determine a further vector 402. A second matrix-vector product between a further matrix B 1200 and a further vector 402 can be equal to the result of the first matrix-vector product.

[0207] Due to the further features of vector 402, the second matrix-vector product can be integer-approximated by the matrix-vector product 404 between a given prediction matrix C 405, the further vector 402, and the further offset 408. The further vector 402 and the further offset 408 can consist of integer or fixed-point values. All components of the further offset are, for example, the same. The given prediction matrix 405 may be a quantized matrix or a matrix that is quantized. The result of the matrix-vector product 404 between the given prediction matrix 405 and the further vector 402 can be understood as the prediction vector 406.

[0208] Further details regarding this integer approximation are provided below. Possible solutions according to Embodiment I: Subtraction and addition of average values Expressions usable in the above scenario

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0209] The specified value of 1400 is not necessarily the average value.

number

number

number

number

number

number

number

number

number

number

[0210] Alternatively, the predetermined value 1400 is the default value, or the value signaled in the data stream in which the picture is encoded. The predetermined value 1400 is, for example, 2 bitdepth-1 It is equal to this. In this case, the further vector 402 is y0=2 if i>0. bitdepth-1 and y i =x i It can be defined by -x0.

[0211] Alternatively, the predetermined component 1500 becomes a constant obtained by subtracting the predetermined value 1400. The constant is, for example, 2 bitdepth-1 It is equal to. According to the embodiment, a predetermined component of the further vector y 402

number

number

[0212] According to one embodiment, the apparatus 1000 is configured to include a plurality of reversible linear transformations 403, each associated with a component of a further vector 402. Furthermore, the apparatus is configured to, for example, select a predetermined component 1500 from the components of a sample value vector 400, and use a reversible linear transformation 403 from among the plurality of reversible linear transformations associated with that predetermined component 1500 as the predetermined reversible linear transformation. This is due, for example, to different positions of the i0th row, i.e., the row of the reversible linear transformation 403 corresponding to the predetermined component, depending on the position of the predetermined component in the further vector 402. For example, if the first component of the further vector 402, i.e., y1, is the predetermined component, then the i o The line replaces the first line of the invertible linear transformation.

[0213] As shown in Figure 10b, the column 412 of a predetermined prediction matrix 405 corresponding to a predetermined component 1500 of the further vector 402, i.e., the matrix component 414 of the predetermined prediction matrix C 405 in the i0th column, is, for example, all zero. In this case, the device is configured to calculate the matrix-vector product 404 by performing a multiplication by calculating the matrix-vector product 407 between the reduced prediction matrix C' 405 resulting from the predetermined prediction matrix C 405 by leaving column 412, and the further vector 410 resulting from the further vector 402 by leaving a predetermined component 1500, as shown in Figure 10c. Thus, the prediction vector 406 can be calculated with fewer multiplications.

[0214] As shown in Figures 9, 10b, and 10c, the device 1000 can be configured to calculate the sum of each component of the prediction vector 406 with a predetermined value 1400 when predicting a sample of a given block based on the prediction vector 406. This sum can be represented by the sum of the prediction vector 406 and vector 409, as shown in Figures 9 and 10c, where all components of vector 409 are equal to the predetermined value 1400. Alternatively, the sum can be represented by the sum of the prediction vector 406 and the matrix-vector product 1310 between the integer matrix M 1300 and a further vector 402, as shown in Figure 10b, where the matrix components of the integer matrix 1300 are 1 in the column of the integer matrix 1300 corresponding to a predetermined component 1500 of the further vector 402, i.e., the i0th column, and all other components are, for example, 0.

[0215] The sum of the predetermined prediction matrix 405 and the integer matrix 1300 is equal to or approximates, for example, a further matrix 1200 shown in Figure 9. In other words, the matrix obtained by summing each matrix component of the given prediction matrix C 405 in column 412, i.e., the i0th column, corresponding to a given component 1500 of the further vector 402, with 1 (i.e., matrix B) times the reversible linear transformation 403, i.e., the further matrix B 1200, corresponds, for example, to a quantized version of the machine learning prediction matrix A 1100, as shown in Figures 9, 10a, and 10b. Summarizing each matrix component of the given prediction matrix C 405 in column 412 with 1 can correspond to the sum of the given prediction matrix 405 and the integer matrix 1300, as shown in Figure 10b. As shown in Figure 9, the machine learning prediction matrix A 1100 can be equal to the result of multiplying the further matrix 1200 by the reversible linear transformation 403. This is,

number

[0216] Matrix multiplication using only integer arithmetic. For low-complexity implementations (in terms of the complexity of adding and multiplying scalar values, as well as the memory required for entries in the parting matrix), it is preferable to perform matrix multiplication 404 using only integer arithmetic.

number

number

number

number

number

[0217] A matrix-vector product 404 with a size m × n matrix, i.e., a given prediction matrix 405, can be performed as shown in this pseudocode, where <<,>> are arithmetic binary left and right shift operations, and +, -, and * are performed only on integer values.

[0218] (1) Final_Offset = 1 << (Right_Shift_Result - 1); Regarding i in 0...m-1 { Accumulator = 0 For j in 0...n-1 { Accumulator := Accumulator + y[j]*C[i,j] } z[i] = (accumulator + final offset) >> right shift result; }

[0219] Here, array C, i.e., a predetermined prediction matrix 405, stores fixed-point numbers, for example, as integers. The final addition of the last offset and the right shift operation by the result are rounded down to obtain the fixed-point format required for the output. To enable an increase in the range of real values ​​that can be represented by integers in C, two additional matrices are used, as shown in the embodiments of Figures 11 and 12.

number

number

number

number

number

number

number

number

number

[0220] In other words, the device 1000 is a predictive parameter, for example, an integer value.

number

number

number

[0221] According to the embodiment, the prediction parameters include weights, each associated with the corresponding matrix component of the prediction matrix. In other words, a given prediction matrix can be replaced by, for example, prediction parameters, or represented by prediction parameters. The weights are, for example, integers and / or fixed point values.

[0222] According to the embodiment, the prediction parameter is one or more scaling factors, for example, the value scale. i,jIt further includes, each of which is a weight associated with one or more corresponding matrix components of a given prediction matrix 405, e.g., an integer value.

number

number

number

number

[0223] For example, in one preferred embodiment,

number

number

number

number

[0224] According to the embodiment,

number

number

number

number

[0225] Offset representation

number

number

[0226] These solutions and their broader embodiments The above solution means the following embodiment: 1. A prediction method as described in Section I, wherein in step 2 of Section I, the following is performed for integer approximations of the matrix-vector product involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0227] 2. A prediction method as described in Section I, wherein in step 2 of Section I, the following is performed on the integer approximation of the matrix-vector product involved:

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

number

[0228] 3. A prediction method as described in Section I, wherein the matrix-vector product

number

number

number

[0229] 4. A prediction method as described in Section I, wherein step 2 uses one of K matrices so that multiple prediction modes can be calculated, each being a different matrix having k=0...K-1

number

number

number

number

[0230] In other words, according to embodiments of the present application, the encoder and decoder function as follows to predict a given block 18 of picture 10, with reference to Figure 9: Multiple reference samples are used for prediction. As outlined above, embodiments of the present application are not limited to intra-coding, and therefore the reference samples are not limited to adjacent samples, i.e., samples of adjacent blocks 18 of picture 10. In particular, the reference samples are not limited to those arranged along the outer edge of block 18, such as samples abutting the outer edge of the block. However, this situation is certainly one embodiment of the present application.

[0231] To make predictions, a sample value vector 400 is formed from reference samples such as reference samples 17a and 17c. Possible formations are described above. The formation may include averaging, which can reduce the number of samples 102 or the number of components in vector 400 compared to the reference samples 17 that contribute to the formation. The formation may also depend in some way on the dimensions or size of block 18, such as the width and height of block 18, as described above.

[0232] This vector 400 is the one that should undergo an affine or linear transformation to obtain the prediction for block 18. Different nomenclature is used above. The goal is to perform the prediction by applying vector 400 to matrix A by matrix-vector product, within the range of performing addition with offset vector b using the most recent method. The offset vector b is arbitrary. The affine or linear transformation determined by A or A and b can be determined for the prediction by the encoder and decoder, or more precisely, based on the size and dimensions of block 18 as already described above.

[0233] However, in order to achieve the computational efficiency improvements outlined above, or to make predictions more effective, the affine or linear transformations are quantized, and the encoder and decoder, or their predictor, use C and T applied in the manner described above to represent and perform the linear or affine transformation, using C and T which represent the quantized version of the affine transformation. In particular, the predictor in the encoder and decoder applies the transformation by subjecting the vector 402 obtained from the sample value vector 400 to mapping via a given reversible linear transformation T, rather than directly applying the vector 400 to matrix A. The transformation T used here is the same as long as the vector 400 has the same size, i.e., it is independent of the dimensions of the block, i.e., width and height, or at least the same for different affine / linear transformations. Above, the vector 402 is denoted by y. The exact matrix for performing the affine / linear transformation determined by machine learning was B. However, instead of performing B exactly, predictions in the encoder and decoder are made by its approximation or quantized version. In particular, the representation is done by appropriately representing C in the manner outlined above, where C+M represents the quantized version of B.

[0234] Therefore, prediction in the encoder and decoder is further performed by calculating the matrix-vector product 404 between vector 402 and a predetermined prediction matrix C appropriately represented and stored in the encoder and decoder in the manner described above. The vector 406 resulting from this matrix-vector product is then used to predict sample 104 of block 18. As described above, for prediction, each component of vector 406 can be added with parameter a, as shown in 408, to compensate for the corresponding definition of C. An optional addition of vector 406 with offset vector b can also be included in the derivation of the prediction for block 18 based on vector 406. As described above, each component of vector 406, and therefore each component of the sum of vector 406, all vectors a shown in 408, and an optional vector b directly correspond to sample 104 of block 18 and can therefore represent the predicted value of the sample. It is also possible that only a subset of sample 104 of the block is predicted in this way, and the remaining samples of block 18, such as 108, are derived by interpolation.

[0235] As mentioned above, there are different embodiments for setting a. For example, it may be the arithmetic mean of the components of vector 400. For that case, see Figure 10. The inverse linear transformation T can be as shown in Figure 10, where i0 are predetermined components of the sample value vector and vector 402, respectively, and are replaced by a. However, as also shown above, there are other possibilities. However, as far as the representation of C is concerned, it has also been shown above that it can be embodied differently. For example, the matrix-vector product 404 can, in its actual calculation, be the actual calculation of a smaller matrix-vector product having a lower dimension. In particular, as mentioned above, by the definition of C, the entirety of its i0th column 412 is 0, so the actual calculation of product 404 is the components

number

[0236] The weights of C or C', i.e., the components of this matrix, can be represented and stored in fixed-point representation. However, these weights 414 may also be stored in a manner related to different scales and / or offsets, as described above. The scales and offsets may be defined for the entire matrix C, i.e., equal for all weights 414 of matrix C or matrix C', or they may be defined to be constant or equal for all weights 414 in the same row or column of matrices C and C', respectively. In this regard, Figure 11 shows that the calculation of the matrix-vector product, i.e., the result of the product, can actually be performed in a slightly different way, i.e., by shifting the multiplication with the scale toward vector 402 or 404, thereby reducing the number of further multiplications that must be performed. Figure 12 shows the case where one scale and one offset are used for all weights 414 of C or C', as done in calculation (2) above.

[0237] According to embodiments, the apparatus described herein for predicting a given block of a picture can be configured to use matrix-based in-sample prediction, which includes the following features: The device is configured to form a sample value vector pTemp[x]400 from a plurality of reference samples 17. Assuming that pTemp[x] is 2*boundarySize, pTemp[x] can be populated, for example, by direct copying, or by subsampling or pooling, by redT[x] with the adjacent sample at the beginning of a given block, x=0..boundarySize-1, followed by redL[x] with the adjacent sample at the left of the given block, x=0..boundarySize-1 (e.g., if transpose=0), or vice versa in the case of transposition (e.g., if transpose=1).

[0238] An input value p[x] x=0..inSize-1 is derived, that is, the device is configured to derive a further vector p[x] from the sample value vector pTemp[x], to which the sample value vector pTemp[x] is mapped by a predetermined invertible linear transformation, or more specifically, a predetermined invertible affine linear transformation, as follows. -If mipSizeId is equal to 2, the following applies: p[x]=pTemp[x+1]-pTemp[0] -Otherwise (mipSizeId is less than 2), the following applies: p[0]=(1<<(BitDepth-1))-pTemp[0] p[x] = pTemp[x] - pTemp[0] If x = 1..inSize-1

[0239] Here, the variable mipSizeId indicates the size of a given block. That is, according to this embodiment, the reversible transformation used to derive further vectors from the sample value vector depends on the size of the given block. The dependency can be given as follows:

[0240] [Table 3]

[0241] predSize indicates the number of predicted samples in a given block, 2*boundarySize indicates the size of the sample value vector, and inSize relates to the size of a further vector according to inSize, i.e., inSize = (2*boundarySize) - (mipSizeId == 2) ? 1:0. More precisely, inSize indicates the number of components of the further vector that actually participate in the calculation. inSize is the same size as the size of the sample value vector for small block sizes, and one component is smaller for large block sizes. In the former case, one component, i.e., the component corresponding to a given component of the further vector, such as the matrix-vector product calculated later, may be ignored, and the contribution of the corresponding vector component will be 0 in any case and therefore does not need to be actually calculated. The dependency on block size can remain in the case where only one of the two alternative examples is necessarily used, i.e., the alternative embodiment is used regardless of block size (either the option corresponding to mipSizeId is less than 2, or the option corresponding to mipSizeId is equal to 2).

[0242] In other words, a given inverse linear transformation is defined such that, for example, a given component of a further vector p is a, and all other components correspond to the components of the sample value vector minus a, e.g., a = pTemp[0]. In the first option, corresponding to mipSizeId equal to 2, this appears straightforward, as only the separately formed components of the further vector are further considered. That is, in the first option, the further vector is actually {p[0...inSize];pTemp[0]}, where pTemp[0] is a, and the actually computed part of the matrix-vector multiplication to yield the matrix-vector product, i.e., the result of the multiplication, is limited to only the inSize components of the further vector and the corresponding columns of the matrix, since the matrix has a 0 column that does not require computation. In other cases, for mipSizeId less than 2, a=pTemp[0] is selected as all components of the further vector except p[0], i.e., each of the other components p[x] of the further vector p (for x=1..inSize-1) excluding the given component p[0] is equal to the corresponding component minus a of the sample value vector pTemp[x], but p[0] is selected to be the constant minus a. Then the matrix-vector product is calculated. The constant is the average of the representable values, i.e., 2 x-1 (i.e., 1 << (BitDepth-1)), where x indicates the bit depth of the computational representation used. Note that if p[0] is chosen to be pTemp[0] instead, the computed product deviates from the product computed using p[0] as above (p[0] = (1 << (BitDepth-1)) - pTemp[0]) by only a constant vector that can be considered when predicting the inside of a block based on the product, i.e., the prediction vector. Thus, the value a is a predetermined value, for example, pTemp[0]. The predetermined value pTemp[0] in this case is, for example, a component of the sample value vector pTemp corresponding to a predetermined component p[0]. It can be the sample closest to the upper left corner of a given block, adjacent to a given block above or to the left of a given block.

[0243] For example, in the case of an in-sample prediction process involving predModeIntra, such as specifying an intra-prediction mode, the instrument is configured to apply the following steps, for example, by performing at least the first step: 1. The matrix-based intra-prediction sample predMip[x][y] where x=0..predSize-1 and y=0..predSize-1 is derived as follows: - The variable modeId is set to be equal to predModeIntra. The weight matrix mWeight[x][y] with -x=0..inSize-1 and y=0..predSize*predSize-1 is derived by calling the MIP weight matrix derivation process with mipSizeId and modeId as inputs. The matrix-based intra-prediction sample predMip[x][y] with -x=0..predSize-1 and y=0..predSize-1 is derived as follows: oW=32-32*(

number

number

[0244] In other words, the device is configured to compute a matrix-vector product between further vectors p[i] to obtain prediction vectors already assigned to an array of block positions {x,y} distributed inside a given block to yield the array predMip[x][y], or, if mipSizeId is equal to 2, {p[i];pTemp[0]} and a given prediction matrix mWeight, or, if mipSizeId is less than 2, a prediction matrix mWeight having additional zero weights corresponding to the omitted components of p. The prediction vectors correspond to the concatenation of rows or columns of predMip[x][y], respectively.

[0245] According to the embodiment, or according to a different interpretation, the component (((

number

[0246] The device optionally performs the following steps when predicting a sample of a given block based on the prediction vector, for example, predMip or ((

number

[0247] 2. A matrix-based intra-prediction sample predMip[x][y] where x=0..predSize-1 and y=0..predSize-1 can be clipped as follows, for example: predMip[x][y]=Clip1(predMip[x][y])

[0248] 3. If the transpose is true, the predSize × predSize array predMip[x][y] where x = 0..predSize-1 and y = 0..predSize-1 is transposed as follows, for example: predTemp[y][x]=predMip[x][y] predMip=predTemp

[0249] 4. Predicted samples predSamples[x][y] where x=0..nTbW-1 and y=0..nTbH-1 can be derived, for example, as follows: -If nTbW, which specifies the transformed block width, is greater than predSize, or nTbH, which specifies the transformed block height, is greater than predSize, the MIP predictive upsampling process is invoked with input block size predSize, x=0..predSize-1, y=0..predSize-1, inputs a matrix-based intra predictive sample predMip[x][y], inputs a transformed block width nTbW, transformed block height nTbH, upper reference sample refT[x] with x=0..nTbW-1, and left reference sample refL[y] with y=0..nTbH-1, and outputs a predictive sample array predSamples. -Otherwise, predSamples[x][y] where x=0..nTbW-1 and y=0..nTbH-1 is set to be equal to predMip[x][y]. In other words, the device is configured to predict the samples predSamples of a given block based on the prediction vector predMip.

[0250] Figure 13 shows a method 2000 for predicting a given block of a picture using multiple reference samples, the method comprising: forming a sample value vector from the multiple reference samples 2100; deriving a further vector from the sample value vector to which the sample value vector is mapped by a predetermined inverse linear transformation 2200; calculating a matrix-vector product between the further vector and a predetermined prediction matrix to obtain a prediction vector 2300; and predicting a sample of the given block based on the prediction vector 2400.

[0251] References [1] P. Helle et al., "Non-linear weighted intraprediction", JVET-L0199, Macau, China, October 2018. [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, "JVET common test conditions and software reference configurations for SDR video", JVET-K1010, Ljubljana, Slovenia, July 2018.

[0252] Further embodiments and examples Generally, an example can be implemented as a computer program product comprising program instructions, which, when the computer program product is executed on a computer, function to perform one of the methods. The program instructions may be stored, for example, in a machine-readable medium. Another example comprises a computer program stored in a machine-readable carrier for performing one of the methods described herein.

[0253] Therefore, an example of the method is a computer program that, when executed on a computer, has program instructions for performing one of the methods described herein.

[0254] Therefore, a further example of the method is a data carrier medium (or digital storage medium, or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier medium, digital storage medium, or recording medium is tangible and / or non-temporary, rather than intangible and transient signals. Therefore, a further example of the method is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals can be transmitted, for example, over a data communication connection such as the Internet.

[0255] Further examples include processing means, such as a computer or a programmable logic device, that perform one of the methods described herein. A further example includes a computer on which a computer program for performing one of the methods described herein is installed.

[0256] Further examples include an apparatus or system for transferring (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transferring the computer program to the receiver.

[0257] In some examples, programmable logic devices (e.g., field-programmable gate arrays) can be used to perform some or all of the functions of the method herein. In some examples, a field-programmable gate array can work with a microprocessor to perform one of the methods herein. In general, the method may be performed by any suitable hardware device.

[0258] The examples described above are merely illustrative of the principles described above. It will be understood that modifications and variations of the configurations and details described herein will be obvious. Therefore, it is intended that these are not limited by the specific details presented as part of the examples and descriptions herein, but rather by the imminent claims. Equal or equivalent elements, or elements having equal or equivalent functions, are indicated in the following description by equal or equivalent reference numerals, even if they occur in different figures.

Claims

1. A device for predicting a given block of a picture using multiple reference samples, A sample value vector (pTemp[x]) is formed from the above-mentioned multiple reference samples. The size index (mipSizeId), which is an indication of the size of the predetermined block, is obtained. Based on the size index (mipSizeId), the input vector (p[x]) to which the sample value vector (pTemp[x]) is mapped by a predetermined inverse linear transformation is derived. Using fixed-point arithmetic, the matrix-vector product between the input vector (p[x]) and a predetermined prediction matrix (mWeight[x][y]) is calculated to obtain a prediction array (predMip[x][y]). A device configured to predict samples of a predetermined block based on the prediction array (predMip[x][y]).

2. The aforementioned input vector (p[x]) is, If the aforementioned size index (mipSizeId) is equal to 2, p[x]=pTemp[x+1]−pTemp[0] Derived according to, If the aforementioned size index (mipSizeId) is less than 2, p[x]=pTemp[x]−pTemp[0], Derived according to, Here, x = 1, 2, ..., in Size - 1, inSize is the number of components of the input vector (p[x]) that are involved in the derivation of the input vector (p[x]). The apparatus according to feature 1.

3. The aforementioned prediction array (predMip[x][y]) is [Math 1] Derived according to, Here, predSize indicates the number of predicted samples within the predetermined block. inSize is the number of components in the input vector (p[x]). The apparatus according to feature 1.

4. The apparatus according to claim 1, configured to calculate the matrix-vector product without floating-point arithmetic.

5. The apparatus according to claim 1, configured to store a fixed-point representation of the predetermined prediction matrix (mWeight[x][y]).

6. The predetermined prediction matrix (mWeight[x][y]) is represented using the prediction parameters, The system is configured to calculate the matrix-vector product by performing multiplication and addition on the components of the input vector (p[x]), the prediction parameters, and the intermediate results arising therefrom. The apparatus according to claim 1, wherein the absolute value of the prediction parameter can be represented by an n-bit fixed-point number representation where n is 14 or less, or 10 or less, or 8 or less.

7. The apparatus according to claim 1, wherein the plurality of reference samples are arranged within the picture along the outer edge of the predetermined block.

8. (Independent Item - Method) A method for predicting blockiness in a picture using multiple reference samples, Obtaining one or more sample values ​​(pTemp[x]) from multiple adjacent samples arranged along the boundary of the block, Based on the block size index (mipSizeId), the input value (p[x]) is determined from one or more sample values ​​(pTemp[x]), The process involves applying a predetermined prediction matrix (mWeight) to the determined input value (p[x]) to determine a matrix-based intra-prediction sample (predMip[x][y]), Predicting the sample of the block based on the determined matrix-based intra-predicted sample (predMip[x][y]), In predicting the aforementioned sample, interpolation is used to calculate at least one sample position in the block based on the matrix-based intra-predicted sample (predMip[x][y]), wherein each component of the matrix-based intra-predicted sample is associated with a corresponding position in the block. Methods that include...

9. Determining the input value (p[x]) based on the size index (mipSizeId) of the block is: If the size index (mipSizeId) is equal to 2, p[x] is determined from the one or more sample values ​​(pTemp[x]) according to p[x] = pTemp[x+1] - pTemp[0], If the size index (mipSizeId) is less than 2, the p[x] is determined according to p[x] = pTemp[x] - pTemp[0], and this includes the following: Here, x = 1, 2, ..., in Size - 1, inSize is the number of components of the input value (p[x]) that are involved in the derivation of the input value (p[x]). The method according to feature 8.

10. Determining the matrix-based intra-predicted sample (predMip[x][y]) by applying the predetermined prediction matrix (mWeight) is: [Math 2] This includes deriving according to, Here, predSize indicates the number of predicted samples within the block. pTemp[0] includes the sample values ​​of one or more sample values ​​(pTemp[x]), inSize corresponds to the number of components in the input value (p[x]), The method according to claim 8.

11. The method according to claim 8, wherein the plurality of adjacent samples are arranged along the upper and left boundaries of the block.

12. Forming the aforementioned sample value (pTemp[x]) means that for each component of the aforementioned sample value (pTemp[x]), One of the aforementioned multiple reference samples is selected as one of the components of the sample value (pTemp[x]), and / or Averaging two or more reference samples to obtain each component of the sample value (pTemp[x]), The method according to claim 8, including the method described in claim 8.

13. A method for encoding a picture, comprising: obtaining a prediction signal using the method of claim 8; and encoding the prediction residual of the block to correct the prediction signal.

14. A method for decoding a picture, comprising: obtaining a prediction signal using the method of claim 8; decoding the prediction residual of the block; and correcting the prediction signal using the prediction residual.

15. An apparatus for encoding a picture, comprising: an apparatus for predicting a predetermined block of the picture according to claim 1 in order to acquire a prediction signal; and an entropy encoder configured to encode the prediction residual of the predetermined block in order to correct the prediction signal.

16. A device for decoding a picture, comprising: a device for predicting a predetermined block of the picture as described in claim 1 in order to acquire a prediction signal; an entropy decoder configured to decode the prediction residual of the predetermined block; and a prediction corrector configured to correct the prediction signal using the prediction residual.

17. A computer program having program code for performing the method described in claim 8 when executed on a computer.

18. A computer-readable storage medium storing a computer program having program code for performing the method described in claim 8 when executed on a computer.

Images