Intra prediction using linear or affine transformations with adjacent sample reduction

Abstract

To provide a method and an apparatus for performing intra prediction and optimizing video delivery (for example, broadcast, streaming, file playback) for video and / or virtual reality applications.SOLUTION: A decoder or encoder predicts a given block of a picture by using a plurality of adjacent samples 17 to reduce a plurality of adjacent samples 17 to obtain a reduced set of sample values having a smaller number of samples compared to the plurality of adjacent samples, and subjecting the reduced set of sample values to a linear or affine-linear transformation to obtain a predicted value of the given sample of the given block 18.SELECTED DRAWING: Figure 7.1

Description

Technical Field

[0001] 1 Introduction The following describes different embodiments, implementations, and aspects of the present invention. At least some of these examples, implementations, and aspects relate, inter alia, to methods and / or apparatuses for video coding and / or for performing intra prediction using, for example, linear or affine transforms with adjacent sample reduction and / or for optimizing video delivery (e.g., broadcast, streaming, file playback, etc.) for, for example, video applications and / or virtual reality applications. Further, the examples, implementations, and aspects relate to High Efficiency Video Coding (HEVC) or successors. Additionally, further embodiments, examples, and aspects are defined by the appended claims.

Background Art

[0002] Note that any embodiment, example, and aspect defined by the claims can be supplemented by any of the details (features and functions) described in the following chapters.

[0003] Also, the embodiments, examples, and aspects described in the following chapters can be used individually and can also be supplemented by any of the features of another chapter or any feature included in the claims.

[0004] Note also that the individual examples, embodiments, and aspects described herein can be used individually or in combination. Thus, details can be added to each of the examples, embodiments, and individual aspects without adding details to another one of the aspects. Note that the present disclosure also explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0005] Furthermore, the features and functions disclosed herein in connection with the method can also be used in an apparatus. Further, any features and functions disclosed herein in connection with the apparatus can also be used in a corresponding method. In other words, the methods disclosed herein can be complemented by any of the features and functions described in connection with the apparatus. Also, any of the features and functions described herein can be implemented in hardware or software, or a combination of hardware and software, as described in other sections such as "Further Embodiments and Examples".

[0006] Furthermore, any of the features described within parentheses ("(...) or "[...]") may be considered optional in some examples, embodiments, or aspects.

Summary of the Invention

Problems to be Solved by the Invention

[0007]

Means for Solving the Problems

[0008] 1.1 Overview According to one aspect, a decoder for decoding a picture from a data stream, using a plurality of adjacent samples to

[0009] reduce the plurality of adjacent samples (e.g., by averaging or downsampling) to obtain a set of sample values with fewer samples as compared to the plurality of adjacent samples, and subject the set of reduced sample values to a linear or affine linear transformation to obtain a predicted value for a predetermined sample of a predetermined block thereby providing a decoder configured to predict a predetermined block of a picture.

[0010] In an example, the decoder may be further configured to perform, for example, averaging of downsampling of a plurality of adjacent samples to obtain a reduced set of sample values with fewer samples as compared to the plurality of adjacent samples.

[0011] In some cases, the decoder may also derive predicted values of additional samples of a given block based on a given sample and predicted values of a plurality of adjacent samples, for example, by interpolation. Accordingly, an upsampling operation may be applied. According to one aspect, an encoder for encoding a picture from a data stream, using a plurality of adjacent samples,

[0012] reducing the plurality of adjacent samples (e.g., by averaging or downsampling) to obtain a reduced set of sample values with fewer samples as compared to the plurality of adjacent samples, and subjecting the reduced set of sample values to a linear or affine linear transformation to obtain predicted values of given samples of a given block thereby providing an encoder configured to predict a given block of a picture.

[0013] In an example, the encoder may be further configured to perform the reduction by downsampling the plurality of adjacent samples to obtain a reduced set of sample values with fewer samples as compared to the plurality of adjacent samples.

[0014] In some cases, the encoder may also derive predicted values of additional samples of a given block based on a given sample and predicted values of a plurality of adjacent samples, for example, by interpolation. Accordingly, an upsampling operation may be applied.

[0015] In an example, a system comprising an encoder as described above and / or a decoder as described above may be provided. In some examples, the hardware of the encoder and / or at least some of the procedural routines may be the same as those of the decoder.

[0016] In an embodiment, in order to obtain a reduced set of sample values with fewer samples as compared to a plurality of adjacent samples, a plurality of adjacent samples are reduced, for example, by downsampling or averaging, to predict a predetermined block of a picture using the plurality of adjacent samples, and to subject the reduced set of sample values to a linear or affine-linear transformation to obtain a predicted value for a predetermined sample of a predetermined block A decoding method may be provided that includes the above.

[0017] In an embodiment, in order to obtain a reduced set of sample values with fewer samples as compared to a plurality of adjacent samples, a plurality of adjacent samples are reduced, for example, by downsampling or averaging, to predict a predetermined block of a picture using the plurality of adjacent samples, and to subject the reduced set of sample values to a linear or affine-linear transformation to obtain a predicted value for a predetermined sample of a predetermined block An encoding method may be provided that includes the above. In an embodiment, a non-transitory storage unit that stores instructions for causing a processor to execute the above method when executed by the processor may be provided. 1.3 Drawings

Brief Description of the Drawings

[0018]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7.1

Figure 7.2

Figure 7.3

Figure 7.4

Figure 8.1

Figure 8.2

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Best Mode for Carrying Out the Invention

[0019] Two encoders, decoders In the following, various examples that may assist in achieving more effective compression when using intra prediction are described. Some examples achieve an improvement in compression efficiency by consuming a set of intra prediction modes. The latter may be added to other intra prediction modes designed heuristically, for example, or may be provided exclusively. Also, in still other examples, both of the aforementioned technical fields are utilized.

[0020] To facilitate understanding of the following examples of this application, the description begins with the presentation of a possible encoder and a decoder adapted thereto that can construct the examples of this application outlined subsequently. FIG. 1 shows an apparatus for encoding picture 10 into data stream 12 in block units. The apparatus is indicated using reference numeral 14 and may be a still image encoder or a video encoder. In other words, if encoder 14 is configured to encode video 16 including picture 10 into data stream 12, picture 10 may be the current picture of video 16, or encoder 14 may encode picture 10 into data stream 12 exclusively.

[0021] As described above, encoder 14 performs encoding in block units or block-based. For this, encoder 14 sub-divides picture 10 into blocks, and in units of blocks, encoder 14 encodes picture 10 into data stream 12. Examples of possible sub-divisions of picture 10 into blocks 18 are described in more detail below. Generally, the sub-division can end up with blocks 18 of a fixed size, such as an array of blocks arranged in rows and columns, or blocks 18 of different block sizes, such as by using a hierarchical multi-tree sub-division that starts from the entire picture area of picture 10 or from a pre-division of picture 10 into an array of tree blocks, to a multi-tree sub-division. These examples are not treated as excluding other possible ways of sub-dividing picture 10 into blocks 18.

[0022] Furthermore, encoder 14 is a predictive encoder configured to predictively encode picture 10 into data stream 12. In the case of a particular block 18, this means that encoder 14 determines a prediction signal for block 18 and encodes the prediction residual, i.e., the prediction error by which the prediction signal deviates from the actual picture content within block 18, into data stream 12.

[0023] Encoder 14 may support different prediction modes to derive the prediction signal of a certain block 18. The prediction mode that is important in the following example is the intra prediction mode, according to which the inside of block 18 is spatially predicted from adjacent already encoded samples of picture 10. The encoding of the data stream 12 of picture 10, and thus the corresponding decoding procedure, can be based on a specific encoding order 20 defined among blocks 18. For example, the encoding order 20 can traverse the blocks 18 in a raster scan order such as from top to bottom in row units while traversing each row from left to right. In the case of hierarchical multi-tree based sub-division, the raster scan ordering may be applied within each hierarchical level, or a depth-first traversal order may be applied, i.e., the leaf nodes within a block of a specific hierarchical level may precede the blocks of the same hierarchical level having the same parent block according to the encoding order 20. Depending on the encoding order 20, the adjacent already encoded samples of block 18 can usually be located on one or more sides of block 18. In the case of the example presented herein, for example, the adjacent already encoded samples of block 18 are located above and to the left of block 18.

[0024] The intra prediction mode may not be the only mode supported by encoder 14. For example, if encoder 14 is a video encoder, encoder 14 may also support an inter prediction mode in which block 18 is temporarily predicted from previously encoded pictures of video 16. Such an inter prediction mode may be a motion compensated prediction mode, according to which motion vectors are signaled for, e.g., block 18 indicating the relative spatial offset of the part where the prediction signal of block 18 is derived as a copy. Additionally or alternatively, other non-intra prediction modes may be available, together with the inter prediction mode in the case where encoder 14 is a multi-view encoder, or a non-prediction mode in which the inside of block 18 is encoded as is, i.e., without prediction.

[0025] Before beginning to focus the description of this application on the intra prediction mode, as described with respect to FIG. 2, a more specific example of a possible block-based encoder, i.e., a possible implementation of encoder 14, is shown, and then two corresponding examples of decoders that conform to FIGS. 1 and 2, respectively, are shown.

[0026] FIG. 2 shows a possible implementation of the encoder 14 of FIG. 1, i.e., the encoder is configured to use transform coding to encode the prediction residual, which is merely an example, and the present application is not limited to that kind of prediction residual coding. According to FIG. 2, the encoder 14 subtracts a corresponding prediction signal 24 from an inbound signal, i.e., picture 10, or the current block 18 on a block-by-block basis, to obtain a prediction residual signal 26, which is then provided with a subtractor 22 configured to be encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 is composed of an irreversible coding stage 28a and a reversible coding stage 28b. The irreversible stage 28a receives the prediction residual signal 26 and includes a quantizer 30 that quantizes the samples of the prediction residual signal 26. As already described above, this example uses transform coding of the prediction residual signal 26, and thus the irreversible coding stage 28a includes a transform stage 32 connected between the subtractor 22 and the quantizer 30, and the quantization of the quantizer 30 performed on the transformed coefficients presenting the residual signal 26 transforms the spectrally decomposed prediction residual 26. The transform may be a DCT, DST, FFT, Hadamard transform, or the like. The transformed and quantized prediction residual signal 34 is then subjected to reversible coding by the reversible coding stage 28b, which is the entropy-coded quantization prediction residual signal 34 of the entropy encoder, to become the data stream 12. The encoder 14 further includes a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30, whereby the prediction residual signal is reconstructed from the transformed and quantized prediction residual signal 34 in a manner available to the decoder, i.e., considering the coding loss in the quantizer 30. For this purpose, the prediction residual reconstruction stage 36 includes an inverse quantizer 38, which performs the inverse of the quantization of the quantizer 30, followed by an inverse transformer 40, which performs an inverse transform with respect to the transform performed by the transformer 32, such as the inverse of the spectral decomposition for any of the specific transform examples described above. The encoder 14 includes an adder 42 that adds the reconstructed prediction residual signal output by the inverse transformer 40 and the prediction signal 24 to output a reconstructed signal, i.e., a reconstructed sample. This output is supplied to the predictor 44 of the encoder 14, and the predictor determines the prediction signal 24 based thereon.This is the predictor 44 that supports all prediction modes already described above with respect to FIG. 1. FIG. 2 also shows the case where the encoder 14 is a video encoder, and the encoder 14 may also include an in-loop filter 46 that filters the fully reconstructed picture to form a reference picture for the predictor 44 regarding the inter prediction block after filtering.

[0027] As already described above, the encoder 14 operates in a block-based manner. For the following description, the block of interest is to sub-divide the picture 10 into blocks, and for each block, an intra prediction mode is selected from a set or plurality of intra prediction modes supported by the predictor 44 or the encoder 14 respectively, and the selected intra prediction mode is executed individually. However, there may be other types of blocks into which the picture 10 is sub-divided. For example, the above determination of whether the picture 10 is inter-coded or intra-coded can be made at a granularity or in units of blocks deviating from the block 18. For example, the inter / intra mode decision may be performed at the level of the coding blocks where the picture 10 is sub-divided and each coding block is sub-divided into prediction blocks. Prediction blocks having coding blocks determined to use intra prediction are each sub-divided for an intra prediction mode decision. In contrast, for each of these prediction blocks, it is determined which supported intra prediction mode should be used for each prediction block. These prediction blocks here form the block 18 of interest. Prediction blocks within the coding blocks associated with inter prediction are processed differently by the predictor 44. They are inter-predicted from the reference picture by determining a motion vector and replicating the prediction signal of this block from the position within the reference picture indicated by the motion vector. Another block sub-division relates to the sub-division into transform blocks at the unit where the transformation by the transformer 32 and the inverse transformer 40 is performed. The transformed block can be, for example, the result of further sub-dividing the coding block. Of course, the embodiments described herein should not be treated as limiting, and there are other embodiments. For the sake of completeness, the sub-division into coding blocks may use, for example, multi-tree sub-division, and the prediction blocks and / or the transform blocks may also be obtained by further sub-dividing the coding blocks using multi-tree sub-division.

[0028] A decoder 54 or apparatus for block-based decoding adapted to the encoder 14 of FIG. 1 is shown in FIG. 3. This decoder 54 performs the opposite of the encoder 14, that is, it decodes from the data stream 12 picture 10 in block units, and for this purpose, supports a plurality of intra prediction modes. The decoder 54 may comprise, for example, a residual provider 156. All other possibilities described above with respect to FIG. 1 are also valid for the decoder 54. In contrast, the decoder 54 may be a still image decoder or a video decoder, and all prediction modes and predictabilities are also supported by the decoder 54. The difference between the encoder 14 and the decoder 54 lies mainly in the fact that the encoder 14 selects encoding decisions according to some optimization in order to minimize some cost function that may depend on, for example, the coding rate and / or coding distortion. One of these coding options or coding parameters may include the selection of the intra prediction mode used for the current block 18 from among the available or supported intra prediction modes. The selected intra prediction mode may then be signaled by the encoder 14 of the current block 18 in the data stream 12 by the decoder 54 re-executing the selection using this signaling within the data stream 12 of the block 18. Similarly, the sub-division of the picture 10 into blocks 18 may be subject to optimization within the encoder 14, and the corresponding sub-division information may be transmitted within the data stream 12 by the decoder 54 restoring the sub-division of the picture 10 into blocks 18 based on the sub-division information. Summarizing the above, the decoder 54 may be a prediction decoder operating in block-based manner, and in addition to the intra prediction mode, the decoder 54 may support other prediction modes such as the inter prediction mode, for example, if the decoder 54 is a video decoder. In decoding, the decoder 54 may also use the coding order 20 described with respect to FIG. 1, and since this coding order 20 is observed by both the encoder 14 and the decoder 54, the same adjacent samples are available for the current block 18 in both the encoder 14 and the decoder 54.Therefore, to avoid unnecessary repetition, the description of the operating mode of encoder 14 shall also apply to decoder 54 as far as the sub-division of picture 10 into blocks is concerned, for example, as far as prediction is concerned, and as far as the coding of prediction residuals is concerned. The difference is that encoder 14, by optimization, selects some coding options or coding parameters and signals, or inserts coding parameters into data stream 12, which are then derived from data stream 12 for decoder 54 to re-execute prediction, sub-division, etc.

[0029] Figure 4 shows a possible implementation of decoder 54 of Figure 3, i.e., an implementation that conforms to the implementation of encoder 14 of Figure 1 shown in Figure 2. Since many elements of encoder 54 in Figure 4 are the same as those occurring in the corresponding encoder of Figure 2, the same reference signs with apostrophes are used in Figure 4 to denote these elements. In particular, adder 42’, optional in-loop filter 46’ and predictor 44’ are connected to the prediction loop in the same way as the encoder of Figure 2. The reconstructed, i.e., inverse quantized and inverse transformed, prediction residual signal applied to adder 42’ is derived by a sequence of entropy decoder 56 that reverses the entropy coding of entropy encoder 28b, followed by a residual signal reconstruction stage 36’ composed of inverse quantizer 38’ and inverse transformer 40’, as in the case of the coding side. The output of the decoder is the reconstruction of picture 10. The reconstruction of picture 10 may be directly available at the output of adder 42’ or may be available at the output of in-loop filter 46’. To subject the reconstruction of picture 10 to some post-filtering to improve picture quality, some post-filters may be placed at the output of the decoder, but this option is not shown in Figure 4.

[0030] Here too, with respect to FIG. 4, the description given above with respect to FIG. 2 is equally valid for FIG. 4, except that only the encoder performs the relevant decisions regarding the optimization task and the coding options. However, all the descriptions regarding block sub-division, prediction, inverse quantization, and re-conversion are also valid for the decoder 54 of FIG. 4. 3 ALWIP

[0031] Some non-limiting examples regarding ALWIP are described herein, even if ALWIP is not necessarily required to implement the techniques described herein.

[0032] This application relates, inter alia, to an improved intra prediction mode concept for block-based picture coding, such as being usable in a video codec such as HEVC or any successor of HEVC.

[0033] Intra prediction modes are widely used in picture and video coding. In video coding, intra prediction modes compete with other prediction modes such as inter prediction modes like motion compensated prediction mode. In an intra prediction mode, the current block is predicted based on adjacent samples, i.e., samples that have already been encoded on the encoder side and already decoded on the decoder side. The adjacent sample values are extrapolated to the current block to form the prediction signal for the current block, and the prediction residual is transmitted in the data stream of the current block. The better the prediction signal, the smaller the prediction residual, and thus the fewer bits required to encode the prediction residual.

[0034] To be effective, several aspects should be considered to form an effective framework for intra prediction in a block-based picture coding environment. For example, the more intra prediction modes supported by the codec, the greater the side information rate consumption for signaling the selection to the decoder. On the other hand, the set of supported intra prediction modes must be able to provide a good prediction signal, i.e., a prediction signal with a low prediction residual.

[0035] When using an improved intra prediction mode concept, an intra prediction mode concept that enables more efficient compression of block-based picture codecs is required.

[0036] This objective is achieved, among other things, by means of a so-called affine linear weighted intra predictor (ALWIP) transform. An apparatus (encoder or decoder) for decoding pictures from a data stream in block units is disclosed, which supports at least one intra prediction mode in which the intra prediction signal of a block of a predetermined size of the picture is determined by applying a first template of neighboring samples of the current block to an affine linear predictor called an affine linear weighted intra predictor (ALWIP) in the sequence.

[0037] The apparatus may have at least one of the following characteristics (the same may be implemented in a method or another technique, for example, when executed by a processor, the instructions for causing the processor to implement the method and / or operate as an apparatus are stored in a non-transitory storage unit). 3.1 The proposed predictor may be complementary to other predictors

[0038] The intra prediction modes supported by the apparatus are complementary to other intra prediction modes of the codec. Thus, they can be complementary to the DC prediction mode, planar prediction mode, or angular prediction mode defined in the HEVC codec response. JEM reference software. Hereinafter, the latter three types of intra prediction modes are referred to as conventional intra prediction modes. Therefore, for a given block of the intra mode, a decoder needs to parse a flag indicating whether one of the intra prediction modes supported by the apparatus should be used. 3.2 Two or more proposed prediction modes

[0039] The apparatus can include two or more ALWIP modes. Thus, if the decoder knows that one of the ALWIP modes supported by the apparatus should be used, the decoder needs to parse additional information indicating which of the ALWIP modes supported by the apparatus should be used.

[0040] The signaling of the supported modes can have the property that the encoding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes require fewer bins and which modes require more bins can depend on information that can be extracted from the already decoded bitstream or can be fixed in advance. 4 Some aspects

[0041] FIG. 2 shows a decoder 54 for decoding a picture from a data stream 12. The decoder 54 can be configured to decode a predetermined block 18 of the picture. In particular, the predictor 44 can be configured to map a set of P adjacent samples adjacent to the predetermined block 18 to a set of Q predicted values of the samples of the predetermined block using a linear or affine linear transformation [e.g., ALWIP].

[0042] As shown in FIG. 5, a predetermined block 18 includes Q predicted values (which will become "predicted values" at the end of the operation). If block 18 has M rows and N columns, then Q = M * N. The Q values of block 18 can be in a spatial region (e.g., pixels) or a transform region (e.g., DCT, discrete wavelet transform, etc.). The Q values of block 18 can generally be predicted based on P values obtained from adjacent blocks 17a - 17c adjacent to block 18. The P values of adjacent blocks 17a - 17c may be in the position closest to block 18 (e.g., adjacent). The P values of adjacent blocks 17a - 17c have already been processed and predicted. The P values are shown as values in parts 17’a - 17’c to distinguish them from the blocks of which they are a part (in some examples, 17’b is not used).

[0043] As shown in FIG. 6, in order to perform prediction, it is possible to operate using a first vector 17P having P entries (each entry associated with a specific position in adjacent parts 17’a - 17’c), a second vector 18Q having Q entries (each entry associated with a specific position in block 18), and a mapping matrix 17M (each row associated with a specific position within block 18, each column associated with a specific position within adjacent parts 17’a - 17’c). Thus, mapping matrix 17M performs the prediction of the P values of adjacent parts 17’a - 17’c to the values of block 18 according to a predetermined mode. Thus, the entries within mapping matrix 17M can be understood as weight coefficients. In the following sections, the adjacent parts of the boundary are referred to using signs 17a - 17c instead of 17’a - 17’c.

[0044] In the relevant art, several conventional modes such as DC mode, planar mode, and 65 - directional prediction mode are known. For example, 67 modes are known.

[0045] However, it should be noted that it is also possible here to utilize a different mode called linear or affine-linear transformation. The linear or affine-linear transformation includes P*Q weight coefficients, at least 1 / 4 of the P*Q weight coefficients of which are non-zero weight values, and for each of the Q predicted values, includes a series of P weight coefficients associated with that respective predicted value. When arranged vertically in raster scan order among the samples of a given block, successively, an envelope that is non-linear in all directions is formed.

[0046] FIG. 13 shows an example of FIG. 70 that maps the P positions (templates) of adjacent values 17’a to 17’c, the Q positions of adjacent samples 17’a to 17’c, and the values of the P*Q weight coefficients of matrix 17M. Plane 72 is an example of the envelope of the DC transformation (a plane for the DC transformation). The envelope is clearly a plane and is thus excluded by the definition of linear or affine-linear transformation (ALWIP). Another example is a matrix that results in an emulation of the angular mode, and the envelope is excluded from the ALWIP definition and, plainly speaking, appears like a hill slanting diagonally from top to bottom along a direction in the P / Q plane. The plane mode and the 65-directional prediction mode have different envelopes, which are linear in at least one direction, i.e., for example, all directions of the illustrated DC, and in the direction of the hill of the angular mode, for example.

[0047] Conversely, the envelope of the linear or affine transformation is not linearly in all directions. It is understood that such a type of transformation may be optimal for performing the prediction of block 18 depending on the situation. It should be noted that it is preferable that at least 1 / 4 of the weight coefficients are different from 0 (i.e., at least 25% of the P*Q weight coefficients are different from 0).

[0048] The weight coefficients may be independent of each other according to any regular mapping rule. Thus, matrix 17M may be such that the values of its entries do not have an obvious recognizable relationship. For example, the weight coefficients cannot be described by any analytical function or differential function.

[0049] In an embodiment, for the ALWIP conversion, the average of the maximum values of the cross-correlations between a first series of weight coefficients associated with respective predicted values and a second series of weight coefficients associated with predicted values other than the respective predicted values, or a reversed version of the latter, may be lower than a predetermined threshold value (e.g., 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold value within the range of 0.05 to 0.035), regardless of whether the maximum value becomes higher. For example, for each combination (i1, i2) of rows of the ALWIP matrix 17M, the cross-correlation may be calculated by multiplying the P value of the i1-th row by the P value of the i2-th row. For each obtained cross-correlation, a maximum value may be obtained. Thus, an average value can be obtained for the entire matrix 17M (i.e., the maximum values of the cross-correlations in all combinations are averaged). Thereafter, the threshold value may be, for example, 0.2 or 0.3 or 0.35 or 0.1, e.g., a threshold value within the range of 0.05 to 0.035.

[0050] The P adjacent samples of blocks 17a to 17c may be arranged along a one-dimensional path extending along the boundary of a predetermined block 18 (e.g., 18c, 18a). For each of the Q predicted values of the predetermined block 18, a series of P weight coefficients associated with the respective predicted values may be ordered so as to traverse the one-dimensional path in a predetermined direction (e.g., from left to right, from top to bottom, etc.). In an example, the ALWIP matrix 17M may be non-diagonal or non-block diagonal. An example of the ALWIP matrix 17M for predicting a 4×4 block 18 from four already predicted adjacent samples may be as follows: { {37, 59, 77, 28}, {32, 92, 85, 25}, {31, 69, 100, 24}, {33, 36, 106, 29}, {24, 49, 104, 48}, {24, 21, 94, 59}, {29, 0, 80, 72}, {35, 2, 66, 84}, {32, 13, 35, 99}, {39, 11, 34, 103}, {45, 21, 34, 106}, {51, 24, 40, 105}, {50, 28, 43, 101}, {56, 32, 49, 101}, {61, 31, 53, 102}, {61, 32, 54, 100} [[ID=18}]}.

[0051] (Here, {3, 7, 5, 9} is the first row, {3, 2, 8, 5} is the second row, and {6, 1, 3, 8} is the 16th row of the 17M matrix. ) The 17M matrix has dimensions 16x4 and contains 64 weight coefficients (as a result of 16 * 4 = 64). This is because the 17M matrix has dimensions QxP, where Q = M * N is the number of samples of the prediction target block 18 (block 18 is a 4x4 block), and P is the number of samples of the already predicted samples. Here, M = 4, N = 4, Q = 16 (as a result of M * N = 4 * 4 = 16), and P = 4. The matrix is non - diagonal and non - block - diagonal and is not described by a specific rule.

[0052] As can be seen, less than 1 / 4 of the weight coefficients are 0 (in the case of the above matrix, one of the 64 weight coefficients is 0). The envelope formed by these values forms an envelope that is non - linear in all directions when arranged vertically in raster scan order.

[0053] Even though the above explanation is mainly given with reference to a decoder (e.g., decoder 54), the same may be performed by an encoder (e.g., encoder 14).

[0054] In some examples, for each block size (in a set of block sizes), the ALWIP transform of the intra prediction modes within a second set of intra prediction modes for each block size is different from each other. Additionally or alternatively, the densities of the second set of intra prediction modes for block sizes in a set of block sizes may match, but the associated linear transforms or affine linear transforms of the intra prediction modes within the second set of intra prediction modes for different block sizes may not be transferable to each other by scaling.

[0055] In some examples, the ALWIP transform may be defined as "having nothing in common" with a conventional transform (e.g., even if the ALWIP transform is mapped via one of the above mappings, it "has nothing" in common with the corresponding conventional transform).

[0056] In an example, the ALWIP mode is used for both luma and chroma components, while in other examples, the ALWIP mode is used for the luma component but not for the chroma component. 5 Acceleration of the Encoder for Affine Linear Weighted Intra Prediction Mode (e.g., Test CE3-1.2.1:) 5.1 Description of the Method or Apparatus

[0057] The Affine Linear Weighted Intra Prediction (ALWIP) mode tested in CE3-1.2.1 may be the same as that proposed in JVET-L0199 under Test CE3-2.2.2, except for the following changes.

[0058] Multiple Reference Line (MRL) intra prediction, particularly the consistency with encoder estimation and signaling, i.e., MRL is not combined with ALWIP, and the transmission of the MRL index is restricted to non-ALWIP blocks.

[0059] Subsampling is mandatory for all blocks, with W×H≧32×32 (previously optional for 32×32), and the additional tests in the encoder and the transmission of subsampling flags are removed.

[0060] ALWIP for 64×N and N×64 blocks (N≦32) is added by downsampling to 32×N and N×32 respectively and applying the corresponding ALWIP mode. Furthermore, Test CE 3-1.2.1 includes the following encoder optimizations for ALWIP,

[0061] Combined mode estimation: The conventional mode and the ALWIP mode use a shared Hadamard candidate list for full RD estimation, i.e., the ALWIP mode candidates are added to the same list as the conventional (and MRL) mode candidates based on the Hadamard cost. EMT intra fast and PB intra fast are supported for the combined mode list, and there are additional optimizations to reduce the number of full RD checks. Only the MPM of the available left and upper blocks is added to the list for full RD estimation of ALWIP following the same method as in the conventional mode. 5.2 Complexity evaluation

[0062] Excluding the calculations that call the discrete cosine transform, in Test CE3-1.2.1, a maximum of 12 multiplications per sample were required to generate the prediction signal. Furthermore, a total of 136492 parameters of 16 bits each were required. This corresponds to 0.273 megabytes of memory. 5.3 Experimental results

[0063] The test evaluation was performed according to the common test conditions JVET-J 1010[2] for the intra-only (AI) and random access (RA) configurations using the VTM software version 3.0.1. The corresponding simulation was executed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, turbo boost off) with the Linux (registered trademark) OS and GCC 7.2.1 compiler.

[0064]

Table 1

[0065] The technique tested in CE2 is related to the "affine linear intra prediction" described in JVET-L0199[1], but simplifies it in terms of memory requirements and computational complexity.

[0066] · There can only be three different sets of prediction matrices (e.g., S0, S1, S2, see also below) and bias vectors (e.g., to provide offset values) that cover all block shapes. As a result, the number of parameters is reduced to 14400 10-bit values, which requires less memory than storing in a 128×128 CTU.

[0067] · The input and output sizes of the predictor are further reduced. Furthermore, instead of transforming the boundaries via DCT, averaging or downsampling can be performed on the boundary samples, and linear interpolation can be used to generate the prediction signal instead of the inverse DCT. As a result, up to 4 multiplications per sample may be required to generate the prediction signal. 6. Examples Here, a method of performing several predictions (e.g., as shown in Figure 6) using ALWIP prediction will be described.

[0068] In principle, referring to FIG. 6, in order to obtain the Q = M * N values of the predicted MxN block 18, the multiplication of the Q * P samples of the QxP ALWIP prediction matrix 17M and the P samples of the Px1 adjacent vector 17P should be performed. Thus, generally, in order to obtain each of the Q = M * N values of the MxN block 18 to be predicted, at least P = M + N value multiplications are required.

[0069] These multiplications have highly undesirable effects. The dimension P of the boundary vector 17P generally depends on the number M + N of (e.g., horizontal) boundary samples (bins or pixels) 17a, 17c adjacent to the MxN block 18 to be predicted. This is because when the size of the block 18 to be predicted is large, the number of boundary pixels M + N (17a, 17c) accordingly becomes large, so the dimension P = M + N of the Px1 boundary vector 17P, the length of each row of the QxP ALWIP prediction matrix 17M, and thus the required number of multiplications (generally speaking, Q = M * N = W * H, where W (width) is another symbol for N and H (height) is another symbol for M, and P = M + N = H + W when the boundary vector is formed by only one row and / or one column of samples) will increase.

[0070] This problem is generally exacerbated by the fact that in a microprocessor-based system (or other digital processing system), multiplication is generally an operation that consumes power. It can be inferred that a large number of multiplications performed on a very large number of samples of a large number of blocks generally cause an undesirable waste of computing power. Therefore, it is preferable to reduce the number of multiplications Q * P required to predict the MxN block 18.

[0071] It is understood that by intelligently selecting an operation that is easier to process instead of multiplication, it is possible to reduce the computing power required for each intra prediction of each predicted block 18 in some form.

[0072] In particular, referring to Figures 7.1-7.4, 8.1, and 8.2, the encoder or decoder uses multiple adjacent samples (e.g., 17a, 17c) to

[0073] reducing (e.g., by averaging or downsampling) (e.g., step 811) the plurality of adjacent samples to obtain a reduced set of sample values that has fewer samples compared to the plurality of adjacent samples (e.g., 17a, 17c);

[0074] subjecting the reduced set of sample values to a linear or affine-linear transformation (e.g., step 812) to obtain a predicted value for a given sample of a given block; It will be appreciated that a given block (e.g., 18) of a picture may be predicted by

[0075] In some cases, the decoder or encoder may also derive, for example by interpolation, predicted values for further samples of a given block based on predicted values of the given sample and multiple neighboring samples (e.g., step 813 in Figure 8.1), thus resulting in an upsampling strategy.

[0076] In an example, it is possible to perform several averages (e.g., in step 811) on the samples of the boundary 17 to arrive at a reduced set of samples 102 (FIGS. 7.1-7.4) with a reduced number of samples (at least one of the samples in the reduced number of samples 102 may be an average of two of the original boundary samples, or a selection of the original boundary samples). For example, if the original boundary has P=M+N samples, the reduced set of samples may be P red =M red +N red And P red <MおよびN red <Nの少なくとも1つを伴い、それによりP redIt can become P. Therefore, the boundary vector 17P actually used for prediction (for example, in step 812b) does not have a Px1 entry, and P red which is P red has a Px1 entry. Similarly, the ALWIP prediction matrix 17M selected for prediction does not have a QxP dimension and has at least P red which is P(M red and N red by at least one of N) reduces the number of elements of the matrix to QxP red (or Q red xP red , see below).

[0077] In some examples (for example, FIGS. 7.2, 7.3, and 8.2), the block obtained by ALWIP (in step 812) has a size

Number

Number

Number

Number

[0078] These techniques may be advantageous because while matrix multiplication involves a reduced number of multiplications (Q red *P red or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (or avoiding) multiplications. For example, downsampling, averaging, and / or interpolation can be performed (e.g., in steps 811 and / or 813) by employing binary operations that do not require computational capabilities such as addition and shift.

[0079] Here, an example of a shift operation at the processor level will be described. FIG. 9 shows the number 580 encoded in binary (1001000100b) within a 10-bit register 910. The 10-bit register 910 has 10-bit registers 910a, and each bit register 910a stores a 1-bit value (e.g., 1 or 0). The value 1001000100b represented in binary encoded by the 2-byte register 910 is shown as 901 as the "value to be shifted" in FIG. 9a. The binary value indicated by the symbol 902 in FIG. 9b is the right-shifted version of the binary value indicated by the symbol 901. As can be seen, the binary value 902 (encoded as 0100100010b) is a version of the binary value 901 after each value encoded in each bit register 910a has been simply moved to the bit register in the respective right position. The least significant bit of the binary value 901 is lost in the binary value 902, and a 0 is added as the most significant bit of the shifted binary value 902. When the shifted binary value 902 is converted to a decimal, it can be seen that the decimal value 290, which is half of 580, is obtained. This can be a technique for bisecting a binary number (when the value 901 is odd, only quantization error exists). This operation is extremely easy to execute and does not require high computing power. In particular, by shifting multiple times, division by a power of 2 can be obtained. For example, by shifting 2 times, division by 4 can be obtained, by shifting 3 times, division by 8 can be obtained, and generally, by shifting r times, division by 2 r (2^r) can be obtained. This can also be represented by the notation f>>r. Thus, f>>1 represents f / 2, f>>2 represents f / 4, and so on (f is an integer). This operation is also known as a right rotation. Similarly, the left rotation operation f<<r multiplies f by 2 rMeans multiplying by (or 2^r). The shift operation, to avoid the need to perform multiplication, does not require computational power for the processor and is simply obtained by moving bits within different registers. In this example, register 910 is represented as a 10-bit register. However, in the example, register 910 may have a different number of bit registers 910a, for example, an 8-bit register 910 (in which case register 910 is an 8-bit register). Also, addition is a very simple operation that can be easily performed without much computational effort.

[0080] This shift operation can be used, for example, to average two boundary samples, and / or to interpolate two samples (support values) of a reduced prediction block (or obtained from the boundary) in order to obtain the final prediction block. (For interpolation, two sample values are required. Within the block, there are always two predetermined values, but along the left and upper boundaries of the block, there is only one predetermined value, as shown in FIG. 7.2, and thus boundary samples are used as support values for interpolation.) There may be cases where a two-step procedure as follows is used: First, sum the values of the two samples. Then, divide the sum value by half (e.g., by a right shift), etc. Alternatively, First divide each of the samples by half (e.g., by a left shift). Then, sum the values of the two halved samples. This is possible.

[0081] Since it is only necessary to select one sample amount and a group of samples (e.g., adjacent samples to each other), easier operations can be performed during downsampling (e.g., in step 811).

[0082] Therefore, it is possible here to define techniques (plural) for reducing the number of multiplications to be performed. Some of these techniques may be based, inter alia, on at least one of the following principles:

[0083] Even if the size of the actually predicted block 18 is MxN, the block is reduced (in at least one of the two dimensions), and the reduced Q red xP red (

Number

Number

[0084] Additionally or alternatively, instead of predicting all Q = M*N values of the block 18 to be predicted by multiplication, a reduced block with reduced dimensions (e.g.,

Number

[0085] An example that can be understood as generally describing process 810 is provided by FIG. 8.1, and a specific case thereof is shown in FIG. 7.1. In this case, a 4×4 block 18 (M = 4, N = 4, Q = M*N = 16) is to be predicted, and the neighborhoods 17 of samples 17a (a vertical matrix with 4 already predicted samples) and 17c (a horizontal row with 4 already predicted samples) have already been predicted in the previous iteration (neighborhoods 17a and 17c can be collectively denoted as 17). Preferably, by using the formula shown in FIG. 5, the prediction matrix 17M should be a QxP = 16x8 matrix (by Q = M*N = 4*4 and P = M+N = 4+4 = 8), and the boundary vector 17P should have an 8x1 dimension (by P = 8). However, this leads to the need to perform 8 multiplications for each of the 16 samples of the 4x4 block 18 to be predicted, and thus to the need to perform a total of 16*8 = 128 multiplications. (Note that the average number of multiplications per sample is a good measure of computational complexity. In conventional intra prediction, 4 multiplications per sample are required, which increases the computational effort involved. Therefore, it is possible to use this as the upper limit of ALWIP, and it is guaranteed that the complexity is reasonable and does not exceed that of conventional intra prediction.)

[0086] Nevertheless, by using the present technology, in step 811, the number of adjacent samples 17a and 17c to the predicted block 18 is changed from P to P redIt is understood that it is possible to reduce to . In particular, in order to obtain a reduced boundary 102 having two horizontal rows and two vertical columns, it is possible to average adjacent boundary samples (17a, 17c) with each other (for example, at 100 in FIG. 7.1), and thus it is understood that operating as block 18 is a 2×2 block (the reduced boundary formed by the average value). Alternatively, it is possible to perform downsampling, and thus select two samples for row 17c and two samples for column 17a. Thus, the horizontal row 17c is processed (e.g., averaged samples) as having two samples instead of having four original samples, and the vertical column 17a, which originally has four samples, is processed as having two samples (e.g., averaged samples). After sub-dividing the row 17c and the column 17a into groups 110 of two samples each, it can also be understood that a single sample is maintained (e.g., the average of the samples in group 110 or a simple selection between the samples in group 110). Thus, by having only a set 102 of four samples (M red =2, N red =2, P red =M red +N red =4, P red <P), a so-called reduced set 102 of sample values is obtained.

[0087] It is understood that it is possible to perform operations (such as averaging or downsampling 100) without performing an excessive number of multiplications at the processor level, and the averaging or downsampling 100 performed in step 811 can be easily obtained by operations that do not require simple and computational capabilities such as addition and shift.

[0088] At this point, it is understood that it is possible to apply a reduced set of sample values 102 to a linear or affine linear (ALWIP) transform 19 (for example, using a prediction matrix such as matrix 17M of FIG. 5). In this case, the ALWIP transform 19 directly maps four samples 102 to the sample values 104 of block 18. In this case, interpolation is not required.

[0089] In this case, the ALWIP matrix 17M has dimensions QxP red = 16x4. This follows from the fact that all Q = 16 samples of block 18 to be predicted are directly obtained by ALWIP multiplication (interpolation is not required).

[0090] Accordingly, in step 812a, an appropriate ALWIP matrix 17M having dimensions QxP red is selected. The selection may be based at least in part on signaling from data stream 12, for example. The selected ALWIP matrix 17M may also be denoted as A k where k is an index that may be signaled in data stream 12 (in some cases, the matrix may be denoted as follows

Number

[0091] In step 812b, the selected QxP red ALWIP matrix 17M (denoted as A k as well) is multiplied by the P red x1 boundary vector 17P.

[0092] In step 812c, an offset value (e.g., b k ) can be added to all the acquired values 104 of the vector 18Q acquired, for example, by ALWIP. The value of the offset (b k , or in some cases even

Number

[0093] Therefore, the comparison between using this technology and not using this technology resumes here: When not using this technology: Block 18 to be predicted, dimension M = 4, N = 4, Q = M * N = 4 * 4 = 16 values to be predicted, P = M + N = 4 + 4 = 8 boundary samples P = 8 multiplications for each of the Q = 16 values to be predicted, P * Q = 8 * 16 = 128 total multiplications, When using this technology, Block 18 to be predicted, dimension M = 4, N = 4, Finally Q = M * N = 4 * 4 = 16 values to be predicted, Reduced dimension of the boundary vector: P red = M red + N red = 2 + 2 = 4, P = 4 multiplications for each of the Q = 16 values to be predicted by ALWIP, red = 4 multiplications, Total number P red * Q = 4 * 16 = 64 multiplications (half of 128!) The ratio of the number of multiplications to the number of final values obtained is, P red * Q / Q = 4, that is, P = half of 8 multiplications for each sample to be predicted!

[0094] As can be understood, it is possible to obtain appropriate values in step 812 by relying on operations that do not require direct and computational capabilities such as averaging (in some cases, addition and / or shifting and / or downsampling).

[0095] Referring to FIGS. 7.2 and 8.2, the predicted block 18 is here an 8×8 block of 64 samples (M = 8, N = 8). Here, preferably, the prediction matrix 17M should have a size QxP = 64x16 (Q = M*N = 8*8 = 64, Q = 64 by M = 8 and N = 8, and P = M + N = 8 + 8 = 16). Thus, preferably, for each of the Q = 64 samples of the predicted 8x8 block 18, P = 16 multiplications are required, reaching 64*16 = 1024 multiplications for the entire 8x8 block 18!

[0096] However, as seen in FIGS. 7.2 and 8.2, instead of using all 16 samples at the boundary, a method 820 can be provided in which only 8 values (e.g., 4 of the horizontal boundary row 17c and 4 of the vertical boundary column 17a between the original samples at the boundary) are used. From the boundary row 17c, instead of 8 samples, 4 samples may be used (e.g., they may be a 2×2 average and / or a selection of 1 out of 2 samples). Thus, the boundary vector becomes not a Px1 = 16x1 vector, but only a P red x1 = 8x1 vector (P red = M red + N red = 4 + 4). Instead of the original P = 16 samples, it is understood that it is possible to select or average (e.g., 2×2) the samples of the horizontal row 17c and the samples of the vertical column 17a so as to have only P red = 8 boundary values, forming a reduced set 102 of sample values. This reduced set 102 makes it possible to obtain a reduced version of the block 18, and the reduced version has a Q red = M red * N redHas 4 * 4 = 16 samples. Size M red xN red An ALWIP matrix for predicting a 4x4 block can be applied. The reduced version of block 18 includes the samples shown in gray in scheme 106 of FIG. 7.2, and the samples shown by the gray squares (including samples 118' and 118'') are the Q obtained in the target step 812 red Forms a 4×4 reduced block having 16 values. The 4x4 reduced block is obtained by applying the linear transformation 19 in the target step 812. After obtaining the values of the 4x4 reduced block, it is possible to obtain the values of the remaining samples (the samples shown as white samples in scheme 106), for example, by interpolation.

[0097] Regarding method 810 of FIGS. 7.1 and 8.1, this method 820 is for the remaining Q - Q of the predicted MxN = 8x8 block 18 red Step 813 of deriving the predicted values of 64 - 16 = 48 samples (white squares), for example, by interpolation, can be further included. The remaining Q - Q red = 64 - 16 = 48 samples are the Q directly obtained by interpolation (the interpolation can also utilize the values of boundary samples, for example). red = 16 samples. As can be seen in FIG. 7.2, samples 118' and 118'' are obtained in step 812 (as shown by the gray squares), while sample 108' (which is in the middle of samples 118' and 118'' and is shown by a white square) is obtained by interpolation between samples 118' and 118'' in step 813. It is understood that the interpolation can also be obtained by an operation similar to that for averaging such as shifting and adding. Thus, in FIG. 7.2, the value 108' can generally be determined (and can be an average) as the value intermediate between the value of sample 118' and the value of sample 118''.

[0098] By performing interpolation, it is also possible to reach the final version of the MxN = 8x8 block 18 based on the plurality of sample values shown in 104 in step 813. Therefore, comparing using this technology with not using it, When not using this technology: The block 18 to be predicted has dimensions M = 8, N = 8, and has Q = M*N = 8*8 = 64 samples within the block 18 to be predicted. P = M + N = 8 + 8 = 16 samples within the boundary 17. For each of the Q = 64 values to be predicted, P = 16 multiplications. A total of P*Q = 16*64 = 1028 multiplications. The ratio of the number of multiplications to the number of final values obtained is P*Q / Q = 16. When using this technology, The block 18 to be predicted has dimensions M = 8, N = 8. Finally, Q = M*N = 8*8 = 64 values to be predicted.

[0099] However, Q red xP red The ALWIP matrix is used, and P red = M red + N red , Q red = M red * N red , M red = 4, N red = 4, and P within the boundary red = M red + N red = 4 + 4 = 8 samples, where P red < P. For each of the Q = 16 values of the 4x4 reduced block to be predicted, P red = 8 multiplications (formed by the gray squares in scheme 106). red The total number of P red * Q red = 8*16 = 128 multiplications (far less than 1024!), The ratio of the number of multiplications to the number of final values obtained is P red*Q red / Q = 128 / 64 = 2 (much less than 16 obtained without using this technology!). Therefore, the technology presented in this specification requires 8 times less computing power than the previous technology.

[0100] Figure 7.3 shows another example (obtainable based on method 820), where the predicted block 18 is a rectangular 4×8 block (M = 8, N = 4) with Q = 4 * 8 = 32 predicted samples. The boundary 17 is composed of a horizontal row 17c of N = 8 samples and a vertical column 17a of M = 4 samples. Therefore, preferably, the boundary vector 17P has dimension Px1 = 12x1, but the predicted ALWIP matrix should be a QxP = 32x12 matrix, and thus Q * P = 32 * 12 = 384 multiplications are required.

[0101] However, for example, it is possible to average or downsample at least 8 samples of the horizontal row 17c to obtain a reduced horizontal row of only 4 samples (e.g., the averaged samples). In some examples, the vertical column 17a remains as it is (e.g., without averaging). In total, the reduced boundary has dimension P red = 8, and P red < P. Therefore, the boundary vector 17P has dimension P red x1 = 8x1. The ALWIP prediction matrix 17M becomes a matrix with dimension M * N red *P red = 4 * 4 * 8 = 64. The 4x4 reduced block (formed by the gray columns of scheme 107) directly obtained in the target step 812 has size Q red = M * N red = 4 * 4 = 16 samples (instead of Q = 4 * 8 = 32 of the original 4x8 block 18 to be predicted). When the reduced 4×4 block is obtained by ALWIP, the offset value b kAdd (step 812c), and interpolation can be performed in step 813. As can be seen in step 813 of FIG. 7.3, the reduced 4x4 block is expanded to a 4x8 block 18, and the value 108' not obtained in step 812 is obtained in step 812 by interpolating the values 118' and 118'' (gray squares) obtained in step 813.

[0102] Therefore, comparing using this technology with not using it, When not using this technology: The block to be predicted 18 has dimensions M = 4, N = 8, The Q values to be predicted are Q = M * N = 4 * 8 = 32, There are P = M + N = 4 + 8 = 12 samples within the boundary, For each of the Q = 32 predicted values, there are P = 12 multiplications, The total number of multiplications is P * Q = 12 * 32 = 384. The ratio of the number of multiplications to the number of final values obtained is P * Q / Q = 12. When using this technology, The block to be predicted 18 has dimensions M = 4, N = 8, Finally, the Q values to be predicted are Q = M * N = 4 * 8 = 32,

[0103] However, Q red xP red = 16x8 ALWIP matrix can be used, M = 4, N red = 4, Q red = M * N red = 16, P red = M + N red = 4 + 4 = 8, and The P within the boundary red = M + N red = 4 + 4 = 8 samples, where P red < P, The Q of the reduced block to be predicted red = For each of the 16 values, there are P red = 8 multiplications, The total number of P red *Q redMultiplication of =8*16 = 128 (less than 384!) The ratio of the number of multiplication factors to the number of final values obtained is P red *Q red / Q = 128 / 32 = 4 (much less than 12 obtained without using this technology!). Therefore, in this technology, the computational effort is reduced to 1 / 3.

[0104] Figure 7.4 shows an example of a block 18 to be predicted with dimensions MxN = 16x16. The finally predicted value is Q = M*N = 16*16 = 256, and P = M + N = 16 + 16 = 32 boundary samples. This results in a prediction matrix with dimensions QxP = 256x32, which means 256*32 = 8192 multiplications!

[0105] However, by applying method 820, in step 811, it is possible to reduce the number of boundary samples, for example, from 32 to 8 (e.g., by averaging or downsampling). For example, for each group 120 of 4 consecutive samples in row 17a, a single sample (e.g., selected from the 4 samples or the average of the samples) remains. Also, for each group consisting of 4 consecutive samples in column 17c, one sample (e.g., selected from the 4 samples or the average of the samples) remains.

[0106] Here, the ALWIP matrix 17M is Q red xP red = 64x8 matrix. This is due to the fact that it is selected to P red = 8, and the fact that the reduced block predicted in step 812 is an 8x8 block (in scheme 109, the gray square is 64).

[0107] Therefore, when 64 samples of the reduced 8×8 block are obtained in step 812, in step 813, the remaining Q - Q of the predicted block 18 redIt is possible to derive 192 values 104, where 192 = 256 - 64.

[0108] In this case, in order to perform interpolation, it has been selected to use only all the samples of the boundary column 17a and the alternative samples of the boundary row 17c. Other selections may be made.

[0109] In this method, the ratio between the number of multiplications and the number of finally obtained values is Q red *P red / Q = 8 * 64 / 256 = 2, which is much less than 32 multiplications for each value without using this technology! Comparing using this technology with not using it, When not using this technology: The block to be predicted 18 has dimensions M = 16, N = 16, The number of values Q to be predicted is Q = M * N = 16 * 16 = 256, The number of samples P within the boundary is P = M + N = 16 + 16 = 32, For each of the Q = 256 predicted values, P = 32 multiplications, The total number of multiplications is P * Q = 32 * 256 = 8192 The ratio of the number of multiplications to the number of finally obtained values is P * Q / Q = 32. When using this technology, The block to be predicted 18, the block has dimensions M = 16, N = 16, The number of values Q finally predicted is Q = M * N = 16 * 16 = 256,

[0110] However, Q red xP red = 64x8 ALWIP matrix can be used, M red = 4, N red = 4, and what is predicted by ALWIP is Q red = 8 * 8 = 64 samples, P red = M red + N red = 4 + 4 = 8 P within the boundary red = M red + N red= 4 + 4 = 8 samples, where P red <which is P Q of the predicted reduction blocks red For each of the = 64 values, P red = Multiplication by 8 Total number Q red * P red = Multiplication of 64 * 4 = 256 (less than 8192!) The ratio of the number of multiplications to the number of final values obtained is P red * Q red / Q = 8 * 64 / 256 = 2 (much less than 32 obtained without using this technology!) Therefore, the computing power required by this technique is 16 times smaller than that of the conventional technique!

[0111] Therefore, using a plurality of adjacent samples (17), compared with a plurality of adjacent samples (17), to obtain a set (102) of sample values with fewer samples and decreased sample values, reducing a plurality of adjacent samples (100, 813), and

[0112] To obtain the predicted values of the predetermined samples (104, 118’, 188’’) of the predetermined block (18), subjecting the set (102) of decreased sample values to a linear or affine linear transformation (19, 17M) (812), and It is possible to predict a predetermined block (18) of the picture.

[0113] In particular, compared with a plurality of adjacent samples (17), it is possible to perform a reduction (100, 813) by downsampling to obtain a set (102) of sample values with fewer samples and decreased sample values.

[0114] Alternatively, compared with a plurality of adjacent samples (17), it is possible to perform a reduction (100, 813) by averaging a plurality of adjacent samples to obtain a set (102) of sample values with fewer samples and decreased sample values.

[0115] Furthermore, by interpolation, it is possible to derive (813) predicted values of further samples (108, 108') of a predetermined block (18) based on predicted values of a predetermined sample (104, 118', 118'') and a plurality of adjacent samples (17).

[0116] The plurality of adjacent samples (17a, 17c) may extend one-dimensionally along two sides of a predetermined block (18) (e.g., toward the right and bottom in FIGS. 7.1 to 7.4). The predetermined sample (e.g., the one obtained by ALWIP in step 812) may also be arranged in rows and columns and may be arranged at every nth position from samples (112) of predetermined samples 112 adjacent to two sides of the predetermined block 18 along at least one of the rows and columns.

[0117] Based on the plurality of adjacent samples (1, it is possible to determine a support value (118) of one position (118) among a plurality of adjacent positions aligned with at least one of the rows and columns respectively. By interpolation, it is also possible to derive a predicted value 118 of further samples (108, 108') of a predetermined block (18) based on the predicted values of a predetermined sample (104, 118', 118'') and the support values of adjacent samples (118) aligned with at least one row and column.

[0118] The predetermined sample (104) may be arranged at every nth position from samples (112) of predetermined samples 112 adjacent to two sides of the predetermined block 18 along a row, and the predetermined sample may be arranged at every mth position from samples (112) of predetermined samples 112 adjacent to two sides of the predetermined block (18) along a column, where n, m > 1. In some cases, n = m (e.g., in FIGS. 7.2 and 7.3, the samples 104, 118', 118'' directly obtained by ALWIP in 812 and shown by gray squares are alternated with the samples 108, 108' subsequently obtained in step 813 along the rows and columns).

[0119] For each support value along at least one of a row (17c) and a column (17a), it may be possible to perform the determination of the support value by, for example, downsampling or averaging (122) a group (120) of adjacent samples within a plurality of adjacent samples including the adjacent samples (118) for which each support value is determined. Thus, in FIG. 7.4, in step 813, it is possible to obtain the value of sample 119 by using the value of a predetermined sample 118''' (previously obtained in step 812) and the values of adjacent samples 118 as support values.

[0120] The plurality of adjacent samples may extend one-dimensionally along two sides of a predetermined block (18). It may be possible to perform a reduction (811) by grouping a plurality of adjacent samples (17) into one or more groups (110) of consecutive adjacent samples and performing downsampling or averaging on each of the one or more groups (110) of adjacent samples having two or more adjacent samples.

[0121] In an example, a linear or affine-linear transformation can include P red *Q [[ID={11]] red or P red *Q weight coefficients, where P red is the number (102) of sample values in a reduced set of sample values, and Q red or Q is the number (18) of predetermined samples within a predetermined block. At least 1 / 4 of P red *Q red or 1 / 4 of P red *Q weight coefficients are non-zero weight values. P red *Q red or P red *Q weight coefficients can include, for each of Q or Q red predetermined samples, a series of P red weight coefficients for each respective predetermined sample, and a series of P redWhen the weight coefficients are arranged vertically according to the raster scan order between predetermined samples of a predetermined block (18), they form an envelope that is non-linear in all directions. P red *Either Q or P red *Q red The weight coefficients may be independent of each other via a regular mapping rule. The average of the maximum values of the cross-correlation between a first series of weight coefficients associated with each predetermined sample and a second series of weight coefficients associated with predetermined samples other than each predetermined sample, or the inverted version of the latter, is lower than a predetermined threshold, regardless of whether the maximum value becomes higher. The predetermined threshold may be 0.3 (or in some cases 0.2 or 0.1). P red The adjacent samples (17) are arranged along a one-dimensional path extending along two sides of a predetermined block (18), either Q or Q red For each of the predetermined samples, a series of P associated with each predetermined sample red The weight coefficients can be ordered to traverse the one-dimensional path in a predetermined direction. Description of the method and apparatus

[0122]

Number

Number

[0123] 1. Among the boundary samples 17, the sample 102 (for example, 4 samples when W = H = 4 and / or 8 samples in other cases) can be extracted by averaging or downsampling (for example, step 811).

[0124] 2. Matrix-vector multiplication followed by addition of an offset can be performed with the averaged samples (or the samples remaining from downsampling) as input. The result can be a reduced prediction signal (for example, step 812) on a set of subsampled samples within the original block.

[0125] 3. The prediction signals at the remaining positions can be generated from the prediction signals on the subsampled set, for example, by upsampling, for example, by linear interpolation (for example, step 813).

[0126] Thanks to step 1 (811) and / or 3 (813), the total number of multiplications required for the calculation of the matrix-vector product is always

Number

[0127] In some examples, the matrix (for example, 17M) and offset vector (for example, b k ) required to generate the prediction signal can be a set of matrices (for example, 3 sets) stored in the memory units of the decoder and encoder, for example,

Number

[0128] In some examples, the set

Number

[0129] In some examples, the set [Number] For executing the technique according to FIG. 7.2 or FIG. 7.3, each has 16 and 8 columns, and 18 offset vectors each of size 16 [Number] and may have (e.g., be composed of) [Number] matrix [Number] and may include. This set

Number

Number

[0130] Additionally or alternatively, the set

Number

Number

Number

[0131] Set based on block dimensions

Number

[0132] As described above, the boundary samples (17a, 17c) may be averaged and / or downsampled (e.g., from P samples to P red <P samples).

[0133] In a first step, the input boundary

Number

Number

Number

Number

[0134] and

Number

Number

[0135] In all other cases (e.g., for blocks with a width or height of the wither different from 4), when the block width W is

Number

Number

Number

Number

[0136] In still other cases, the boundary (e.g., by selecting one specific boundary sample from a group of boundary samples) can be downsampled to reduce the number of samples. For example,

Number

Number

Number

[0137] The two reduced boundaries

Number

Number

[0138] Thus, for a specific state [Number] according to, it is possible to distribute the predicted values of the output vector along different scan orders (e.g., one scan order: [Number] ).

[0139] Other strategies may be executed. In other examples, the mode index "mode" is not necessarily in the range from 0 to 35 (other ranges may be defined). Further, each of the three sets S0, S1, S2 has 18 matrices (thus, [Number] Instead of expressions like, the number of matrices for each set of matrices S0, S1, S2 [Number] does not necessarily have to have (it is possible to do so for each). Also, each set may have a different number of matrices (for example, S0 may have 16 matrices, S1 may have 8 matrices, and S2 may have 6 matrices).

[0140] The mode and transpose information are not necessarily stored and / or transmitted as a single combined mode index "mode". In some examples, it may be explicitly signaled as a transpose flag and a matrix index (0 to 15 for S0, 0 to 7 for S1, 0 to 5 for S2).

[0141] In some cases, the combination of the transpose flag and the matrix index may be interpreted as a set index. For example, there may be 1 bit operating as the transpose flag and several bits indicating the matrix index, which are collectively shown as the "set index". 5.5 Generation of a Reduced Prediction Signal by Matrix-Vector Multiplication Here, features are provided regarding step 812.

[0142] Reduced input vector [Number] One of (the boundary vector 17P) is the reduced prediction signal [Number] can generate. The latter signal is [Number] a signal on the downsampled block of. Here,[[]] [Number] can be defined as follows.

[0143]

Number

[0144] The predicted signal that has decreased

Number

Number

[0145] Here,

Number

Number

Number

[0146]

Number

Number

Number

Number

[0147] Matrix A and vector

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0148] Other strategies may be executed. In other examples, the mode index "mode" is not necessarily within the range from 0 to 35 (other ranges may be defined). Further, each of the three sets S0, S1, S2 does not necessarily have 18 matrices (thus,

Number

Number

[0149] For the interpolation of the subsampled prediction signal, in a large block, a second version of the averaged boundary may be required. That is,

Number

Number

Number

Number

Number

Number

Number

[0150]

Number

Number

[0151] The linear interpolation may be given as follows (other examples are possible).

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0152] Here,

Number

Number

[0153] This is an example of interpolation that uses reduced boundary samples for the first interpolation (horizontal or vertical) and the original boundary samples for the second interpolation (vertical or horizontal). Depending on the block size, only the second interpolation or no interpolation is required. When both horizontal and vertical interpolations are required, the order depends on the width and height of the block.

[0154] However, different techniques may be implemented. For example, the original boundary samples may be used for both the first and second interpolations, and the order may be fixed. For example, it may be horizontal first and then vertical (in other cases, vertical first and then horizontal). Therefore, the interpolation order (horizontal / vertical) and the use of reduced / original boundary samples can be changed. 5.7 Description of an Example of the Entire ALWIP Process

[0155] The entire process of averaging, matrix-vector multiplication, and linear interpolation is shown for different shapes in FIGS. 7.1 to 7.4. Note that the remaining shapes are treated as one of the illustrated cases.

[0156] Given a 4×4 block, ALWIP can take two averages along each axis of the boundary by using the technique of FIG. 7.1. The resulting four input samples enter the matrix-vector multiplication. The matrix is the set

Number

[0157] When an 8×8 block is given, ALWIP can take four averages along each axis of the boundary. The resulting eight input samples enter a matrix-vector multiplication using the technique of FIG. 7.2. The matrix is the set

Number

[0158] When an 8×4 block is given, ALWIP can take four averages along the horizontal axis of the boundary and four original boundary values on the left boundary by using the technique of FIG. 7.3. The resulting eight input samples enter a matrix-vector multiplication. The matrix is the set

Number

[0159] When a 16×16 block is given, ALWIP can take four averages along each axis of the boundary. The resulting eight input samples enter the matrix-vector multiplication using the technique of FIG. 7.2. The matrix is obtained from the set

Number

[0160] For a W×8 block, since samples are given at the odd horizontal and each vertical position, only horizontal interpolation is required. Therefore, in these cases, at most (8 * 64) / (16 * 8) = 4 multiplications are performed per sample.

[0161] Finally, for a W×4 block with W>8,

Number

[0162] The parameters required for all possible proposed intra prediction modes are the set

Number

[0163] For luma blocks, for example, 35 ALWIP modes are proposed (other numbers of modes may be used). For each coding unit (CU) of the intra mode, a flag indicating whether the ALWIP mode should be applied to the corresponding prediction unit (PU) is transmitted in the bitstream. The signaling of the latter indicator can be coordinated with the MRL in the same way as the first CE test. When the ALWIP mode is applied, the index of the ALWIP mode

Number

[0164] Here, the derivation of the MPM may be performed using the intra modes of the above and left PUs as follows. Each conventional intra prediction mode

Number

Number

Number

Number

[0165] This indicates from which of the three sets the ALWIP parameters should be interpreted as in Section 4 above. The prediction unit above

Number

Number

Number

Number

[0166] The above PU is available, belongs to the same CTU as the current PU, is in the intra mode, and the conventional intra prediction mode

Number

[0167]

Number

Number

[0168] This means that this mode is not available. Similarly, derive the mode, but without the restriction that the left PU must belong to the same CTU as the current PU:

Number

[0169] Finally, three fixed default lists

Number

Number

[0170] The proposed ALWIP mode can be reconciled with the MPM-based coding of the conventional intra prediction mode as follows. The luma and chroma MPM list derivation processes of the conventional intra prediction mode use a fixed table

Number

Number

[0171] In the case of deriving the Luma MPM list, the ALWIP mode

Number

Number

Number

[0172] The test evaluation was carried out according to the common test conditions JVET-J 1010[2] for the intra-only (AI) and random access (RA) configurations using the VTM software version 3.0.1. The corresponding simulations were executed on an Intel Xeon cluster (E5-2697A v4, AVX2 on, turbo boost off) with a Linux (registered trademark) OS and a GCC 7.2.1 compiler

[0173]

Table 2

[0174]

Table 3

[0175] Figure 10 shows another example that can be interpreted from the examples of FIGS. 1, 2, and 5-9 (in particular, some features may be directly derived from FIG. 2 and are therefore not repeated here).

[0176] FIG. 10 shows an encoder 14 that can be a particular instance of the encoder of FIG. 1. Similar to FIG. 2, the encoder 14 subtracts a corresponding prediction signal 24 (e.g., block 18 having the reconstructed sample 104 obtained in step 812) from an inbound signal, i.e., picture 10, or the current block 18 on a block-by-block basis, to obtain a prediction residual signal 26, which may then include a subtractor 22 configured to be encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 may include an irreversible encoding stage 28a and a reversible encoding stage (entropy encoder) 28b. The irreversible encoding stage 28a may include a quantizer 30 (not shown) configured to receive the prediction residual signal 26 and quantize the samples of the prediction residual signal 26. The obtained prediction residual signal 34 is then subjected to reversible encoding by the reversible encoding stage 28b, which is the entropy encoded quantization prediction residual signal 34 of the entropy encoder, to become the data stream 12. The encoder 14 may further include a prediction residual signal reconstruction stage 36 connected to the output of the irreversible encoding stage 28a, whereby the prediction residual signal can be reconstructed from the transformed and quantized prediction residual signal 34'.

[0177] The encoder 14 may include an adder 42 that adds the reconstructed prediction residual signal 34' output by the stage 36 and the prediction signal 24 (e.g., including block 18 having the reconstructed sample 104 obtained in step 813) to output a reconstructed signal, i.e., the reconstructed sample. This output is supplied to a predictor 44, which can determine the prediction signal 24 based thereon (e.g., by applying the techniques shown in FIGS. 8.1-7.4).

[0178] As can be seen in FIG. 9, method steps 811, 812, 813 are here respectively mapped by stages 811’, 812’, 813’ within predictor 44, and method steps 811, 812, 813 can be implemented by hardware units and / or procedure routines within predictor 44, collectively denoted by 811’, 812’, 813’ or controlled by the predictor. In the example, it is shown that it is possible to skip derivation stage 813’ as in the example of FIG. 7.1.

[0179] In particular, stages 811 and / or 813 can be shown as presenting a register such as register 910 for performing the above-described shift operation (register 910 is not necessarily part of stage 811 or 813, it can be a unit controlled by the target stage). Instead, stage 812 is shown as having or controlling multiplier 1910, where the multiplication performed between P red of the adjacent samples 17 or the averaged samples 102 is

Number

[0180] memory device 1044 is here the ALWIP matrix 17M or

Number

Number

[0181] Even if not shown in the figure, the encoder 14 may determine the dimension of the ALWIP matrix to be used (e.g., a set among the sets S0, S1, S2) based on, for example, the dimension of the block 18. In some cases, as a result of the selection of the dimension of the block 18, there is no need to notify this selection.

[0182] Accordingly, the encoder 14 is further configured to insert the prediction residual of the predetermined block 18 into the data stream 12 from which the predetermined block 18 can be reconstructed using the prediction residual 34 and the predicted value 24 (104) of the predetermined sample obtained in step 812 for the predetermined block 18.

[0183] Additionally or alternatively, for a predetermined block (18), the encoder 14 inserts the prediction residual (26, 34) into a data stream (12) indicating the corresponding residual value for each of the Q or Q red predetermined samples, such that the predetermined block (18) is reconstructed using the prediction residual (26, 34) and the predicted value of the predetermined samples by correcting the predicted value for each of the set of Q or Q red values, such that the corresponding reconstructed values are, optionally, strictly linearly dependent on the P red adjacent samples (102) within the reduced set of sample values (102) of the sample values, excluding the clipping applied after prediction and / or correction.

[0184] Additionally or alternatively, the encoder 14 may be configured to sub-divide a picture (16) into a plurality of blocks of different block sizes each comprising a predetermined block (18). The encoder 14 is such that a selected linear or affine linear transform (19, Ak) for a predetermined block (18) is selected from a first set of linear or affine linear transforms as long as the width W and height H of the predetermined block (18) are within a first set of width / height pairs (e.g., associated with S0), and from a second set of linear or affine linear transforms as long as the width W (also denoted as N) and height H (also denoted as M) of the predetermined block (18) are within a second set of width / height pairs (e.g., associated with S1) that do not belong to the first set of width / height pairs, according to the width W and height H of the predetermined block (18). k ) may be configured to select.

[0185] Additionally or alternatively, the encoder is such that a third set of one or more width / height pairs (e.g., S0) simply comprises one width / height pair W’,H’, and each linear or affine linear transform within the second set of linear or affine linear transforms is for converting N’ sample values into W’*H’ predicted values of a W’xH’ array of sample positions.

[0186] Additionally or alternatively, the encoder is such that each of the first and second sets of width / height pairs comprises a first width / height pair W p ,H p wherein W p is not equal to H p , a first width / height pair W p ,H p and a second width / height pair W q ,H q wherein H q =W p and W q =H p , a second width / height pair W q ,H q and may be configured to include.

[0187] Additionally or alternatively, for each of the first and second sets of width / height pairs, the encoder may be configured such that the encoder further includes a third width / height pair W p ,H p wherein W p equals H p and H p >H q as may be configured.

[0188] Additionally or alternatively, for a given block, the encoder may be configured to insert a set index into the data stream and to select a linear or affine linear transformation from a given set of linear or affine linear transformations according to the set index.

[0189] Additionally or alternatively, the encoder has a plurality of adjacent samples extending one-dimensionally along two sides of a given block, and the encoder groups a first subset of the plurality of adjacent samples adjacent to a first side of the given block into a first group (110) of one or more consecutive adjacent samples, and groups a second subset of the plurality of adjacent samples adjacent to a second side of the given block into a second group (110) of one or more consecutive adjacent samples, and performs a reduction by performing downsampling or averaging on each of the first and second groups of one or more adjacent samples having three or more adjacent samples, such that a first sample value is obtained from the first group and a second sample value is obtained for the second group, and the encoder selects a linear or affine-linear transformation according to a set index from a given set of linear or affine-linear transformations, such that two different states of the set index select one of the linear or affine-linear transformations of the given set of linear or affine-linear transformations, subjects the reduced set of sample values to a given linear or affine-linear transformation, assumes a first state of two different states in the form of a first vector to generate an output vector of predicted values, distributes the predicted values of the output vector to given samples of a given block along a first scan order, assumes a second state of two different states in the form of a second vector, such that the first vector is different from the second vector, such that a component input with one of the first sample values in the first vector has one of the second sample values in the second vector input, and a component input with one of the second sample values in the first vector has one of the first sample values in the second vector input, such that an output vector of predicted values is generated, and the predicted values of the output vector are configured to be distributed to given samples of the given block transposed with respect to the first scan order along a second scan order.

[0190] Additionally or alternatively, the encoder is such that each linear or affine linear transformation within the first set of linear or affine linear transformations is for converting N1 sample values into a predicted value of w1*h1 of a w1xh1 array of sample positions, and each linear or affine linear transformation within the first set of linear or affine linear transformations is for converting N2 sample values into a predicted value of w2*h2 of a w2xh2 array of sample positions, and for a first predetermined one of the first set of width / height pairs, w1 exceeds the width of the first predetermined width / height pair or h1 exceeds the height of the first predetermined width / height pair, and for a second predetermined one of the first set of width / height pairs, neither w1 exceeds the width of the second predetermined width / height pair nor h1 exceeds the height of the second predetermined width / height pair, and the encoder performs the reduction (100) by downsampling or averaging a plurality of adjacent samples to obtain a reduced set (102) of sample values, and as a result, when a predetermined block is of the first predetermined width / height pair and when a predetermined block is of the second predetermined width / height pair, the reduced set (102) of sample values has N1 sample values, and the reduced set of sample values is subjected to the selected linear or affine linear transformation by using only a first sub-part of the selected linear or affine linear transformation associated with the subsampling of the w1xh1 array of sample positions along the width dimension when w1 exceeds the width of one width / height pair or along the height dimension when h1 exceeds the height of one width / height pair when the predetermined block is of the first predetermined width / height pair, and is subjected entirely to the selected linear or affine linear transformation when the predetermined block is of the second predetermined width / height pair, and may be configured to perform so.

[0191] Additionally or alternatively, the encoder may be configured such that each linear or affine linear transformation within the first set of linear or affine linear transformations converts N1 sample values into w1*h1 predicted values of a w1xh1 array at a sample position where w1 = h1, and each linear or affine linear transformation within the first set of linear or affine linear transformations converts N2 sample values into w2*h2 predicted values of a w2xh2 array at a sample position where w2 = h2. 7. Example of FIG. 11 FIG. 11 shows another example that can be interpreted from the examples of FIGS. 3 to 9 (in particular, some features may be directly derived from FIG. 4 and are therefore not repeated here).

[0192] FIG. 11 shows a possible implementation of the decoder 54 of FIG. 4, i.e., an implementation that is compatible with the implementation of the encoder 14 of FIG. 10. In particular, the adder 42' and the predictor 44' can be connected to the prediction loop in the same way as the encoder 14 of FIG. 10. The reconstructed, i.e., inverse quantized and inverse transformed, prediction residual signal applied to the adder 42' may be derived by a sequence of entropy decoders that reverse the entropy encoding of the entropy encoder, followed by a residual signal reconstruction stage composed of an inverse quantizer and an inverse transformer 40' as in the case of the encoding side. The output of the decoder is the reconstruction of picture 10. The reconstruction of picture 10 may be directly available at the output of the adder 42' or may be available at the output of the in-loop filter.

[0193] As can be seen from the figure, the stages 813', 812', 813' may be those of the encoder 14, and the storage unit 1044 may store a set of matrices in the same way as the encoder 14. Therefore, the description is not repeated here. The index 944 (e.g., one or more of the above indices such as i, k, transposed index, set index, etc.) can be directly obtained from the data stream 12. The selection between the sets S0, S1, S2 can follow the size (e.g., H / K or M / N).

[0194] Additionally or alternatively, the decoder may be configured to derive a prediction residual (34'') from the data stream (12) for a given block (18) and to reconstruct the given block (18) (42') using the prediction residual (34'') and the predicted values (24') for given samples (24', 104, 108, 108').

[0195] Additionally or alternatively, the decoder may be configured to derive a prediction residual (34'') from the data stream (12) for a given block (18) and to obtain corresponding residual values for each of Q or Q red sets of given samples, and to correct the predicted values for each of Q or Q red sets of given samples by the corresponding residual values (34'') to reconstruct the given block (18) using the prediction residual (34'') and the predicted values (24', 104) of given samples (118', 118''), such that the corresponding reconstructed values (10) are, optionally, strictly linearly dependent on P red adjacent samples (102) within a set of sample values excluding clipping applied after prediction and / or correction. It may be so configured.

[0196] Additionally or alternatively, the decoder may be configured such that the decoder subdivides the picture (10) into a plurality of blocks of different block sizes including the given block (18), and the decoder selects a linear or affine-linear transform (19, 17M, A k ) according to the width W and height H of the given block (18), such that the linear or affine-linear transform selected for the given block (18) is selected from a first set of linear or affine-linear transforms as long as the width W and height H of the given block (81) are within a first set of width / height pairs, and is selected from a second set of linear or affine-linear transforms as long as the width W and height H of the given block are within a second set of width / height pairs separated from the first set of width / height pairs. It may be so configured.

[0197] Additionally or alternatively, the decoder is configured such that the decoder subdivides a picture (10) into a plurality of blocks of different block sizes including a predetermined block (18), and the decoder performs a linear or affine linear transformation (19, 17M, A k ) according to the width W and height H of the predetermined block (18), such that the linear or affine linear transformation selected for the predetermined block (18) is selected from a first set of linear or affine linear transformations as long as the width W and height H of the predetermined block (18) are within a first set of width / height pairs, and is selected from a second set of linear or affine linear transformations as long as the width W and height H of the predetermined block (18) are within a second set of width / height pairs separated from the first set of width / height pairs, and is configured to be selected from a third set of linear or affine linear transformations as long as the width W and height H of the predetermined block (18) are within one or more third sets of width / height pairs separated from the first and second sets of width / height pairs.

[0198] Additionally or alternatively, the decoder may be configured such that the third set of one or more width / height pairs simply comprises one width / height pair W’, H’ and each linear or affine linear transformation within the first set of linear or affine linear transformations is for converting N’ sample values to W’*H’ predicted values of a W’xH’ array of sample positions.

[0199] Additionally or alternatively, the decoder may be configured such that each of the first and second sets of width / height pairs comprises a first width / height pair W p , H p where W p is not equal to H p of a first width / height pair W p , H p and a second width / height pair W q , H q where H q =W p and W q =H p of a second width / height pair W q , H qmay be configured to include.

[0200] Additionally or alternatively, for each of the first and second sets of width / height pairs, the decoder may further include a third width / height pair W p ,H p where W p equals H p and H p >H q . Additionally or alternatively, for a given block (18), the decoder may be configured to read a set index (k) from the data stream (12) and select a linear or affine-linear transformation according to the set index (k) from a given set of linear or affine-linear transformations.

[0201] Additionally or alternatively, the decoder has a plurality of adjacent samples (17) extending one-dimensionally along two sides of a predetermined block (18), and the decoder groups a first subset of the plurality of adjacent samples adjacent to a first side of the predetermined block into a first group (110) of one or more consecutive adjacent samples, and groups a second subset of the plurality of adjacent samples adjacent to a second side of the predetermined block into a second group (110) of one or more consecutive adjacent samples, and performs a reduction (811) by performing downsampling or averaging on each of the first and second groups of one or more adjacent samples having three or more adjacent samples, such that a first sample value is obtained from the first group and a second sample value is obtained for the second group, and the decoder selects a linear or affine-linear transformation according to a set index from a predetermined set of linear or affine-linear transformations, such that two different states of the set index select one of the linear or affine-linear transformations of the predetermined set of linear or affine-linear transformations, subjects the reduced set of sample values to a predetermined linear or affine-linear transformation, assumes a first state of two different states in the form of a first vector to generate an output vector of predicted values, distributes the predicted values of the output vector to predetermined samples of the predetermined block along a first scan order, and assumes a second state of two different states in the form of a second vector, such that the first vector is different from the second vector, and as a result, a component input with one of the first sample values in the first vector has one of the second sample values in the second vector input, and a component input with one of the second sample values in the first vector has one of the first sample values in the second vector input, and as a result, generates an output vector of predicted values and distributes the predicted values of the output vector to predetermined samples of the predetermined block transposed with respect to the first scan order along a second scan order.

[0202] Additionally or alternatively, for each linear or affine linear transformation within the first set of linear or affine linear transformations, the decoder is configured to transform N1 sample values into w1*h1 predicted values of a w1xh1 array of sample positions, and for each linear or affine linear transformation within the first set of linear or affine linear transformations, the decoder is configured to transform N2 sample values into w2*h2 predicted values of a w2xh2 array of sample positions, and for a first predetermined one of the first set of width / height pairs, w1 exceeds the width of the first predetermined width / height pair or h1 exceeds the height of the first predetermined width / height pair, and for a second predetermined one of the first set of width / height pairs, w1 does not exceed the width of the second predetermined width / height pair and h1 does not exceed the height of the second predetermined width / height pair, and the decoder is configured to perform the reduction (100) by downsampling or averaging a plurality of adjacent samples to obtain a reduced set of sample values (102), such that, if a predetermined block is of the first predetermined width / height pair, and if a predetermined block is of the second predetermined width / height pair, the reduced set of sample values (102) has N1 sample values, and the reduced set of sample values is subjected to only the first sub-part of a selected linear or affine linear transformation associated with subsampling of a w1xh1 array of sample positions along the width dimension if w1 exceeds the width of one width / height pair or along the height dimension if h1 exceeds the height of one width / height pair when the predetermined block is of the first predetermined width / height pair, and is subjected completely to the selected linear or affine linear transformation when the predetermined block is of the second predetermined width / height pair.

[0203] Additionally or alternatively, the decoder may be configured such that each linear or affine linear transformation within the first set of linear or affine linear transformations transforms N1 sample values into the w1*h1 predicted values of a w1xh1 array at a sample position where w1=h1, and each linear or affine linear transformation within the first set of linear or affine linear transformations transforms N2 sample values into the w2*h2 predicted values of a w2xh2 array at a sample position where w2=h2. 8. Consideration of the effects of the present technology

[0204] Note that in some examples, other effects may be obtained that exceed the effective use of bit shifts, independent of operations such as bit shifts for averaging and / or interpolation (which ultimately result in the effect of reducing computational effort).

[0205] In particular, in this embodiment, the prediction mode can be shared across different block shapes such that the selection of the ALWIP matrix 17M (e.g., in step 812a) is performed for a limited number of sets. For example, there may be a set of ALWIP matrices that is less than the possible dimensions (e.g., height / width pairs) of the block 18 to be predicted. Referring to FIG. 12, different width / height pairs of the predicted block 18 can be mapped to one of sets S0 (e.g., n0 matrices, e.g., n0 = 16), S1 (e.g., n1 matrices, e.g., n1 = 8), and S2 (e.g., n2 matrices, e.g., n2 = 6) (different subdivisions may be possible).

[0206] For example, the 16×8 matrix of set S1 has dimensions It may be shared by the prediction mode of a block having any one of 4×8, 4×16, 4×32, 4×64, 8×4, 8×8, 16×4, 32×4, and 64×4. The 64×8 matrix of set S2 may be shared by the prediction mode of a block having any dimension of 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×6. The P red It is only necessary to execute the technique as described for the reduction step 811 (referenced above) to reduce the dimension of the boundary 17 for the number of samples in set 102, but in step 812, the original dimension of the predicted block 18 is irrelevant. In step 813 (if implemented), it is possible to reach the complete prediction of the block simply by performing interpolation.

[0207] It should be noted that this technique makes it possible to reduce the storage space required in the storage space 1044 in an unexpected dimension of 16*16*4 + 8*16*8 + 6*64*8 = 5120 values (for example, each value is, for example, an 8-bit value).

[0208] In comparison, in the prior art, it is necessary to use a set of matrices for each width / height pair. As can be easily understood from FIG. 12, 25 sets are required! It can be easily understood that 25 sets of matrices require much more than the storage space for 5120 values. Therefore, in order to reduce the required storage space, it is necessary to reduce the number of matrices in each set, but if there are few matrices that can be freely used for prediction, the quality will deteriorate!

[0209] The reduction of the storage space considering the sharing technique is further amplified by the reduction of the size of the stored matrix itself. For example, the prediction of an MxN = 64x64 block would require a matrix of size QxP = (M*N)x(M + N), that is, (64*64)*(64 + 64) = 524288 values to be stored in the storage space! Therefore, in this technique, it is possible to save even more storage space than expected. Thus, the present technology makes it possible to reduce the number of parameters that need to be stored in unit 1044.

[0210] Regardless of whether bit shifting is actually used, the storage resources available for free use by the encoder or decoder can be reduced, or conversely, more prediction modes can be used for the parity of the storage space.

[0211] Nevertheless, the optimal effect is achieved by combining the bit shifting technique (in steps 811 and / or 813) with one that shares the same prediction mode for multiple modes (in step 812).

[0212] Regarding the conventional approach of using 25 different sets for 25 different height / width pairs, the present technology may apparently be interpreted as increasing complexity (since steps 811 and / or 813 are not considered in the prior art). However, the introduction of steps 811 and / or 813 can be more compensated for by the reduction in multiplication.

[0213] Furthermore, regarding the conventional approach of using 25 different sets for 25 different height / width pairs, the instructions required to control this process require more storage space (since additional instructions for steps 811 and / or 813 are stored). However, the need to store the instructions for steps 811 and / or 813 can be further compensated for by the reduction in space implied by the reduction in the number of matrices stored. 9. Further Embodiments and Examples

[0214] Generally, an embodiment can be implemented as a computer program product having program instructions that operate to execute one of the present methods when the computer program product operates on a computer. The program instructions can be stored, for example, on a machine-readable medium. Other embodiments include a computer program for performing one of the methods described herein, stored on a machine-readable carrier.

[0215] In other words, an embodiment of the method of the present invention is a computer program having program instructions for performing one of the methods described herein when the computer program is executed on a computer.

[0216] Accordingly, a further embodiment of the method of the present invention includes a computer program for performing one of the methods described herein, a data carrier medium (or digital storage medium or computer-readable medium) on which it is recorded. The data carrier medium, digital storage medium, or recording medium is tangible and / or non-transitory, not an intangible and transient signal.

[0217] Accordingly, a further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals can be transferred, for example, via a data communication connection, such as the Internet. A further embodiment includes processing means, such as a computer or a programmable logic device, for performing one of the methods described herein. A further embodiment includes a computer on which a computer program for performing one of the methods described herein is installed.

[0218] A further embodiment according to the present invention includes an apparatus or system for transferring (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver can be, for example, a computer, a mobile device, a memory device, etc. This apparatus or system can include, for example, a file server for transferring the computer program to the receiver.

[0219] In some examples, a programmable logic device (e.g., a field programmable gate array) can be used to execute some or all of the functions of the methods described herein. In some embodiments, the field programmable gate array can cooperate with a microprocessor to execute one of the methods described herein. Generally, these methods can be executed by any suitable hardware device.

[0220] The above examples are merely illustrative of the above principles. It will be understood that changes and modifications to the configurations and details described herein are obvious. Therefore, it is intended to be limited by the impending claims and not by the specific details shown by the description and illustration of the examples herein.

[0221] Elements that are equal or equivalent or have functions that are equal or equivalent are indicated by equal or equivalent reference numerals in the following description, even if they occur in different figures. References

[0222] [1] P. Helle et al., “Non-linear weighted intra prediction”, JVET-L0199, Macao, China, October 2018.

[0223] [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, “JVET common test conditions and software reference configurations for SDR video”, JVET-K1010, Ljubljana, SI, July 2018.

Claims

1. A method for predicting at least a part of a picture, comprising: the method comprising: downsampling a set of sample values adjacent to a block of the picture, wherein the picture includes a plurality of blocks; selecting a matrix of weight values according to the width and height of the block, wherein the matrix is selected from one of the following sets of matrices; when both the width and the height of the block are 4, a first set of matrices; when both the width and the height of the block are 8, or when only one of the width and the height is 4, a second set of matrices; otherwise, a third set of matrices; generating a plurality of predicted values based at least in part on the downsampled set of sample values, wherein the generating includes applying the matrix of weight values; The method as described above.

2. Deriving additional predicted sample values for the block by upsampling the plurality of predicted values. The method according to claim 1, further comprising the above.

3. The set of sample values adjacent to the block of the picture extends one-dimensionally along the upper part of the block and one-dimensionally along the left part of the block. The method according to claim 1.

4. Decoding data corresponding to the picture from a data stream. The method according to claim 1, further comprising the above.

5. Encoding data corresponding to the picture into a data stream. The method according to claim 1, further comprising the above.

6. A non-transitory computer-readable medium including instructions that, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 5.

7. An apparatus for predicting at least a part of a picture, comprising: a non-transitory computer-readable medium; and at least one processor communicatively connected to the non-transitory computer-readable medium, wherein the at least one processor is configured to read instructions from the non-transitory computer-readable medium and perform the method according to any one of claims 1 to 5. The apparatus as described above.

Images