Intra-prediction using linear or affine transformations with neighbor reduction

Abstract

A technique for encoding / decoding a video signal is provided, for example, implemented in a decoder, an encoder, a method, and a non-transitory storage unit storing instructions for performing the method. The decoder or the encoder may be configured to predict a predetermined block of an image by reducing a plurality of neighboring samples to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples, subjecting the reduced set of sample values ​​to a linear or affine linear transformation to obtain a predicted value for a predetermined sample of the predetermined block.

Description

Technical Field

[0001] The present application relates to methods and / or devices for video encoding and / or for performing intra prediction. Background Art

[0002] 1 Introduction

[0003] In the following, different examples, embodiments and aspects of the invention will be described. At least some of these examples, embodiments and aspects relate in particular to methods and / or devices for video encoding and / or for performing intra-prediction, for example, using linear or affine transformations with neighboring sample reduction, and / or for optimizing video delivery (e.g., broadcasting, streaming, file playback, etc.), for example, for video applications and / or virtual reality applications. In addition, examples, embodiments and aspects may generally refer to technologies of High Efficiency Video Coding (HEVC) or subsequent versions. Moreover, further embodiments, examples and aspects will be defined by the appended claims.

[0004] It should be noted that any embodiment, example and aspect defined by the claims may be supplemented by any details (features and functions) described in the following sections.

[0005] Furthermore, the embodiments, examples, and aspects described in the following sections may be used independently and may also be supplemented by any features in another section or any features included in the claims.

[0006] In addition, it should be noted that the various examples, embodiments and aspects described herein can be used alone or in combination. Therefore, details can be added to each of the examples, embodiments and aspects without adding details to another aspect.

[0007] It should also be noted that the present disclosure explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0008] In addition, the features and functions disclosed herein with respect to the method may also be used in the device. In addition, any features and functions disclosed herein with respect to the device may also be used in the corresponding method. In other words, the method disclosed herein may be supplemented by any features and functions described with respect to the device. In addition, as in other sections, such as “ Further implementation Examples and Examples ", etc., any features and functions described herein can be implemented in hardware or software, or using a combination of hardware and software.

[0009] Furthermore, any features described within brackets ("(...)" or "[...]") may be considered optional in some examples, embodiments, or aspects. Summary of the invention

[0010] According to one aspect, there is provided a decoder for decoding an image from a data stream, configured to predict a predetermined block of the image using a plurality of neighboring samples by the following steps:

[0011] reducing (e.g., by averaging and downsampling) a plurality of neighboring samples to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples,

[0012] The reduced set of sample values ​​is subjected to a linear or affine linear transform to obtain a prediction value for a predetermined sample of a predetermined block.

[0013] In an example, the decoder is further configured to perform downscaling of the plurality of neighboring samples, such as by averaging, downsampling, to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples.

[0014] In some cases, the decoder may further derive the prediction value for the further samples of the predetermined block by interpolation based on the prediction value for the predetermined sample with the plurality of neighboring samples. Thus, an upsampling operation is applied.

[0015] According to one aspect, an encoder for encoding an image into a data stream is provided. The encoder is configured to predict a predetermined block of the image using a plurality of neighboring samples by the following steps:

[0016] reducing (e.g., by averaging and downsampling) a plurality of neighboring samples to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples,

[0017] The reduced set of sample values ​​is subjected to a linear or affine linear transform to obtain a prediction value for a predetermined sample of a predetermined block.

[0018] In an example, the encoder is further configured to perform downscaling of the plurality of neighboring samples, such as by averaging, downsampling, to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples.

[0019] In some cases, the encoder may also derive prediction values ​​for further samples of the predetermined block by interpolation based on the prediction value for the predetermined sample and a plurality of neighboring samples. Thus, an upsampling operation is applied.

[0020] In an example, a system may be provided that includes the encoder described above and / or the decoder described above. In some examples, the hardware and / or at least some program routines of the encoder may be the same as those of the decoder.

[0021] In an example, a decoding method may be provided, comprising:

[0022] Predicting a predetermined block of an image using a number of neighboring samples is done by:

[0023] For example, by averaging and downsampling, a plurality of neighboring samples are reduced to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples,

[0024] The reduced set of sample values ​​is subjected to a linear or affine linear transform to obtain a prediction value for a predetermined sample of a predetermined block.

[0025] In an example, a coding method may be provided, comprising:

[0026] Predicting a predetermined block of an image using a number of neighboring samples is done by:

[0027] For example, by averaging and downsampling, a plurality of neighboring samples are reduced to obtain a reduced set of sample values ​​that is less in number of samples than the plurality of neighboring samples,

[0028] The reduced set of sample values ​​is subjected to a linear or affine linear transform to obtain a prediction value for a predetermined sample of a predetermined block.

[0029] In an example, a non-transitory storage unit storing instructions may be provided, which, when executed by a processor, cause the processor to perform the above-described method. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 and 2 An encoder example is shown.

[0031] Figure 3 and 4 A decoder example is shown.

[0032] Figure 5 Shows the prediction graph for a block.

[0033] Figure 6 Shows matrix operations.

[0034] Figure 7.1-7.4 An operation example according to the example is shown.

[0035] Figure 8.1 and 8.2 An example of a method according to an example is shown.

[0036] Figure 9a and 9b An example of a component is shown.

[0037] Fig.10 An encoder example is shown.

[0038] Fig.11 A decoder example is shown.

[0039] Fig.12 A scheme related to the dimension of a block to be predicted by a prediction mode is shown.

[0040] Fig.13 Aspects useful for understanding the present invention are shown. DETAILED DESCRIPTION

[0041] 2. Encoder and Decoder

[0042] Various examples are described below that can help achieve more efficient compression when using intra prediction. Some examples achieve an increase in compression efficiency by spending a set of intra prediction modes. For example, the latter can be added to other intra prediction modes designed heuristically, or the latter can be provided exclusively. Even other examples utilize both of the characteristics just discussed.

[0043] In order to simplify the understanding of the following examples of the present application, the description starts by presenting a possible encoder and a decoder adapted thereto, in which the examples subsequently outlined in the present application can be built. Figure 1 An apparatus is shown for encoding an image 10 into a data stream 12 in a block-wise manner. The apparatus is indicated using reference numeral 14 and may be a still image encoder or a video encoder. In other words, when the encoder 14 is configured to encode a video 16 including the image 10 into the data stream 12, the image 10 may be the current image in the video 16, or the encoder 14 may encode the image 10 exclusively into the data stream 12.

[0044] As mentioned above, the encoder 14 performs encoding in a block-by-block manner or on a block basis. To this end, the encoder 14 subdivides the image 10 into blocks, i.e. the encoder 14 encodes the image 10 as units of the data stream 12. Examples of possible subdivisions of the image 10 into blocks 18 will be explained in more detail below. In general, the subdivision may result in blocks 18 of constant size, such as an array of blocks arranged in rows and columns, or in blocks 18 of different block sizes, such as by using a hierarchy multi-tree subdivision, which subdivides into an array of tree blocks starting from the entire image area of ​​the image 10 or starting from a pre-segmentation of the image 10, wherein these examples should not be regarded as excluding other possible ways of subdividing the image 10 into blocks 18.

[0045] Furthermore, the encoder 14 is configured as a predictive encoder to predictively encode the image 10 into the data stream 12. For a particular block 18, this means that the encoder 14 determines a prediction signal for the block 18 and encodes a prediction residual, i.e. a prediction error of the prediction signal derived from the actual image content at the block 18, into the data stream.

[0046] The encoder 14 may support different prediction modes in order to derive a prediction signal for a particular block. The prediction mode that is important in the following example is the intra-prediction mode, which is based on spatial prediction of the interior of which block 18 from samples already encoded from neighboring images. The encoding of the image 10 into a data stream 12 and the corresponding decoding process can be based on a specific coding order 20 defined between the blocks 18. For example, the coding order 20 can traverse the blocks in a raster scan order, for example, traversing each row from top to bottom and from left to right row by row. If in the case of a hierarchical multi-tree based subdivision, a raster scan order can be applied within each hierarchical level, where a depth-first traversal order can be applied, i.e., a leaf node within a block at a particular hierarchical level can be located before a block of the same hierarchy with the same kinship block according to the coding order 20. Depending on the coding order 20, the samples of the neighboring, already encoded blocks 18 can generally be located on one side or multiple sides of the block 18. In the case of the example presented herein, for example, the samples of the neighboring, encoded blocks of the block 18 are arranged at the top and left side of the block 18.

[0047] The intra-prediction mode may not be the only mode supported by the encoder 14. For example, in case the encoder 14 is a video encoder, the encoder 14 may also support an inter-prediction mode based on temporally predicting the block 18 from an image of a previously encoded video. Such an intra-prediction mode may be a motion compensated prediction mode, according to which a motion vector is signaled for such a block 18, the motion vector indicating the relative spatial offset of the portion from which the prediction signal for the block 18 is to be derived as a copy. Additionally or alternatively, other non-intra-prediction modes may also be available, such as an inter-prediction mode in case the encoder 14 is a multi-view encoder, or a non-prediction mode, according to which the interior of the block 18 is encoded without any prediction.

[0048] Before starting to focus the description of the present application on the intra-prediction mode, a more specific example of a possible block-based encoder, that is, a possible implementation of the encoder 14, as described with respect to Figure 2 described, and then presented separately with Figure 1 and 2 Figure 2. Two corresponding examples of the adapted decoder.

[0049] Figure 2 Show Figure 1 A possible implementation of the encoder 14 is that the encoder is configured to encode the prediction residual using transform coding, although this is essentially an example and the application is not limited to such prediction residual coding. Figure 2, the encoder 14 comprises a subtractor 22 configured to subtract the corresponding prediction signal 24 from the inbound signal, i.e. the image 10, or the current block 18 in the block-based case, so as to obtain a prediction residual signal 26, which is then encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 consists of a lossy encoding stage 28a and a lossless encoding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and comprises a quantizer 30, which quantizes the samples of the prediction residual signal 26. As mentioned above, the present example uses transform coding of the prediction residual signal 26, and therefore, the lossy encoding stage 28a comprises a transform stage 32 connected between the subtractor 22 and the quantizer 30, so as to quantize this spectrally decomposed prediction residual 26, wherein the quantization of the quantizer occurs on the transform coefficients representing the residual signal 26. The transform may be DCT, DST, FFT, Hadamard transform, etc. The transformed and quantized prediction residual signal 34 is then losslessly encoded by a lossless encoding stage 28b, which is an entropy encoder that entropy encodes the quantized prediction residual signal 34 into a data stream 12. The encoder 14 further comprises a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 in order to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner also usable at the decoder, i.e. taking into account the coding losses in the quantizer 30. To this end, the prediction residual reconstruction stage 36 comprises a dequantizer 38 that performs the inverse of the quantization of the quantizer 30, followed by an inverse transformer 40 that performs an inverse transform with respect to the transform performed by the transformer 32, e.g. the inverse of the spectral decomposition, e.g. the inverse of any of the specific transform examples described above. The encoder 14 comprises an adder 42 that adds the reconstructed prediction residual signal as output of the inverse transformer 40 to the prediction signal 24 to output a reconstructed signal, i.e. reconstructed samples. This output is fed to the predictor 44 of the encoder 14, which then determines the prediction signal 24 based thereon. The predictor 44 supports the above description of Figure 1 All the prediction models already discussed. Figure 2 It is also shown that, in the case where the encoder 14 is a video encoder, the encoder 14 may also include an in-loop filter 46 having the filter fully reconstruct the picture which, after having been filtered, forms a reference picture for the predictor 44 with respect to the inter-prediction blocks.

[0050] As described above, the encoder 14 operates on a block basis. For the subsequent description, the block basis of interest is the subdivision of the image 10 into blocks, which are blocks of an intra-prediction mode selected from a set or multiple intra-prediction modes supported by the predictor 44 or the encoder 14, respectively, and the selected intra-prediction mode is performed separately. However, there may also be other types of blocks into which the image 10 is subdivided. For example, the above-mentioned determination of whether the image 10 is inter-coded or intra-coded may be performed at a granularity or in units of blocks different from the block 18. For example, the inter / intra mode determination may be performed at the level of the coding blocks into which the image 10 is subdivided, and each coding block is subdivided into prediction blocks. The prediction blocks of the coding blocks that have been determined to use intra-prediction are respectively subdivided into intra-prediction mode determinations. To this end, for each of these prediction blocks, it is determined which supported intra-prediction mode is applied to each prediction block. These prediction blocks will form the block 18 of interest here. The prediction blocks within the coding blocks associated with inter-prediction will be processed differently by the predictor 44. They are inter-predicted from the reference image by determining the motion vector and copying the prediction signal for this block, the prediction signal for this block coming from the position in the reference image pointed to by the motion vector. At the unit where the transformation is carried out by the transformer 32 and the inverse transformer 40, another block subdivision is associated with the subdivision into transform blocks. For example, the transformed block may be the result of a further subdivision of the coding block. Naturally, the examples set forth here should not be considered restrictive and other examples also exist. For the sake of completeness only, it should be noted that, for example, the subdivision into coding blocks may use a multi-tree subdivision and that prediction blocks and / or transform blocks may also be obtained by further subdividing the coding block using a multi-tree subdivision.

[0051] exist Figure 3 Description for Figure 1 The decoder 54 or device for block-by-block decoding of the encoder 14. This decoder 54 performs the opposite operation of the encoder 14, that is, it decodes the image 10 from the data stream 12 in a block-by-block manner. And for this purpose, multiple intra prediction modes are supported. For example, the decoder 54 may include a residual provider 156. Figure 1All other possibilities discussed are also valid for the decoder 54. For this purpose, the decoder 54 can be a still image decoder or a video decoder, and the decoder 54 also supports all prediction modes and prediction possibilities. The difference between the encoder 14 and the decoder 54 is mainly that the encoder 14 selects the encoding decision according to some optimization, for example, in order to minimize some cost function that can depend on the coding rate and / or coding distortion. One of these encoding options or encoding parameters may involve selecting the intra-prediction mode to be used for the current block 18 among the available or supported intra-prediction modes. The selected intra-prediction mode can then be signaled by the encoder 14 for the current block 18 in the data stream 12, and the decoder 54 uses the signaling in the data stream 12 for the block 18 to reselect. Similarly, the subdivision of the image 10 into blocks 18 can be optimized in the encoder 14, and the corresponding subdivision information can be transmitted in the data stream 12, and the decoder 54 restores the subdivision of the image 10 into blocks 18 based on the subdivision information. In summary, the decoder 54 may be a block-based prediction decoder, and in addition to the intra prediction mode, the decoder 54 may also support other prediction modes, such as the inter prediction mode, as in the case where the decoder 54 is a video decoder. In decoding, the decoder 54 may also use the Figure 1 The coding order 20 discussed is followed both at the encoder 14 and at the decoder 54, so that the same neighboring samples for the current block 18 are available at the encoder 14 and at the decoder 54. Therefore, in order to avoid unnecessary repetitions, the description of this mode of operation of the encoder 14 should also apply to the decoder 54, as far as it concerns the subdivision of the image 10 into blocks, for example, concerning prediction, and concerning the coding of the prediction residual. The difference lies in the fact that the encoder 14 selects some coding options or coding parameters and signals in the data stream 12 by optimization, and then inserts the coding parameters derived from the data stream 12 by the decoder 54, in order to redo the prediction, subdivision, etc.

[0052] Figure 4 show Figure 3 A possible implementation of the decoder 54 is, that is, Figure 2 shown Figure 1 The implementation of the encoder 14 is adapted to the decoder implementation. Figure 4 Many components of the decoder 54 are similar to those in Figure 2 To indicate these components, the same reference symbols followed by a prime are used. Figure 4 In particular, the adder 42', the optional in-loop filter 46' and the predictor 44' are used in conjunction with their Figure 2is connected to the prediction loop in the same way as in the encoder of . The reconstructed, i.e. dequantized and retransformed prediction residual signal applied to the adder 42' is derived by a series of entropy decoders 56 which reverse the entropy coding of the entropy encoder 28b, followed by a residual signal reconstruction stage 36', which consists of a dequantizer 38' and an inverse transformer 40', just as in the case at the encoding end. The output of the decoder is a reconstruction of the image 10. The reconstruction of the image 10 can be used directly at the output of the adder 42' or, alternatively, at the output of the in-loop filter 46'. It is possible to provide some post-filters at the output of the decoder so that the reconstruction of the image 10 is subjected to some post-filtering in order to improve the quality of this image, but this option is not provided in the specification. Figure 4 Depicted in.

[0053] Again, about Figure 4 , the above about Figure 2 The description proposed for Figure 4 It should also be valid, except that only the encoder performs the optimization tasks and the associated decisions about coding options. However, all descriptions of block subdivision, prediction, dequantization and retransformation are for Figure 4 The decoder 54 is also effective.

[0054] 3. ALWIP

[0055] Even though it is not always necessary to specifically implement the techniques discussed herein, ALWIP, some non-limiting examples of ALWIP are discussed herein.

[0056] The present application relates to an improved intra prediction mode concept for block-by-block image coding, such as used in video codecs such as HEVC or any subsequent versions of HEVC.

[0057] Intra-prediction mode is widely used in image and video coding. In video coding, intra-prediction mode competes with other prediction modes, such as inter-prediction mode, such as motion compensated prediction mode. In intra-prediction mode, the current block is predicted based on neighboring samples, that is, samples have been encoded as far as the encoder side is concerned, and samples have been decoded as far as the decoder side is concerned. The neighboring sample values ​​are extrapolated into the current block to form a prediction signal for the current block, where the prediction residual is sent in the data stream for the current block. The better the prediction signal, the lower the prediction residual, and therefore, fewer bits are needed to encode the prediction residual.

[0058] To be effective, several aspects should be considered to form an efficient frame operation for intra prediction in a block-by-block image coding environment. For example, the greater the number of intra prediction modes supported by the codec, the greater the side information rate consumption in order to signal the selection to the decoder. On the other hand, the set of supported intra prediction modes should be able to provide a good prediction signal, i.e. a prediction signal that results in a low prediction residual.

[0059] If one seeks to use the improved intra prediction mode concept, the intra prediction mode concept allows for a more efficient compression of a block-by-block image codec.

[0060] This object is achieved by a so-called Affine Linear Weighted Intra Predictor (ALWIP) transform. A device (encoder or decoder) for decoding an image block by block from a data stream is disclosed, the device supporting at least one intra prediction mode according to: determining an intra prediction signal for a block of a predetermined size of the image by applying a first template of samples neighboring a current block to an affine linear predictor, the affine linear predictor being referred to hereinafter as Affine Linear Weighted Intra Predictor (ALWIP).

[0061] The apparatus may have at least one of the following properties (the same properties may apply to a method or another technology, for example, being implemented in a non-transitory storage unit storing instructions which, when executed by a processor, cause the processor to implement the method and / or operate as the apparatus).

[0062] 3.1 The proposed predictor can complement other predictors

[0063] The intra prediction modes supported by the device are complementary to the other intra prediction modes of the codec. Therefore, they can be complementary to the DC, planar or angular prediction modes defined in the HEVC codec. JEM reference software. From now on, the latter three types of intra prediction modes will be referred to as traditional intra prediction modes. Therefore, for a given block in intra mode, a flag needs to be parsed by the decoder, which indicates whether one of the intra prediction modes supported by the device is to be used.

[0064] 3.2 More than one proposed prediction model

[0065] The device may include more than one ALWIP mode. Therefore, in case the decoder knows which of the ALWIP modes supported by the device is to be used, the decoder needs to parse additional information indicating which of the ALWIP modes supported by the device is to be used.

[0066] The signaling of supported modes may have the following characteristics: the encoding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes requires fewer bins and which modes require more bins may depend on information extracted from an already decoded bitstream, or may be fixed in advance.

[0067] 4Some aspects

[0068] Figure 2 A decoder 54 is shown for decoding an image from the data stream 12. The decoder 54 may be configured to decode a predetermined block 18 of the image. In particular, the predictor 44 may be configured to map a set of P neighboring samples adjacent to the predetermined block 18 onto Q predicted values ​​for a set of samples of the predetermined block using a linear or affine linear transform [e.g., ALWIP].

[0069] like Figure 5 As shown, a predetermined block 18 includes a Q value to be predicted (which will be a "predicted value" at the end of the operation). If block 18 has M rows and N columns, then Q=M*N. The Q value of block 18 may be in the spatial domain (e.g., pixels) or in a transform domain (e.g., DCT, discrete wavelet transform, etc.). The Q value of block 18 may be predicted based on the P value obtained from neighboring blocks 17a-17c, which are typically adjacent to block 18. The P value of neighboring blocks 17a-17c may be the closest location to block 18 (e.g., adjacent). The P value of neighboring blocks 17a-17c has been processed and predicted. The P values ​​are represented as numerical values ​​in parts 17'a-17'c to distinguish them from the blocks to which they belong (in some examples, 17'b is not used).

[0070] like Figure 6 As shown, in order to perform the prediction, it is possible to operate with a first vector 17P having P entries (each associated with a specific position in the neighboring part 17'a-17'c), a second vector 18Q having Q entries (each associated with a specific position in the block 18), and a mapping matrix 17M (each row associated with a specific position in the block 18 and each column associated with a specific position in the neighboring part 17'a-17'c). Therefore, the mapping matrix 17M performs the prediction of the P value of the neighboring part 17'a-17'c of the value entering the block according to a predetermined pattern. Therefore, the entries in the mapping matrix 17M may be understood as weighting factors. In the following paragraphs, we will use the symbols 17a-17c to replace the neighboring part of 17'a-17'c referring to the boundary.

[0071] In the art, several conventional modes are known, such as DC mode, planar mode and 65 directional prediction modes. For example, 67 modes are known.

[0072] However, it has been noted that a different mode may also be used, referred to herein as a linear or affine linear transformation. The linear or affine linear transformation comprises P*Q weighting factors, wherein at least 1 / 4P*Q weighting factors are non-zero weighting values, and for each of the Q prediction values, comprises a series of P weighting factors associated with the respective prediction value. When the series are arranged one after another according to a raster scan order between samples of a predetermined block, an omnidirectional nonlinear envelope is formed.

[0073] Fig.13 An example of a graph 70 is shown, which maps the P positions of neighboring values ​​17'a-17'c (templates), the Q positions of neighboring samples 17'a-17'c and the values ​​of P*Q weighting factors in the matrix 17M. Plane 72 is an example of the envelope of the series for a DC transform (it is a plane for a DC transform). The envelope is obviously planar and is therefore excluded by the definition of a linear or affine linear transform (ALWIP). Another example is a simulation matrix that results in the simulation of an angular mode: the envelope will be excluded from the ALWIP definition and, frankly, will look like a ramp, sloping from top to bottom in one direction of the P / Q plane. The planar mode will have a different envelope than the 65 directional prediction modes, but it will be linear in at least one direction, i.e., for example, all directions for the DC of the example, and a ramp direction for the angular mode.

[0074] Conversely, the envelope of a linear or affine transformation will not be omnidirectionally linear. It will be appreciated that in certain circumstances this type of transformation may be optimal for performing the prediction of block 18. It has been noted that it is preferred that at least 1 / 4 of the weighting factors be different from zero (i.e. at least 25% of the P*Q weighting factors be different from zero).

[0075] The weighting factors may be independent of each other according to any conventional mapping rules. Thus, the matrix 17M may be such that the values ​​of its entries have no clearly identifiable relationship. For example, the weighting factors cannot be described by any analytical or differential function.

[0076] In an example, the ALWIP transformation is such that the average of the maximum values ​​of the cross-correlations between a first series of weighting factors associated with respective predicted values ​​and a second series of weighting factors associated with predicted values ​​other than respective predicted values ​​or an inverse version of the latter series, whether or not it results in a higher maximum value, may be below a predetermined threshold value (e.g., 0.2 or 0.3 or 0.35 or 0.1, for example, the threshold value is in the range of 0.05 to 0.035). For example, for each pair of rows (i1, i2) of the ALWIP matrix 17M, the cross-correlation may be calculated by multiplying the P value of the i1-th row by the P value of the i2-th row. For each obtained cross-correlation, the maximum value may be obtained. Thus, the average value (average value) for the entire matrix 17M may be obtained (that is, the maximum values ​​of the cross-correlations in all combinations are averaged). Thereafter, the threshold value may be, for example, 0.2 or 0.3 or 0.35 or 0.1, for example, a threshold value in the range of 0.05 to 0.035.

[0077] The P neighboring samples of the blocks 17a-17c may be located along a one-dimensional path that extends along a boundary (e.g., 18c, 18a) of the predetermined block 18. For each of the Q predicted values ​​of the predetermined block 18, the P weighting factors for the series of the respective predicted values ​​may be ordered in a predetermined direction (e.g., from left to right, from top to bottom, etc.) across the one-dimensional path.

[0078] In an example, the ALWIP matrix 17M may be non-diagonal or non-block diagonal.

[0079] An example of an ALWIP matrix 17M for predicting a 4x4 block 18 from 4 already predicted neighboring samples may be:

[0080] {

[0081] {37,59,77,28},

[0082] {32,92,85,25},

[0083] {31,69,100,24},

[0084] {33,36,106,29},

[0085] {24,49,104,48},

[0086] {24,21,94,59},

[0087] {29,0,80,72},

[0088] {35,2,66,84},

[0089] {32,13,35,99},

[0090] {39,11,34,103},

[0091] {45,21,34,106},

[0092] {51,24,40,105},

[0093] {50,28,43,101},

[0094] {56,32,49,101},

[0095] {61,31,53,102},

[0096] {61,32,54,100}

[0097] }.

[0098] (Here, {37, 59, 77, 28} is the first row of matrix 17M; {32, 92, 85, 25} is the second row; and {61, 32, 54, 100} is the 16th row.) Matrix 17M has dimensions 16x4 and includes 64 weighting factors (as a result of 6*4=64). This is because matrix 17M has dimensions QxP, where Q=M*N, which is the number of samples of block 18 to be predicted (block 18 is a 4x4 block), and P is the number of samples of samples that have been predicted. Here, M=4, N=4, Q=16 (as a result of M×N=4×4=16), and P=4. The matrix is ​​off-diagonal and non-block-diagonal, and is not described by a particular rule.

[0099] It can be seen that less than 1 / 4 of the weighting factors are zero (in the case of the matrix shown above, one of the sixty-four weighting factors is zero). The envelope formed by these values, when arranged one below the other in raster scan order, forms an omnidirectional non-linear envelope.

[0100] Even though the above description is primarily discussed with reference to a decoder (eg, decoder 54), the same may be performed at an encoder (eg, encoder 14).

[0101] In some examples, for each block size (in a set of block sizes), ALWIP transforms of intra-prediction modes within the second set of intra-prediction modes for the respective block sizes are different from each other. Additionally or alternatively, for block sizes within the set of block sizes, cardinalities of the second set of intra-prediction modes may overlap, but related linear or affine linear transforms of intra-prediction within the second set of intra-prediction modes for different block sizes may not be convertible to each other by scaling.

[0102] In some examples, ALWIP transforms may be defined in such a way that they “share nothing” with legacy transforms (eg, an ALWIP transform “shares nothing” with a corresponding legacy transform, even if they have been mapped via one of the mappings above).

[0103] In an example, the ALWIP mode is used for both the luma component and the chroma components, but in other examples, the ALWIP mode is used for the luma component but not for the chroma components.

[0104] 5. Affine linear weighted intra prediction mode with encoder acceleration (e.g., test CE3-1.2.1:)

[0105] 5.1 Description of method or device

[0106] The affine linear weighted intra prediction (ALWIP) mode tested in CE3-1.2.1 may be the same as the mode proposed in JVET-L0199 under test CE3-2.2.2, except for the following changes:

[0107] Coordination with Multiple Reference Line (MRL) intra prediction, in particular encoder estimation and signaling, ie MRL is not combined with ALWIP and transmission of MRL indices is restricted to non-ALWIP blocks.

[0108] Subsampling is now mandatory for all blocks WxH ≥ 32x32 (it was optional for 32x32 before); therefore, the extra test at the encoder and the subsampling flag sending have been removed.

[0109] ALWIP for 64xN and Nx64 blocks (N≤32) has been added by downsampling 32xN and Nx32 respectively and applying the corresponding ALWIP modes.

[0110] Additionally, test CE3-1.2.1 includes the following encoder optimizations for ALWIP:

[0111] Combined mode estimation: Conventional and ALWIP modes use a shared Hadamard candidate list for full RD estimation, i.e., ALWIP mode candidates are added to the same list as conventional (and MRL) mode candidates based on Hadamard costs.

[0112] The combined mode list supports EMT in-screen fast and PB in-screen fast, reducing the number of complete RD checks with additional optimization.

[0113] Following the same approach as the conventional mode, only the available MPMs of the left and upper blocks are added to the list of complete RD estimates for ALWIP.

[0114] 5.2 Complexity Evaluation

[0115] In test CE3-1.2.1, excluding the calculations that cause the discrete cosine transform, a maximum of 12 multiplications per sample are required to produce the prediction signal. In addition, a total of 136492 parameters of 16 bits each are required. This corresponds to 0.273MB of memory.

[0116] 5.3 Experimental Results

[0117] The evaluation was performed according to the common test conditions JVET-J1010 [2] for both internal only (AI) and random access (RA) configurations with VTM software version 3.0.1. The corresponding simulations were performed on an Intel Xeon cluster (E5-2697Av4, AVX2 on, turbo boost off) with Linux OS and GCC 7.2.1 compiler.

[0118] Table 1. Results for CE3-1.2.1 configured for VTMAI

[0119]

[0120] Table 2. Results for CE3-1.2.1 configured for VTM RA

[0121]

[0122] 5.4 Test CE3-1.2.2: Affine Linear Weighted Intra-Prediction with Reduced Complexity

[0123] The techniques tested in CE2 are related to the “affine linear intra prediction” described in JVET-L0199 [1], but are simplified with regard to memory requirements and computational complexity:

[0124] There are only three different sets of prediction matrices (e.g. S0, S1, S2, see also below) and offset vectors (e.g. for providing offset values) to cover all block shapes. As a result, the number of parameters is reduced to 14400 10-bit values,

[0125] Less memory is stored than in 128×128 CTU.

[0126] The input and output sizes of the predictor are further reduced. In addition, instead of transforming the boundaries by DCT, averaging or downsampling can be performed on the boundary samples, and the generation of the prediction signal can use linear interpolation instead of inverse DCT. Therefore, four times the maximum value per sample may result in the prediction signal having to be generated.

[0127] 6. Examples

[0128] Here we discuss how to use ALWIP prediction to perform some predictions (e.g. Figure 6 shown).

[0129] In principle, reference Figure 6 , in order to obtain the Q=M×N value of the M×N block 18 to be predicted, a multiplication of Q×P samples of the Q×PALWIP prediction matrix 17M by P samples of the Px1 neighboring vector 17P should be performed. Therefore, in general, in order to obtain each Q=M×N value of the M×N block 18 to be predicted, at least P=M+N value multiplications are required.

[0130] These multiplications have an extremely detrimental effect. The dimension P of the boundary vector 17P usually depends on the number M+N of boundary samples (bins or pixels) 17a, 17c of the adjacent (e.g., adjacent to) MxN block 18 to be predicted. This means that if the size of the block 18 to be predicted is large, the number M+N of boundary pixels (17a, 17c) is correspondingly large, thus increasing the dimension P=M+N of the Px1 boundary vector 17P, as well as the length of each row of the QxPALWIP prediction matrix 17M, and thus also the number of multiplications (in general, Q=M*N=W*H, where for N, W (width) is another symbol and for M, H (height) is another symbol; in the case where the boundary vector is formed by only one row and / or one column of samples, P is P=M+N=H+W).

[0131] Typically, this problem is exacerbated by the fact that multiplication is generally a power-hungry operation in a microprocessor-based system (or other digital processing system). As you can imagine, a large number of multiplications for a very large number of samples in a large number of blocks will result in a waste of computational power, which is generally undesirable.

[0132] Therefore, it is preferable to reduce the number of multiplications Q*P required to predict the MxN block 18.

[0133] It has been appreciated that it is possible to somehow reduce the computational power required for each intra prediction of each block 18 to be predicted by intelligently selecting an alternative multiplication operation that is more easily processed.

[0134] In particular, refer to Figure 7.1-7.4 , Figure 8.1 and 8.2 It can be understood that the encoder or decoder can predict a predetermined block (e.g., 18) of an image using a plurality of neighboring samples (e.g., 17a, 17c) by the following steps:

[0135] reducing (100, 813) the plurality of neighboring samples to obtain a reduced set (102) of sample values ​​that is less in number than the plurality of neighboring samples (17),

[0136] Subject the reduced set (102) of sample values to (812) a linear or affine-linear transformation (19, 17M) to obtain predicted values for predetermined samples (104, 118’, 188”) of a predetermined block (18).

[0137] In some cases, the decoder or encoder may also derive (e.g., step 813 in Figure 8.1) predicted values for additional samples of the predetermined block based on the predicted values of the predetermined samples and a plurality of neighboring samples. Thus, an upsampling strategy can be obtained.

[0138] In an example, some averaging is performed (e.g., in step 811) on the samples at the boundary 17 to obtain a reduced set 102 with a reduced number of samples (at least one of the samples in the reduced number of samples 102 can be an average of two samples of the original boundary samples or a selection of the original boundary samples) Figure 7.1-7.4 ) of samples. For example, if the original boundary has P = M + N samples, the samples of the reduced set may have M red < M and N red < at least one of N of P red = M red + N red , such that P red < P. Thus, the boundary vector 17P actually used for prediction (e.g., in step 812b) will not have Px1 terms, but will have P red < P of P red x1 terms. Similarly, the ALWIP prediction matrix 17M selected for prediction will not have a QxP dimension, but will have QxP red (or Q red xP red , see below), at least because P red < P (due to M red < M and N red < at least one of N).

[0139] In some examples (e.g., Figures 7.2, 7.3, and 8.2), if the block obtained by ALWIP (at step 812) is a reduced block with dimensions M′ red × N′ red , where M′ red < M and / or N′ red < N (i.e., the number of samples directly predicted by ALWIP is less than the number of samples of the block 18 to be actually predicted), then the number of multiplications may even be further reduced. Thus, in the case of setting Q red = M′ red * N′ red , this will be achieved by using Q red * Pred a multiplication where Q red *P red <Q*P red <Q*P) replaces Q*P red a multiplication, which will obtain the ALWIP prediction. This multiplication will predict a reduced block of dimension nM′ red ×N′ red Despite this, it is possible to perform (e.g., at subsequent step 813) an upsampling from the reduced M′ red ×N′ red prediction block to the final MxN prediction block (e.g., obtained by interpolation).

[0140] These techniques may be advantageous because although matrix multiplication involves a reduced number of multiplications (Q red *P red or Q*P red ), both the initial reduction (e.g., averaging or downsampling) and the final transformation (e.g., interpolation) can be performed by reducing (or even avoiding) multiplications. For example, downsampling, averaging, and / or interpolation (e.g., at steps 811 and / or 813) can be performed by using binary operations that do not require computational power, such as addition and shift.

[0141] Examples of shift operations at the processor level are discussed here. Figure 9a and 9b shows the number 580 encoded in binary (1001000100b) in the ten-bit register 910, which has ten one-bit registers 910a, each storing a one-bit value (e.g., 1 or 0). The value 1001000100b encoded in the two-byte register 910 is represented as 901 in Figure 9a as the "value to be shifted". The binary bit represented by the number 902 in Figure 9b is a one-right-shifted version of the binary bit represented by the number 901. It can be seen that the binary bit 902 (encoded as 0100100010b) is the version of the binary bit 901 after each value encoded in each one-bit register 910a has simply been moved to the one-bit register in the respective right position. The least significant bit of the binary bit 901 is lost in the binary bit 902; and the most significant bit of the shifted binary bit 902 is added as zero. If the shifted binary bit 902 is transformed to decimal, we will see that the decimal value 290, which is half of 580, has been obtained. This may be a technique for dividing a binary number by 2 (if the value 901 is odd, there is only a quantization error). This operation is very easy to perform and does not require high computational power. It is worth noting that by shifting multiple times, division by a power of 2 is obtained: by shifting twice, division by 4 is obtained. By shifting three times, division by 8 is obtained, and generally by shifting r times, division by 2r Division by (2^r). This can also be denoted by the symbol f>>r, such that f>>1 represents f / 2, f>>2 represents f / 4, and so on (where f is an integer). This operation is also known as a right rotation. Similarly, the left rotation operation f<<r represents f multiplied by 2r (or 2^r). For processors, this shift operation is not computationally demanding because they avoid the need to perform multiplication and instead obtain the result by simply shifting bits in different registers. In the example, register 910 is represented as a 10-bit register. However, in an instance, register 910 can have a different number of bits, such as a 8-bit register 910 (in which case, register 910 is an 8-bit register).

[0142] Moreover, addition is a very easy operation and can be easily performed without a large amount of computational work.

[0143] For example, this shift operation can be used to average two boundary samples and / or to interpolate two samples (support values) of the reduced prediction block (or obtained from the boundary) to obtain the final prediction block. (For interpolation, two sample values are necessary. Inside the block, we always have two predetermined values, but for interpolating samples along the left and upper boundaries of the block, we only have one predetermined value, as shown in Figure 7.2. Therefore, we use the boundary samples as support values for interpolation.)

[0144] A two-step process can be used, such as:

[0145] First, sum the values of the two samples;

[0146] Then, halve the sum value (e.g., by right shifting).

[0147] Alternatively, one can:

[0148] First, halve each of the samples (e.g., by left shifting);

[0149] Then, sum the values of the two halved samples.

[0150] When downsampling (e.g., in step 811), even easier operations may be performed because only one sample amount needs to be selected from a set of samples (e.g., samples adjacent to each other).

[0151] Therefore, techniques for reducing the number of multiplications to be performed can now be defined. Some of these techniques can in particular be based on at least one of the following principles:

[0152] Even if the block 18 to be actually predicted has dimensions MxN, the block can be reduced (in at least one of the two dimensions), and a prediction with reduced dimensions Q red xP red (where Qred = M' red * N' red , P red = N red + M red , where M' red <M and / or N' red <N and / or M red <M and / or N red <N) of the ALWIP matrix. Thus, the boundary vector 17P will have dimension P red x1, implying P red <P multiplications (where P red = M red + N red , and

[0153] P = M + N).

[0154] This P red x1 boundary vector 17P can be easily obtained from this original boundary 17, for example:

[0155] By downsampling (e.g., only selecting some samples of the boundary); and / or

[0156] By averaging multiple samples of the boundary (which can be easily obtained by addition and shift without multiplication).

[0157] Additionally or alternatively, instead of predicting all Q = M * N values of the block 18 to be predicted by multiplication, a reduced block with reduced dimensions can be predicted (e.g., Q red = M' red * N' red , where M' red <M and / or N' red <N). By using, for example, Q red个 samples as support values for the remaining Q - Q red values to be predicted for interpolation, the remaining samples of the block 18 to be predicted are obtained.

[0158] Figure 8.1 An example is provided, which can be understood as generally describing the process 810, and its specific cases are illustrated in Figure 7.1 In this case, the 4x4 block 18 to be predicted (M = 4, N = 4, Q = M * N = 16), and the neighboring regions 17 of the samples 17a (a vertical column with four already predicted samples) and 17c (a horizontal row with four already predicted samples) have been predicted in a previous iteration (the neighboring regions 17a and 17c can be represented by 17 together). A priori, by using Figure 5For the equation shown, the prediction matrix 17M should be a QxP = 16x8 matrix (since Q = M*N = 4*4 and P = M+N = 4+4 = 8), and the boundary vector 17P should have an 8x1 dimension (since P = 8). However, this would drive the necessity of performing 8 multiplications for each of the 16 samples of the 4×4 block 18 to be predicted, thus resulting in a total of 16×8 = 128 multiplications to be performed. (Note that the average number of multiplications per sample is a good measure of computational complexity. For traditional intra prediction, four multiplications per sample are required, and this increases the computational load involved. Thus, using this as an upper bound for ALWIP would ensure reasonable complexity and not exceed the complexity of traditional intra prediction.)

[0159] Nonetheless, it has been understood that by using the present technique, at step 811, the number of samples 17a and 17c adjacent to the block 18 to be predicted can be reduced, from P to P red <P. In particular, it has been understood that the boundary samples (17a, 17c) adjacent to each other can be averaged (e.g., at 100 in Figure 7.1 ) to obtain a reduced boundary 102 having two horizontal rows and two vertical columns. Thus, the operation performed on the block 18 is a 2×2 block (the reduced boundary is formed by the average values). Alternatively, downsampling can be performed, so that two samples are selected for the row 17c and two samples are selected for the column 17a. Thus, instead of having four original samples, the horizontal row 17c is processed to have two samples (e.g., average samples), while the vertical column 17a, which originally had four samples, is processed to have two samples (e.g., averaged samples). It can also be understood that after subdividing the row 17c and the column 17a into groups of two samples 110, a single sample is retained (e.g., the average of the samples in the group 110 or a simple selection among the samples). Thus, since the set 102 only has four samples (M red = 2, N red = 2, P red = M red + N red = 4, where P red <P), a so-called reduced set 102 of sample values can be obtained.

[0160] It has been understood that the operations (such as averaging or downsampling 100) can be performed at the processor level without performing too many multiplications: the averaging or downsampling 100 performed at step 811 can be simply obtained by direct and computationally non-power-consuming operations, such as addition and shift.

[0161] It has been understood that at this point, the reduced set 102 of sample values can be subjected to a linear or affine linear (ALWIP) transformation 19 (e.g., using a prediction matrix, such as Figure 5In this case, the ALWIP transform 19 maps the four samples 102 directly onto the sample values ​​104 of the block 18. In the present case, no interpolation is required.

[0162] In this case, the ALWIP matrix 17M has dimensions QxP red =16×4: This follows from the fact that all Q=16 samples of block 18 to be predicted are directly obtained by ALWIP multiplication (no interpolation is required).

[0163] Therefore, in step 812a, a red For example, the selection may be based at least in part on a signal from the data stream 12. The selected ALWIP matrix 17M may also be selected using A k , where k can be understood as an index that can be signaled in the data stream 12 (in some cases, the matrix is ​​also represented as See below). Fig.12 The described scheme performs this selection: for each dimension (e.g., a height / width pair of the block 18 to be predicted), the ALWIP matrix 17M is selected, for example, between three groups of matrices S0, S1, S2 (each of the three groups S0, S1, S2 is able to group multiple ALWIP matrices 17M of the same dimension, and the ALWIP matrix to be selected for prediction will be one of them).

[0164] In step 812b, the selected QxP is executed red ALWIP matrix 17M (also denoted as A k ) and P red Multiplication between x1 boundary vector 17P.

[0165] At step 812c, the offset value (e.g., b k ) may be added to all obtained values ​​104 of the vector 18Q obtained, for example, by ALWIP. The value of the offset (b k Or in some cases denoted, see below) can be combined with a specific selected ALWIP matrix (A k ) and may be based on an index (e.g., the index may be signaled in data stream 12).

[0166] So here goes the comparison between using this technique and not using it:

[0167] Without this technology:

[0168] A block 18 to be predicted, the block having dimensions M=4, N=4;

[0169] Q = M*N = 4*4 = 16 values ​​to be predicted;

[0170] P = M + N = 4 + 4 = 8 boundary samples

[0171] P=8 multiplications for each of the Q=16 values ​​to be predicted,

[0172] Total number P*Q=8*16=128 multiplications;

[0173] Using this technology, we can achieve:

[0174] A block 18 to be predicted, the block having dimensions M=4, N=4;

[0175] Finally, Q = M*N = 4*4 = 16 values ​​to be predicted;

[0176] Reduced dimension of the boundary vector: P red =M red +N red =2+2=4;

[0177] P for each of the Q=16 values ​​to be predicted by ALWIP red = 4 multiplications,

[0178] Total P red * Q = 4 * 16 = 64 multiplications (half of 128!)

[0179] The ratio of the number of multiplications to the number of final values ​​to be obtained is P red *Q / Q=4, which is half of P=8 multiplications for each sample to be predicted!

[0180] It will be appreciated that it is possible to obtain appropriate values ​​at step 812 by relying on straightforward and computationally non-power demanding operations, such as averaging (and, in the case of addition and / or shifting and / or downsampling).

[0181] refer to Figure 7.2 and 8.2 , the block 18 to be predicted is here an 8×8 block of 64 samples (M=8, N=8). Here, a priori, the prediction matrix 17M should have the size QxP=64x16 (Q=64 due to Q=M*N=8*8=64, M=8 and N=8 and due to P=M+N=8+8=16). Therefore, a priori, for each of the Q=64 samples of the 8×8 block 18 to be predicted, P=16 multiplications will need to be performed, so as to reach 64×16=1024 multiplications for the entire 8×8 block 18!

[0182] However, refer to Figure 7.2 and 8.2, a method 820 may be provided according to which, instead of using all 16 samples of the boundary, only 8 values ​​are used (e.g., 4 in the horizontal boundary row 17c and 4 in the vertical boundary column 17a between the original samples of the boundary). From the boundary row 17c, 4 samples may be used instead of 8 samples (e.g., they may be a 2 by 2 average and / or one sample is selected from two samples). Thus, the boundary vector is not a Px1=16x1 vector, but only a Px2=16x1 vector. red x1 = 8x1 vector (P red =M red +N red =4+4). It is understood that the samples of the horizontal rows 17c and the samples of the vertical columns 17a may be selected or averaged (eg, two by two) to have only P red = 8 boundary values ​​instead of the original P = 16 samples, forming a reduced set 102 of sample values. This reduced set 102 will allow a reduced version of block 18 to be obtained, which has Q red =M red *N red = 4*4 = 16 samples (instead of Q = M*N = 8*8 = 64). The ALWIP matrix can be applied to predict red xN red =4x4 blocks. The reduced version of block 18 includes Figure 7.2 Samples indicated in gray in scheme 106: Samples indicated in gray squares (including samples 118' and 118") form a Q obtained in step 812. red = 16 values ​​of the 4x4 reduced block. The 4x4 reduced block is obtained by applying the linear transformation 19 in step 812. After obtaining the values ​​of the 4x4 reduced block, the values ​​of the remaining samples (the samples represented by white samples in scheme 106) can be obtained, for example by interpolation.

[0183] Relative to Figure 7.1 and 8.1 The method 810 may additionally include a deriving step 813, for example, by interpolation, to derive the residual QQ for the MxN=8x8 block 18 to be predicted. red =64-16=48 samples (white squares) of predicted values. By interpolation (for example, the interpolation can also use the values ​​of the boundary samples) from Q red =16 The remaining QQ can be obtained from the samples directly obtained red =64-16=48 samples. Figure 7.2It can be seen that, although samples 118' and 118" (as shown by the grey squares) have been obtained in step 812, sample 108' is obtained by interpolation between samples 118' and 118" in step 813 (between samples 118' and 118" and indicated by the white squares). It is understood that interpolation can also be obtained by similar operations to those used for averaging, such as shifting and adding. Therefore, in Figure 7.2 In the example, value 108' may generally be determined as a midpoint (which may be an average) between the value of sample 118' and the value of sample 118".

[0184] By performing interpolation, in step 813 , a final version of the MxN=8x8 block 18 may also be derived based on the plurality of sample values ​​indicated in 104 .

[0185] Therefore, the comparison between using this technology and not using this technology is:

[0186] This technology is not used:

[0187] The block to be predicted 18 has dimensions M=8, N=8 and

[0188] There are Q=M*N=8*8=64 samples in the block 18 to be predicted.

[0189] P in boundary 17 = M + N = 8 + 8 = 16 samples.

[0190] P=16 multiplications for each of the Q=64 values ​​to be predicted.

[0191] The total number P*Q=16*64=1028 multiplications.

[0192] The ratio between the number of multiplications and the number of final values ​​to be obtained is P*Q / Q=16.

[0193] Use this technology:

[0194] The block to be predicted 18 has dimensions M=8 and N=8.

[0195] The final number of values ​​to be predicted is Q=M*N=8*8=64.

[0196] However, Q red xP red ALWIP matrix, where P red =M red +N red ,Q red =M red *N red ,M red =4,N red =4

[0197] P within the boundary red = M red + N red = 4 + 4 = 8 samples, where P red < P.

[0198] Q for the 4x4 reduced block to be predicted red = P for each of the 16 values red = 8 multiplications (formed by the gray squares in Scenario 106).

[0199] Total P red * Q red = 8 * 16 = 128 multiplications (far less than 1024!)

[0200] The ratio between the number of multiplications and the number of final values to be obtained is P red * Q red / Q = 128 / 64 = 2 (far less than 16 obtained without using this technology).

[0201] Therefore, the technology proposed here requires 8 times less power than the previous technology.

[0202] Figure 7.3 Shows another example (which may be based on Method 820), where the block 18 to be predicted is a rectangular 4×8 block (M = 8, N = 4), with Q = 4×8 = 32 samples to be predicted. The boundary 17 is formed by the horizontal row 17c with N = 8 samples and the vertical column 17a with M = 4 samples. Therefore, a priori, the boundary vector 17P will have dimension Px1 = 12x1, however the predicted ALWIP matrix should be QxP = 32x12 matrix, thus resulting in the need for Q * P = 32 * 12 = 384 multiplications.

[0203] However, for example, at least 8 samples of the horizontal row 17c can be averaged or downsampled to obtain a reduced horizontal row with only 4 samples (e.g., average samples). In some examples, the vertical column 17a will remain as it is (e.g., not averaged). Overall, the reduced boundary will have dimension P red = 8, and P red < P. Therefore, the boundary vector 17P will have dimension P red x1 = 8x1. The ALWIP prediction matrix 17M will be a matrix with dimension M * N red * P red = 4 * 4 * 8 = 64. The 4x4 reduced block (formed by the gray columns in Scenario 107) directly obtained after undergoing Step 812 will have size Q red = M * N red=4*4=16 samples (instead of Q=4*8=32 of the original 4x8 block 18 to be predicted). Once the downscaled 4x4 block is obtained by ALWIP, the offset value b can be added k (Step 812c) and interpolation is performed in step 813. Figure 7.3 As shown in step 813 in FIG. 1 , the reduced 4×4 block is expanded to a 4×8 block 18 , wherein in step 813 the value 108 ′ not obtained in step 812 is obtained by interpolating the value 118 ′ and 118 ″ (gray square) obtained in step 812 .

[0204] Therefore, the comparison between using this technology and not using this technology is:

[0205] This technology is not used:

[0206] The block to be predicted is 18, the dimensions of the block are M=4, N=8

[0207] Q = M * N = 4 * 8 = 32 values ​​to be predicted

[0208] P = M + N = 4 + 8 = 12 samples in the boundary

[0209] P = 12 multiplications for each of the Q = 32 values ​​to be predicted

[0210] Total number P*Q = 12*32 = 384 multiplications

[0211] The ratio between the number of multiplications and the number of final values ​​to be obtained is P*Q / Q=12.

[0212] Use this technology:

[0213] The block to be predicted is 18, the dimensions of the block are M=4, N=8

[0214] Finally, Q = M*N = 4*8 = 32 values ​​to be predicted

[0215] However, Q red xP red =16x8 ALWIP matrix, where M=4, N red =4,Q red =M*N red =16,P red =M+N red =4+4=8

[0216] P red =M+N red =4+4=8 samples are within the boundary, of which P red <P

[0217] Q for the reduced block to be predicted red= Each of the 16 values, P red = 8 multiplications

[0218] Total Q red Q red *P red =16*8=128 multiplications (less than 384!)

[0219] The ratio of the number of multiplications to the number of final values ​​to be obtained is P red *Q red / Q = 128 / 32 = 4 (much less than the 12 obtained without using this technique!).

[0220] Therefore, using this technology, the amount of calculation is reduced to one third.

[0221] Figure 7.4 Show the case of a block 18 to be predicted, which has dimensions MxN=16x16 and ultimately has Q=M*N=16*16=256 values ​​to be predicted, of which P=M+N=16+16=32 boundary samples. This will result in a prediction matrix with dimensions QxP=256x32, which means 256*32=8192 multiplications!

[0222] However, by applying the method 820, at step 811, the number of boundary samples may be reduced (e.g., by averaging or downsampling), for example, from 32 to 8: For example, for each group 120 of four consecutive samples of row 17a, one sample is retained (e.g., selected from four samples, or the average of the samples). For each group 120 of four consecutive samples of column 17c, one sample is retained (e.g., selected from four samples, or the average of the samples).

[0223] Here, the ALWIP matrix 17M is Q red xP red =64x8 matrix: This comes from the selected P red =8 (by using 8 averaged or selected samples from 32 of the boundaries) and the fact that the reduced block to be predicted in step 812 is an 8×8 block (in scheme 109, the grey square is 64).

[0224] Therefore, once the 64 samples of the reduced 8×8 block are obtained at step 812, the residual QQ of the block 18 to be predicted can be derived at step 813 red =256-64=192 values ​​104.

[0225] In this case, for performing the interpolation, it has been chosen to use all samples of the boundary column 17a and only to use replacement samples in the boundary row 17c. Other choices may be made.

[0226] When using this method, the ratio between the number of multiplications and the number of values ​​finally obtained is Q red *P red / Q = 8*64 / 256 = 2, which is much less than the 32 multiplications for each value without using this technique!

[0227] The comparison between using this technology and not using this technology is:

[0228] This technology is not used:

[0229] The block to be predicted is 18, the dimensions of the block are M=16, N=16

[0230] Q = M * N = 16 * 16 = 256 values ​​to be predicted;

[0231] P = M + N = 16 * 16 = 32 samples are at the boundary;

[0232] P = 32 multiplications for each of the Q = 256 values ​​to be predicted,

[0233] Total number P*Q=32*256=8192 multiplications;

[0234] The ratio between the number of multiplications and the number of final values ​​is P*Q / Q=32.

[0235] Use this technology:

[0236] The block to be predicted is 18, the dimensions of the block are M=16, N=16

[0237] Q = M * N = 16 * 16 = 256 final values ​​to be predicted;

[0238] But the Q to be used red xP red =64x8 ALWIP matrix, where M red =4,N red =4,Q red =8*8=64 samples to be predicted by ALWIP, P red =M red +N red =4+4=8

[0239] P in the border red =M red +N red =4+4=8 samples, where P red <P

[0240] Q for the reduced block to be predicted red = 64 values ​​each, P red = 8 multiplications,

[0241] Total Qred *P red =64*4=256 multiplications (less than 8192!)

[0242] The ratio between the number of multiplications and the number of final values ​​to be obtained is P red *Q red / Q = 8*64 / 256 = 2 (much less than the 32 obtained without using this technique!).

[0243] As a result, this technology requires 16 times less computing power than traditional techniques!

[0244] Therefore, a predetermined block (18) of an image may be predicted using a plurality of neighboring samples (17) by the following steps:

[0245] reducing (100, 813) the plurality of neighboring samples to obtain a reduced set (102) of sample values ​​that is less in number than the plurality of neighboring samples (17),

[0246] The reduced set of sample values ​​(102) is subjected (812) to a linear or affine linear transformation (19, 17M) to obtain predicted values ​​for predetermined samples (104, 118', 188") of a predetermined block (18).

[0247] In particular, the reduction (100, 813) may be performed by downsampling the plurality of neighboring samples to obtain a reduced set of sample values ​​(102) that is less in number of samples than the plurality of neighboring samples (17).

[0248] Alternatively, the reduction (100, 813) may be performed by averaging a plurality of neighboring samples to obtain a reduced set of sample values ​​(102) that is less in number of samples than the plurality of neighboring samples (17).

[0249] Furthermore, further samples (108, 108') for the predetermined block (18) may be derived (813) by interpolation based on the predicted value for the predetermined sample (104, 118', 118") and a plurality of neighboring samples (17).

[0250] The plurality of neighboring samples (17a, 17c) may be arranged along both sides of the predetermined block (18) (e.g., Figure 7.1-7.4 The predetermined samples (e.g., the samples obtained by ALWIP in step 812) may also be arranged in rows and columns, and along at least one of the rows and columns, the predetermined samples are located at every nth position from the sample (112) of the predetermined sample 112 adjacent to both sides of the predetermined block.

[0251] A support value (118) for one (118) of a plurality of neighboring positions may be determined for each of at least one of the rows and columns based on a plurality of neighboring samples (17), the one (118) of the plurality of neighboring positions being aligned with a corresponding one of the at least one of the rows and columns. Additional samples (108, 108') of the predetermined block (18) may also be derived by interpolation based on a predicted value for the predetermined sample (104, 118', 118") and a support value for the neighboring sample (118) aligned with at least one of the rows and columns.

[0252] The predetermined sample (104) may be located at every n-th position from the sample (112) adjacent to both sides of the predetermined block 18 along the row, and the predetermined sample may be located at every m-th position from the sample (112) of the predetermined sample adjacent to both sides of the predetermined block (18) along the column, where n, m>1. In some cases, n=m (e.g., in Figure 7.2 and 7.3 , where the samples 104 , 118 ′, 118 ″ directly obtained by ALWIP at 812 and represented by gray squares are arranged alternately along rows and columns to the samples 108 , 108 ′ subsequently obtained at step 813 .

[0253] The determination of the support value may be performed along at least one of the rows (17c) and columns (17a) by, for each support value, downsampling or averaging (122) a set (120) of neighboring samples within the plurality of neighboring samples, including the neighboring sample (118) for which the corresponding support value is determined. Figure 7.4 In step 813, the value of sample 119 may be obtained by using predetermined sample 118'" (previously obtained in step 812) and the values ​​of neighboring samples 118 as support values.

[0254] The plurality of neighboring samples may extend one-dimensionally along both sides of the predetermined block (18). The reduction (811) may be performed by grouping the plurality of neighboring samples (17) into groups (110) having one or more consecutive neighboring samples, and performing downsampling or averaging on each of the groups (110) having one or more neighboring samples having two or more neighboring samples.

[0255] In an example, a linear or affine linear transformation may include P red *Q red or P red *Q weighting factors, where P red is the number of sample values ​​(102) in the reduced set (102) of sample values, and Q red Or Q is the number of predetermined samples within the predetermined block (18). At least 1 / 4P red *Qred or 1 / 4P red *Q weighting factors are non-zero weight values. red For each of the predetermined samples, the P red *Q red or P red The Q weighting factors may include a series of P associated with the corresponding predetermined samples. red weighting factors, wherein when the series are arranged one below the other according to a raster scan order between predetermined samples of the predetermined block (18), the series form an omnidirectional nonlinear envelope. red *Q red or P red *The Q weighting factors may all be independent of each other by any conventional mapping rule. The average of the maximum values ​​of a cross-correlation between a first series of weighting factors associated with each predetermined sample and a second series of weighting factors associated with predetermined samples other than the respective predetermined sample, or an inverse version of the latter series, whichever results in a higher maximum value, is below a predetermined threshold. The average of the maximum values ​​of the cross-correlation between the first series of weighting factors associated with the respective predetermined sample and the second series of weighting factors associated with predetermined samples other than the respective predetermined sample or an inverse version of the latter series, whichever results in a higher maximum value, is below a predetermined threshold. The predetermined threshold may be 0.3 [or in some cases 0.2 or 0.1]. red个 The neighboring samples (17) may be arranged along a one-dimensional path extending along both sides of the predetermined block (18), and for Q or Q red Each of the predetermined samples is traversed in a predetermined direction along the one-dimensional path, and the P associated with the corresponding predetermined sample is red The series of weighting factors are sorted.

[0256] Description of the method and apparatus

[0257] To predict samples of a rectangular block of width W (also denoted by N) and height H (also denoted by M), affine linear weighted intra prediction (ALWIP) can take as input a column of H reconstructed neighboring boundary samples to the left of the block and W reconstructed neighboring boundary samples above the block. If the reconstructed samples are not available, they can be generated as done in traditional intra prediction.

[0258] The generation of the prediction signal (e.g., the value of the complete block 18) may be based on at least one or some of the following three steps:

[0259] 1. Outside the boundary samples 17, samples 102 (eg, four samples in the case of W=H=4, and / or eight samples in other cases) may be extracted by averaging or downsampling (eg, step 811).

[0260] 2. Perform a matrix vector multiplication using the averaged samples (or samples remaining from downsampling) as input, and then add the offset. The result can be a reduced prediction signal over a subsampled set of samples in the original block (e.g., step 812).

[0261] 3. The prediction signals at the remaining positions may be generated from the prediction signals over the subsampled set, for example by linear interpolation (eg step 813), for example by upsampling.

[0262] Due to step 1.(811) and / or step 3.(813), the total number of multiplications required in the calculation of the matrix-vector product can be made so that it is always less than or equal to 4*W*H. In addition, the averaging operation on the boundary and the linear interpolation of the reduced prediction signal are performed only by using additions and bit shifts. In other words, in the example, a maximum of four multiplications per sample are required for the ALWIP mode.

[0263] In some examples, the matrix (e.g., 17M) and the offset vector (e.g., b k ) can be taken from a set of matrices (e.g. three sets), such as S0, S1, S2, which can be stored in storage units of a decoder and an encoder, for example.

[0264] In some examples, the set S0 may include (eg, be composed of) n0 (eg, n0=16 or n0=18 or other number) matrices Each matrix can have 16 rows and 4 columns and 18 offset vectors 16 per size, to perform according to Figure 7.1 This set of matrices and offset vectors is used for blocks of size 4×4 18. Once the boundary vector has been reduced to P red =4 vector (such as Figure 7.1 In step 811), the P of the reduced set 102 of samples may be red =4 samples are directly mapped to Q=16 samples of the 4x4 block 18 to be predicted.

[0265] In some examples, the set S1 may include (eg, be composed of) n1 (eg, n1=8 or n1=18 or other number) matrices Each matrix may have 16 rows and 8 columns and 18 offset vectors 16 per size, to perform according to Figure 7.2 Or the technology of 7.3. The matrix and offset vector of this set S1 can be used for blocks of sizes 4×8, 4×16, 4×32, 4×64, 8×4, 8×8, 16×4, 32×4, and 64×4. Additionally, it can also be used for blocks of size WxH with max(W,H)>4 and min(W,H) = 4, that is, for blocks of size 4×16 or 16×4, 4×32 or 32×4, and 4×64 or 64×4. The 16×8 matrix references a reduced version of block 18, which is a 4×4 block, as Figure 7.2 and 7.3 obtained.

[0266] Additionally or alternatively, the set S2 may include (e.g., consist of) n2 (e.g., n2 = 6 or n2 = 18 or other quantity) matrices Each matrix may have 64 rows and 8 columns and 18 offset vectors Each of size 64. The 64x8 matrix references a reduced version of block 18, which is an 8x8 block, for example as Figure 7.4 obtained. The matrices and offset vectors of this set can be used for sizes 8×16, 8×32, 8×64, 16×8, 16×16, 16×32, 16×64, 32×8, 32×16, 32×32, 32×64, 64×8, 64×16, 64×32, 64×64.

[0267] In Fig.12 summarizes examples of the selection of sets S0, S1, S2 based on block dimensions (which can be performed in step 812a). In alternative examples, it is possible for each set to have a different number of matrices. Additionally or alternatively, matrices of different sizes may be used for different sets.

[0268] The matrices and offset vectors of the set or a part of these matrices and offset vectors can be used for all other block shapes.

[0269] 5.4 Averaging or downsampling of boundaries

[0270] Here, features regarding step 811 are provided.

[0271] As described above, boundary samples (17a, 17c) can be averaged and / or downsampled (e.g., from P samples to P red < P samples).

[0272] In the first step, the input boundaries bdry top (e.g., 17c) and bdry left (e.g., 17a) can be reduced to smaller boundaries and To reach the reduced set 102. Here, in the case of a 4x4 block, and In all cases, it consists of 2 samples, and in all other cases, it consists of 4 samples.

[0273] In the case of a 4x4 block, we can define

[0274]

[0275] And similarly define therefore, is an average value obtained using, for example, a bit shift operation.

[0276] In all other cases (e.g., for blocks with width or height not equal to 4), if the block width is given as W=4*2 k , for 0≤i<4, define

[0277]

[0278] and similarly defined

[0279] In other cases, the boundary can be downsampled (e.g., by selecting a particular boundary sample from a set of boundary samples) to achieve a reduced number of samples. Available at bdry top [0] With bdry top [1] and Available at bdry top [2] With bdry top [3] Select. It can also be similarly defined

[0280] Two reduction boundaries and Can be cascaded into a reduced boundary vector bdry red (associated with the reduced set 102), also denoted 17P. The reduced boundary vector bdry red So we can have a size of four (P red =4)( Figure 7.1 for example) and for all other shapes of blocks of size eight (P red =8)( Figure 7.2-7.4 ).

[0281] Here, if mode < 18 (or the number of matrices in the matrix set), then we can define

[0282]

[0283] If mode ≥ 18, which corresponds to the transposed mode of mode-17, then we can define

[0284]

[0285] Therefore, according to a specific state (one state: mode<18; another state: mode≥18), different scanning orders (for example, a scanning order: Another scan sequence: The predicted values ​​of the distribution output vector.

[0286] Other strategies may be implemented. In other examples, the mode index 'mode' need not necessarily be in the range of 0 to 35 (other ranges may be defined). Furthermore, each of the three sets S0, S1, S2 need not have 18 matrices (thus, instead of an expression such as mode≥18, it may be mode≥n0,n1,n2, which is the number of matrices for each matrix set S0, S1, S2, respectively). Furthermore, the sets may each have a different number of matrices (e.g., it may be that S0 has 16 matrices, S1 has 8 matrices, and S2 has 6 matrices).

[0287] The modulus and transposition information does not have to be stored and / or sent as a combined modulus index 'mode': in some examples it is possible to explicitly signal it as a transposition flag and a matrix index (0-15 for S0, 0-7 for S1, 0-5 for S2).

[0288] In some cases, the combination of the transposition flag and the matrix index may be interpreted as a set index. For example, there may be one bit used as a transposition flag and some bits indicating the matrix index, collectively referred to as a "set index".

[0289] 5.5 Generating Reduced Prediction Signals via Matrix-Vector Multiplication

[0290] Here, features regarding step 812 are provided.

[0291] The input vector bdry can be reduced from red (Boundary vector 17P) generates a reduced prediction signal pred red The latter signal may be about a signal having a width W red With height H red Here, W red With H red Can be defined as:

[0292] W red =4,H red =4; if max(W,H)≤8,

[0293] W red =min(W,8),H red =min(H,8); otherwise.

[0294] Reduced prediction signal pred red This can be calculated by taking the matrix-vector product and adding the offset:

[0295] pred red =A.bdry red +b.

[0296] Here, A is a matrix (eg, prediction matrix 17M), which may have W in the case of W=H=4. red *H red rows and 4 columns, and in all other cases 8 columns, and b may have size W red *H red Vector.

[0297] If W=H=4, then A may have 4 columns and 16 rows, and thus in this case 4 multiplications per sample may be required to compute pred red In all other cases, A can have 8 rows, and it can be verified that in these cases there are 8*W red *H red ≤4*W*H, that is, in these cases, at most 4 multiplications are required per sample to calculate pred red .

[0298] The matrix A and the vector b can be taken from one of the sets S0, S1, S2 as shown below. The index idx=idx(W,H) can be defined by setting idx(W,H)=0 if W=H=4, idx(W,H)=1 if max(W,H)=8, and idx(W,H)=2 in all cases. In addition, m=mode can be set if mode<18, otherwise m=mode-17. Then, if idx≤1 or idx=2 and min(W,H)>4, as well as In the case where idx = 2 and min(W,H) = 4, let A be The matrix generated by averaging each row of W = 4 corresponds to the odd x-coordinates in the downsampled block, or the odd y-coordinates in the downsampled block, in the case of H = 4. If mode ≥ 18, the downscaled prediction signal is replaced by its transposed signal. In alternative examples, different strategies may be implemented. For example, instead of reducing the size of the larger matrix ("leaving out"), using a matrix with W red =4 and H redSmaller matrix of S1(idx = 1) where = 4. That is, such blocks are now distributed to S1 instead of S2.

[0299] Other strategies may be executed. In other examples, the modulo index'mode' need not necessarily be in the range of 0 to 35 (other ranges can be defined). Additionally, each of the three sets S0, S1, S2 need not have 18 matrices (thus, instead of an expression like mode < 18, it can be mode < n0, n1, n2, and n0, n1, n2 are the number of matrices for each matrix set S0, S1, S2 respectively). Additionally, the sets can each have a different number of matrices (for example, S0 can have 16 matrices, S1 can have eight matrices, and S2 can have six matrices).

[0300] 5.6 Linear interpolation to generate the final prediction signal

[0301] Here, the features regarding step 812 are provided.

[0302] Interpolating the subsampled prediction signal on large blocks, a second version of the averaged boundary may be needed. That is, if min(W, H) > 8 and W ≥ H, then write W = 8 * 2 l , and for 0 ≤ i < 8, define

[0303]

[0304] If min(W, H) > 8 and H > W, define similarly

[0305] Additionally or alternatively, there may be "hard downsampling", where equals

[0306]

[0307] And can be defined similarly

[0308] At the sample positions omitted in the generation of pred red , the final prediction signal can be generated by linear interpolation from pred red (for example, in Figure 7.2-7.4 and step 813 in the example of 8.2). In some examples, if W = H = 4 (for example, Figure 7.1 and 8.1 example), then this linear interpolation may not be necessary.

[0309] The linear interpolation can be given as follows (although other examples are possible). Assume W ≥ H. Then, if H > H red , pred can be executed redIn this case, pred red It can be extended up one column as shown below. If W = 8, pred red May have a width W red =4, and can be obtained by averaging the boundary signal Extends to the top, as defined above. If W>8, pred red With width W red = 8, and it passes through the averaged boundary signal Extends to the top, as defined above. red The first line of can be written as pred red [x][-1]. Then in the red With height 2*H red The signal on the block It can be given as follows

[0310]

[0311] Where 0≤x <W red and 0≤y <H red The latter process can be performed k times until 2 k *H red =H. Therefore, if H=8 or H=16, it can be performed at least once. If H=32, it can be performed twice. If H=64, it can be performed three times. Next, the horizontal upsampling operation can be applied to the vertically upsampled result. The subsequent upsampling operation can use the full boundaries of the prediction signal. Finally, if H>W, it can be performed in a similar manner by first upsampling in the horizontal direction (if necessary) and then in the vertical direction.

[0312] This is an interpolation example that uses reduced boundary samples for the first interpolation (horizontal or vertical) and original boundary samples for the second interpolation (horizontal or vertical). Depending on the block size, only the second interpolation or no interpolation is needed. If horizontal and vertical interpolation are needed, the order depends on the width and height of the block.

[0313] However, different techniques may be implemented: for example, the original boundary samples may be used for both the first and second interpolation, and the order may be fixed, for example, horizontal then vertical (in other cases, vertical then horizontal).

[0314] Therefore, the interpolation order (horizontal / vertical) and the use of downscaled / original boundary samples may vary.

[0315] 5.7 Example of the entire ALWIP process

[0316] exist Figure 7.1-7.4The whole process of averaging, matrix-vector multiplication, and linear interpolation is illustrated in Figure 1 using different shapes. Note that the remaining shapes are handled as one of the cases shown.

[0317] If a 4×4 block is given, ALWIP can be implemented by using Figure 7.1 The technique takes two averages along each axis of the boundary. The resulting four input samples enter a matrix-vector multiplication. The matrix is ​​taken from set S0. After adding the offset, this can produce 16 final prediction samples. Linear interpolation is not necessary to produce the prediction signal. Therefore, a total of (4*16) / (4*4)=4 multiplications are performed per sample. For example, see Figure 7.1 and 8.1 .

[0318] Given an 8×8 block, ALWIP can take four averages along each axis of the boundary. By using Figure 7.2 The resulting eight input samples enter the matrix-vector multiplication using the technique of . The matrix is ​​taken from set S1. This produces 16 samples at odd positions of the prediction block. Therefore, a total of (8*16) / (8*8)=2 multiplications are performed per sample. After adding the offset, these samples can be interpolated vertically, for example by using the top boundary, and horizontally, for example by using the left boundary. For example, see Figure 7.2 and 8.2 .

[0319] Given an 8×4 block, ALWIP might use Figure 7.3 The technique takes four averages along the horizontal axis of the boundary and four original boundary values ​​at the left boundary. The resulting eight input samples enter the matrix-vector multiplication. The matrix is ​​taken from set S1. This produces 16 samples at each odd horizontal and vertical position of the prediction block. Therefore, a total of (8*16) / (8*4)=4 multiplications are performed for each sample. After adding the offset, these samples are horizontally interpolated, for example by using the left boundary. For example, see Figure 7.3 and 8.2 .

[0320] The transposed case will be handled accordingly.

[0321] Given a 16×16 block, ALWIP can take four averages along each axis of the boundary. By using Figure 7.2 The resulting eight input samples enter the matrix-vector multiplication using the technique. The matrix is ​​taken from set S2. This produces 64 samples at odd positions in the prediction block. Therefore, a total of (8*64) / (16*16)=2 multiplications are performed per sample. After adding the offset, for example, these samples are interpolated vertically using the top boundary and horizontally using the left boundary. For example, see Figure 7.2 and8.2 For example, see Figure 7.4 and 8.2 .

[0322] For larger shapes the process is probably essentially the same, and it's easy to check that the number of multiplications per sample is less than two.

[0323] For Wx8 blocks, horizontal interpolation is only required when samples are given at odd horizontal positions and every vertical position. Thus, in these cases, at most (8*64) / (16*8)=4 multiplications are performed per sample.

[0324] Finally, for W×4 blocks with W>8, A k is the matrix produced by discarding each column corresponding to odd entries along the horizontal axis of the downsampled block. Thus, the output size can be 32, and again, only horizontal interpolation is left to be performed. At most (8*32) / (16*4)=4 multiplications can be performed per sample.

[0325] The transposed case may be handled accordingly.

[0326] 5.8 Required Parameter Number and Complexity Evaluation

[0327] The matrices and offset vectors belonging to the sets S0, S1, S2 may include the parameters required for all possible proposed intra-prediction modes. All matrix coefficients and offset vectors may be stored as 10-bit values. Therefore, according to the above description, a total of 14400 parameters may be required for the proposed method, each with 10 bits of precision. This corresponds to 0.018MB of memory. It should be noted that, currently, in standard 4:2:0 chroma subsamples, a CTU of size 128×128 consists of 24576 values, each with 10 bits. Therefore, the memory requirements of the proposed intra-prediction tool do not exceed the memory requirements of the current image reference tool adopted at the last meeting. In addition, it should be noted that due to the 4-tap interpolation filter of the PDPC tool or the angular prediction mode with fractional angular positions, the traditional intra-prediction mode requires four multiplications per sample. Therefore, in terms of computational complexity, the proposed method does not exceed the traditional intra-prediction mode.

[0328] 5.9 Signaling of the Proposed Intra-Prediction Model

[0329] For luma blocks, for example, 35 ALWIP modes are proposed (it is possible to use other numbers of modes). For each coding unit (CU) in intra mode, a flag is sent in the bitstream indicating whether the ALWIP mode is applied to the corresponding prediction unit (PU). The signaling of the latter index can be coordinated with the MRL in the same way as for the first CE test. If the ALWIP mode is to be applied, the index predmode of the ALWIP mode can be signaled using an MPM list with 3 MPMSs.

[0330] Here, the derivation of MPM can be performed using the above intra mode and the left PU as described below. There can be a table, for example, for each traditional intra prediction mode predmode Angular Assign three fixed tables of ALWIP mode map_angular_to_alwip idx ,idx∈{0,1,2}.

[0331] predmode ALWIP =map_angular_to_alwip idx [predmode Angular ].

[0332] For each PU of width W and height H, define and index

[0333] idx(PU)=idx(W,H)∈{0,1,2}

[0334] It indicates from which of the three sets the ALWIP parameters are obtained, as described in Section 4 above. above is available, belongs to the same CTU as the current PU, and is in intra mode, if idx(PU) = idx(PU above ) and if ALWIP is applied in a PU above If

[0335]

[0336] If the above prediction unit PU is available, belongs to the same CTU as the current PU, and is in intra mode, and if the traditional intra prediction mode is applied to the above PU, then set

[0337]

[0338] In all other cases, set

[0339]

[0340] This means that this mode is not available. In the same way, but without the restriction that the left PU needs to belong to the same CTU as the current PU, the export mode

[0341]

[0342] Finally, three fixed default lists are provided: idx ,idx∈{0,1,2}, each of which contains three different ALWIP modes. From the default list list idx(PU) And the mode and Among them, three different MPMs are constructed by substituting -1 according to the default value and eliminating duplication.

[0343] 5.10 Export of adapted MPM lists for conventional luma and chroma intra prediction modes

[0344] As described below, the proposed ALWIP mode can be coordinated with the MPM-based coding of the traditional intra prediction mode. The luma and chroma MPM list derivation process for the traditional intra prediction mode can use a fixed table map_lwip_to_angular idx ,idx∈{0,1,2},given ALWIP mode predmode on the PU LWIP One that maps to the traditional intra prediction mode.

[0345] predmode Angular =map_lwip_to_angular idx(PU) [predmode LWIP ].

[0346] For luma MPM list export, whenever ALWIP mode predmode is used LWIP The neighboring luminance block can be regarded as if the traditional intra prediction mode predmode Angular For chroma MPM list derivation, whenever the current luma block uses LWIP mode, the same mapping can be used to transform ALWIP mode to conventional intra prediction mode.

[0347] 5.11 Experimental Results

[0348] The evaluation was performed on the internal-only (AI) and random-access (RA) configurations with VTM software version 3.0.1 according to the common test conditions JVET-J1010 [2]. The corresponding simulations were performed on an Intel Xeon cluster (E5-2697Av4, AVX2 on, turbo boost off) with Linux OS and GCC 7.2.1 compiler.

[0349] Table 1. Results for CE3-1.2.2 configured for VTMAI

[0350]

[0351] Table 2. Results for CE3-1.2.2 configured for VTM RA

[0352]

[0353] 5.12 Additional Results Exploiting Further Encoder Speedups

[0354] We additionally provide two further results for tests that rely on the same syntax as CE 3-1.2.2, but with optimized encoder search.

[0355] Table 1. Results of CE3-1.2.2 for VTMAI configuration, first encoder acceleration

[0356]

[0357] Table 2. Results of CE3-1.2.2 for VTM RA configuration, first encoder acceleration

[0358]

[0359] Table 3. Results of CE3-1.2.2 for VTMAI configuration, second encoder acceleration

[0360]

[0361] Table 4. Results of CE3-1.2.2 for VTM RA configuration, second encoder acceleration,

[0362]

[0363] 6. Fig.10 Encoder

[0364] Fig.10 Another example is shown which can be explained from the examples of FIGS. 1, 2 and 5-9. (In particular, some features can be directly derived from Figure 2 exported, so it is not repeated here).

[0365] Fig.10 An encoder 14 is shown, which may be, for example, Figure 1 A special case of an encoder. Similar to Figure 2, the encoder 14 may include a subtractor 22 configured to subtract a corresponding prediction signal 24 (e.g., a block 18 with reconstructed samples 104 obtained at step 812) from an incoming signal, i.e., an image 10, or a current block 18 in a block-based case, so as to obtain a prediction residual signal 26, which is then encoded into the data stream 12 by a prediction residual encoder 28. The prediction residual encoder 28 may include a lossy encoding stage 28a and a lossless encoding stage (entropy encoder) 28b. The lossy encoding stage 28a may receive the prediction residual signal 26 and include a quantizer 30 (not shown) configured to quantize samples of the prediction residual signal 26. The obtained prediction residual signal 34 is then subjected to lossless encoding by a lossless encoding stage 28b. The lossless encoding stage 28b is an entropy encoder that entropy encodes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 may further comprise a prediction residual signal reconstruction stage 36 connected to the output of the lossy encoding stage 28a in order to reconstruct from the transformed and quantized prediction residual signal 34'.

[0366] The encoder 14 may include an adder 42 for adding the reconstructed prediction residual signal 34' output by the stage 36 to the prediction signal 24 (e.g., including the block 18 with the reconstructed samples 104 obtained in step 813) in order to output a reconstructed signal, i.e., the reconstructed samples. This output is fed to the predictor 44, which may then determine the prediction signal 24 based thereon (e.g., by applying Figure 8.1-7.4 technology shown).

[0367] It can be seen that in Figure 9a and Figure 9b In the example, method steps 811, 812, 813 are respectively mapped by stages 811', 812', 813' in the predictor 44: method steps 811, 812, 813 can be implemented in the predictor 44 as hardware units and / or program routines, collectively represented by 811', 812', 813', or controlled by the predictor. Figure 7.1 As in the example of , the export stage 813' is skipped.

[0368] In particular, stages 811 and / or 813 may be depicted as presenting a buffer, such as buffer 910, for performing the shift operation described above (buffer 910 need not be part of stage 811 or 813: it may be a unit controlled by the stage being subjected to it). Alternatively, stage 812 is depicted as having or controlling multiplier 1910, wherein the P of a selected or averaged sample 102 of neighboring samples 17 is multiplied by the P of the selected or averaged sample 102 of neighboring samples 17. red The multiplication performed between elements is by the matrix 17M Q or Q redWeighting factors For the sake of simplicity, other elements (such as adders, etc.) are not shown in stages 811', 812', and 813'.

[0369] The storage device 1044 is indicated here as storing the ALWIP matrix 17M or (e.g., in the set S0, S1, S2) and the offset vector (Also indicated as b in the above k ). The matrix and / or offset index 944 (e.g., one or more of the indices discussed above, such as i, k, transpose index, set index) may be encoded in the data stream 12. Q or Q red The weighting factors are usually not signaled in the data stream 12: this is because the decoder already has the Q or Q of the ALWIP matrix 17M. red The concept of a weighting factor (eg, having a copy of the data stored on storage device 1044) thus reduces the payload.

[0370] Even though not shown in the figure, encoder 14 may determine the dimensions of the ALWIP matrix to be used (e.g., the dimensions provided in set S0, S1, S2), for example, based on the dimensions of block 18. In some cases, since the dimensions of block 18 are selected, it is not necessary to signal this selection.

[0371] Therefore, the encoder 14 is configured to insert the prediction residual 34 into the data stream 12 for the predetermined block 18 using the prediction residual obtained in step 812 and the prediction value 24 ( 104 ) of the predetermined sample, and the predetermined block 18 can be reconstructed from the data stream 12 .

[0372] Additionally or alternatively, the encoder 14 may be configured to insert prediction residuals (26, 34) into the data stream (12) for the predetermined block (18), the prediction residuals for Q or Q red Each of the predetermined samples indicates a corresponding residual value, so that the group Q or Q red The method further comprises reconstructing a predetermined block (18) using the prediction residuals (26, 34) and the predicted values ​​of the predetermined samples so that the corresponding reconstructed values, except for clipping applied optionally after prediction and / or correction, strictly linearly depend on the P values ​​in the reduced set of sample values ​​(102). red Neighboring samples (102).

[0373] Additionally or alternatively, the encoder 14 may be configured to subdivide the image (16) including the predetermined block (18) into a plurality of blocks of different block sizes. The encoder 14 may be configured to select a linear or affine linear transformation (19, A) based on a width W (also denoted by N) and a height H (also denoted by M) of the predetermined block (18). k ), so that as long as the width W and the height H of the predetermined block (18) are within a first set of width / height pairs (e.g. associated with S0), a linear or affine linear transformation (19, A) selected for the predetermined block (18) is selected from the first set of linear or affine linear transformations. k ), and as long as the width W and height H of the predetermined block (18) are within a second set of width / height pairs (e.g. associated with S1), a linear or affine linear transformation selected for the predetermined block (18) is selected from the second set of linear or affine linear transformations, and the second set of width / height pairs is disjoint from the first set of width / height pairs.

[0374] Additionally or alternatively, the encoder may be configured such that a third set of one or more width / height pairs (e.g., S0) includes only one width / height pair, W', H', and each linear or affine linear transform within the second set of linear or affine linear transforms is for transforming the N' sample values ​​into W'*H' prediction values ​​for a W'*H' array for the sample position.

[0375] Additionally or alternatively, the encoder may be configured such that each of the first and second sets of width / height pairs includes a width / height pair having W p Equal to H p The first width / height pair W p , H p , and H q =W p With W q =H p The second width / height pair W q , H q .

[0376] Additionally or alternatively, the encoder may be configured such that each of the first and second sets of width / height pairs includes a width / height pair having W p Equal to H p And H p >H q The third width / height pair W p , H p .

[0377] Additionally or alternatively, the encoder may be configured to insert a set index into the data stream for a predetermined block, the linear or affine linear transform being selected from a predetermined set of linear or affine linear transforms according to the set index.

[0378] Additionally or alternatively, the encoder may be configured such that the plurality of neighboring samples extend one-dimensionally along both sides of the predetermined block, and the decoder is configured to perform reduction by the following steps: for a plurality of neighboring samples of a first subset of a first side adjacent to the predetermined block, grouping the first subset into a first group (110) having one or more consecutive neighboring samples, and for a plurality of neighboring samples of a second subset of a second side adjacent to the predetermined block, grouping the second subset into a second group (110) having one or more consecutive neighboring samples, and in order to obtain a first sample value from the first group and a second sample value from the second group, downsampling or averaging is performed on each group of more than two neighboring samples of the first and second groups (110) having one or more neighboring samples, and the decoder is configured to select a linear or affine linear transformation from a predetermined set according to a set index, such that the set index results in two different states. Selecting one of a plurality of linear or affine linear transforms of a predetermined set, and subjecting a reduced set of sample values ​​to the predetermined linear or affine linear transform, generating an output vector of prediction values ​​if the set index takes the first of two different states in the form of a first vector, and distributing the prediction values ​​of the output vector to predetermined samples of a predetermined block along a first scanning order, and generating an output vector of prediction values ​​if the set index takes the second of two different states in the form of a second vector, and distributing the prediction values ​​of the output vector to predetermined samples of a predetermined block along a second scanning order, the second scanning order being transposed relative to the first scanning order, the first vector being different from the second vector, such that one of the second sample values ​​in the second vector fills a filled component of one of the first sample values ​​in the first vector, and one of the first sample values ​​in the second vector fills a filled component of the second sample value in the first vector.

[0379] Additionally or alternatively, the encoder may be configured such that each linear or affine linear transform within a first set (S1) of linear or affine linear transforms is used to transform N1 sample values ​​into w1*h1 prediction values ​​for a w1xh1 array for a sample position, and each linear or affine linear transform within a second set of linear or affine linear transforms is used to transform N2 sample values ​​into w2*h2 prediction values ​​for a w2xh2 array for a sample position, and wherein, for a first predetermined one of the first set of multiple width / height pairs, w1 exceeds the width of the first predetermined width / height pair, or h1 exceeds the height of the first predetermined width / height pair, and for a second predetermined one of the first set of multiple width / height pairs, w1 does not exceed the width of the second predetermined width / height pair, and h1 does not exceed the height of the second predetermined width / height pair, and the encoder may be configured to Downscaling (100) a plurality of neighboring samples by downsampling or averaging to obtain a reduced set of sample values ​​(102) such that the reduced set of sample values ​​(102) has N1 sample values ​​if the predetermined block has a first predetermined width / height pair and if the predetermined block has a second predetermined width / height pair, and performing the step of subjecting the reduced set of sample values ​​to a selected linear or affine linear transform by using only a first sub-portion of the selected linear or affine linear transform associated with a subsampling of the w1xh1 array of sample positions along the width dimension if w1 exceeds the width of one width / height pair or along the height dimension if h1 exceeds the height of one width / height pair in the case where the predetermined block has the first predetermined width / height pair, and by using the selected linear or affine linear transform in its entirety in the case where the predetermined block has the second predetermined width / height pair.

[0380] Additionally or alternatively, the encoder may be configured such that each linear or affine linear transform within a first set of linear or affine linear transforms is used to transform N1 sample values ​​into w1*h1 prediction values ​​of a w1xh1 array for a sample position where w1=h1, and each linear or affine linear transform within a second set of linear or affine linear transforms is used to transform N2 sample values ​​into w2*h2 prediction values ​​of a w2xh2 array for a sample position where w2=h2.

[0381] 7. Fig.11 Example

[0382] Fig.11 Another example is shown which can be explained from the examples 3-9. (In particular, some features can be directly derived from Figure 4 , so it is not repeated here).

[0383] Fig.11 show Figure 4 A possible implementation of the decoder 54, namely a method similar to Fig.10 The decoder implementation is adapted to the implementation of the encoder 14. In particular, the adder 42' and the predictor 44' can be adapted to the implementation of the encoder 14. Fig.10 The reconstructed, i.e. dequantized and retransformed prediction residual signal applied to the adder 42' can be derived through a series of entropy decoders which reverse the entropy coding of the entropy encoder, followed by a residual signal reconstruction stage consisting of a dequantizer and an inverse transformer 40', just as in the encoding. The output of the decoder is a reconstruction of the image 10. The reconstruction of the image 10 can be used directly at the output of the adder 42' or, alternatively, at the output of an in-loop filter.

[0384] It can be seen that the stages 813', 812', 813' can act as encoders 14, and the storage device 1044 can store sets of matrices as in encoders 14. Therefore, the discussion is not repeated here. The index 944 (e.g., one or more of the indices discussed above, such as i, k, transposed index, set index) can be directly obtained from the data stream 12. The selection between the sets S0, S1, S2 can follow the size (e.g., H / K or M / N).

[0385] Additionally or alternatively, the decoder may be configured to derive a prediction residual (34") from the data stream (12) for a predetermined block (18), and to reconstruct (42') the predetermined block (18) using the prediction residual (34") and a prediction value (24') for the predetermined samples (24', 104, 108, 108').

[0386] Additionally or alternatively, the decoder may be configured to generate a decoder for Q or Q red Each of the predetermined samples obtains a corresponding residual value, deriving a prediction residual (34") from the data stream (12) for the predetermined block (18), and using the corresponding residual value (34") for the set Q or Q red Each of the predetermined samples is reconstructed using the prediction residual (34") and the prediction value (24', 104) to obtain a corresponding reconstructed value (10) by correcting the prediction value, so that the corresponding reconstructed value (10) depends strictly linearly on P in the reduced set of sample values ​​(102). red Neighboring samples (102), except for a shearing that may be optionally applied after prediction and / or correction.

[0387] Additionally or alternatively, the decoder may be configured such that the decoder is configured to subdivide the image (10) comprising the predetermined block (18) into a plurality of blocks of different block sizes, wherein the decoder is configured to select a linear or affine linear transformation (19, 17M, A) according to a width W and a height H of the predetermined block (18).k ), such that as long as the width W and the height H of the predetermined block (18) are within a first set of width / height pairs, the linear or affine linear transformation selected for the predetermined block (18) is selected from the first set of linear or affine linear transformations, and as long as the width W and the height H of the predetermined block are within a second set of width / height pairs, the linear or affine linear transformation selected for the predetermined block (18) is selected from the second set of linear or affine linear transformations, and the second set of width / height pairs is disjoint from the first set of width / height pairs.

[0388] Additionally or alternatively, the decoder may be configured such that the decoder is configured to subdivide the image (10) comprising the predetermined block (18) into a plurality of blocks of different block sizes, wherein the decoder is configured to select a linear or affine linear transformation (19, 17M, A) depending on a width W and a height H of the predetermined block (18). k ), so that as long as the width W and height H of the predetermined block (18) are within the first set of width / height pairs, the linear or affine linear transformation selected for the predetermined block (18) is selected from the first set of linear or affine linear transformations (S1), as long as the width W and height H of the predetermined block (18) are within the second set of width / height pairs, the linear or affine linear transformation selected for the predetermined block (18) is selected from the second set of linear or affine linear transformations (S2), the second set of width / height pairs is disjoint with the first set of width / height pairs, and as long as the width W and height H of the predetermined block (18) are within the third set of width / height pairs, the linear or affine linear transformation selected for the predetermined block (18) is selected from the third set of linear or affine linear transformations (S3), the third set of width / height pairs is disjoint with the first and second sets of width / height pairs.

[0389] Additionally or alternatively, the decoder may be configured such that a third set (S0) of one or more width / height pairs comprises only one width / height pair, W', H', and each linear or affine linear transform within the third set (S0) of linear or affine linear transforms is used to transform the N' sample values ​​into W'*H' predicted values ​​for a W'xH' array for the sample position.

[0390] Additionally or alternatively, the decoder may be configured such that each of the first and second sets (S1, S2) of width / height pairs comprises a width / height pair having W p Equal to H p The first width / height pair W p , H p , and H q =W p With W q =H p The second width / height pair Wq , H q .

[0391] Additionally or alternatively, the decoder may be configured such that each of the first and second sets (S1, S2) of width / height pairs comprises a width / height pair having W p Equal to H p And H p >H q The third width / height pair W p , H p .

[0392] Additionally or alternatively, the decoder may be configured to read a set index (k) from the data stream (12) for a predetermined block (18), and select the linear or affine linear transform from a predetermined set of linear or affine linear transforms according to the set index (k).

[0393] Additionally or alternatively, the decoder may be configured such that the plurality of neighboring samples (17) extend one-dimensionally along both sides of the predetermined block (18), and the decoder is configured to perform reduction (811) by: for a plurality of neighboring samples of a first subset of a first side adjacent to the predetermined block, grouping the first subset into a first group (110) having one or more consecutive neighboring samples, and for a plurality of neighboring samples of a second subset of a second side adjacent to the predetermined block, grouping the second subset into a second group (110) having one or more consecutive neighboring samples, and in order to obtain a first sample value (110) from the first group and a second sample value (102) from the second group, downsampling or averaging is performed on each of the first and second groups (110) having one or more neighboring samples, each of the first and second groups (110) having more than two neighboring samples, and the decoder is configured to select a linear or affine linear transform from a predetermined set of linear or affine linear transforms according to a set index. , so that the set index of two different states results in selecting one of the linear or affine linear transforms in a predetermined set of linear or affine linear transforms, and subjecting the reduced set of sample values ​​(102) to the predetermined linear or affine linear transform, if the set index takes the first of the two different states in the form of a first vector, an output vector of predicted values ​​is generated, and the predicted values ​​of the output vector are distributed over predetermined samples of a predetermined block along a first scan order, and if the set index takes the second of the two different states in the form of a second vector to generate an output vector of predicted values, and the predicted values ​​of the output vector are distributed over predetermined samples of a predetermined block along a second scan order, the second scan order is transposed relative to the first scan order, the first vector is different from the second vector, so that one of the second sample values ​​in the second vector fills a filled component of the first sample value in the first vector, and one of the first sample values ​​in the second vector fills a filled component of the second sample value in the first vector.

[0394] Additionally or alternatively, the decoder may be configured such that each linear or affine linear transform within a first set (S1) of multiple linear or affine linear transforms is used to transform N1 sample values ​​into w1*h1 prediction values ​​for a w1xh1 array for a sample position, and each linear or affine linear transform within a second set (S2) of linear or affine linear transforms is used to transform N2 sample values ​​into w2*h2 prediction values ​​for a w2xh2 array for a sample position, and wherein, for a first predetermined one of the first set (S1) of multiple width / height pairs, w1 exceeds the width of the first predetermined width / height pair, or h1 exceeds the height of the first predetermined width / height pair, and for a second predetermined one of the first set (S1) of width / height pairs, w1 does not exceed the width of the second predetermined width / height pair, and h1 does not exceed the height of the second predetermined width / height pair, and wherein the decoder is configured The method further comprises performing a downscaling (100, 811) of a plurality of neighboring samples by downsampling or averaging to obtain a reduced set of sample values ​​(102), so that if the predetermined block has a first predetermined width / height pair and if the predetermined block has a second predetermined width / height pair, the reduced set of sample values ​​(102) has N1 sample values, and performing a step of subjecting (813) the reduced set of sample values ​​to a selected linear or affine linear transform by using only a first sub-portion of the selected linear or affine linear transform associated with a subsampling of the w1xh1 array of sample positions, along the width dimension if w1 exceeds the width of one width / height pair, or along the height dimension if h1 exceeds the height of one width / height pair, if the predetermined block has the second predetermined width / height pair, and by using the selected linear or affine linear transform in its entirety, if the predetermined block has the second predetermined width / height pair.

[0395] Additionally or alternatively, the decoder may be configured such that each linear or affine linear transform within a first set of linear or affine linear transforms is used to transform N1 sample values ​​into w1*h1 prediction values ​​for a w1xh1 array for a sample position where w1=h1, and each linear or affine linear transform within a second set of linear or affine linear transforms is used to transform N2 sample values ​​into w2*h2 prediction values ​​for a w2xh2 array for a sample position where w2=h2.

[0396] 8. Discuss the effects of this technology

[0397] With respect to independent use of operations such as bit shifting for averaging and / or interpolation (particularly to achieve the effect of reducing computational effort), it should also be noted that other effects may be achieved, which in some examples may even be superior to the efficient use of bit shifting.

[0398] In particular, for this example, prediction modes may be shared between different block shapes such that the selection of the ALWIP matrix 17M (e.g., at step 812a) is performed on a limited number of sets. For example, the set of ALWIP matrices may be fewer than the possible dimensions (e.g., height / width pairs) of the block 18 to be predicted. Fig.12 , mapping different width / height pairs of the block 18 to be predicted to one of a set S0 (e.g., having n0 matrices, e.g., having n0=16), a set S1 (e.g., having n1 matrices, e.g., having n1=8) and S2 (e.g., having n2 matrices, e.g., having n2=6), as described above (different partitionings are possible).

[0399] For example, multiple 16x8 matrices of set S1 may be shared by prediction modes for blocks having any of dimensions 4x8, 4x16, 4x32, 4x64, 8x4, 8x8, 16x4, 32x4, and 64x4, and multiple 64x8 matrices of set S2 may be shared by prediction modes for blocks having any of dimensions 8x16, 8x32, 8x64, 16x8, 16x16, 16x32, 16x64, 32x8, 32x16, 32x32, 32x64, 64x8, 64x16, 64x32, 64x64. It is only necessary to perform techniques such as those discussed above for reduction step 811 to reduce the dimensions of boundary 17 to the necessary P matrix of samples to form set 102. red The number of blocks 18 to be predicted is not relevant in step 812, but the original dimensions of the block 18 to be predicted are irrelevant. In step 813 (if implemented), the complete prediction of the block will be obtained by simply performing an interpolation.

[0400] It has been noted that this method allows reducing the storage space necessary at the memory location by an unexpected dimension of 16*16*4+8*16*8+6*64*8=5120 values ​​(each value being, for example, an 8-bit value).

[0401] In contrast, conventional techniques would require a set of matrices for each width / height pair. Fig.12 It is easy to understand that 25 groups will be needed! It is easy to understand that 25 groups of matrices are much more than the storage space for 5120 values. In order to reduce the necessary storage space, it is necessary to reduce the number of matrices per group: however, if only a few matrices are available for prediction, the quality will be reduced!

[0402] The reduction in storage space taking into account the sharing technique can even be amplified by reducing the size of the stored matrix itself. For example, the prediction of an MxN=64x64 block will require a matrix of size QxP=(M*N)x(M+N), i.e., (64*64)*(64+64)=524288 values ​​to be stored in the storage space! Thus, using this technique, more storage space can be saved than expected.

[0403] Thus, the present technique allows the number of parameters that need to be stored in unit 1044 to be reduced.

[0404] With or without actually using bit shifting, the storage resources available to an encoder or decoder may be reduced, or conversely, more prediction modes may be used for parity checking of storage space.

[0405] The best results are obtained though by combining the bit shifting technique (at steps 811 and / or 813) with the one that shares the same prediction mode for multiple modes (at step 812).

[0406] With respect to the conventional method of using 25 different sets for 25 different height / width pairs, it is obvious that the present technique can be interpreted as an increased complexity (because the conventional technique cannot imagine steps 811 and / or 813). However, the introduction of steps 811 and / or 813 can be compensated by multiplicative reduction.

[0407] Furthermore, with respect to the conventional method of using 25 different sets for 25 different height / width pairs, the instructions required for the control process require more storage space (because the extra instructions of steps 811 and / or 813 are to be stored). However, the necessity of storing the instructions of steps 811 and / or 813 can be offset by the space implied by reducing the number of reduced matrices to be stored.

[0408] 9. Further embodiments and examples

[0409] Generally, the examples can be implemented as a computer program product with program instructions, which, when the computer program product runs on a computer, can be operated to perform one of the methods. For example, the program instructions may be stored on a machine-readable medium.

[0410] Other examples comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0411] In other words, therefore, one example of a method is a computer program having program instructions for performing one of the methods described herein, when the computer program runs on a computer.

[0412] Therefore, a further example of the method is a data carrier medium (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods herein. The data carrier medium, the digital storage medium or the recorded medium are tangible and / or non-transitory, as opposed to intangible and transitory signals.

[0413] Therefore, a further example of a method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may be transmitted, for example, via a data communication connection, for example via the Internet.

[0414] Further examples comprise a processing device, for example a computer or a programmable logic device, for performing one of the methods herein.

[0415] A further example comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0416] A further example comprises an apparatus or system for transferring (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0417] In some examples, a programmable logic device (e.g., a field programmable logic gate array) can be used to perform some or all of the functions of the methods described herein. In some examples, a field programmable logic gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods can be performed by any appropriate hardware device.

[0418] The above examples are merely examples of the above principles. It should be understood that modifications and variations of the arrangements and details described herein will be apparent. Therefore, it is intended that the present invention be limited by the scope of the appended claims rather than by the specific details given by the description and explanation of the examples herein.

[0419] In the following description, the same or equivalent components or components having the same or equivalent functions are denoted by the same or equivalent reference numerals even if they appear in different drawings.

[0420] References

[0421] [1] P. Helle et al., “Non-linear weighted intraprediction”, JVET-L0199, Macau, China, October 2018.

[0422] [2] F. Bossen, J. Boyce, K. Suehring, X. Li, V. Seregin, “JVET generic test conditions and software reference configuration for SDR video”, JVET-K1010, Ljubljana, SI, July 2018.

Claims

1. A decoder for decoding an image from a data stream, wherein the decoder is configured to: subdivide the image including a predetermined block into a plurality of blocks with different block sizes, and predict the predetermined block of the image using a plurality of neighboring samples by the following steps: The plurality of neighboring samples are reduced to obtain a reduced sample value set having a smaller number of samples than the plurality of neighboring samples, wherein: The decoder is configured to perform the reduction by grouping the plurality of neighboring samples into groups of one or more consecutive neighboring samples, and performing downsampling or averaging on each of the groups of one or more neighboring samples having two or more neighboring samples; The matrix is ​​selected according to the width W and the height H of the predetermined block, so that the matrix selected for the predetermined block is selected from the following set: As long as the width W and height H of the predetermined block are both 4, select from the first matrix set, As long as the width W and the height H of the predetermined block are both 8, or only one of the width W or the height H is 4, select from the second matrix set; and otherwise, selecting from a third set of matrices; and The selected matrix and offset value are applied to the set of downscaled sample values ​​to obtain prediction values ​​for predetermined samples of the predetermined block. 2 . The decoder of claim 1 , further configured to perform the reduction by downsampling. The decoder of claim 1 , further configured to perform the reduction by averaging. The decoder of claim 3 , wherein the averaging comprises bit shifting.

5. The decoder of claim 1, further configured to derive a prediction value for further samples of the predetermined block by interpolation based on the prediction value for the predetermined sample and the plurality of neighboring samples.

6. The decoder of claim 5, configured to perform the interpolation by bit shifting.

7. The decoder according to claim 1, wherein: The plurality of neighboring samples extend one-dimensionally along both sides of the predetermined block.

8. The decoder according to claim 1, Wherein the matrix includes P red *Q red or P red *Q weighting factors, where P red is the number of sample values ​​in the reduced sample value set, and Q red Or Q is the number of predetermined samples within the predetermined block, At least 1 / 4P red *Q red or 1 / 4P red *Q weighting factors are non-zero weight values, Wherein for said Q or Q red For each of the predetermined samples, the P red *Q red or P red *Q weighting factors include a series of P related to the corresponding predetermined samples red weighting factors, where When the series are arranged one below the other according to a raster scanning order between predetermined samples of the predetermined block, the series form an omnidirectional nonlinear envelope.

9. The decoder according to claim 1, configured to derive a prediction residual from the data stream for the predetermined block, and The predetermined block is reconstructed using the prediction residual and the prediction value for the predetermined sample.

10. A decoding method for decoding a picture including a predetermined block subdivided into a plurality of blocks having different block sizes, the method comprising predicting the predetermined block of the picture using a plurality of neighboring samples by the following steps: The plurality of neighboring samples are reduced by downsampling or averaging to obtain a reduced sample value set having a smaller number of samples than the plurality of neighboring samples, wherein: The reducing includes grouping the plurality of neighboring samples into groups of one or more consecutive neighboring samples, and performing downsampling or averaging on each of the one or more groups of neighboring samples having two or more neighboring samples to perform the reducing; The matrix is ​​selected according to the width W and the height H of the predetermined block, such that the matrix selected for the predetermined block is selected from the following set: As long as the width W and the height H of the predetermined block are both 4, select from the first matrix set; As long as the width W and the height H of the predetermined block are both 8, or only one of the width W or the height H is 4, select from the second matrix set; and otherwise, selecting from a third set of matrices; and The selected matrix and offset value are applied to the set of downscaled sample values ​​to obtain prediction values ​​for predetermined samples of the predetermined block.

11. An encoder for encoding an image into a data stream, the encoder being configured to subdivide an image including a predetermined block into a plurality of blocks having different block sizes, the encoder being configured to predict the predetermined block of the image using a plurality of neighboring samples by the following steps: The plurality of neighboring samples are reduced to obtain a reduced sample value set having a smaller number of samples than the plurality of neighboring samples, wherein: The encoder is configured to perform the reduction by grouping the plurality of neighboring samples into groups of one or more consecutive neighboring samples, and performing downsampling or averaging on each of the groups of one or more neighboring samples having two or more neighboring samples; The matrix is ​​selected according to the width W and the height H of the predetermined block, such that the matrix selected for the predetermined block is selected from the following set: As long as the width W and height H of the predetermined block are both 4, select from the first matrix set, As long as the width W and the height H of the predetermined block are both 8, or only one of the width W or the height H is 4, select from the second matrix set; and otherwise, selecting from a third set of matrices; and The selected matrix and offset value are applied to the set of downscaled sample values ​​to obtain prediction values ​​for predetermined samples of the predetermined block.

12. Encoder according to claim 11, configured to derive the prediction value for further samples of the predetermined block by interpolation based on the prediction value for the predetermined sample and the plurality of neighboring samples.

13. The encoder of claim 12, configured to perform the interpolation by bit shifting.

14. The encoder according to claim 12, wherein: The plurality of neighboring samples extend one-dimensionally along both sides of the predetermined block.

15. The encoder according to claim 12, Wherein the matrix includes P red *Q red or P red *Q weighting factors, where P is the number of sample values ​​in the reduced set of sample values, and Q is the number of predetermined samples in the predetermined block, At least 1 / 4P red *Q red or 1 / 4P red *Q weighting factors are non-zero weight values, Wherein for said Q or Q red For each of the predetermined samples, the P red *Q red or P red *Q weighting factors include a series of P related to the corresponding predetermined samples red weighting factors, where When the series are arranged one below the other according to a raster scanning order between predetermined samples of the predetermined block, the series form an omnidirectional nonlinear envelope.

16. The encoder according to claim 11, configured to insert a prediction residual into the data stream for the predetermined block, the predetermined block being reconstructible from the data stream using the prediction residual and a prediction value of the predetermined sample.

17. A coding method for coding an image comprising a predetermined block subdivided into a plurality of blocks having different block sizes, the method comprising predicting the predetermined block of the image using a plurality of neighboring samples by the following steps: Reducing the plurality of neighboring samples to obtain a reduced sample value set having a smaller number of samples than the plurality of neighboring samples by downsampling or averaging, wherein the reducing comprises grouping the plurality of neighboring samples into groups of one or more consecutive neighboring samples, and performing downsampling or averaging on each of the one or more groups of neighboring samples having two or more neighboring samples; The matrix is ​​selected according to the width W and the height H of the predetermined block, so that the matrix selected for the predetermined block is selected from the following set: As long as the width W and height H of the predetermined block are both 4, select from the first matrix set, As long as the width W and the height H of the predetermined block are both 8, or only one of the width W or the height H is 4, select from the second matrix set; and otherwise, selecting from a third set of matrices; and The selected matrix and offset value are applied to the set of downscaled sample values ​​to obtain prediction values ​​for predetermined samples of the predetermined block.

18. A non-transitory storage medium storing instructions which, when executed by a processor, cause the processor to perform the method according to claim 10 or 17.

Images