Affine linear weighted intra prediction

Abstract

To provide an apparatus and a method for video encoding and decoding that uses the concept of an intra-prediction mode to enable more efficient compression.SOLUTION: A method for predicting a block of a picture includes determining that a luma prediction signal is generated using an affine linear weighted intra prediction (ALWIP) mode, mapping the ALWIP mode to a different intra prediction mode, and using the mapping to obtain a corresponding chroma prediction signal.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] This specification relates to techniques including, among other things, Affine Linear Weighted Intra Prediction (ALWIP).

Background Art

[0002] Various embodiments and aspects of the present invention will be described. At least some of these embodiments refer to methods and / or apparatuses for video coding, such as, among other things, High Efficiency Video Coding (HEVC).

[0003] Furthermore, additional embodiments are defined by the appended claims.

[0004] Note that any embodiment defined by the claims may optionally be complemented by any of the details (features and functionality) described in the following chapters in some cases.

[0005] Also, the embodiments described in the following chapters may be used individually and may be complemented by any of the features of another chapter or any of the features included in the claims.

[0006] Note also that the individual aspects described in this specification may be used individually or in combination. Thus, details may be added to each of these aspects without adding details to different aspects of these individual aspects.

[0007] It should also be noted that this disclosure also explicitly or implicitly describes features of decoding and / or encoding systems and / or methods.

[0008] Moreover, the features and functionalities described herein with respect to the method may be used in an apparatus. Further, any features and functionalities disclosed herein with respect to an apparatus may also be used in a corresponding method. In other words, the methods disclosed herein may be complemented by any of the features and functionalities described with respect to these apparatuses.

[0009] Also, any of the features and functionalities described herein may be implemented using hardware or software or a combination of hardware and software, as described in the "Alternative implementations" section.

[0010] Moreover, in some examples, any of the features described within parentheses (either “(...)” or “[…]”) may be optional.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0011] The achievement of further implementation techniques for predicting blocks to be encoded and / or decoded has always been a goal of this technique. For example, in order to shorten the code length, it is preferable to efficiently compress the data stream.

MEANS FOR SOLVING THE PROBLEMS

[0012] According to one aspect, a decoder for decoding a picture from a data stream, wherein a predetermined block of the picture is predicted by mapping a set of P neighboring samples in the vicinity of the predetermined block to a set of Q predicted values for the samples of the predetermined block using a linear or affine linear transformation, the linear or affine linear transformation includes P * Q weight coefficients, at least 1 / 4P of which *The Q weight coefficients are non-zero weight values, and for each of the Q predicted values, a series of P weight coefficients associated with each predicted value is formed. When the series is arranged one by one in raster scan order among the samples of a predetermined block and goes down sequentially, a decoder that forms an envelope that is non-linear in all directions is disclosed.

[0013] The decoder is P * The Q weight coefficients may be such that they are independent of each other according to any regular mapping rule.

[0014] The decoder may be such that the maximum mean of the cross-correlation between the first series of weight coefficients associated with each predicted value and the one that produces the higher maximum value among the second series of weight coefficients associated with predicted values other than each predicted value or the inverted version of the second series is lower than a predetermined threshold.

[0015] The decoder may be such that the predetermined threshold is 0.3.

[0016] The decoder may be such that P neighboring samples are arranged along a one-dimensional path extending along the boundary of a predetermined block, and for each of the Q predicted values, the series of P weight coefficients associated with each predicted value is ordered to traverse the one-dimensional path in a predetermined direction.

[0017] The decoder Derives a prediction residual from a data stream for a predetermined block to obtain a corresponding residual value for each of the Q predicted values. Reconstructs a predetermined block by correcting each of the Q predicted values with the corresponding residual value to obtain a corresponding reconstructed value such that the corresponding reconstructed value depends linearly entirely on the P neighboring samples. May be configured as follows.

[0018] The decoder The decoder is configured to subdivide a picture into a plurality of blocks of different block sizes including a predetermined block, For each of the set of block sizes, the decoder a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, wherein for each of the second set of intra prediction modes, according to each intra prediction mode, a block of each block size maps a set of P neighboring samples in the neighborhood of the block of each block size to a set of Q prediction values for the samples of the block of each block size, thereby predicting, and supports the second set of intra prediction modes, For each block size of the set of block sizes, the associated linear or affine linear transformation of the intra prediction mode within the second set of intra prediction modes for each block size is different from each other, and the cardinality of the second set of intra prediction modes for the block sizes within the set of block sizes is the same, but the associated linear or affine linear transformation of the intra prediction mode within the second set of intra prediction modes for different block sizes within the set of block sizes is not transferable to each other by scaling.

[0019] According to one aspect, a method for decoding a picture from a data stream, comprising: mapping, using a linear or affine linear transformation, a set of P neighboring samples in the neighborhood of a predetermined block to a set of Q prediction values for the samples of the predetermined block, The linear or affine linear transformation includes P * Q weight coefficients, at least 1 / 4P of which *The Q weight coefficients are non-zero weight values and constitute a series of P weight coefficients associated with each of the Q predicted values. When the series are arranged one by one sequentially downward in raster scan order among the samples of a predetermined block, a method for forming an envelope that is non-linear in all directions is disclosed.

[0020] According to one aspect, an encoder for encoding a data stream from a picture, configured to map a set of P neighboring samples in the neighborhood of a predetermined block to a set of Q predicted values for the samples of the predetermined block using a linear or affine linear transformation, The linear or affine linear transformation includes P * Q weight coefficients, at least 1 / 4P of which * The Q weight coefficients are non-zero weight values and, for each of the Q predicted values, constitute a series of P weight coefficients associated with each predicted value. When the series are arranged one by one sequentially downward in raster scan order among the samples of a predetermined block, an encoder for forming an envelope that is non-linear in all directions is disclosed.

[0021] The encoder includes P * The Q weight coefficients may be such that they are independent of each other according to any normal mapping rule.

[0022] The encoder may be such that the maximum average of the cross-correlation between a first series of weight coefficients associated with each predicted value and the one that produces the higher maximum value among a second series of weight coefficients associated with predicted values other than each predicted value or a reversed version of the second series is lower than a predetermined threshold.

[0023] The encoder may be configured to use a linear or affine linear transformation for the luma component of a predetermined block.

[0024] According to one aspect, an encoding method, mapping a set of P neighboring samples in the neighborhood of a given block to a set of Q predicted values for the samples of the given block using a linear or affine linear transformation, The linear or affine linear transformation includes P * including Q weight coefficients, at least 1 / 4P of which * The Q weight coefficients are non-zero weight values, For each of the Q predicted values, a series of P weight coefficients associated with the respective predicted value is constructed, and when the series is arranged sequentially one by one downward according to the raster scan order among the samples of the given block, it forms an envelope that is non-linear in all directions. A coding method is disclosed.

[0025] According to one aspect, a decoder for decoding a picture from a data stream supporting a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, assigning a given block of a picture to the first set or the second set based on a first signalization in the data stream, sorting the assigned set of intra prediction modes according to the intra prediction modes used for neighboring blocks in the neighborhood of the given block to obtain a list of intra prediction modes, deriving an index into the list of intra prediction modes from the data stream for the given block, predicting the given block using the intra prediction mode pointed to by the index and configured to decode by The decoder is The decoder is When the assigned set is the second set of intra prediction modes, when sorting the assigned set, use a first mapping that maps each intra prediction mode of the first set of prediction modes to one representative intra prediction mode within the second set of intra prediction modes, and / or When the assigned set is the first set of intra prediction modes, when sorting the assigned set, use a second mapping that maps each intra prediction mode of the second set of prediction modes to one representative intra prediction mode within the first set of intra prediction modes A decoder configured as such is disclosed.

[0026] According to one aspect, a decoder for decoding a picture from a data stream that supports a set of intra prediction modes for each of different block sizes, For a predetermined block of a picture To obtain a list of intra prediction modes, sort the set of intra prediction modes for the block size of the predetermined block according to the intra prediction modes used for neighboring blocks in the neighborhood of the predetermined block, Derive an index into the list of intra prediction modes from the data stream for the predetermined block, Predict the predetermined block using the intra prediction mode pointed to by the index And is configured to decode by doing so, The decoder When one of the neighboring blocks is of a different size than the predetermined block, when sorting the assigned set, use a mapping that maps each intra prediction mode of the set of prediction modes for the block size of one neighboring block to one representative intra prediction mode within the set of intra prediction modes for the block size of the predetermined block A decoder configured as such is disclosed.

[0027] The decoder may be further configured to decode an index from a data stream using a variable length code such that the code length depends on the rank of the intra prediction mode within the list of intra prediction modes pointed to by the index.

[0028] The variable length code may be of a type such as a unary code.

[0029] The decoder may be configured to sort the assigned set of intra prediction modes such that the intra prediction mode used for neighboring blocks within the assigned set of intra prediction modes or targeted by the intra prediction mode used for neighboring blocks via the first or second mapping is placed first in the list.

[0030] The decoder may be configured to sort the set of intra prediction modes for a given block size of a given block such that the intra prediction mode used for neighboring blocks within this set of intra prediction modes or targeted by the intra prediction mode used for neighboring blocks by mapping is placed first in the list.

[0031] The decoder may be configured such that a given block is predicted by mapping a set of P neighboring samples in the neighborhood of the given block to a set of Q predicted values for the samples of the given block according to each of a second set of intra prediction modes, for each of the second set of intra prediction modes, the linear or affine linear transformation includes P * Q weight coefficients, at least 1 / 4P of which *The Q weight coefficients are non-zero weight values, and for each of the Q prediction values, a series of P weight coefficients associated with each respective prediction value is formed, and when the series is arranged one by one sequentially downward in raster scan order among the samples of a predetermined block, an envelope that is non-linear in all directions is formed.

[0032] The decoder may be configured such that, according to each of a set of intra prediction modes, by using a linear or affine linear transformation to map a set of P neighboring samples in the vicinity of a predetermined block to a set of Q prediction values for the samples of the predetermined block, the predetermined block is predicted. For each of the second set of intra prediction modes, the linear or affine linear transformation includes P * Q weight coefficients, of which at least 1 / 4P * The Q weight coefficients are non-zero weight values, and for each of the Q prediction values, a series of P weight coefficients associated with each respective prediction value is formed, and when the series is arranged one by one sequentially downward in raster scan order among the samples of a predetermined block, an envelope that is non-linear in all directions is formed.

[0033] The decoder is configured such that the decoder subdivides a picture into a plurality of blocks of different block sizes including a predetermined block. is configured such that for each of a set of block sizes, it supports a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes. The decoder When one of the neighboring blocks is of the same block size as a predetermined block within the set of block sizes but is assigned to a different one of the first and second sets of intra prediction modes, when sorting the assigned set, the first and second mappings are used. If one of the neighboring blocks has a block size different from a predetermined block within the set of block sizes, when sorting the assigned set, if the set assigned to the predetermined block is a second set for the block size of the predetermined block and the set assigned to one neighboring block is a first set for the block size of one neighboring block, use a third mapping that maps each intra prediction mode in the first set of prediction modes for the block size of one neighboring block to a representative one intra prediction mode within the second set of intra prediction modes for the block size of the predetermined block, and / or if the set assigned to the predetermined block is a first set for the block size of the predetermined block and the set assigned to one neighboring block is a second set for the block size of one neighboring block, use a fourth mapping that maps each intra prediction mode in the second set of prediction modes for the block size of one neighboring block to a representative one intra prediction mode within the first set of intra prediction modes for the block size of the predetermined block, and / or If one of the neighboring blocks has the same block size as a predetermined block within the set of block sizes, when the set assigned to the predetermined block is a second set for the block size of the predetermined block and the set assigned to one neighboring block is a second set for the block size of one neighboring block, use a fifth mapping that maps each intra prediction mode in the second set of prediction modes for the block size of one neighboring block to a representative one intra prediction mode within the second set of intra prediction modes for the block size of the predetermined block may be configured as such.

[0034] The decoder is configured to subdivide the picture into a plurality of blocks of different block sizes including the predetermined block, The decoder is configured to support a first set of intra prediction modes including, for each of a set of block sizes, a plurality of directional prediction modes and at least one of a DC mode and a planar mode, as well as a second set of intra prediction modes. The decoder, if a given block is not square, when the given block is oriented in a first direction, apply a linear or affine linear transformation associated with the intra prediction mode pointed to by the index to a series of neighboring samples of the neighborhood of the given block. when the given block is oriented in a second direction orthogonal to the first direction, obtain a prediction block and then apply a reversed version of the series of neighboring samples of the neighborhood of the given block to a linear or affine linear transformation other than the linear or affine linear transformation associated with the intra prediction mode pointed to by the index to an intra prediction mode of a second set of intra prediction modes for transposing the prediction block. It may be configured to perform prediction of a given block using the intra prediction mode pointed to by the index.

[0035] The decoder, the second set of intra prediction modes includes first and second subsets of intra prediction modes, and the given block is square and may be configured as such, The decoder, when the intra prediction mode pointed to by the index is included within the first subset, apply a linear or affine linear transformation associated with the intra prediction mode pointed to by the index to a series of neighboring samples of the neighborhood of the given block. when the intra prediction mode pointed to by the index is included within the second subset, obtain a prediction block and then apply a reversed version of the series of neighboring samples of the neighborhood of the given block to a linear or affine linear transformation associated with an intra prediction mode of the first subset of intra prediction modes for transposing the prediction block. perform prediction of a given block using the intra prediction mode pointed to by the index is configured to be.

[0036] The decoder may be such that, for at least one of the second set of intra prediction modes, a prediction using one intra prediction mode requires prediction of a given block from a spatial region to a transform region or from a transform region to a transform region.

[0037] The decoder may be configured to use a first or second set of transforms for the luma component and a first set of transforms for the chroma component.

[0038] The decoder may be configured to map the second set of modes used for the luma component to the first set of modes for the luma component.

[0039] The decoder may be such that the second mapping maps a plurality of modes of the second set to a single mode of the first set.

[0040] The decoder may be such that the second mapping maps all modes of the second set to a single mode of the first set.

[0041] The decoder may be such that a single mode of the first set is the planar mode.

[0042] According to one aspect, a method for decoding a picture from a data stream supporting a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, comprising: for a given block of the picture, assign the given block to the first set or the second set based on a first signal in the data stream, To obtain a list of intra prediction modes, sort the assigned set of prediction modes according to the intra prediction modes used for neighboring blocks in the neighborhood of a given block, derive an index into the list of intra prediction modes from a data stream for a given block, predict using the intra prediction mode pointed to by the index including a step of decoding by The decoding step is when the assigned set is the second set of intra prediction modes, when sorting the assigned set, mapping each prediction mode of the first set of prediction modes to a representative one intra prediction mode within the second set of intra prediction modes, a first mapping, and / or when the assigned set is the first set of intra prediction modes, when sorting the assigned set, using a second mapping that maps each intra prediction mode of the second set of prediction modes to a representative one intra prediction mode within the first set of intra prediction modes, a method is disclosed.

[0043] According to one aspect, an encoder for encoding a picture on a data stream that supports a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, assign a given block of the picture to either the first set or the second set based on a first signal in the data stream, and encode the given block, To obtain a list of intra prediction modes, sort the assigned set of prediction modes according to the intra prediction modes used for neighboring blocks in the neighborhood of a given block, determine an index into the list of intra prediction modes from the data stream for a given block, predict the given block using the intra prediction mode pointed to by the index configured to encode by The encoder when the assigned set is the second set of intra prediction modes, when sorting the assigned set, use a first mapping that maps each intra prediction mode of the first set of prediction modes to one representative intra prediction mode within the second set of intra prediction modes, and / or disclosed is an encoder configured to use a second mapping that maps each intra prediction mode of the second set of prediction modes to one representative intra prediction mode within the first set of intra prediction modes when sorting the assigned set, when the assigned set is the first set of intra prediction modes.

[0044] According to one aspect, an encoder for encoding a picture on a data stream that supports a set of intra prediction modes for each of different block sizes, a predetermined block of the picture, to obtain a list of intra prediction modes, sort the set of intra prediction modes for the block size of the predetermined block according to the intra prediction modes used for neighboring blocks in the neighborhood of the predetermined block, for the predetermined block, insert an index into the data stream into the list of intra prediction modes, predict the predetermined block using the intra prediction mode pointed to by the index configured to encode by The encoder when one of the neighboring blocks is a different size than the predetermined block, when sorting the assigned set, is configured to use a mapping that maps each intra prediction mode of the set of prediction modes for the block size of one neighboring block to one representative intra prediction mode within the set of intra prediction modes for the block size of the predetermined block. Disclosed is an encoder.

[0045] The encoder may be further configured to encode an index in a data stream using a variable length code such that the code length monotonically depends on the rank of the intra prediction mode in the list pointed to by the index.

[0046] The variable length code may be a unary code.

[0047] The encoder may be configured to sort the prediction modes according to the probabilities of those prediction modes for previous uses for other blocks and / or the history data.

[0048] The encoder may be configured to write a second signalization into the data stream by continuously repeating the first symbol up to the second symbol so as to derive an index into the list based on the length of symbol repetitions.

[0049] The encoder may be configured to sort the assigned set of intra prediction modes such that the intra prediction mode used for neighboring blocks within the assigned set of intra prediction modes or targeted by the intra prediction mode used for neighboring blocks via the first or second mapping is placed first in the list.

[0050] The encoder may be configured to sort the set of intra prediction modes for a block size of a given block such that the intra prediction mode used for neighboring blocks within this set of intra prediction modes or targeted by the intra prediction mode used for neighboring blocks by the mapping is placed first in the list.

[0051] The encoder may be configured such that, for each of a second set of intra prediction modes, a set of P neighboring samples in the neighborhood of a given block is mapped to a set of Q prediction values for the samples of the given block by using a linear or affine linear transformation, whereby the given block is predicted. For each of the second set of intra prediction modes, the linear or affine linear transformation includes P * weight coefficients, at least 1 / 4P of which * are non-zero weight values, and for each of the Q prediction values, a sequence of P weight coefficients associated with that prediction value is formed, and when the sequences are arranged one by one in raster scan order among the samples of the given block, they form an envelope that is non-linear in all directions.

[0052] The encoder may be configured such that, for each of a set of intra prediction modes, a set of P neighboring samples in the neighborhood of a given block is mapped to a set of Q prediction values for the samples of the given block by using a linear or affine linear transformation, whereby the given block is predicted. For each of the second set of intra prediction modes, the linear or affine linear transformation includes P * weight coefficients, at least 1 / 4P of which * are non-zero weight values, and for each of the Q prediction values, a sequence of P weight coefficients associated with that prediction value is formed, and when the sequences are arranged one by one in raster scan order among the samples of the given block, they form an envelope that is non-linear in all directions.

[0053] The encoder may be of a type configured to subdivide a picture into a plurality of blocks of different block sizes including the given block. The encoder is configured to support a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode for each of the block sizes, as well as a second set of intra prediction modes. The encoder When one of the neighboring blocks has the same block size as a predetermined block within the set of block sizes but is assigned to a different one of the first and second sets of intra prediction modes, the first and second mappings are used when sorting the assigned sets. When one of the neighboring blocks has a block size different from that of a predetermined block within the set of block sizes (e.g., they are not the same), when sorting the assigned sets, When the set assigned to the predetermined block is the second set for the block size of the predetermined block and the set assigned to one neighboring block is the first set for the block size of the one neighboring block, a third mapping is used to map each intra prediction mode of the first set of prediction modes for the block size of the one neighboring block to a representative one intra prediction mode within the second set of intra prediction modes for the block size of the predetermined block, and / or When the set assigned to the predetermined block is the first set for the block size of the predetermined block and the set assigned to one neighboring block is the second set for the block size of the one neighboring block, a fourth mapping is used to map each intra prediction mode of the second set of prediction modes for the block size of the one neighboring block to a representative one intra prediction mode within the first set of intra prediction modes for the block size of the predetermined block, and / or If one of the neighboring blocks has the same block size as a predetermined block within the set of block sizes, the set assigned to the predetermined block is the second set for the block size of the predetermined block, and if the set assigned to one neighboring block is the second set for the block size of one neighboring block, configured to use a fifth mapping that maps each intra prediction mode of the second set of prediction modes for the block size of one neighboring block to a representative one intra prediction mode within the second set of intra prediction modes for the block size of the predetermined block.

[0054] The encoder is configured such that the encoder divides a picture into a plurality of blocks of different block sizes including a predetermined block, is configured to support, for each of the set of block sizes, a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, The encoder if the predetermined block is not square, if the predetermined block is oriented in a first direction, apply a linear or affine linear transformation associated with the intra prediction mode pointed to by the index to a series of neighboring samples of the neighborhood of the predetermined block, if the predetermined block is oriented in a second direction orthogonal to the first direction, obtain a prediction block and then apply a linear or affine linear transformation other than the linear or affine linear transformation associated with the intra prediction mode pointed to by the index to an inverted version of the series of neighboring samples of the neighborhood of the predetermined block for the intra prediction mode of the second set of intra prediction modes for which the prediction block is to be transposed so as to be configured to perform prediction of the predetermined block using the intra prediction mode pointed to by the index.

[0055] The encoder is configured such that a second set of intra prediction modes includes first and second subsets of intra prediction modes, a predetermined block is square, and the encoder is configured such that when the intra prediction mode pointed to by the index is included in the first subset, a series of neighboring samples of a neighborhood of the predetermined block is applied to a linear or affine linear transform associated with the intra prediction mode pointed to by the index, when the intra prediction mode pointed to by the index is included in the second subset, a prediction block is obtained, and then a reversed version of the series of neighboring samples of a neighborhood of the predetermined block is applied to a linear or affine linear transform associated with the intra prediction mode of the second subset of intra prediction modes in order to transpose the prediction block so as to be configured to perform prediction of a predetermined block using the intra prediction mode pointed to by the index.

[0056] The encoder may be such that for at least one of the intra prediction modes of the second set, prediction using one intra prediction mode requires prediction of a predetermined block from a spatial region to a transform region or from a transform region to a transform region.

[0057] According to one aspect, a method for encoding a picture on a data stream supporting a first set of intra prediction modes including a plurality of directional prediction modes and at least one of a DC mode and a planar mode, and a second set of intra prediction modes, the method comprising: assigning and encoding a predetermined block to the first set or the second set based on a first signal in the data stream; sorting the assigned set of prediction modes according to the intra prediction modes used for neighboring blocks of a neighborhood of the predetermined block to obtain a list of intra prediction modes; Determining, for a given block, an index into a list of intra prediction modes from a data stream; Predicting the given block using the intra prediction mode pointed to by the index; Encoding the data stream; and including; The encoder is; When the assigned set is the second set of intra prediction modes, when sorting the assigned set, using a first mapping that maps each intra prediction mode of the first set of prediction modes to one representative intra prediction mode within the second set of intra prediction modes, and / or When the assigned set is the first set of intra prediction modes, configured to use a second mapping that maps each intra prediction mode of the second set of prediction modes to one representative intra prediction mode within the first set of intra prediction modes when sorting the assigned set; A method is disclosed.

[0058] According to one aspect, a method for decoding a picture from a data stream, comprising: Subjecting a given block of the picture to a transformation [e.g., FFT, DCT,...] for obtaining a first set of transform coefficients for a first set of neighboring samples in the neighborhood of the given block on a first side of the given block and / or subjecting to a transformation for obtaining a second set of transform coefficients for a second set of neighboring samples in the neighborhood of the given block on a second side of the given block; Subjecting a combination of the first and second sets of transform coefficients, or a combination of the first set of neighboring samples and the second set of transform coefficients, or a combination of the second set of neighboring samples and the first set of transform coefficients to a linear or affine linear transformation to predict; A method is disclosed. Including steps of predicting;

[0059] The decoder is; Divide the picture into a plurality of blocks of different sizes including a predetermined block, It can be configured to arrange the first and second sets of neighborhood samples so as not to depend on the size of the neighborhood block including the first and second sets of neighborhood samples. [It does not necessarily use the entire block, but only uses stripes]

[0060] The decoder It can be configured to arrange the first and second sets of neighborhood samples such that the first and second sets of neighborhood samples are arranged along a one-dimensional path side by side with the first and second sides respectively, and the first and second sets of transform coefficients represent a one-dimensional transform.

[0061] The decoder can be configured to form a combination by taking out a first proper subset of the transform coefficients from the first set of transform coefficients and / or a second proper subset of the transform coefficients from the second set of transform coefficients such that the combination of the first and second sets of transform coefficients does not depend on the unextracted parts of the first and / or second sets of transform coefficients respectively.

[0062] The decoder can be configured such that subjecting the combination of the first and second sets of transform coefficients to a linear or affine-linear transform results in a predictor of a predetermined block in the transform domain.

[0063] The decoder can be configured such that subjecting the combination of the first and second sets of transform coefficients to a linear or affine-linear transform results in a predicted value for a proper subset of the transform coefficients of the transform of a predetermined block [while, for example, the others are set to zero by default].

[0064] The decoder can be configured such that subjecting the combination of the first and second sets of transform coefficients to a linear or affine-linear transform results in a predictor of a predetermined block in the spatial domain.

[0065] According to one aspect, a method for decoding a picture from a data stream, comprising: causing a predetermined block of the picture to undergo a transform [e.g., FFT, DCT...] for obtaining a first set of transform coefficients in a first set of neighboring samples in the neighborhood of the predetermined block on a first side of the predetermined block and / or undergo a transform for obtaining a second set of transform coefficients in a second set of neighboring samples in the neighborhood of the predetermined block on a second side of the predetermined block, and predicting by subjecting a combination of the first and second sets of transform coefficients, or a combination of the first set of neighboring samples and the second set of transform coefficients, or a combination of the second set of neighboring samples and the first set of transform coefficients, to a linear or affine-linear transform is disclosed. [e.g., transforming each of these parts from the spatial domain to the transform domain and only then applying a (e.g., linear or affine) transform].

[0066] According to one aspect, an encoder for encoding a picture on a data stream, comprising: causing a predetermined block of the picture to undergo a transform [e.g., FFT, DCT...] for obtaining a first set of transform coefficients in a first set of neighboring samples in the neighborhood of the predetermined block on a first side of the predetermined block and / or undergo a transform for obtaining a second set of transform coefficients in a second set of neighboring samples in the neighborhood of the predetermined block on a second side of the predetermined block, and being configured to predict by subjecting a combination of the first and second sets of transform coefficients, or a combination of the first set of neighboring samples and the second set of transform coefficients, or a combination of the second set of neighboring samples and the first set of transform coefficients, to a linear or affine-linear transform is disclosed. [For example, each of these parts is converted from a spatial domain to a transform domain, and only then is a (e.g., linear or affine) transform applied.]

[0067] The encoder subdivides the picture into a plurality of blocks of different sizes including a predetermined block, and may be configured to arrange the first and second sets of neighborhood samples so as not to depend on the size of the neighborhood block including the first and second sets of neighborhood samples.

[0068] The encoder may be configured to arrange the first and second sets of neighborhood samples such that the first and second sets of neighborhood samples are respectively arranged along a one-dimensional path alongside the first and second sides, and the first and second sets of transform coefficients represent a one-dimensional transform.

[0069] The encoder may be configured to form a combination by taking a first true subset of the transform coefficients from the first set of transform coefficients and / or a second true subset of the transform coefficients from the second set of transform coefficients such that the combination of the first and second sets of transform coefficients does not depend on the non-extracted parts of the first and / or second sets of transform coefficients.

[0070] The encoder may be configured such that subjecting the combination of the first and second sets of transform coefficients to a linear or affine linear transform results in a predictor of a predetermined block within the transform domain.

[0071] The encoder may be configured such that subjecting the combination of the first and second sets of transform coefficients to a linear or affine linear transform results in a predicted value for a true subset of the transform coefficients of the transform of a predetermined block.

[0072] The encoder can be configured to subject a combination of first and second sets of transformation coefficients to a linear or affine-linear transformation so as to produce a predictor of a given block within a spatial region.

[0073] According to one aspect, a method for encoding a picture from a data stream, wherein a given block of the picture is subjected to a transformation [e.g., FFT, DCT...] for obtaining a first set of transformation coefficients for a first set of neighboring samples in the neighborhood of the given block at a first side of the given block and / or subjected to a transformation for obtaining a second set of transformation coefficients for a second set of neighboring samples in the neighborhood of the given block at a second side of the given block, and the method includes predicting by subjecting a combination of the first and second sets of transformation coefficients, or a combination of the first set of neighboring samples and the second set of transformation coefficients, or a combination of the second set of neighboring samples and the first set of transformation coefficients, to a linear or affine-linear transformation is disclosed. [e.g., by transforming each of these parts from the spatial domain to the transform domain and only then applying the (e.g., linear or affine) transformation]. BRIEF DESCRIPTION OF THE DRAWINGS

[0074]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10a

Figure 10b

Figure 10c

Figure 10d

Figure 10e

Figure 11a

Figure 11b

Figure 11c

Figure 11d

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0075] [Encoder and Decoder by Example] Hereinafter, various examples that help achieve more effective compression when using intra prediction will be described. Some examples achieve an increase in compression efficiency by expending a set of intra prediction modes. The latter may be added to, for example, other intra prediction modes designed heuristically, or may be provided exclusively. Also, some other examples utilize both of the features just described.

[0076] To facilitate the understanding of the following examples of the present application, the description begins with presenting possible encoders and decoders that are compatible with and can construct the examples outlined hereinafter in the present application.

[0077] FIG. 1 shows an apparatus for encoding picture 10 into data stream 12 (also shown as 80 in some examples, see below) in block units. The apparatus is shown using reference numeral 14 and can be a still image encoder or a video encoder. In other words, picture 10 can be the current picture from video 16 when encoder 14 is configured to encode video 16 including picture 10 into data stream 12, or encoder 14 can encode picture 10 into data stream 12 exclusively. The following examples can utilize apparatus 14.

[0078] As described above, encoder 14 performs encoding in a block unit method or in a block-based manner. For this, encoder 14 subdivides picture 10 into blocks and, in those units, encoder 14 encodes picture 10 into data stream 12. Possible examples of subdividing picture 10 into blocks 18 are described in detail below. Generally, the subdivision will ultimately result in blocks 18 of a certain size, such as an array of blocks arranged in rows and columns, or will start with a hierarchical multi-tree subdivision from the entire picture area of picture 10 or from a pre-partitioning of picture 10 into an array of tree blocks, resulting in blocks 18 of different block sizes, but these examples should not be treated as excluding other possible ways of subdividing picture 10 into blocks 18.

[0079] Furthermore, encoder 14 is a predictive encoder configured to predictively encode picture 10 into data stream 12. For a certain block 18, this means that encoder 14 determines a prediction signal for block 18 and encodes the prediction residual, i.e., the prediction error by which the prediction signal deviates from the actual picture content within block 18, into data stream 12.

[0080] Encoder 14 may support various prediction modes to derive a prediction signal for a certain block 18. An important prediction mode in the following example is the intra prediction mode, according to which the inside of block 18 is spatially predicted from already encoded samples in the vicinity of picture 10. The encoding of picture 10 into data stream 12 and, correspondingly, the corresponding decoding procedure may be based on a certain coding order 20 defined among blocks 18. For example, coding order 20 may traverse block 18 in a raster scan order, such as traversing each row from left to right and then proceeding row by row from top to bottom. In the case of hierarchical multi-tree based subdivision, a raster scan order may be applied within each hierarchical level, and at this time a depth-first traversal order may be applied, i.e., leaf notes within a certain hierarchical level may precede blocks of the same hierarchical level having the same parent block according to coding order 20. Depending on coding order 20, the already encoded samples in the vicinity of block 18 may usually be arranged on one or more sides of block 18. In the case of the example presented in this specification, for example, the already encoded samples in the vicinity of block 18 are arranged above and to the left of block 18.

[0081] The intra prediction mode does not have to be the only supported prediction mode by the encoder 14. For example, when the encoder 14 is a video encoder, the encoder 14 may also support an intra prediction mode in which the block 18 is temporarily predicted from a picture encoded before the video 16. Such an intra prediction mode may be a motion compensated prediction mode, and according to this prediction mode, a motion vector indicating a relative spatial offset of a portion from which the prediction signal of the block 18 is derived as a copy is signaled for such a block 18. Additionally or alternatively, other non-intra prediction modes may be similarly available, such as an inter-view prediction mode when the encoder 14 is a multi-view encoder, or a non-prediction mode in which the inside of the block 18 is coded as it is, i.e., without any prediction.

[0082] Before directing the focus of the description of the present application to the intra prediction mode, a more detailed example of a possible block-based encoder, i.e., a possible implementation form of the encoder 14 as described with respect to FIG. 2, will be described next by presenting two corresponding examples of decoders that conform to FIGS. 1 and 2, respectively.

[0083] Figure 2 shows a possible implementation of the encoder 14 of FIG. 1, i.e., an encoder configured to use transform coding to encode prediction residuals, which is merely an example, and the present application is not limited to the type of prediction residual coding. According to FIG. 2, the encoder 14 (which may be used for the following example) then obtains a prediction residual signal 26 to be encoded into the data stream 12 by the prediction residual encoder 28, and may include a subtractor 22 configured to subtract the corresponding prediction signal 24 from the incoming signal, i.e., the picture 10, or, in a block-based manner, from the current block 18. The prediction residual encoder 28 consists of a lossy coding stage 28a and a lossless coding stage 28b. The lossy stage 28a receives the prediction residual signal 26 and includes a quantizer 30 that quantizes the samples of the prediction residual signal 26. As already described above, this example uses transform coding of the prediction residual signal 26, and thus, in the lossy coding stage 28a, quantization by the quantizer 30 occurs for the transformed coefficients presenting the residual signal 26, and includes a transform stage 32 connected between the subtractor 22 and the quantizer 30 to transform the prediction residual 26 decomposed in such a spectrum. The transform may be a DCT, DST, FFT, Hadamard transform, etc. The transformed and quantized prediction residual signal 34 then undergoes lossless coding by the lossless coding stage 28b, and the lossless coding stage 28b is an entropy encoder that entropy-codes the quantized prediction residual signal 34 into the data stream 12. The encoder 14 further includes a prediction residual signal reconstruction stage 36 connected to the output of the quantizer 30 to reconstruct the prediction residual signal from the transformed and quantized prediction residual signal 34 in a manner available also in the decoder, i.e., taking into account the coding loss in the quantizer 30. For this purpose, the prediction residual signal reconstruction stage 36 includes an inverse quantizer 38 that performs the inverse of the quantization of the quantizer 30, followed by an inverse transform device 40 that performs an inverse transform for the transform performed by the transform device 32, such as an inverse of any of the specific transform examples described above, or an inverse of the spectral decomposition.The encoder 14 includes an adder 42 that adds the predicted residual signal reconstructed as an output by the inverse converter 40 and the prediction signal 24 in order to output a reconstructed signal, that is, a reconstructed sample. This output is supplied to the predictor 44 of the encoder 14, and the predictor 44 then determines the prediction signal 24 based on its output. It is the predictor 44 that supports all the prediction modes already discussed above with respect to FIG. 1. FIG. 2 also shows that when the encoder 14 is a video encoder, the encoder 14 may include an in-loop filter 46 that filters the fully reconstructed picture, and this fully reconstructed picture forms a reference picture for the predictor 44 for the inter prediction block after being filtered.

[0084] As already described above, the encoder 14 operates in a block-based manner. For the following description, the block-based of interest is the one where the intra prediction mode is selected from a plurality of intra prediction modes or a set of intra prediction modes supported by the predictor 44 or the encoder 14 respectively, and the picture 10 is subdivided into blocks where the selected intra prediction mode is individually performed. However, other types of blocks into which the picture 10 can be subdivided may equally exist. For example, the above determination as to whether the picture 10 is inter-coded or intra-coded can be made at the granularity or unit of blocks deviating from the block 18. For example, the inter / intra mode determination can be made at the level of coding blocks where the picture 10 is subdivided and each coding block is further subdivided into prediction blocks. Prediction blocks associated with coding blocks for which intra prediction is determined to be used are each subdivided for intra prediction mode determination. For this purpose, for each of these prediction blocks, it is determined which supported intra prediction mode should be used for each prediction block. These prediction blocks will form the block 18 of interest here. Prediction blocks within coding blocks associated with inter prediction will be treated differently by the predictor 44. These prediction blocks will be inter-predicted from a reference picture by determining a motion vector and replicating a prediction signal for this block from the location within the reference picture pointed to by the motion vector. Another block subdivision relates to subdividing into transform blocks at the unit where the transformation by the transformer 32 and the inverse transformer 40 is performed. The transformed blocks can be, for example, the result of further subdivision of coding blocks. Of course, the examples described herein should not be treated as limiting, and other examples equally exist. For the sake of completeness only, the subdivision into coding blocks may, for example, use multi-tree subdivision, and it should be noted that prediction blocks and / or transform blocks can equally be obtained by further subdividing coding blocks using multi-tree subdivision.

[0085] A decoder or apparatus for block-based decoding that conforms to the encoder 14 of FIG. 1 (for example, as used in these examples) is shown in FIG. 3. This decoder 54 performs the reverse of the encoder 14, that is, the decoder 54 decodes the picture 10 from the data stream 12 in a block-based manner and, for this purpose, supports a plurality of intra prediction modes. The decoder 54 may comprise, for example, a residual provider 156. All the other possibilities discussed above with respect to FIG. 1 are also valid for the decoder 54. For this purpose, the decoder 54 may be a still image decoder or a video decoder, and all prediction modes and predictabilities are similarly supported by the decoder 54. The difference between the encoder 14 and the decoder 54 is mainly that the encoder 14 selects or chooses coding decisions according to some optimization, for example, to minimize some cost functions that may depend on the coding rate and / or coding distortion. One of these coding options or coding parameters may require the selection of the intra prediction mode to be used for the current block 18 among the available or supported intra prediction modes. The selected intra prediction mode may then be signaled for the current block 18 in the data stream 12 by the encoder 14, and the decoder 54 uses this signalization in the data stream 12 for the block 18 to make this selection again. Similarly, the subdivision of the picture 10 into blocks 18 may be optimized in the encoder 14, and the corresponding subdivision information is transmitted in the data stream, and the decoder 54 may recover the subdivision of the picture 10 into blocks 18 based on the subdivision information. Summarizing the above, the decoder 54 may be a prediction decoder operating on a block basis, and in addition to the intra prediction mode, the decoder 54 may support other prediction modes, for example, the inter prediction mode if the decoder 54 is a video decoder.When decoding, decoder 54 can also use the coding order 20 discussed with respect to FIG. 1. Since this coding order 20 is adhered to in both the encoder 14 and the decoder 54, the same neighboring samples are available for the current block 18 in both the encoder 14 and the decoder 54. Therefore, to avoid unnecessary repetition, the description of the operation mode of the encoder 14 applies to the decoder 54 as well, as far as the subdivision of the blocks of picture 10 is concerned, for example, as far as prediction is concerned, and as far as the coding of the prediction residue is concerned. The difference is that the encoder 14, by optimization, selects some coding option or coding parameter, signals the coding parameter within the data stream 12 or inserts it therein, and the coding parameter is then derived from the data stream 12 by the decoder 54 to perform prediction, subdivision, etc. again.

[0086] FIG. 4 shows one possible implementation of the decoder 54 of FIG. 3 (e.g., as used in these examples), i.e., one that is adapted to the implementation of the encoder 14 of FIG. 1 as shown in FIG. 2. Since many elements of the decoder 54 of FIG. 4 are the same as the elements that occur in the corresponding encoder of FIG. 2, the same reference numerals given with apostrophes are used in FIG. 4 to denote these elements. Specifically, the adder 42', the optional in-loop filter 46', and the predictor 44' are connected within the prediction loop in the same way as in the encoder of FIG. 2. The reconstructed, i.e., inverse quantized and inverse transformed prediction residual signal applied to the adder 42' is derived by a sequence of the entropy decoder 56 that inverts the entropy encoding of the entropy encoder 28b, as in the case of the encoding side, followed by a residual signal reconstruction stage 36' consisting of an inverse quantizer 38' and an inverse transformer 40'. The output of the decoder is the reconstruction of picture 10. The reconstruction of picture 10 may be directly available at the output of the adder 42' or at the output of the in-loop filter 46'. Some post-filters may be arranged at the output of the decoder to subject the reconstruction of picture 10 to some post-filtering to improve the picture quality, but this option is not shown in FIG. 4.

[0087] Again, with respect to FIG. 4, the explanations presented above with respect to FIG. 2 are equally valid for FIG. 4, except that the encoder makes optimization tasks and associated decisions regarding coding options. However, all explanations regarding block subdivision, prediction, inverse quantization, and inverse transformation are also valid for the decoder 54 of FIG. 4.

[0088] [Example] In some of the above and below examples, the encoder and / or decoder may be for reconstructing a given block (18) by correcting each of a set of Q predicted values by a corresponding residual value to obtain a corresponding reconstructed value such that (except for clipping which may be added optionally after prediction correction) the corresponding reconstructed value (24') depends linearly completely on P neighboring samples (templates).

[0089] In some cases, it is possible to refer to a "set of block sizes" including various sizes that can be used. For example, the size MxN is a different size from MxN1 if N≠N1. Some modes target only a specific block size (which is one of the block sizes of the set of block sizes).

[0090] Furthermore, reference is made to a "first set 121 of legacy modes" including legacy modes. Also, reference is made to a "second set 122 of ALWIP modes" including ALWIP modes (examples of ALWIP are provided below).

[0091] This specification relates, inter alia, to an improved intra prediction mode concept for block-based picture coding that can be used in a video codec, such as HEVC or some successor technology to HEVC.

[0092] Intra prediction modes are widely used in picture coding and video coding. In video coding, intra prediction modes compete with other prediction modes such as inter prediction modes like motion compensation prediction mode. In an intra prediction mode, a current block is predicted based on neighboring samples, i.e., samples that have already been encoded as far as the encoder side is concerned and already decoded as far as the decoder side is concerned. The neighboring sample values are extrapolated to the current block to form a prediction signal for the current block, and the prediction residual is transmitted within the data stream for the current block. The better the prediction signal is, the lower the prediction residual will be, and thus the fewer bits required to code the prediction residual.

[0093] To be effective, several aspects should be considered to form an effective framework for intra prediction in a block-based picture coding environment. For example, the more intra prediction modes supported by a codec, the higher the side information rate consumption for signaling the selection to the decoder. On the other hand, the set of supported intra prediction modes should be able to provide a good prediction signal, i.e., a prediction signal that results in a low prediction residual.

[0094] This specification is intended to provide, among other things, an intra prediction mode concept that enables more effective compression of block-based picture codecs if the concept of improved intra prediction modes is used.

[0095] This object is achieved by the subject matter of the independent claims of this application.

[0096] An apparatus and method for decoding (or encoding) a picture (e.g., 10) in block units from a data stream (e.g., 12, 80) are disclosed. The apparatus and method support at least one intra prediction mode in which an intra prediction signal for a block of a predetermined size of the picture is determined accordingly by applying a first template of neighboring samples of the current block to an affine linear predictor, ultimately called an Affine Linear Weighted Intra Predictor (ALWIP).

[0097] This apparatus and method may have at least one of the characteristics discussed below.

[0098] [Examples of predictors complementary to other predictions] The intra prediction modes supported by the present apparatus and method may, in some examples, complement other intra prediction modes of the codec. These may complement the DC prediction mode, Planar prediction mode, or angular prediction mode defined in the HEVC codec resp. JEM reference software. The latter three types of intra prediction modes are here referred to as "conventional intra prediction modes". Thus, for a given block of the intra mode, a flag (e.g., encoded in a field later denoted as "81") indicating whether one of the intra prediction modes supported by the apparatus or method will be used can be parsed by the decoder.

[0099] [Two or more proposed prediction modes] The apparatus and method may include, for example, two or more ALWIP modes stored in a memory unit (alternatively, in some cases, these may be obtained on-the-fly). Thus, if the decoder knows that one of the ALWIP modes supported by the encoder device is to be used, the decoder may parse additional information (e.g., encoded in a field later denoted as "82") that may indicate which of the ALWIP modes supported by the apparatus of the method is to be used.

[0100] The signalization of the supported modes may have the property that the coding of some ALWIP modes may require fewer bins than other ALWIP modes. Which of these modes require fewer bins and which modes require more bins may both depend on information extractable from the already decoded bitstream 12 (or 80), or may be predefined.

[0101] [Sharing predictors between different block sizes using downsampling / upsampling] Some of the examples discussed here may be illustrated specifically by considering the example of FIG. 13 (see also the following discussion).

[0102] In some cases, an encoder or a decoder can perform a conversion between predictors of different block sizes, for example, by downsampling and / or upsampling. This can occur when the ALWIP mode is provided for a specific size (e.g., MxN), but the block to be predicted (e.g., 18 or B1) is predicted to have dimensions M1xN1 different from MxN (e.g., at least one of M and N becomes such that M≠M1 and / or N≠N1). Hereinafter, the "second template 1160" may refer to a group of already predicted neighboring samples (also denoted as 17'a, 17'b, 17'c, see below) used to perform intra prediction related to block sizes for which the encoder or decoder cannot freely use the ALWIP mode. The "first template 1161" may refer to a template having dimensions required for prediction related to block sizes for which the encoder or decoder can actually freely use the ALWIP mode. Here, an example will be discussed where it is allowed to "jump" from the second template 1160 to the first template 1161, then perform prediction using the first template 1161, and finally return to the original block size after predicting block 18 (B1).

[0103] Referring to FIG. 13, for a current block 18 (B1) that is different from a predetermined size provided to the decoder (or encoder) in the ALWIP mode, the apparatus and method resample a second template (1160) of neighboring samples (17'a, 17'b, 17'c) of the current block (B1, 18) to match a first template (1161) in order to obtain a resampled template (1136), apply the resampled template (1161) to an ALWIP predictor (1138a) to obtain a preliminary intra prediction, resample the preliminary intra prediction signal to match the current block (1140) to obtain an intra prediction signal for the current block (18). may be configured as follows.

[0104] The apparatus may be configured to resample a second template (1160) by downsampling (D) to obtain a first template (1161).

[0105] The apparatus may be configured to resample a preliminary intra prediction signal by upsampling the preliminary intra prediction signal. The apparatus may be configured to convert the preliminary intra prediction signal from the spatial domain to the transform domain and resample the preliminary intra prediction signal within the transform domain. The apparatus may be configured to resample the preliminary intra prediction signal in the transform domain by scaling the coefficients of the preliminary intra prediction signal.

[0106] In some examples, the apparatus or method increases the size of the intra prediction signal to match the size of the current block, zero-pads the coefficients of the additional coefficients of the preliminary intra prediction signal, where the additional coefficients are related to high-frequency bins, thereby resampling the preliminary intra prediction signal in the transform domain, may be configured as follows.

[0107] The apparatus may be configured to create a preliminary intra prediction signal in the transform domain with an inverse quantized version of the prediction residual signal. The apparatus may be configured to resample the preliminary intra prediction signal within the spatial domain.

[0108] The apparatus may be configured to resample the preliminary intra prediction signal by performing bilinear interpolation. The apparatus may be configured to encode information regarding the use of resampling and / or affine linear prediction for different sizes within the data field.

[0109] In some cases, it is possible to use a mapping that maps the modes required for the M1xN1 block 18 (B1) to mode 1138a.

[0110] Although this example describes the ALWIP mode, it may be implemented for the legacy mode or for other types of modes.

[0111] [Prediction for the Conversion Region] As will be apparent from the following text (see also FIG. 12), it is possible to perform intra prediction within the spatial region and / or within the conversion region. There are several considerations by the encoder device and / or decoder device within the conversion region as follows.

[0112] An apparatus for decoding a picture in block units from a data stream, by applying a first set of neighboring samples of the current block to ALWIP to obtain a prediction of a set of transform coefficients for the transformation of the current block, supporting at least one intra prediction mode in which an intra prediction signal for the current block of the picture is determined accordingly is also disclosed.

[0113] One of the apparatuses may be configured to inverse-transform the prediction to obtain the reconstructed signal. One of the apparatuses may be configured to decode an index from the data stream using a variable length code and make a selection using the index. One of the apparatuses may be configured to determine a ranking of a set of intra prediction modes and then resample a second template.

[0114] A method comprising: resampling a second template of neighboring samples of the current block to match a first template to obtain a resampled template; To obtain a preliminary intra prediction signal, applying a resampled template of the samples to ALWIP; To obtain an intra prediction signal for the current block, resampling the preliminary intra prediction signal to match the current block; A method including the above is disclosed.

[0115] A method for decoding pictures in blocks from a data stream, Applying a first set of neighboring samples of the current block to ALWIP to obtain a prediction of a set of transform coefficients of the transform of the current block; A method including the above is disclosed.

[0116] The above and / or the following methods may use an apparatus comprising at least one of the above and / or the following.

[0117] When the proposed predictor predicts transform coefficients, it may be inferred that the non-predicted transform coefficients are zero. Which transform coefficients will be predicted may depend only on a given mode, not on the input (e.g., not on neighboring blocks). Additionally, for a given transform, it may be pre-determined that all high-frequency components of the prediction signal starting from a certain point are inferred to be zero.

[0118] In an example, a mapping may be defined that maps a mode to other modes that are the objects for which resampling must be performed.

[0119] The above examples mainly discuss the ALWIP mode, but these examples may be generalized to conventional modes and other types of modes.

[0120] [Prediction from the transform domain] Some prediction modes of the present application can be configured to first apply a transform having energy compression characteristics to natural images (e.g., discrete cosine transform (DCT) or wavelet transform), and use only some of the resulting transform coefficients as input to the affine linear prediction supported by the device. The coefficients to be used may be predetermined (e.g., only low-frequency coefficients) or derived from the transform signal itself (e.g., only frequency coefficients having the maximum amplitude).

[0121] If it is predetermined which frequency coefficients will be used, only a partial transform, e.g., a discrete cosine transform that calculates only low-frequency coefficients or one or more stages of a low-pass filter corresponding to a given wavelet transform, can be applied to the input signal.

[0122] [Transposition of the proposed prediction mode] For the examples discussed here, it will be described in detail later with particular reference to FIGS. 11a and 11b.

[0123] For a given block 18 of N rows and M columns, if the ALWIP mode is already supported by the device for the block and for a block of M rows and N columns, the device can be configured to support a new prediction mode for the block of M rows and N columns as follows. First, map a template consisting of neighboring already reconstructed samples for the block of M rows and N columns to a template that functions as input for affine linear intra prediction for the block of N rows and M columns. Second, apply the ALWIP mode to the block of N rows and M columns. Third, transpose the result of the latter prediction so that it becomes the prediction signal for the block of M rows and N columns. Note that M and N may be equal.

[0124] Here, to further explain the first step, for example, if a template consisting of already reconstructed samples in the neighborhood for a block of M rows and N columns consists of k rows above the block and l columns to the left of the block, and a template that functions as an input for affine linear prediction for a block containing N rows and M columns consists of l rows above the block and k columns to the left of the block, then the p-th row above the block of M rows and N columns can be mapped to the p-th column to the left of the block of N rows and M columns, and the q-th column to the left of the block of M rows and N columns can be mapped to the q-th row above the block of N rows and M columns.

[0125] For example, it is possible to perform a mapping operation from a mode applicable to an MxN block to an associated mode applicable to an NxM block.

[0126] The above examples mainly refer to the ALWIP mode, but these can also be effective for conventional modes and / or other modes.

[0127] [Mapping between mode indices] For a given block shape and for the ALWIP intra prediction for that block shape which is part of the proposed apparatus and method, there may exist a mapping that maps each ALWIP mode to a conventional intra prediction mode (DC, Planar, or Angular) existing within the codec on which it is based (in the example, vice versa).

[0128] This mapping can be used in the signalization of legacy intra prediction modes that exist within the underlying codec. That is, in a decoder, if a list is generated that ranks between legacy intra prediction modes, and this list is used in their signalization, and the generation of the latter list is realized according to defined rules using the legacy intra prediction modes of surrounding already reconstructed blocks, then this rule first applies the mapping to each of these modes and then, when generating the list, treats them as legacy intra prediction modes, so as to also include surrounding already reconstructed blocks that use one of the proposed ALWIP modes within the underlying device.

[0129] Moreover, this mapping can be used as follows. When a luma prediction signal is generated using one of the ALWIP modes for its block shape, the chroma prediction signal can be obtained by using, via the mapping, the legacy intra prediction mode corresponding to the ALWIP.

[0130] For a given block shape that is part of the proposed device and a given ALWIP mode for that block shape, there may exist a mapping that maps each of the legacy intra prediction modes that exist within the underlying codec to the ALWIP mode for that block shape that is part of the proposed device.

[0131] For a given first block shape that is part of the proposed device and a given ALWIP intra prediction mode for the first block shape, and for a given second block shape that is part of the proposed device and a given ALWIP intra prediction mode for the second block shape, there may exist a mapping that maps each ALWIP mode for the first block shape to the ALWIP for the second block shape.

[0132] This mapping can be achieved by first applying the mapping described in the above paragraph of this section and then applying the mapping of the second paragraph of this section to the result (or vice versa). Both of the last two mappings just described may be either in a conventional intra prediction mode or an ALWIP mode, and can be used to generate rankings of all available ALWIP modes for a given block using the intra prediction mode of neighboring blocks. First, the mapping is applied to each of the neighboring intra prediction modes that give a set of ALWIP modes corresponding to the shape of the given block. Then, all possible ALWIP modes are ranked according to a predefined rule, and the ALWIP modes occurring within the set corresponding to the surrounding blocks can be ranked differently from other ALWIP modes.

[0133] The latter ranking can be used during the signalization of the ALWIP mode by coding the ALWIP mode using a different number of bins corresponding to that ranking.

[0134] [Explanation of Example Aspects Regarding ALWIP Conversion] FIG. 2 shows a decoder 54 for decoding a picture from a data stream 12. The decoder 54 can be configured to decode a predetermined block 18 of a picture. Specifically, the predictor 44 can be configured to map a set of P neighboring samples in the neighborhood of the predetermined block 18 to a set of Q predicted values for the samples of the predetermined block using a linear or affine linear transformation [e.g., ALWIP].

[0135] As shown in FIG. 5, the predetermined block 18 includes Q values (which will become "predicted values" at the end of the operation) to be predicted. If the block 18 has M rows and N columns, the number of values to be predicted is Q = M *There will be N values. The Q values of block 18 may be in a spatial region (e.g., a pixel) or in a transform region (e.g., DCT, etc.). The Q values of block 18 can be predicted based on the P values taken from the neighboring blocks 17a - 17c adjacent to block 18. The P values of the neighboring blocks 17a - 17c can be at the positions closest to block 18 (e.g., adjacent). The P values of the neighboring blocks 17a - 17c have already been processed and predicted. The P values are shown as the values in the parts 17'a - 17'c (forming a so - called "template") so as to be distinguishable from the blocks of which they are a part (in some examples, 17'b is not used).

[0136] As shown in FIG. 6, for prediction, a first vector 17P having P components (each component is a specific position within the neighboring parts 17'a - 17'c), a second vector 18Q having Q components (each component is a specific position within block 18), and a mapping matrix 17M (each row is associated with a specific position within block 18 and each column is associated with a specific position within the neighboring parts 17'a - 17'c) can be used for calculation. The mapping matrix 17M thus performs the prediction of the P values of the neighboring parts 17'a - 17'c (template) to the values of block 18 according to a predetermined mode. The components of the mapping matrix 17M are thus weight coefficients. The matrix 17M (which can be associated with a conventional mode or an ALWIP mode) is predefined and can be stored in a storage unit (e.g., a register, a memory, etc.) of the decoder and / or encoder, or can be obtained on - the - fly. The matrix 17M (and also the associated mode) is generally associated with a specific size. For example, the size MxN is generally not M1xN1 when (M≠M1 and / or N≠N1), and the matrix 17M associated with the size MxN is generally different from the matrix 17M associated with the size M1xN1 for a mode associated with the size MxN. In the case of a mode associated with the size MxN, the matrix 17M will have Q rows, where Q = M* It is N. In the case of a mode associated with size M1xN1, matrix 17M will have Q1 rows, where Q1 = M1 * It is N1.

[0137] In the relevant art, there are several known conventional modes, such as the DC mode, the planar mode, and 65 directional prediction modes. For example, there may be 67 known modes. For discussions regarding conventional modes, see the following.

[0138] However, it has been found that it is also possible to use different modes (in addition to conventional modes). The different additional modes presented here are herein referred to as linear or affine linear transformations. The linear or affine linear transformation (associated with matrix 17M) has P * including Q weight coefficients, at least 1 / 4P of which * The Q weight coefficients are non-zero weight values, and for each of the Q predicted values, a series of P weight coefficients associated with that respective predicted value is formed. The series forms an envelope that is non-linear in all directions when arranged sequentially one by one downward according to the raster scan order among the samples of a given block 18.

[0139] FIG. 7 shows P positions of neighboring values 17'a to 17'c (templates), Q positions of blocks 17'a to 17'c, and P of matrix 17M *An example of a chart 70 that maps the values of Q weight coefficients is shown. Plane 72 is an envelope of a series for DC conversion (a plane for DC conversion). This envelope is clearly a plane and is thus excluded by the definition of linear or affine-linear transformation (ALWIP). The planar mode and 65 directional prediction modes will have different envelopes, but these will be linear in all directions. In contrast, the envelope of a linear or affine transformation will not be linear in all directions. It is understood that such types of transformations may be optimal for making predictions about block 18 depending on the situation. At least 1 / 4 of the weight coefficients are different from zero (i.e., P * It has been found that it is preferable that at least 25% of the Q weight coefficients are different from 0). The weight coefficients can be independent of each other according to any normal mapping rule. Thus, matrix 17M can be such that the values of its components do not have an obvious recognizable relationship.

[0140] In an example, the ALWIP transformation is such that the maximum average of the cross-correlation between a first series of weight coefficients related to each predicted value and the higher of a second series of weight coefficients related to predicted values other than each predicted value or the inverted version of the latter series is lower than a predetermined threshold value (e.g., a threshold value within the range between 0.2 or 0.3 or 0.35 or 0.1, e.g., 0.05 to 0.035).

[0141] The P neighboring samples (17'a to 17'c) of blocks 17a to 17c can be arranged along a one-dimensional path extending along the boundary of a predetermined block 18 (e.g., 18c, 18a). For each of the Q predicted values of a predetermined block 18, the series of P weight coefficients related to each predicted value may be ordered to traverse the one-dimensional path in a predetermined direction (e.g., from left to right, from top to bottom, etc.).

[0142] In an example, the ALWIP matrix 17M can be non-diagonal or non-block diagonal.

[0143] An example of the ALWIP matrix 17M for predicting the 4x4 block 18 from 4 already predicted neighboring samples can be as follows: { {37, 59, 77, 28}, {32, 92, 85, 25}, {31, 69, 100, 24}, {33, 36, 106, 29}, {24, 49, 104, 48}, {24, 21, 94, 59}, {29, 0, 80, 72}, {35, 2, 66, 84}, {32, 13, 35, 99}, {39, 11, 34, 103}, {45, 21, 34, 106}, {51, 24, 40, 105}, {50, 28, 43, 101}, {56, 32, 49, 101}, {61, 31, 53, 102}, {61, 32, 54, 100} }。 (Here, {37, 59, 77, 28} is the first row, {32, 92, 85, 25} is the second row, and {61, 32, 54, 100} is the 16th row of the matrix 17M.) The matrix 17M has dimensions 16x4 and (as a result of 16 * 4 = 64) contains 64 weight coefficients. This is because the matrix 17M has dimensions QxP, where Q = M * N, which is the number of samples of the block 18 to be predicted (the block 18 is a 4x4 block), and P is the number of samples of the already predicted samples (e.g., 17'a to 17'c). Here, M = 4, N = 4, (as a result of M * N = 4 * 4 = 16) Q = 16, P = 4. The matrix is non-diagonal and non-block diagonal and is not described by a specific rule.

[0144] As can be seen, less than 1 / 4 of the weight coefficients are 0 (in this case, 1 out of 64 weight coefficients is zero). The envelopes formed by these values form an envelope that is non-linear in all directions when arranged one by one downward in raster scan order.

[0145] The above description pertains to the decoder, but the same can be done in the encoder (e.g., encoder 14).

[0146] In some examples, for each block size (within the set of block sizes), the ALWIP transformation of the intra prediction modes within the second set 122 of intra prediction modes for each block size is different from each other. Additionally or alternatively, the density of the second set 122 of intra prediction modes for the block sizes within the set of block sizes may match, but the associated linear or affine-linear transformations of the intra prediction modes within the second set of intra prediction modes for different block sizes may not be convertible with respect to each other by scaling.

[0147] In some examples, the ALWIP transformation can be defined such that those transformations have "nothing in common" with conventional transformations (e.g., the ALWIP transformation is mapped by one of the above mappings, but the ALWIP transformation may not have "anything in common" with the corresponding conventional transformation).

[0148] In examples, the ALWIP mode is used for the luma component, but the ALWIP mode can be avoided for the chroma component.

[0149] Hereafter, a first set 121 of conventional intra prediction modes including a plurality of directional prediction modes and at least one of the DC mode and the planar mode (the set can have, for example, 67 modes); and a second set 122 of intra prediction modes (e.g., the linear or affine prediction mode (ALWIP) discussed above) will be mainly referred to.

[0150] [Signaling, Mapping] Here, for example, it will be described how to reduce the size of a bitstream (e.g., 12, 80) that encodes and signals a prediction mode to be selected using, for example, a unary variable length code or another variable length code.

[0151] This selection may be made by an encoder that encodes a variable length code of type "000...1" (which may be unary) within the bitstream, where "1" is located after a sequence of "0". (More generally, even if not necessarily unary, a less expensive code is given by more frequently occurring intra prediction modes.) The shortest field may thus be "1" (indicating the first prediction mode), the second shortest field may be "01" (indicating the second prediction mode), and so on (a string containing 66 "0"s and 1 "1" within the 66th position may indicate, for example, the last prediction mode of 67 prediction modes). Since this code can be signaled for each block of a large number of blocks to be predicted, it is preferable to avoid fields having dozens of "0"s while the field is short for each block (i.e., the mode to be used is indicated by a short string such as "1", "01", "001", etc.). Thus, strategies based on ranking between modes have been developed. According to such a strategy, the length of the code monotonically depends on the rank of the intra prediction mode within the list of intra prediction modes in order to recognize an index indicating a specific prediction mode within the list. The list can be understood as a list of the most accurate modes, and although the list is not directly signaled by the encoder to the decoder, it can be interpreted by a rule common to the encoder and the decoder.

[0152] FIG. 8 shows a data (video) stream 80 (e.g., 12) that encodes an image (e.g., 10 of FIGS. 1-4). A portion of the data stream 80 may refer to a predetermined block 18 that is to be decoded.

[0153] The data stream 80 may include a first signalization 81 for assigning the block 18 to a first set 121 or a second set 122 of intra prediction modes. The signal 81 may require one single bit in some examples.

[0154] A second signalization 82 (variable length code) within the data stream 80 may include a unary variable length code of type "000...1" or another variable length code that can assign less expensive codes (e.g., codes that require fewer bits) to more frequently occurring intra prediction modes, as discussed above.

[0155] When the decoder 54 reads the first signal 81, the decoder 54 understands that the block 18 is to be predicted in either the first set of intra prediction modes or the second set of intra prediction modes (i.e., through the flag 81, the block 18 is assigned to either the first set 121 of conventional modes or the second set 122 of ALWIP modes).

[0156] In that case, in some examples, the decoder 54 may sort the assigned set of prediction modes (indicated within the first signal 81) according to an intra prediction mode (such as that previously used for neighboring blocks 17a-17c). Thus, a list 90 of intra prediction modes ("most probable mode list") may be obtained. The list 90 may be stored in registers within the decoder 54 and the encoder 14. The list 90 may thus provide a particular order, here indicated by a first position 91, a second position 92, a third position 93, a fourth position 94, and a fifth position 95. For example, other positions may of course be provided to cover all the modes of the assigned set. However, in some examples, the list 90 need not have as many positions as there are modes (either conventional modes or ALWIP modes) within the set and may have fewer. In an example, the list 90 may have less than 10 positions, such as a number of positions between 3 and 7, such as 5. Within the list 90, the first position 91 is here utilized by the prediction mode "23", the second position 92 is utilized by the prediction mode "15", etc. (the mode numbering may be stored, for example, within a look-up table (LUT)).

[0157] Note that in the example, the assignment for a particular position may not be signaled within the data stream 80(12) and may be determined by the decoder based on the prediction made previously for blocks 17a - 17c. In some examples, the previous most used intra prediction mode (or, in any case, the statistically more frequent intra prediction mode) can obtain the highest rank within list 90 (the highest rank can be understood as the highest ranked position). In this case, mode "23" is the previously most used prediction mode, and thus, the first position 91 is given. It should also be noted that the same sorting is performed in the encoder. The encoder will obtain a copy of list 90 (based on the same historical data regarding the most used prediction mode). Thus, the encoder and decoder share the same list without even having to signal that list within the data stream 80. Other techniques are possible.

[0158] It should be understood that the size of the data stream 80 can be reduced by assigning the shortest (cheaper) codes (e.g., "1", "01", "001",...) within the data stream 80 (e.g., 12) to the highest ranks (91, 92, 93,...) within list 90. This conclusion is possible based on the consideration that the most used prediction modes (e.g., "23", "15", "18",...) are also the most accurate prediction modes for the current block 18. (Alternatively, it is the most statistically certain mode that is given the highest rank within list 90.) Thus, by assigning short codes to the most accurate prediction modes, a reduction in the size of the data stream 80 is obtained.

[0159] Note that the second signal 82 is not necessarily encoded as a unary code. For example, a truncated binary code can be used. An example of a truncated binary code is provided in the following table:

[0160] [Table 1]

[0161] Index 0 is associated with the highest index 91 (associated with the next most accurate mode), index 1 is associated with the second highest index (associated with the next second most accurate mode), index 10 is associated with the third highest index (associated with the next third most accurate mode), and so on. As can be seen, the third most accurate mode is associated with an index that is not less expensive than the indices associated with the first and second most accurate modes (index 10 for the third most accurate mode requires 2 bits in the bitstream 80, while indices 0 and 1 for the first and second most accurate modes each require only 1 bit for encoding).

[0162] The second signal 82 may thus include an encoded index indicating a particular position. For example, if the second signal 82 includes "1", it will indicate that the first position 91 is pointed to, and thus that mode "23" will be used. If the second signal 82 includes "01" (with a reduced probability), it will indicate that the second position 92 is pointed to, and thus that mode "15" will be used, and so on.

[0163] Thus, it is possible to predict a given block 18 that will be used for the encoder and decoder using the intra prediction mode indicated by the index.

[0164] In some cases, several problems may occur. One example is when a given block 18 is predicted using a legacy mode (i.e., the first signal 81 indicates that the legacy mode is used), but one of the neighboring blocks 17a - 17c has been previously predicted using the ALWIP mode. This implies that one of the indices 91 - 95 in the list 90 will indicate the ALWIP mode that cannot be used (since the first signal 81 requires the legacy mode). Thus, there will be a redundant indication within the list 90.

[0165] However, it should be understood that it is possible to map some ALWIP modes to legacy modes. Thus, when a particular ALWIP mode ALWIP1 is being used, the mapping will allow for the derivation of a particular legacy mode CONV1 (the mapping of ALWIP1 by the mapping). Therefore, when the ALWIP mode is used for the previous blocks 17a, 17b, or 17c, one of the indices 91 - 95 will then indicate the mapped legacy mode associated with the previously used ALWIP mode. Thus, the indices 91 - 95 of the list 90 are not wasted and will indicate unusable modes. The mapping is predefined and may be known to both the encoder and the decoder.

[0166] The same may apply by mapping legacy modes to ALWIP modes. However, note that in some examples, the list 90 of the most likely modes is used only for a particular set (e.g., the first set 121 of legacy modes), and such a list is not used for other sets (or the list of the most likely modes may be predefined and fixed). This makes some mappings unnecessary in such examples (e.g., in some cases, the list of the most likely ALWIP modes may be predefined based on a pre - assumed probability and will never be changed on - the - fly, and thus, in these cases, no mapping is given).

[0167] The mapping may refer to the block size, and different mappings may be used for different sizes. Thus, multiple mappings may be stored for different sizes (i.e., the mapping for an MxN size may be different from the mapping for an M1xN1 size).

[0168] It is possible to define a mapping from a first size to a second size. This may be the case, for example, when a neighboring, previously predicted block 17a, 17b, or 17c has a different size from the block 18 that is to be predicted.

[0169] Although mapping the ALWIP mode, it may also be possible to associate it with different sizes.

[0170] Here, the consideration regarding mapping will be described.

[0171] When the assigned set is the second set 122 of the intra prediction mode (ALWIP), for example, when sorting the assigned set, a first mapping that maps each intra prediction mode of the first set of prediction modes to a representative one intra prediction mode within the second set of intra prediction modes may be used. When the assigned set is the second set of the intra prediction mode, when sorting the assigned set, a second mapping that maps each intra prediction mode of the second set of prediction modes to a representative one intra prediction mode within the first set of intra prediction modes will be used.

[0172] Hereinafter, at least one of the following mappings may be used: The first mapping (101a, Figure 10a'): from ALWIP to the conventional type (same size); The second mapping (102a in Figure 10a', 102b in Figure 10b): from the conventional type to ALWIP (same size); The third mapping (103, Figure 10c): from ALWIP to the conventional type (from the size of the neighboring blocks predicted through the conventional mode to the different sizes of block 18 that would be predicted through the ALWIP mode); The fourth mapping (104, Figure 10d): from the conventional type to ALWIP (from the size of the neighboring blocks predicted through the ALWIP mode to the different sizes of block 18 that would be predicted through the conventional mode); The fifth mapping (105, Figure 10a'''): from ALWIP to ALWIP (different sizes); The generalized fifth mapping (106, Figure 10e): from one size to another size (general example).

[0173] For a more detailed examination, it is described here. Figure 10a is subdivided into Figure 10a', Figure 10a'', Figure 10a'''.

[0174] Figure 10a' shows an example of the first mapping 101a in which different conventional modes (CONV1, CONV2, CONV3) are mapped to different ALWIP modes (ALWIP1, ALWIP2, ALWIP3) through different mappings. In the example, the different conventional modes and different ALWIP modes have the same size. In an example where list 90 is not updated on-the-fly for ALWIP (however, an index is given based on a predefined probability), this mapping is not used.

[0175] Figure 10a shows an example of a second mapping 102a in which different ALWIP modes (ALWIP1, ALWIP2, ALWIP3) of the second set 122 are mapped to different legacy modes (CONV1, CONV2, CONV3) of the first set 121. In the example, the different legacy modes and different ALWIP modes have the same size. This second mapping can be used when the list 90 of the most accurate modes is updated on-the-fly at least for the legacy modes (thus, the list 90 may sometimes be referred to as the "list 90 of the most accurate legacy modes"). Thus, if the previous block 17a, 17b, or 17c was predicted using ALWIP2, the list 90 will indicate the legacy mode CONV1 in one of its indices (i.e., even if CONV1 has not been used yet because ALWIP1 has already been used and the second mapping 102a maps ALWIP2 to CONV1, CONV1 becomes one of the most accurate legacy modes).

[0176] Figure 10c shows an example of a third mapping 103 in which the legacy modes of the first set 121 (size 152) are mapped to the ALWIP modes of the second set 122 (size 151). In an example where the list 90 is not updated on-the-fly for ALWIP (however, an index is provided based on a predefined probability), this mapping is not used.

[0177] Figure 10d shows an example of a fourth mapping 104 in which the ALWIP modes of the second set 122 (size 151) are mapped to the legacy modes of the first set 121 (size 152). In an example where the list 90 is not updated on-the-fly for the ALWIP modes (however, an index is provided based on a predefined probability), this fourth mapping 104 is not used.

[0178] Figure 10a' shows that different ALWIP modes (ALWIP11, ALWIP12, ALWIP13) of the same size (size 1 or 151) are different mappings (1051, 105 n) shows an example of a fifth mapping 105 that is mapped to different ALWIP modes (ALWIP11', ALWIP12', ALWIP13') of different sizes through (0). Other ALWIP modes (ALWIPn1, ALWIPn2, ALWIPn3) of the same size (size n or 15n) can be mapped to different ALWIP modes (ALWIPn1', ALWIPn2', ALWIPn3') of different sizes. Thus, it is possible to map any ALWIP mode to a different other ALWIP mode.

[0179] FIG. 10e shows a mapping 106 that is a more general example than the fifth mapping 105. Here, different modes used for size 151 are mapped to modes for different sizes 152. This mapping not only maps ALWIP modes to ALWIP modes, but also maps legacy modes to legacy modes, legacy modes to ALWIP modes, and ALWIP modes to legacy modes.

[0180] For a particular case of the second mapping 102a, we discuss here with reference to the second mapping 102b in FIG. 10b. In some examples, multiple (or all) ALWIP modes of the same size (within the set of sizes) can be mapped to one single legacy mode. Here, a first set 121 of legacy modes 131 is shown. Here, for the first set 121 of legacy modes, only one mode 131 is represented, and each of these modes is associated with a particular size (for example, CONV11 is one particular legacy mode 131 for size 1, CONV21 is one particular legacy mode 131 for size 2, CONV1 is one particular legacy mode 131 for size n, etc.). The first set 121 may include other legacy modes 131 not shown here. However, the legacy modes 131 (CONV11, CONV12, CONV13) shown here may be of the same type (for example, all planar modes). A second set 122 of ALWIP modes is also shown. The second set 122 includes multiple modes (for size 1: ALWIP11, ALWIP12, ALWIP13; for size 2: ALWIP21, ALWIP22, ALWIP23; for size n: ALWIPn1, ALWIPn2, ALWIPn3) for each size (for example, size 1, size 2, …, size n). The second mapping 102b can be defined. For example, multiple (or all) ALWIP modes of the same size (size 1) (i.e., ALWIP11, ALWIP12, ALWIP13) can be mapped to the same legacy mode CONV11 (which may have the same size as ALWIP11, ALWIP12, ALWIP13). Thus, there are other legacy modes 131 (not shown) within the first set 121 having the same size (size 1), but multiple (or all) ALWIP modes are mapped to the same legacy mode conv11. The same can apply to the ALWIP modes (ALWIP21, ALWIP22, ALWIP23) of size 2, which are all mapped to the same legacy mode CONV21.The same can apply to ALWIPs of size n (ALWIPn1, ALWIPn2, ALWIPn3) that are all mapped to the same conventional mode convn1. In some examples, the conventional mappings CONV11, CONV21, CONV2 to which multiple ALWIP modes are mapped can all be in planar mode. Thus, multiple different ALWIP modes can be mapped to a single conventional mode. (In certain examples, different ALWIP modes can be mapped to multiple conventional modes.).

[0181] In some examples, at least one of the following techniques can be implemented: If one of the neighboring blocks (17a - 17c) has the same block size as a predetermined block (18) within the set of block sizes, but is assigned to a different one of the first and second sets (121, 122) of intra prediction modes, use the first or second mapping (101a, 101b, 102b). If one of the neighboring blocks (17a - 17c) has a block size different from that of a predetermined block (18) within the set of block sizes If the set assigned to the predetermined block (18) is the second set (122) for the block size of the predetermined block, and the set assigned to one of the neighboring blocks (17a - 17c) is the first set (121) for the block size of one of the neighboring blocks, use a third mapping (103) that maps each intra prediction mode of the first set (121) of prediction modes for the block size of one of the neighboring blocks to a representative intra prediction mode within the second set (122) of intra prediction modes for the block size of the predetermined block, and / or When the set assigned to a given block (18) is a first set (121) with respect to the block size of the given block, and the set assigned to one neighboring block (17a to 17c) is a second set (122) with respect to the block size of one neighboring block, a fourth mapping (104) is used to map each intra prediction mode of the second set (122) of prediction modes with respect to the block size of one neighboring block to one representative intra prediction mode within the first set (121) of intra prediction modes with respect to the block size of the given block, and / or When one of the neighboring blocks (17a to 17c) has a block size different from that of the given block (18) within the set of block sizes, and the set assigned to the given block (18) is a second set (122) with respect to the block size of the given block, and the set assigned to one neighboring block (17a to 17c) is a second set (122) with respect to the block size of one neighboring block, a fifth mapping (105) is used to map each intra prediction mode of the second set (122) of prediction modes with respect to the block size of one neighboring block to one representative intra prediction mode within the second set (122) of intra prediction modes with respect to the block size of the given block.

[0182] The mapping from the conventional type to ALWIP (e.g., 102a, 102b) can be used for the chroma component when the ALWIP mode is used for the luma component.

[0183] As described above, in some examples, list 90 is static. This can be, for example, when flag 81 requires the use of the ALWIP mode. In these cases, in some examples, there is no list 90 that is updated on the fly, and there is a list with a predefined relationship.

[0184] In the above example, often a list 90 referring to five modes through five indices 91 - 95 is referenced, although a different number (more or fewer) of indices may be defined.

[0185] In the example, the first signal 81 may require one single bit (e.g., to signal a selection between, for example, an "ALWIP mode" and a "conventional mode"). In the example, the second signal 82 may require a variable length: for example, a less expensive code (e.g., having a narrower length) may be associated with the most statistically frequent mode. In some cases, the code length may depend (e.g., monotonically) on the rank of the intra prediction mode that the index refers to, with a higher rank being associated with the most frequent index (and also the most frequent mode). Other codes may be used, but one example may be a unary code (see above).

[0186] [Transpose] Next, referring to FIGS. 11a, 11b, and 11c, mention is made of the "transpose of the proposed prediction mode" discussed above. Here, a given block 18 of size MxN (M≠N) or NxM (M≠N) is to be predicted from previously predicted neighboring blocks 17a, 17b, 17c, specifically from a series of P neighboring samples 17'a - 17'c that form a template 170. (Here, block 17b shown in FIG. 5 is not used, although it is imagined that block 17b may be used in some examples).

[0187] The inventors assume that the decoder 56 or the encoder 14 has a specific ALWIP mode (hereinafter referred to as ALWIP1) that is suitable for a size of NxM but not for a size of MxN. In other words, the NxM ALWIP matrix 17M is stored, but the MxN ALWIP matrix 17M is not stored. (In principle, it can be imagined that it would be preferable to have and store another MxN matrix for the ALWIP mode. However, in order to reduce memory and / or signaling, it may be preferable to reduce the amount of ALWIP mode stored in the encoder and / or decoder.)

[0188] In some cases, the first signal 81 in the stream 80(12) may indicate that the ALWIP mode is used for a given block 18. The second signal 82 may have an encoded index associated with ALWIP1 (or in either case, indicating ALWIP1). Thus, different operations can be performed according to the orientation of the block 18. As shown in FIG. 11a, when a given block 18 to be predicted has a size MxN (corresponding to the size of ALWIP1), the mode ALWIP1 is applied (140) to the P samples of the template 170 to predict the P samples of the block 18. As shown in FIG. 11b, when a given block 18 to be predicted has a size MxN (opposite to the size of the stored ALWIP1), since ALWIP1 has a size NxM different from the size MxN of the given block 18, it is not possible to simply apply ALWIP1 (assuming that the NxM matrix 17M is not stored). However, it should be understood that it is possible to adopt a technique in which the encoder 14 or the decoder 54 then performs the following steps: 1) Reverse the series of templates 170 (141) (oriented according to direction R170) to obtain a reversed template 170 (oriented according to direction R170T), 2) To obtain a predicted block 18T of size NxM that conforms to ALWIP1, apply ALWIP1 to the inverted version 170T of template 170 (142), 3) To obtain a predetermined block 18, transpose the predicted block 18T (144).

[0189] In particular, whether the MxN ALWIP mode or the NxM ALWIP mode is to be used, it is not necessary to provide signaling within stream 12(80), nor is it necessary to signal that the transposition is to be performed. It is simply possible to use the fifth mapping 105 of FIG. 10a. In fact, to reconstruct block 18: 1) The second signal 82 can signal that a particular ALWIP mode (e.g., ALWIP1) is used for the MxN block 18 (either directly or by indices 91 - 95 of list 90), 2) However, in the decoder, the MxN matrix for ALWIP1 is not stored (ALWIP1 can be regarded as a part of the subset 120N of the non - stored ALWIP modes), 3) Nevertheless, in the decoder, the NxM ALWIP mode (which can be regarded as a part of the subset 120S of the stored ALWIP modes) is stored, 4) Mapping 105 maps the non - stored MxN ALWIP1 to the stored NxM ALWIP mode, 5) The procedure of FIG. 11b may be performed, and thus reconstruction of block 18 is reached.

[0190] The above example is proposed for a rectangular block MxN where M≠N. However, a similar procedure may be performed for a square (orthogonal) block MxM. In fact, in some cases, exactly (although M=N) as in Fig. 11b, it should be noted that there may be a possibility of simply applying the inverted version of template 170, making a prediction using the stored transformation, and finally transposing the obtained block 180T. As shown in Fig. 11d, the set 120Q of modes (ALWIP and / or legacy) may include the following subsets: A first subset 120QS of modes stored for the square block (for these modes, matrix 17M is stored), and A second subset 120QN of modes that are not stored for the square block but can be predicted by using the method of Fig. 11b (where M=N).

[0191] Therefore, if the mode is directly indicated within the first subset 120QS of modes for which index 91 - 95 or the second signal is stored (case 1), that mode is directly called (also, the procedure of Fig. 11a where M=N is performed).

[0192] Otherwise, if the mode is directly indicated within the second subset 120QN of modes for which index 91 - 95 or the second signal is not stored (case 2), after being mapped from the modes of subset 120QN, the mode from the first subset 120QS is called.

[0193] The examples discussed above and below may be within the spatial domain and / or the transform domain.

[0194] [Examples in the transform domain] FIG. 12 shows the prediction of a spatial region block 18 from previously predicted neighboring blocks 17c and 17a (which can be one of the predetermined blocks 18 as described above, such as in FIG. 5). Here, the prediction is performed in the transform domain (for example, after applying transforms such as fast Fourier transform (FFT), and / or discrete cosine transform (DCT), wavelets, etc.).

[0195] FIG. 12' shows a predetermined block 18 that is to be predicted. The Q values of block 18 are to be predicted from first and second sets (for example, proper subsets) 17'c and 17'a of neighboring blocks 17c and 17a (the first and second sets 17'c and 17'a can represent lines or stripes or arrays that are adjacent to, for example, adjacent to, sides 18c and 18a of the block 18 to be predicted and can include samples that have already been predicted). The values of the first and second sets 17'c and 17'a can be selected independently of the sizes of the neighboring blocks 17c and 17a, and regardless of the dimensions of the neighboring blocks 17c and 17a, the dimensions of the first and second sets 17'c and 17'a are based on the dimensions of block 18. The values of the first and second sets 17'c and 17'a (templates) can be selected independently of the sizes of blocks 17c and 17a. In the example, the first and second sets 17'c, 17'a can be arranged along one-dimensional paths along the first and second sides 18c, 18a, respectively. As discussed above, subsampling can be performed, for example, by utilizing only some specific transform coefficients.

[0196] A transformation 160 (e.g., FFT, DCT, wavelet, etc.) can be applied to at least some of the samples of the first and second sets 17'c and 17'a. The first and second sets 17'cT, 17'aT of transform coefficients (templates in the transform domain) can represent the one-dimensional transformation of the samples of the first and second sets 17'c, 17'a, and the latter are arranged along a one-dimensional path. It is not necessary to transform all the neighboring blocks 17c and 17a, and only the low-frequency part (or in any case, the reduced part) needs to be transformed. Advantageously, there is no problem even if the sizes of the first and second sets 17'c and 17'a do not match the sizes of the sides 18c and 18a of the block 18: the lengths of the sets 17'c and 17'a can be longer or shorter than the lengths of the sides 18c and 18a.

[0197] FIG. 12" shows that the first and second sets 17'c and 17'a (formed by the "transform coefficients") are then in the transform domain and are thus denoted by 17'cT and 17'aT, respectively.

[0198] The ALWIP intra prediction transform 162 is applied to the transform coefficients of the sets 17'cT and 17'aT. Thus, in FIG. 12'', the prediction block 18T of the block 18 in the transform domain is obtained. The ALWIP intra prediction transform 162 may follow an intra prediction mode signaled in the signal 81 and / or 82. In particular, the prediction version 18T in the transform domain of the block 18 may have too few transform coefficients: in that case, it is simply possible to perform a zero-padding operation before the inverse transform 164: the remaining transform coefficients (e.g., high-frequency bins) can be set to 0. Thus, after the inverse transform 164, the entire set of values of the block 18 can be obtained. Thus, the prediction block 18T in the transform domain may be smaller than the prediction block 18 in the spatial domain. The prediction block 18T in the transform domain may also be larger than the prediction block 18. In that case, a low-pass filter can be applied later.

[0199] To obtain the prediction block 18 within the spatial region of FIG. 12'', an inverse FFT (IFFT), or inverse DCT (IDCT), or another inverse transform 164 is applied to the prediction block 18T.

[0200] In some examples, it is possible to generate a predictor for a given block 18 within the spatial region.

[0201] [Conventional Intra Prediction Mode] The following is a discussion regarding the conventional intra prediction mode, at least some of which may be included within the first set 121 of conventional intra prediction modes (e.g., conv1, conv2, etc.).

[0202] The conventional intra prediction mode can be represented by matrix-vector multiplications and filtering operations applied to the input vector resp. output vector. However, the inventors point out that the matrix used in the latter calculation of the matrix-vector product has a very special structure, as the inventors will explain next.

[0203] To set the notation, the inventors assume that an M×N block (M rows, N columns) is given, in which the intra prediction signal (within the luma component) is to be calculated by the conventional intra prediction mode. The reference samples that function as the input to the prediction consist of samples that have already been reconstructed. Generally, the reference samples may consist of N+q samples at the top of the block (some samples in the upper right of the block may not be directly available, but can be generated by a defined padding operation), and M+p samples to the left of the block (some samples in the lower left of the block may not be directly available, but can be generated by a defined padding operation).

[0204] In the case of the conventional intra prediction mode, the reference sample is regarded as a vector ref of size M + p + N + q. The first M + p components of the vector consist of the reference samples to the left of the block, and the last N + p components of the vector consist of the samples above the block.

[0205] The inventors first explain the case where the conventional intra prediction mode is a directional intra prediction mode also called the angular intra prediction mode.

[0206] Here, in the first step, an interpolation filtering operation is applied to the reference sample ref. The latter operation generates sample values for non-integer sample positions between the true sample positions at a fixed resolution. Thus, for a fixed integer k and each i ∈ {1, …, M + p + N + q - 1}, k sample values are generated whose sample positions are between the i-th and (i + 1)-th components of ref. Optionally, a smoothing filtering operation is performed before or after the interpolation filtering operation. The overall result of the filtering operations can also be regarded, in this case too, as a vector of size (M + p + N + q)*(k + 1) - k, denoted as ref fltr It is pointed out that for some directional modes, such as the horizontal intra prediction mode or the vertical intra prediction mode, interpolation may not be necessary, i.e., the number of k may be equal to zero.

[0207] In the second step, for a fixed matrix A corresponding to the conventional intra prediction mode and having M * N rows and (M + p + N + q)*(k + 1) - k columns, the predicted signal pred is calculated as a matrix-vector product pred = A·ref fltr where · denotes matrix-vector multiplication. Here, each row of the matrix A has only one non-zero component consisting of 1. In other words, each predicted sample value is exactly one ref fltr value.

[0208] In a final step, a second filtering operation may be applied to the signal which is an extension of pred by the samples already reconstructed in the upper resp. left block of the block in order to generate an overall directional intra prediction signal.

[0209] Next, the inventors will explain the case where the conventional intra prediction mode is the planar mode. Here, the vector ref fltr In order to generate, an operation of only smoothing without interpolation may be applied to the reference samples. The (unfiltered) planar prediction signal pred planar When it is calculated by matrix-vector multiplication pred planar =A planar ·ref fltr In that case, the matrix A planar is planar such that each row of A planar consists of only 4 non-zero components. In other words, each predicted sample value is calculated as a linear combination of 4 reference sample values. The column positions of the 4 non-zero components of a fixed row of A fltr correspond to positions in ref

[0210] In a final step, a second filtering operation may be applied to the signal which is an extension of pred by the samples already reconstructed in the upper resp. left block of the block in order to generate an overall planar intra prediction signal. planar

[0211] ​Finally, the inventors will explain the case where the conventional intra prediction mode is the DC mode. Here, the (unfiltered) DC prediction signal pred DC is calculated by a matrix-vector product, pred DC =A DC ·ref fltr and in that case, the matrix A DC has the property that all rows of A DC are equal. In the final step, a second filtering operation may be applied to the signal that is an extension of pred DC by the already reconstructed samples at the top of the block resp. on the left to generate the overall DC intra prediction signal.

[0212] It is pointed out that the matrices used in the ALWIP intra prediction of the present application are not a priori restricted to one of the aforementioned patterns of matrices corresponding to the directional mode, DC mode, planar mode. Rather, these matrices are, for example, the result of an offline data-driven training method that takes into account the possibility of dividing the block into various shapes, a loss function that models the loss (rate or rate-distortion) of a general video encoder, the signaling cost required to signal the mode, and various other features known in the art.

[0213] [Downsampling / Upsampling, Conversion Region for ALWIP] Next, refer to FIG. 13. As described above, in some cases, the encoder or decoder may perform a conversion between predictors of different block sizes, for example, by downsampling and / or upsampling.

[0214] As described above, the "second template 1160" (within the spatial region) is used for intra prediction, but may refer to a group of Q neighboring samples (identified above as 17'a, 17'b, 17'c) for which the encoder or decoder cannot freely use the ALWIP mode. Basically, the "second template 1160" is formed by the samples 17'a, 17'b, 17'c used for ALWIP prediction.

[0215] The following discussion is valid for both ALWIP conversion and conventional conversion, but here mainly focuses on ALWIP conversion.

[0216] FIG. 13 shows a current predetermined spatial region block B1 (size M1xN1) that can be one block 18 in the above example. Block 18 is to be predicted by ALWIP conversion (for example, one of the above). However, in the encoder or decoder, ALWIP conversion for different sizes MxN (where M≠M1 and / or N≠N1) can be freely used.

[0217] Note that the second template 1160 (formed by samples 17'a, 17'b, 17'c) has already been reconstructed (predicted).

[0218] If the ALWIP mode for reconstructing block 18 (B1) is not stored due to the dimensions of block 18 (B1), and if blocks (B) with different dimensions MxN can freely use ALWIP conversion, the following procedure can be implemented. In particular, ALWIP conversion for a block B of size MxN requires a template (here called the "first template 1161") having a size different from the already obtained second template 1160. The technique for overcoming this obstacle is discussed below.

[0219] The conversion operation (hereinafter referred to as D) can be applied, for example, to the second template 1160. The conversion D can provide an element 1136 formed from the converted (resampled) first template 1161 and a block B (1138) of size MxN that is to be predicted.

[0220] For example, a block B1 (18) of size M1xN1 (having unknown coefficients) can be logically converted to a block B (1138) of size MxN (also having unknown coefficients at this point). Since the coefficients of block B (1138) are unknown, the conversion to the conversion region is unnecessary.

[0221] Similarly, the conversion D converts the template 1160 (size MxN) to a different template 1161 (size M1xN1) having different dimensions.

[0222] In some cases, the conversion operation D can be a downsampling operation when M1 > M and N1 > N (and in particular, when M is a multiple of M1 and N is a multiple of N1). For example, M1 = 2 * M and N1 = 2 * N, the conversion operation D can simply be based on hiding some bins in a chess-like pattern.

[0223] At this point, block B is predicted in MxN units by the ALWIP conversion discussed above. The ALWIP conversion (associated with size MxN) is included in the encoder or decoder. In the transition path 1138a, the device 14 or 54 can then use the ALWIP conversion originally defined for the MxN block (for example, in predictors 44, 44'). By applying the transition path discussed above, a prediction is obtained for block B of size MxN.

[0224] The prediction in block B(1138) is obtained for size MxN, and the image to be reconstructed will have size M1xN1. It is simply possible to perform a transformation 1140(U) that transforms block B(1138) from size MxN to block 18 of size M1xN1. This transformation 1140(U) can be, for example, a bilinear interpolation or upsampling operation. This transformation 1140(U) can be performed by introducing coefficients in the M1xN1 block in addition to the coefficients in the MxN block 1138. For example, M1 = 2 * M and N1 = 2 * In the case of N, it is simply possible to perform interpolation (e.g., bilinear interpolation) to approximate ( "infer") the coefficients of what has been discarded by transformation D. Thus, the M1xN1 prediction is obtained as element block 18 and can be used to display the block image as part of image 10.

[0225] In the example, it is possible to perform a mapping from the mode that will be used for block 18 (B1) to mode 1138a.

[0226] [Further Embodiments and Examples] Generally, an example may be implemented as a computer program product including program instructions that, when the computer program product is executed on a computer, are operable to execute one of these methods. The program instructions can be stored, for example, on a machine-readable medium.

[0227] Another example includes a computer program stored on a machine-readable carrier for executing one of the methods described herein.

[0228] In other words, an example of a method is thus a computer program having program instructions for executing one of the methods described herein when the computer program is executed on a computer.

[0229] A further example of a method is thus a data carrier medium (or a digital storage medium, or a computer-readable medium) storing a computer program for performing one of the methods described herein. The data carrier medium, digital storage medium, or recording medium is tangible and / or non-transitory, not an intangible and transitory signal.

[0230] A further example of a method is thus a sequence of data streams or signals representing a computer program for performing one of the methods described herein. The sequence of data streams or signals can be transferred, for example, via a data communication connection, for example, via the Internet.

[0231] A further example includes processing means, for example, a computer or a programmable logic device, for performing one of the methods described herein.

[0232] A further example includes a computer having installed thereon a computer program for performing one of the methods described herein.

[0233] A further example includes an apparatus or system for transferring (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver can be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system can include, for example, a file server for transferring the computer program to the receiver.

[0234] In some examples, a programmable logic device (for example, a field-programmable gate array) can be used to perform some or all of the functionality of the methods described herein. In some examples, the field-programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, these methods can be performed by any suitable hardware device.

[0235] The examples described above are merely examples of the principles discussed above. It should be understood that modifications and variations of the arrangements and details described herein will become apparent. Accordingly, it is intended to be limited not by the specific details presented by the description and illustration of the examples herein, but by the scope of the following claims.

[0236] Equal or equivalent elements with equal or equivalent functionality are designated by equal or equivalent reference numerals in the following description, even if they occur in different drawings.

Description of Reference Numerals

[0237] 10 Picture 12 Data Stream 14 Encoder 16 Video 17a~17c Neighborhood Block 17'a, 17'b, 17'c Neighborhood Sample 17'a Second Set of T-Transform Coefficients 17c Second Set of T-Transform Coefficients 17M Matrix 18 Current Block 18a Second Side 18c First Side 18T Prediction Block 20 Coding Order 22 Subtractor 24 Prediction Signal 24' Reconstructed Value 26 Prediction Residual Signal 28 Prediction Residual Encoder 28a Lossy Encoding Stage 28b Lossless Encoding Stage 30 Quantizer 32 Transformer 34 Transformed and Quantized Prediction Residual Signal 36 Prediction Residual Signal Reconstruction Stage 36' Residual Signal Reconstruction Stage 38, 38' Inverse Quantizer 40, 40' Inverter 42, 42' Adder 44, 44' Predictor 46, 46' In-loop filter 54 Decoder 56 Entropy decoder 70 Chart 72 Plane 80 Data stream 81 First signal 82 Second signal 90 List 91 First position 92 Second position 93 Third position 94 Fourth position 95 Fifth position 101 Mapping 101a First mapping 102a Second mapping 102b Second mapping 103 Third mapping 104 Fourth mapping 105 Fifth mapping 1051 Mapping 105 n Mapping 106 Generalized fifth mapping 120N Subset of ALWIP mode 120Q Set of modes 120QN Second subset 120QS First subset 120S First subset of intra prediction mode 121 First set of conventional intra prediction modes 122 Second set of intra prediction modes 131 Conventional mode 151, 152 Block size 156 Residual provider 162 ALWIP intra prediction transform 164 Inverse transform 170 Template 170T Inverted version 1136 Resampled template 1138 Block B 1138a ALWIP predictor, mode 1140 Conversion 1160 Second template 1161 First template

Claims

1. A method for predicting blocks of a picture, comprising: determining that a luma prediction signal is generated using an affine linear weighted intra prediction (ALWIP) mode; mapping the ALWIP mode to a different intra prediction mode; obtaining a corresponding chroma prediction signal using the mapping; A method comprising the above steps.

2. The method according to claim 1, wherein the ALWIP mode includes a matrix-based mode.

3. The different intra prediction modes include: a planar intra prediction mode; a DC intra prediction mode; an angular intra prediction mode; The method according to claim 1 or 2, including at least one of the above.

4. The method according to any one of claims 1 to 3, wherein the different intra prediction modes include a planar intra prediction mode.

5. The method according to any one of claims 1 to 4, further comprising at least one of the following steps: encoding the picture into a data stream; decoding the picture from the data stream.

6. A non-transitory storage medium storing instructions that, when executed by at least one processor, cause the at least one processor to execute the method according to any one of claims 1 to 5.

7. An apparatus for predicting blocks of a picture, comprising: a non-transitory computer-readable medium; at least one processor configured to cooperate with the non-transitory computer-readable medium to execute the method according to any one of claims 1 to 5. An apparatus including the above components. ​ ​

Images