Determining a set of candidate transforms for video coding.

Abstract

The present invention relates to transform coefficient coding, in which a predetermined transform set may be used. Embodiments of the present invention provide an encoder (10) and a decoder (20) for encoding / decoding an image (12) into / from a data stream (14) using block-based prediction and block-based residual coding, where the encoder (10) and the decoder (20) support a transform set (94) for block-based residual coding. The encoder (10) and decoder (20) are configured to determine (122) a candidate list (98) of transforms for a current residual block (84) from a set of transforms (94) based on the height and width of the current residual block (84), encode / decode a prediction residual (100) in the residual block (84) using one selected transform (102) from the list of selectable transforms (98), and selectively signal the selected transform (102) in / derive the selected transform (102) from a data stream (14).

Description

[Technical field]

[0001] The present invention relates to image and video coding, in particular to block-based image and video coding. The images may be still images or images in a consecutive order within a video image sequence. Some specific examples and embodiments of the present invention relate to a scheme for unified and extended residual coding in the context of video coding, utilizing a mode-dependent set of transform candidates used for transformations to obtain respective residual signals in the spatial domain. [Background technology]

[0002] <1.Technical background> In addition to DCT-II and 4x4DST-VII, used in some standards such as High Efficiency Video Coding (HEVC), other standards, e.g. Joint Exploration Test Model (JEM), may also define two additional transform families: the Adaptive Multiple Core Transform (AMT) and a set of mode-dependent Non-Separable Secondary Transforms (NSST).

[0003] <2.1 Adaptive multi-core conversion[1,8]> AMT extends the set of transforms available for the luma component with transforms from the DCT / DST family, namely DST-VII, DCT-VIII, DST-I, and DCT-V. AMT is applied if the matching CU-level flag is set, otherwise DCT-II is used. For inter-coded blocks, two additional flags determine whether DST-VII or DCT-VIII is used for each horizontal and vertical transform, respectively. If the block is intra-coded, which transforms are available depends on the intra prediction mode. Each intra mode maps to two of three predefined sets, one for each horizontal and vertical transform. Each set consists of two transforms. If the number of significant coefficients is greater than a predefined threshold, an index is signaled for each direction, otherwise the first transform of each set is used.

[0004] <2.2 Mode-dependent non-separable quadratic transformations [1,2]> NSST defines a second-stage transform that is applied to the coefficients of the core transform (primary transform): "If both the width and height of the transform coefficient block are equal to or greater than 8, then an 8x8 non-separable secondary transform is applied to the top-left 8x8 region of the transform coefficient block. Otherwise, if either W or H of the transform coefficient block is equal to 4, then a 4x4 non-separable secondary transform is applied, and a 4x4 non-separable transform is performed on the top-left min(8,W)xmin(8,H) region of the transform coefficient block. The above transform selection rules apply to both luma and chroma components." [1]

[0005] <2.3 Large block size transforms using high frequency zeroing> "JEM allows transforms with larger block sizes up to 128x128 in size, which is primarily useful for high resolution video such as 1080p and 4K sequences. For transform blocks with size (width or height, or both width and height) greater than or equal to 64, the high frequency transform coefficients are zeroed and only the low frequency coefficients are kept. For example, for an MxN transform block, where M is the block width and N is the block height, only the left 32 columns of transform coefficients are kept if M is greater than or equal to 64. Similarly, only the top 32 rows of transform coefficients are kept if N is greater than or equal to 64. When transform skip mode is used for large blocks, the entire block is used without zeroing out any values." [1] Summary of the Invention [Problem to be solved by the invention]

[0006] However, existing techniques suffer from several drawbacks, for example, the encoding effort increases significantly as the image size increases, so it would be desirable to provide a codec with improved efficiency.

[0007] It is therefore proposed to provide a mode-dependent transform candidate set. The identification of the transform used to obtain the residual signal in the spatial domain may exploit a candidate list utilizing already decoded syntax elements. The coefficients of the residual block may be decoded conditional on the available transforms in the candidate list. In so doing, the candidate list itself may be iteratively reduced by removing transforms that contradict already obtained information about the coefficients. In case all coefficients are decoded and multiple transform candidates remain, an index into the candidate list may be signaled.

[0008] The innovative concept of introducing mode-dependent candidate lists may enable flexible and efficient testing and signaling of the transformations of residual blocks. For example, categorizing structured transformations by transformation-induced constraints may allow exploiting dependencies between the selected transformation, the resulting coefficients, and neighboring transformation blocks. [Means for solving the problem]

[0009] <Modes of the Invention> According to a first aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding. The encoder may be configured to determine a transform candidate list of transforms for a current residual block from among the transform set based on a height and a width of the current residual block. The encoder may be further configured to encode the prediction residual in the residual block using one selected transform from among the list of selectable transforms, and selectively signal the selected transform in the data stream. The signaling may possibly selectively indicate that the candidate list only includes one candidate transform, such that the signaling may be omitted.

[0010] According to a first aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to decode a prediction residual in the residual block by determining a transform candidate list of transforms for a current residual block from among the transform set based on a height and width of the current residual block, and selectively deriving a selected transform from the data stream in a domain of one selected transform from among the list of selectable transforms. The derivation may possibly selectively indicate that the candidate list only includes one candidate transform such that its signaling / derivation may be omitted.

[0011] According to a second aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. The encoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set, and to encode transform coefficient data representing a prediction residual in the residual block into the data stream using one selected transform from the list of selectable transforms. The encoder may be further configured to signal the selected transform in the data stream and / or to determine the selectable list in a manner dependent on the transform coefficient data or a portion of the transform coefficient data.

[0012] According to a second aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set, and to decode transform coefficient data representing a prediction residual in the residual block from the data stream using one selected transform from the list of selectable transforms. The decoder may be further configured to derive the selected transform from the data stream and / or to determine the selectable list in a manner dependent on the transform coefficient data or a portion of the data transform coefficient data.

[0013] According to a third aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. The encoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set and to signal the selected transform in the data stream. The encoder may be further configured to encode transform coefficients representing a prediction residual in the residual block using a selected transform from among the list of selectable transforms in a manner that depends on the selected transform or the list of selectable transforms.

[0014] According to a third aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set and derive the selected transform from the data stream. The decoder may be further configured to decode transform coefficients representing a prediction residual in the residual block from the data stream using a selected transform from among the list of selectable transforms in a manner that depends on the selected transform or the list of selectable transforms.

[0015] According to a fourth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. The encoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set, and to encode a prediction residual in the residual block using one selected transform from the list of selectable transforms. The encoder may be further configured to signal the selected transform in the data stream using a transform indicator, the transform indicator being signaled in the data stream using a one or more digit code, each of the one or more digit sets being uniquely associated with a respective transform characteristic.

[0016] According to a fourth aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set, and to decode a prediction residual in the residual block from the data stream using one selected transform from the list of selectable transforms. The decoder may be further configured to derive the selected transform from the data stream using a transform indicator, the transform indicator being derived from the data stream using a one or more digit code, each of the one or more digit sets being uniquely associated with a respective transform characteristic. For example, the code may include a syntax element that can be parsed from the data stream, from which the bins of the transform indicator or the transform indicator that undergoes binary arithmetic encoding are configured.

[0017] According to a fifth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. The encoder may be further configured to predict the intra-predicted block by applying a spatial neighborhood of the intra-predicted block on a selected neural network from among a plurality of neural networks to obtain a prediction of the intra-predicted block, and determine a set of selectable transforms for the residual block covering the intra-predicted block from among the transform set based on the selected neural network and a block size of the residual block. The encoder may be further configured to encode a prediction residual associated with the prediction of the intra-predicted block in the residual block using one selected transform from the set of selectable transforms, and signal the selected transform in the data stream.

[0018] According to a fifth aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to predict an intra-predicted block by applying a spatial neighborhood of the intra-predicted block on a selected neural network from among a plurality of neural networks to obtain a prediction of the intra-predicted block, and determine a set of selectable transforms for the residual block covering the intra-predicted block from among the transform set based on the selected neural network and a block size of the residual block. The decoder may further be configured to decode a prediction residual associated with a prediction of the intra-predicted block in the residual block from the data stream by deriving a selected transform from the data stream in a domain of one selected transform from the set of selectable transforms.

[0019] According to a sixth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding. The encoder may be configured to predict an intra-predicted block by applying a spatial neighborhood of the intra-predicted block on a neural network to obtain a prediction of the intra-predicted block in a domain of a primary transform, encode a prediction residual associated with the prediction of the intra-predicted block in a residual block using a selected transform from a set of selectable transforms, and signal the selected transform in the data stream, where the set of selectable transforms includes at least one multi-stage transform including a primary transform and a secondary transform applied on a subgroup of primary transform coefficients of the primary transform.

[0020] According to a sixth aspect, another embodiment of the innovative concept is a block-based prediction and block-based residual Decryption The decoder may be configured to predict an intra predicted block by applying a spatial neighborhood of the intra predicted block on a neural network to obtain a prediction of the intra predicted block in a domain of a primary transform, and to decode a prediction residual associated with a prediction of the intra predicted block in a residual block from the data stream by deriving a selected transform from the data stream in another domain of a selected transform from a set of selectable transforms, the set of selectable transforms being input with at least one multi-stage transform including a primary transform and a secondary transform applied on a subgroup of primary transform coefficients of the primary transform.

[0021] According to a seventh aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding. The encoder may be configured to predict an intra-predicted block based on a spatial neighborhood of the intra-predicted block using a selected one from a set of intra-prediction modes to obtain a prediction of the intra-prediction mode, and to determine a set of selectable transforms for the residual block covering the intra-predicted block from the transform set based on the selected intra-prediction mode and a block size of the residual block. The encoder may be further configured to encode a prediction residual associated with a prediction of the intra-predicted block in the residual block using one selected transform from the set of selectable transforms, and signal the selected transform in the data stream.

[0022] According to a seventh aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to predict an intra-predicted block based on a spatial neighborhood of the intra-predicted block using a selected one from a set of intra-prediction modes to obtain a prediction of the intra-prediction mode, and determine a set of selectable transforms for the residual block covering the intra-predicted block from the transform set based on the selected intra-prediction mode and a block size of the residual block. The decoder may further be configured to decode a prediction residual associated with a prediction of the intra-predicted block in the residual block from the data stream by deriving a selected transform from the data stream in a domain of one selected transform from the set of selectable transforms.

[0023] According to an eighth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. The encoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set based on a reference transform selected for a neighboring residual block, also from among the transform set, for coding a prediction residual of the neighboring residual block. The encoder may be further configured to encode the prediction residual in the residual block using one selected transform from among the list of selectable transforms and signal the selected transform in the data stream.

[0024] According to an eighth aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set based on a reference transform selected for a neighboring residual block, also from among the transform set, for encoding a prediction residual of the neighboring residual block. The decoder may further be configured to decode a prediction residual in the residual block by deriving the selected transform from the data stream in a domain of one selected transform from the list of selectable transforms.

[0025] According to a ninth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding. The encoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set in a manner dependent on a quantization parameter of the current block. If the quantization parameter corresponds to a quantization finer than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one multi-stage transform including a primary transform applied on a prediction residual in the current residual block and a secondary transform applied on a subgroup of primary transform coefficients of the primary transform. However, if the quantization parameter corresponds to a quantization coarser than a certain threshold, the list of selectable transforms for the current residual block may be populated with at least one modification of the multi-stage transform for zeroing primary transform coefficients beyond the subgroup in addition to or instead of the at least one multi-stage transform. In either case, the encoder may be configured to encode the prediction residual in the residual block using a selected transform from a list of selectable transforms and to signal the selected transform in the data stream.

[0026] According to a ninth aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for block-based residual decoding. The decoder may be configured to determine a list of selectable transforms for a current residual block from among the transform set in a manner dependent on a quantization parameter of the current block. If the quantization parameter corresponds to a quantization finer than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one multi-stage transform including a primary transform applied on a prediction residual in the current residual block and a secondary transform applied on a subgroup of primary transform coefficients of the primary transform. However, if the quantization parameter corresponds to a quantization coarser than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modification of the multi-stage transform in which primary transform coefficients beyond the subgroup are zeroed in addition to or instead of the at least one multi-stage transform. In either case, the decoder may be configured to decode the prediction residual in the residual block by deriving a selected transform from the data stream in the domain of a selected transform from a list of selectable transforms.

[0027] According to a tenth aspect, an embodiment of the innovative concept may provide an encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for the block-based residual coding. One or moreThe encoder may be configured to determine a list of selectable transforms for the current residual block from among the transform sets based on the reference transform selected for the neighboring residual blocks in a manner that depends on the number of significant transform coefficients in the neighboring residual blocks of the current residual block or on their sum. If the number or sum is greater than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one multi-stage transform including a primary transform applied on the prediction residual of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform. However, if the number or sum is less than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modification of the multi-stage transform in which the primary transform coefficients beyond the subgroup are zeroed in addition to or instead of the at least one multi-stage transform. In either case, the encoder may be configured to code the prediction residual in the residual block using one selected transform from the list of selectable transforms and to signal the selected transform in the data stream.

[0028] According to a tenth aspect, another embodiment of the innovative concept may provide a decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a transform set for the block-based residual decoding, the decoder comprising: One or moreThe decoder may be configured to determine a list of selectable transforms for the current residual block from among the transform sets based on the reference transforms selected for the neighboring residual blocks in a manner that depends on the number of significant transform coefficients in the neighboring residual blocks of the current residual block or on their sum. If the number or sum is greater than a certain threshold, the list of selectable transforms for the current residual block includes at least one multi-stage transform that includes a primary transform applied on the prediction residual of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform. However, if the number or sum is less than a certain threshold, the list of selectable transforms for the current residual block includes at least one modification of the multi-stage transform in which the primary transform coefficients beyond the subgroup are zeroed in addition to or instead of the at least one multi-stage transform. In either case, the decoder may be further configured to decode the prediction residual in the residual block by deriving the selected transform from the data stream in the domain of one selected transform from the list of selectable transforms.

[0029] According to another aspect, computer programs are provided, each computer program configured to perform the above encoding and / or decoding methods when executed on a computer or signal processor such that the above method is performed by one of the computer programs.

[0030] Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the figures. [Brief description of the drawings]

[0031] [Figure 1] 1 is a schematic block diagram illustrating an encoder that may be used in one or more embodiments of the present invention. [Diagram 2] FIG. 2 is a schematic block diagram illustrating a decoder that may be used in one or more embodiments of the present invention. [Diagram 3] FIG. 1 is a schematic block diagram illustrating a block-based encoding scheme that may be used in one or more embodiments of the present invention. [Figure 4] 1A-1C are diagrams illustrating some examples of partitioning a residual block into groups and subgroups of transform coefficients when utilizing a transform. [Diagram 5] 5A-5C illustrate some alternative examples of partitioning a residual block into groups and subgroups of transform coefficients when utilizing a transform. [Figure 6] FIG. 2 illustrates several examples of different stages of a multi-stage transformation including a linear transformation and a secondary transformation. [Figure 7] FIG. 1 is a schematic block diagram illustrating block-based residual coding using a set of transforms including one or more neural networks according to an embodiment of the present invention; [Figure 8] FIG. 2 illustrates the concept of a multi-stage transformation including a primary transformation and a secondary transformation according to an embodiment of the present invention. [Figure 9] FIG. 2 illustrates the concept of a multi-stage transform including a linear transform and a secondary transform in the spatial domain that exploits row and column encoding order according to an embodiment of the present invention. [Figure 10] FIG. 2 illustrates the concept of first-order only transformation according to one embodiment of the present invention. [Figure 11] FIG. 1 is a schematic block diagram illustrating a block-based residual coding scheme using a multi-stage transform including a neural network according to an embodiment of the present invention. [Figure 12] FIG. 2 is a schematic block diagram illustrating a block-based residual coding scheme using a transform set and a set of selectable transforms according to an embodiment of the present invention; [Figure 13] FIG. 2 is a schematic block diagram illustrating a residual block including a number of significant and non-significant transform coefficients that are coded in a pre-defined coding order according to an embodiment of the present invention; [Figure 14] FIG. 1 is a schematic block diagram illustrating a block-based residual coding scheme using a transform set and a set of selectable transforms and using transform signaling according to an embodiment of the present invention; [Figure 15]FIG. 1 is a schematic block diagram illustrating a block-based residual coding scheme using a transform set and a set of selectable transforms and using transform signaling according to an embodiment of the present invention; [Figure 16] FIG. 2 is a schematic block diagram illustrating a block-based residual coding scheme in which some transform coefficients are set to zero, according to an embodiment of the present invention; [Figure 17] FIG. 2 illustrates a schematic residual block including a number of primary coefficients, some of which are set to zero before subjecting the remaining non-zero primary coefficients to a secondary transform, in accordance with an embodiment of the present invention. [Figure 18] FIG. 2 is a schematic block diagram illustrating a transform block from a decoder perspective, in which some transform coefficients may be zeroed according to one embodiment; [Figure 19] FIG. 13 is a schematic block diagram illustrating a transform block from a decoder perspective, in which some transform coefficients may be zeroed according to another embodiment; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0032] In the following description, like or equivalent elements or elements having like or equivalent functions are designated with like or equivalent reference numbers.

[0033] Method steps illustrated in and described with reference to block diagrams may be performed in an order different from that illustrated and / or described. Additionally, method steps relating to a particular feature of a device may be interchangeable with that feature of the device, and vice versa.

[0034] The term transformation may be abbreviated herein by the term trafo. The term transformation may be used synonymously for both transformation and the transformed (i.e., the result of the transformation).

[0035] The following diagram description first presents a description of a video encoder and decoder of a block-based predictive codec for encoding video images to form an example of an encoding framework in which the embodiments described herein can be built. The video encoder and decoder are described with reference to Figures 1 to 3. Then, a description of another embodiment of the present application is presented with reference to the figures. The above parts referring to the embodiments described and the appended claims are numbered the same and referenced in the above sections. Although all these embodiments can be built into the video encoder and decoder of Figures 1 and 2, respectively, the embodiments described herein, such as the embodiments described with reference to the subsequent figures, can also be used to form video encoders and decoders that do not operate according to the underlying encoding framework of the video encoder and decoder of Figures 1 and 2.

[0036] 1 shows an apparatus for predictively coding a video 11 consisting of a sequence of pictures 12 into a data stream 14. Block-wise predictive coding is used for this purpose. Furthermore, transform-based residual coding is exemplarily used. The apparatus, i.e. the encoder, is designated using the reference number 10.

[0037] FIG. 2 shows a corresponding decoder 20, i.e., apparatus 20, configured to predictively decode from data stream 14 a video 11' consisting of images 12' in image blocks, again using transform-based residual decoding, the apostrophe indicating that images 12' and video 11 reconstructed by decoder 20 are respectively different from images 12 originally encoded by apparatus 10 with respect to coding losses introduced by quantization of the predicted residual signal. Derivation is used to indicate that

[0038] 1 and 2 exemplarily use transform-based predictive residual coding, but the embodiments of the present application are not limited to this type of predictive residual coding, which also applies to other details described with respect to FIGs.

[0039] The encoder 10 is configured to perform a spatial-spectral transformation on the prediction residual signal and to encode the prediction residual signal thus obtained into a data stream 14. Similarly, the decoder 20 is configured to decode the prediction residual signal from the data stream 14 and to perform a spectral-spatial transformation on the prediction residual signal thus obtained.

[0040] Internally, the encoder 10 generates an original signal, i.e. a video 11 or a prediction signal 26 from a current image 12. Derivation The prediction residuals are measured as signal The prediction residual signal former 22 may include a prediction residual signal former 22 generating a prediction signal 24. The prediction residual signal former 22 may for example be a subtractor which subtracts a prediction signal 26 from the original signal, i.e. the current image 12.

[0041] The encoder 10 then further comprises a transformer 28 which performs a spatial-spectral transformation on the prediction residual signal 24 to obtain a spectral domain prediction residual signal 24' which is then quantized by a quantizer 32 also included in the encoder 10. The prediction residual signal 24'' thus quantized is data It is encoded into stream 14.

[0042] To this end, the encoder 10 may optionally include an entropy encoder 34 that entropy encodes the transformed and quantized prediction residual signal into the data stream 14. signal 24 on Data Stream 14 encoding 2 and is generated by a prediction stage 36 of the encoder 10 based on a prediction residual signal 24 ″ that is decodable from the data stream 14 .

[0043] For this purpose, the prediction stage 36 may internally include an inverse quantizer 38 which inversely quantizes the prediction residual signal 24'', as shown in Fig. 1, to obtain a spectral domain prediction residual signal 24''', which corresponds to the signal 24' excluding the quantization losses, followed by an inverse transformer 40 which applies an inverse transformation, i.e. a spectral-spatial transformation, to the latter prediction residual signal 24''' to obtain a prediction residual signal 24'''', which corresponds to the original prediction residual signal 24 excluding the quantization losses. A combiner 42 of the prediction stage 36 then recombines the prediction signal 26 and the prediction residual signal 24'''', for example by addition, to obtain a reconstructed signal 46, i.e. a reconstruction of the original signal 12. The reconstructed signal 46 may correspond to the signal 12'.

[0044] A prediction module 44 of the prediction stage 36 then generates the predicted signal 26 based on the signal 46 using, for example, spatial prediction, i.e., intra-prediction, and / or temporal prediction, i.e., inter-prediction, as will be described in more detail below.

[0045] Similarly, the decoder 20 may be internally composed of interconnected components corresponding to the prediction stage 36. In particular, an entropy decoder 50 of the decoder 20 entropy decodes a quantized spectral domain prediction residual signal 24″ from the data stream, and then an inverse quantizer 52, an inverse transformer 54, a combiner 56, and a prediction module 58, which are interconnected and cooperate as described above with respect to the modules of the prediction stage 36, recover a reconstructed signal based on the prediction residual signal 24″, such that the output of the combiner 56 is a reconstructed signal, i.e., the video 11′ or current picture 12′, as shown in FIG.

[0046] Although not specifically described above, it is readily apparent that the encoder 10 may set some encoding parameters, including, for example, prediction modes, motion parameters, etc., according to some optimization scheme, such as, for example, using some rate- and distortion-related criteria, i.e., a method that optimizes the encoding cost, and / or some rate control. As will be described in more detail below, the encoder 10 and the decoder 20 and corresponding modules 44, 58 support different prediction modes, such as intra- and inter-coding modes, respectively, or modes in which the former modes form a kind of set or pool of primitive prediction modes on the basis of which the prediction of the image blocks is configured. The granularity with which the encoder and the decoder switch between these prediction configurations may correspond to a subdivision into blocks of the image 12 and the image 12′, respectively. It is noted that some of these blocks are exclusively intra-coded, some are exclusively inter-coded, and optionally, there may even be cases where further blocks are obtained using both inter- and intra-coding, as will be described in more detail below.

[0047] According to the intra-coding modes, the prediction signal of a block is obtained based on the spatial, already coded / decoded neighborhood of the respective block. There may be several intra-coding sub-modes, the choice between which represents, in a pseudo way, a kind of intra-prediction parameter. There may be directional or angular intra-coding sub-modes, which are followed to fill in the prediction signal of each block by extrapolating the neighboring sample values ​​along a specific direction specific to each directional intra-coding sub-mode, into the respective block. The intra-coding sub-modes may be, for example, a DC coding mode, which is followed by the prediction signal of each block to assign DC values ​​to all samples in the respective block, and / or a planar intra-coding mode, which is followed to approximate or determine the prediction signal of each block to be a spatial distribution of sample values ​​described by a two-dimensional linear function over the sample positions of the respective block, by deriving the slope and offset of the plane defined by the two-dimensional linear function based on the neighboring samples. One or more Further sub-modes of the above may also be included.

[0048] In comparison, according to the inter prediction mode, the prediction signal of the block may be obtained, for example, by predicting inside the block temporally. In the parameterization of the inter prediction mode, a motion vector may be signaled in the data stream, which indicates the spatial displacement of the part of the previously coded image of the video 11, where the previously coded / decoded image is sampled to obtain the prediction signal of the respective block. This means that in addition to the residual signal coding contained in the data stream 14, such as the entropy coded transform coefficient levels representing the quantized spectral domain prediction residual signal 24'', the data stream 14 may also code prediction related parameters for assigning to the block prediction mode, prediction parameters for the assigned prediction mode, such as motion parameters for the inter prediction mode, and optionally other parameters that control the configuration of the final prediction signal of the block using the assigned prediction mode and prediction parameters, as will be explained in more detail below. In addition, the data stream 14 may also include parameters that control and signal the subdivision into blocks of the respective images 12 and 12′. The decoder 20 subdivides the image 12 in the same way as the encoder 10, using these parameters to assign the same prediction modes and parameters to the blocks and to perform the same predictions to obtain the same prediction signals.

[0049] 3 shows the relationship between, on the one hand, a reconstructed signal, i.e. the reconstructed image 12′, and, on the other hand, a combination of a prediction residual signal 24″″ and a prediction signal 26 signaled in the data stream 14. As already mentioned above, this combination can be additive.

[0050] The prediction signal 26 is shown in Fig. 3 as a subdivision of the image area into blocks 80 of various sizes, but this is by way of example only. The subdivision may be any subdivision, such as a regular subdivision of the image area into rows and columns of blocks, or a multi-tree subdivision of the image 12 into leaf blocks of various sizes, such as a quad-tree subdivision, a mixture of which is shown in Fig. 3, where the image area is first subdivided into rows and columns of tree root blocks, which are then further subdivided according to a recursive multi-tree subdivision to obtain the blocks 80.

[0051] The prediction residual signal 24'''' in Fig. 3 is also shown as a subdivision of the image area into blocks 84. These blocks may also be called transform or residual blocks in order to distinguish them from the coding blocks 80. In effect, Fig. 3 shows that the encoder 10 and the decoder 20 may use two different subdivisions of the image 12 and the image 12' into blocks, respectively, one subdivision into coding blocks 80 and another subdivision into blocks 84. Although both subdivisions may be the same, i.e. each block 80 simultaneously forms a transform block 84 and vice versa, Fig. 3 shows the case where the subdivision into transform blocks 84 forms an extension of the subdivision into blocks 80, for example, such that any boundary between two blocks 80 overlaps the boundary between two blocks 84, in other words each block 80 coincides with one of the transform blocks 84 or coincides with a cluster of transform blocks 84.

[0052] However, the subdivisions may be determined or selected independently of one another such that the transformation blocks 84 may alternatively cross block boundaries between blocks 80. As far as the subdivision into transformation blocks 84 is concerned, similar statements apply as those presented with regard to the subdivision into blocks 80, i.e. the blocks 84 may be the result of a regular subdivision of the image area into blocks arranged in rows and columns, a recursive multi-tree subdivision of the image area, or a combination thereof or any other kind of segmentation.

[0053] It should be noted, by the way, that the blocks 80 and 84 are not limited to quadratic shapes, rectangular or any other shapes. Moreover, the subdivision of the current image 12 into blocks 80, where the prediction signal is formed, and into blocks 84, where the prediction residual is coded, may not be the only subdivision used for coding / decoding. These subdivisions form the granularity at which the determination of the prediction signal and the residual coding are performed, but firstly, the residual coding may alternatively be performed without subdivision, and secondly, at a granularity other than these subdivisions, the encoder 10 and the decoder 20 may set certain coding parameters, which may include some of the aforementioned parameters, such as prediction parameters, prediction signal configuration control signals, etc.

[0054] 3 shows that the combination of the prediction signal 26 and the prediction residual signal 24'''' directly results in the reconstructed signal 12'. However, it should be noted that multiple prediction signals 26 may be combined with the prediction residual signal 24'''' to result in the image 12' according to alternative embodiments, such as prediction signals obtained from other views or from other coding layers that are encoded / decoded in separate prediction loops with separate DPBs.

[0055] In Fig. 3, the transform blocks 84 shall have the following significance: the transformer 28 and the inverse transformer 54 perform their transform in units of these transform blocks 84. For example, many codecs use some kind of DST or DCT for all transform blocks 84. Some codecs allow to skip the transform for some of the transform blocks 84, so that the prediction residual signal is directly coded in the spatial domain. However, according to the embodiments described below, the encoder 10 and the decoder 20 are configured such that they support multiple transforms. For example, the transforms supported by the encoder 10 and the decoder 20 may include: DCT-II (or DCT-III), where DCT stands for discrete cosine transform DST-IV, DST stands for discrete sine transform DCT-IV DST-VII Identity Transformation (IT)

[0056] Of course, the transformer 28 will support all forward transform versions of these transforms, and the decoder 20 or inverse transformer 54 will support their corresponding backward or inverse versions. Inverse DCT-II (or inverse DCT-III) Reverse DST-IV Inverse DCT-IV Reverse DST-VII Identity Transformation (IT)

[0057] Note that in any case, the supported transformation set may only contain one transformation, such as one spectral-to-spatial or spatial-to-spectral transformation.

[0058] As already mentioned above, Figures 1 to 3 are presented as examples in which the concepts described further below may be implemented to form exemplary embodiments of video encoders and decoders according to the present application. Thus far, the video encoders and decoders of Figures 1 and 2, respectively, represent possible implementations of the video encoders and decoders described further herein.

[0059] Please note that with respect to the claims, "non-separable" refers to vertical and horizontal separability.

[0060] <3. Structured multi-stage conversion> In general, any linear transformation for a residual block of size D=W*H can be defined by D*D parameters. Here, we call these kinds of transformations fully connected transformations. These transformations are very flexible, but their complexity in terms of the number of operations required and the parameter storage required grows quadratically with the size D.

[0061] Thus, fully connected transforms are only applicable to small transform sizes. To reduce the complexity, the transforms can be constrained to have a specific structure that allows efficient implementation and reduces the number of transform parameters. One approach requires that the transforms be factored into multiple transforms, with each transform stage itself separating into independent subgroups. The complexity of these multi-stage transforms can be adjusted by selecting the number of transform stages, the dimensions of the independent subgroups, and also by constraining the subgroup transforms.

[0062] <Example: Reducing the number of multiplications by dividing into subblocks> Consider a block of data with width W and height H. This block of data can be represented as an H×W matrix B that is serialized into a (H*W)×1 vector b for further processing.

[0063] Let a data block be transformed by a fully connected transformation. The transformation is performed by computing a matrix-vector product T*b between the transformation matrix T and the serialized data block vector b. The shape of the transformation matrix is ​​(H*W) x (H*W), resulting in a required number of H*W multiplications per output coefficient. Thus, for H*W output coefficients, there are a total of (H*W) 2 multiplications are required.

[0064] For H=8 and W=8, this results in 4096 multiplications being required to compute the transform coefficients. Next, assuming that both H and W are divisible by 4, the data block is transformed into a 4 × 4 matrix B i Each subblock is then stored as a 16×1 vector b i Then the transform coefficients are calculated to transform each sub-block by a fully connected transform and the results are concatenated. i and the serialized sub-block b i Matrix vector product T between i *b iThe transform is performed again by computing . Since the transform matrix shape in this case is 16x16, a total of 16 multiplications are required per output coefficient. Therefore, 16*16=256 multiplications are required to compute the transform coefficients of a single subblock. Since there are H / 4*W / 4 subblocks, the number of multiplications required for all subblocks is H / 4*W / 4*256=H*W*16. Looking again at the case of H=8, W=8, the number of multiplications is reduced by 75%, from 4096 to 1024, by splitting into subblocks and transforming each subblock independently. By adding another transform stage with a different split, dependencies between blocks can also be taken into account. Assuming the same complexity as the first stage, the total number of multiplications is 2048, so it is only half the complexity of a fully connected 8x8 transform.

[0065] Further reduction in complexity can be achieved by allowing non-bijective transformations. More precisely, these transformations are defined here as transformations that assume certain subgroups of a transformation stage are all zero, thus reducing the dimension of the residual signal. These transformations are called transformations with dimensionality reduction and are a generalization of the transformations with high frequency zeroing as described above. Here, the zeroing of coefficients can be applied in combination with any preceding transformation stage of any size. This includes the case where the zeroing of coefficients is applied to the first stage. These transformations allow not only efficient implementation, as will be described below, but also efficient signaling of non-significant coefficients.

[0066] To fully describe a multi-stage transform, as described above, each transform stage must be given a unique division into subgroups and the transforms applied to those subgroups.

[0067] The reduced complexity of the multi-stage transforms described above arises from the fact that each subgroup transform can be applied independently of the others, each subgroup transform has limited dimensionality, and / or each subgroup transform can be applied using an efficient algorithm. Thus, each subgroup transform is one of the following types: Subgroup transformation: (Identity) Conversion from DCT / DST family Fully combined transformation, for subgroup sizes up to 64 (Permutations +) Pairwise Givens rotations (Permutation +) Pairwise lifting scheme transformation Multi-stage conversion itself

[0068] The division into subgroups can be given by any of the following rules, some of which are illustrated in FIGS. 4 and 5 and are represented in lowercase letters: a) For each row and position, the labels are stored in a ROM that assigns a unique subgroup to the label. b) For each position, the label is stored in a Rom that assigns a unique subgroup to the label. This division is used for all stages of the transformation. c) Given a unique scan order of positions and a list D={d0,d1,dN} of subgroup sizes, the partition is made along this scan by grouping consecutive positions, with the size of each group given by the associated element of D. d) Given a unique scan order of positions, the first d0 consecutive positions are taken as one subgroup and the remaining positions are taken as a second subgroup. e) The division is made horizontally or vertically, such that each row or each column defines a subgroup. f) The division is done horizontally such that each row is further divided into subgroups of size {2, 4, 8, 16, 24, 48, 32, 64}. g) The division is done vertically such that each column is further divided into subgroups of size {2, 4, 8, 16, 24, 48, 32, 64}. h) The division is done by considering the subblock that contains the first position (top left) and has one of the shapes {4x4, 4x8, 4x12, 4x16, 8x4, 8x8, 12x4, 16x4} as one subgroup. The remaining positions form a second group. i) The division is done by considering as one subgroup the subblock that contains the first position (top left) and has one of the shapes {4x4, 4x8, 4x12, 4x16, 8x4, 8x8, 12x4, 16x4}. The remaining positions are divided into groups of size {1x2, 2x1, 2x2, 4x4}. j) The division is into subgroups of shape 4x4 and / or 2x2. k) Subgroup transformation of dimensions {2,4,8,16} l) Subgroup transformation of dimensions {24,32,48,64} m) The partition is into subgroups of geometry sizes {2x2, 4x4, 4x8, 4x12, 4x16, 8x4, 8x8, 12x4, 16x4}. n) The partition is derived from the width and height of the residual block by considering the subgroup of size min(8,width)×min(8,height) containing the first position as one group, and the remaining positions form the second group. o) The partition is derived from the width (W) and height (H) of the residual block by taking a subgroup of size {4x4, 4x8, 4x12, 4x16, 8x4, 8x8, 12x4, 16x4} containing the first position and applying the following rules: If W==H, the subgroup has shape DxD and D=min(W,8). If W>H, the height of the subgroup is set to 4. Its width is chosen by taking the largest size in {8,12,16} less than or equal to W. If H>W, the width of the subgroup is set to 4. Its height is chosen by taking the largest size in {8,12,16} less than or equal to H. p) Permutations stored in ROM + pairwise division. q) Derivation from prediction mode r) Hierarchical structure: 1) N horizontal transformations (fully connected: dimension M) 2) M vertical transforms (fully connected: dimension N)

[0069] Conversion Class: Structured transformations, on the one hand, help to reduce the complexity, but also allow grouping of transformations according to the constraints they introduce. This classification can be useful to compare transformations and to define dependency models for the quantization and / or entropy coding stages.

[0070] As one class of transforms, consider a multi-stage transform that is factored into two transforms, hereafter referred to as a primary transform and a secondary transform, respectively, as exemplarily shown in Figure 6. This name arises from the encoder's point of view, since the primary transform is applied directly to the prediction residual in the spatial domain resulting in the primary transform coefficients, while the secondary transform is applied to a subset of the primary transform coefficients.

[0071] It should be noted that both the linear and secondary transforms can themselves be defined as multi-stage transforms, with the secondary transform being of lower dimensionality compared to the linear transform.

[0072] If a transform cannot be factored into two transforms, such that the second transform is applied only to a subset of the coefficients of the first transform, then the transform is called a linear-only transform.

[0073] Another class of transforms is given by the fact that the transform can be factorized into a vertical transform and a horizontal transform, which can be called a separable transform. This is a particularly simple form of multi-stage transform. In the first stage, a transform is applied row-wise, and in the second stage, a transform is applied column-wise to the output (coefficients) of the first transform stage (or vice versa). Typically, in each stage, the same transform is applied to each row (column).

[0074] A special class of transformations is given by their dimensionality reduction properties, i.e., where the dimensionality is explicitly reduced by the transformation, in which case the exact coefficient positions are assumed to be zero.

[0075] Given that the reconstruction of the residual signal from the coefficients coded in the bitstream involves an inverse transform as part of the entropy coding stage, applied to the coefficients (possibly of different residual blocks) before they are dequantized, it is necessary to have a lossless transform that does not introduce rounding errors. Whether a transform is lossless or not can also be a classification characteristic.

[0076] Further classification of transformations can be made by distinguishing between analytically constructed transformations, such as those from the DCT / DST family, and those that are trained on a specific set of examples (settings). For analytical transformations, classification can be made precisely by their construction rules. These construction rules can be applied with different transform sizes, thus also allowing comparisons between transformations even when the sizes of the transforms are different. This can be useful, for example, to study dependencies between blocks for the derivation of context models that depend on the transformation decisions of neighboring blocks.

[0077] However, data-driven transforms can also be further classified depending on their specialization. Specifically, it is more common to train a transform for intra-predicted residuals, but more specialized transforms are obtained by training a transform precisely for one particular intra-mode given, for example, a precise intra-direction. This specialization can be increased even further by clustering data for a given prediction mode.

[0078] 3.1 Exemplary embodiments of primary and secondary transformations Some examples and embodiments for primary and secondary transformations may be set forth in the following non-exhaustive list.

[0079] Primary transformation: A) 1. Column: division into lines, dedicated transformation for each line 2. Columns: Split into columns, dedicated transformation for each column As mentioned above, each subgroup transformation can be of any type.

[0080] especially: A.1) 1. Column: division into rows, same transformation for each subgroup (row) 2. Columns: Split into columns, same transformation for each subgroup (column) A.2) 1. Column: division into rows, same DCT / DST transformation for each subgroup (row) 2. Rows: Split into columns, same DCT / DST transformation for each subgroup (column) A.3) 1. Stage: Split into rows, same DCT / DST transformation with dimensionality reduction for each subgroup (row) 2. Rows: Split into columns, same DCT / DST transformation for each subgroup (column) A.4) 1. Column: division into rows, same DCT / DST transformation for each subgroup (row) 2. Stage: Split into columns, same DCT / DST transformation with dimensionality reduction for each subgroup (column) A.5) 1. Stage: Split into rows, same DCT / DST transformation with dimensionality reduction for each subgroup (row) 2. Stage: Split into columns, same DCT / DST transformation with dimensionality reduction for each subgroup (column) A.6) 1. Columns: division into rows, identity "transformation" for each subgroup (row) 2. Rows: Split into columns, same DCT / DST transformation for each subgroup (column) A.7) 1. Column: division into rows, same DCT / DST transformation for each subgroup (row) 2. Columns: division into columns, identity "transformation" for each subgroup (column) A.8) 1. Columns: division into rows, identity "transformation" for each subgroup (row) 2. Columns: division into columns, identity "transformation" for each subgroup (column) (equivalent to not applying any transformations) A.9) 1. Column: Split into rows, same full join transformation for each subgroup (row) 2. Rows: Split into columns, and perform the same full join transformation for each subgroup (column) A.10) 1. Column: Split into rows, and perform dedicated join transformation for each subgroup (row) 2. Rows: Split into columns, and perform dedicated full join transformation for each subgroup (column)

[0081] B) 1. Columns: division along rows into subgroups of size S (e.g. 8) 2. Columns: division along columns into subgroups of size S (e.g. 8) As mentioned above, each subgroup transformation can be of any type.

[0082] especially: B.1) 1. Column: Division along the rows into subgroups of size S, half the block width 2. Columns: Split into columns, same transformation for each subgroup (column) B.2) 1. Column: division into rows, same transformation for each subgroup (row) 2. Columns: Division along columns into subgroups of size S, half the block height B.3) 1. Column: Division along the rows into subgroups of size S, half the block width 2. Columns: Division along columns into subgroups of size S, half the block height B.4) 1. Columns: division into subgroups of size S along the rows 2. Columns: division into subgroups of size S along columns The first subgroup transform is of the DCT / DST type and the second subgroup is set to zero. B.5) 1. Columns: division into subgroups of size S along the rows 2. Columns: division into subgroups of size S along columns The first subgroup is set to zero and the second subgroup transform is of the DCT / DST type.

[0083] C) 1. Columns: division along rows into subgroups of size S (e.g. 8) 2. Columns: division along columns into subgroups of size S (e.g. 8) 3. Stages: Hierarchical structures with bypasses (equivalent to subgroups with identity transformations) As mentioned above, each subgroup transformation can be of any type.

[0084] especially: C.1) 1. Column: division into rows, same transformation for each subgroup (row) 2. Columns: Split into columns, same transformation for each subgroup (column) 3. Stage 3: Transformation with dimensionality reduction C.2) 1. Column: division into rows, same DCT / DST transformation for each subgroup (row) 2. Rows: Split into columns, same DCT / DST transformation for each subgroup (column) 3. Stage 3: Transformation with dimensionality reduction

[0085] Secondary transformation: A) A subgroup of primary coefficients used in the secondary transform derived from min(8,W), min(8,H). The secondary transform is of the fully connected transform type. A.1) A subgroup of primary coefficients used in the secondary transformation derived from min(8,W), min(8,H). The secondary transformation is of the fully connected transformation type with dimensionality reduction.

[0086] B) Split the primary coefficients into 4x4 subgroups. A full join transform is applied to a subset of the subgroups and then combined in a hierarchical structure. B.1) Split the linear coefficients into 4x4 subgroups. A fully connected transformation with dimensionality reduction is applied to a subset of the subgroups and then combined in a hierarchical structure.

[0087] C) Division of the primary coefficients into subgroups. A secondary transform is applied to a subset of those subgroups. The transform type (i.e., connectivity) of the secondary transform depends on the position of the subgroup within the overall transform block.

[0088] <4. Conversion candidate set> The decoding process of the selected transformation may consist of the following steps.

[0089] First, a candidate list may be built utilizing already decoded syntax elements.

[0090] Knowledge of the selected transform can support the process of decoding coefficients from the bitstream. Thus, when the candidate list includes multiple transforms, it may be beneficial to reduce the candidate list (or even identify the selected candidate) by explicitly signaling information about the selected transform before the coefficient decoding process.

[0091] On the other hand, already decoded information about the coefficients can be utilized to determine the transformation selected among the candidates. During the coefficient decoding process, the candidate list can be iteratively reduced by removing transformations that contradict the already obtained information about the coefficients. When all coefficients are decoded and multiple transformation candidates are left, an index into the remaining candidate list must be signaled.

[0092] <4.1 Building a candidate list> The transform candidate list for the residual block is built by utilizing already decoded syntax elements. These may include transform block parameters such as the total size of the transform, shape (width and height) and color channel type (luma and chroma), but also information about the prediction method used for the current block. Having different transform types depending on the transform size can already be motivated by the complexity considerations in Section 3.

[0093] 12 illustrates a non-limiting example for encoding an image 12 having multiple residual blocks 84, 84', 84''. As an example shown for residual blocks 84, each residual block includes a particular block dimension 96 that may be defined by a width and height of the respective block 84. According to this non-limiting example, building a transform candidate list may be described with respect to the respective block dimensions 96 as an example.

[0094] Thus, an encoder 10 and a decoder 20 for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding may be provided, the encoder 10 and the decoder 20 supporting a transform set 94 for block-based residual coding. The encoder 10 and the decoder 20 may be configured to determine 122 a transform candidate list 98 of transforms for a current residual block 84 from the transform set 94 based on a size 96, such as a height and width, of the current residual block 84. The encoder 10 may be configured to encode a prediction residual 100 in the residual block 84 using a selected transform 102 from the list of selectable transforms 98 and selectively signal the selected transform 102 in the data stream 14. Furthermore, the decoder 20 may be configured to decode a prediction residual 100 in the residual block 84 by selectively deriving the selected transform 102 from the data stream 14 in a domain of the selected transform 102 from the list of selectable transforms 98.

[0095] With regard to the terms 'domain' and 'codomain', reference is made to FIG. 16 which shows a detailed description of the above residual block 84 which includes a number of samples 105. The number of samples 105 may determine the block dimensions 96, e.g. the width and height of the residual block 84, e.g. a 4x4, 8x8, etc. block. The residual block 84, or the samples 105, respectively, may be subjected to a linear transform, the block dimensions 96 and the samples 105 corresponding to the domain of the linear transform. The samples 105 are transformed into linear transform coefficients 114, One or moremay be included in the subgroup 112 of primary transform coefficients. The output of the primary transform, i.e. the range of the primary transform, corresponds to the size of the subgroup 112 defined by the number of primary transform coefficients 114.

[0096] In the non-limiting example shown in FIG. 16, subgroup 112 includes: One or more Zero-value linear transformation coefficient 114 and One or more and significant (i.e., non-zero-valued) linear transform coefficients 114. One or more Some specific examples in which the linear transform of may be set to zero are discussed below in Section 5.

[0097] For the time being, the general principle will be further explained with continued reference to Figure 16. As shown, at least from the encoder's point of view, subgroup 112 in this non-limiting example has the shape of an inverted 'L', i.e., the left column and top row containing significant coefficients 114 are kept, and the remaining primary transform coefficients 114 outside the inverted 'L' may be set to zero. In more general terms that also apply to the decoder, the domain of the secondary transform is reduced compared to the domain of the primary transform.

[0098] The subgroups 112 of primary transform coefficients 114 may then be subjected to a secondary transform resulting in secondary subgroups 138 of secondary transform coefficients 134. The size of the subgroups 112, i.e., the number of primary transform coefficients 114 (corresponding to the domain of the secondary transform), may be greater than the size of the secondary subgroups 138, i.e., the number of secondary transform coefficients 119 (corresponding to the range of the secondary transform).

[0099] Figure 16 shows a first alternative A, where only the significant primary transform coefficients 114 are transformed by the secondary transform and the zero-valued primary transform coefficients are implicitly reduced and not signaled, i.e., in this case only the transformed significant transform coefficients (secondary subgroup 138) may be coded. Figure 16 shows a second alternative B (in dashed lines), where there may be an entire subblock 117 including the secondary subgroup 138 and zero-valued coefficients outside the subgroup 138, i.e., in this case all coefficients are coded.

[0100] As also shown in FIG. 16, the encoder 10 and decoder 20 may inverse transform the secondary transform coefficients 134, where the domain of the inverse transform may correspond to the range of the secondary transform.

[0101] As mentioned above, the transform candidate list 98 for the residual block can be constructed by utilizing already decoded syntax elements, e.g., transform block parameters such as total size of the transform, shape (width and height), etc., in other words, based on the domain / range of the respective transform.

[0102] Thus, encoder 10 of this embodiment may support multiple block dimensions 96 (i.e., height and width). For each block dimension, transform set 94 may include a subset of transforms whose domain corresponds to the respective block dimension 96. For each transform in transform set 94, encoder 10 may include an inverse transform whose domain matches the range of the respective transform (not necessarily T*T, such as for an inverse transform T involving dimensionality reduction). -1 = 1), and encoder 10 can use the inverse transform to fill the decoded image buffer in the residual block for which the respective transform was selected. Encoder 10, in determining transform candidate list 98, can be further configured to input to the transforms exclusively those from among a subset of transforms whose domain corresponds to the height and width of the current residual block 84.

[0103] Thus, this embodiment of decoder 20 may support multiple block dimensions (i.e., height and width). For each block size, transform set 94 includes a subset of transforms whose ranges correspond to the respective block dimensions. Note that the transforms referred to here are inverse transforms to those performed by encoder 10. Decoder 20 may be further configured, in determining transform candidate list 98, to input to transforms exclusively those from among the subset of transforms whose ranges correspond to the height and width of the current residual block 84.

[0104] In summary, each transform (primary or secondary) may include a domain corresponding to its input and a range corresponding to its output, and the respective ranges of the first transform and the secondary transform may differ from each other. The reduced range is achieved, for example, by setting some primary and / or secondary coefficients to zero, as will be described in more detail below.

[0105] As mentioned at the beginning, computational complexity can increase with transform size. Therefore, it is beneficial to trade off and restrict the transform candidates for transform blocks with sizes above a threshold to simpler transform types. For the time being, a simple example is shown by considering a separable transform defined by one horizontal transform type and one vertical transform type. Next to the identity "transform" with no extra computational complexity, the DCT-II is considered to be a computationally simple transform type due to the efficient algorithms that exploit its butterfly structure.

[0106] As mentioned in Section 3, the complexity of each transform type can be reduced by introducing candidate transforms that implicitly reduce the dimensionality by forcing certain coefficients to zero, thus reducing the number of multiplications required.

[0107] In the following, the linear conversion coefficients One or more and / or of secondary transformations One or moreWe describe several example embodiments of encoder 10 and decoder 20 configured to set coefficients to zero, also referred to as zeroing. Setting coefficients to zero may be applied at the encoder end, and the decoder may avoid decoding coefficients that are zero.

[0108] For example, we define three thresholds (T1, T2, T3). T1 is defined by the size (width or height) according to section 2.3 such that DCT-II is a candidate for the vertical (horizontal) transform of a block only if its height (width) is less than T1. Otherwise, DCT-II with dimensionality reduction is a candidate, with the last height (width)·T1 high frequency coefficients set to zero. In addition to that, T2 defines the threshold size for other transform types (like DST-VII, DCT-III, etc.) such that full-sized transforms are candidates only if their size is less than T2. ​​Otherwise, their corresponding transforms with dimensionality reduction are candidates. Finally, T3 defines the threshold size for identity "transforms" such that they are candidates only if their transform size is less than T3.

[0109] According to one example, encoder 10, in determining transform candidate list 98, If the height and width of the current residual block 84 meet a first criterion (e.g., T1 and T2), then the candidate transform list 98 is populated with a transform candidate from among a subset of transforms whose domain corresponds to the height and width of the current residual block 84 and whose range is smaller than the domain. One or more Enter a transformation (e.g., a "width reduction" transformation), or If the height and width of the current residual block 84 do not meet the first criterion, a transform candidate list 98 is selected from the subset of transforms whose domain corresponds to the height and width of the current residual block 84 and whose range is smaller than the domain. One or more Exclude conversion of.

[0110] Furthermore, the decoder 20, in determining the transformation candidate list 98, If the height and width of the current residual block 84 meet a first criterion (e.g., T1 and T2), then the candidate transform list 98 is populated with a transform candidate from among a subset of transforms whose range corresponds to the height and width of the current residual block 84 and whose domain is smaller than the range (e.g., contains fewer samples / coefficients). One or more (i.e., the inverse of the transformation with "width reduction"), or If the height and width of the current residual block 84 do not meet the first criterion, a transformation candidate list 98 is selected, whose range corresponds to the height and width of the current residual block 84 and whose domain is Value range From the subset of transformations smaller than One or more Exclude conversion of.

[0111] The above One or more The transformation is performed using DCT-II, DST-VII, and DCT-III. One or more It could be.

[0112] According to another example, the encoder 10 and the decoder 20, in determining the transform candidate list 98, If the height and width of the current residual block 84 meet a second criterion (e.g., T3), enter in the transformation candidate list 98 an identity transformation from among the subset of transformations whose domain corresponds to the height and width of the current residual block 84; or If the height and width of the current residual block 84 do not satisfy the second criterion, remove from the candidate transform list 98 the identity transform from the subset of transforms whose domain corresponds to the height and width of the current residual block 84.

[0113] The encoder 10 and decoder 20 may be configured to check the first and / or second criteria by checking whether the height, the width, both the height and the width, and / or the product of the height and the width exceed certain thresholds.

[0114] In addition to independently defining constraints by the width and height of the residual block, rules can be applied by examining their relative shapes. For example, accept a candidate only if either its width or its height exceeds the other. For separable transform candidates, we define this as applying one transform type to the larger transform dimension and one to the smaller. dimension This can be done by assigning a different transform type to . For transforms that involve dimensionality reduction, this information can be used to determine which coefficients are set to zero.

[0115] According to yet another example, the encoder 10 and the decoder 20, in determining the transform candidate list, If the height and width of the current residual block meet a third criterion (e.g., does the height-to-width ratio exceed a certain threshold?), the candidate transform list includes the transforms in the subset whose domain corresponds to the size of the current residual block (84). from Separable Spectra Using the decomposition transformation Enter, If the height and width of the current residual block do not satisfy the third criterion, then select a separable spectrum from the list of candidate transforms among a subset of transforms whose domain corresponds to the size of the current residual block (84). Decomposition Transformation Exclude.

[0116] According to this example, the encoder 10 and decoder 20 may be configured to check the third criterion by checking if the height to width ratio exceeds a certain threshold. For a secondary transform, the width and height of the residual block can be used to determine which coefficients are transformed in further transform steps. Due to the increased complexity from above, more flexible transform structures (than separable transforms) should also be constrained according to the transform block size. Transform stages for smaller block sizes can be defined such that the entire result (all coefficients) of the previous transform stage is further transformed in the subsequent stage (referred to as a primary-only transform in section 3), while multi-stage transforms consisting of a primary + secondary transform can be used for larger block sizes.

[0117] For example, by assuming that secondary transforms are applicable up to 64 dimensions (coefficients of the primary transform), the construction rules for secondary inputs can be formulated as subgroups defined by min(8,width)×min(8,height). To reduce the total number of transforms defined, the shapes of the secondary transform subgroups can be restricted. For example, by defining secondary transforms only for 8×8 and 4×4 shapes, secondary transform subgroups are selected by the larger shape (width*height) applicable for a given transform block size.

[0118] For example, a transform block of size (4x32) has a secondary transform shape of (4x4), and a transform block of size (8x16) has a secondary transform shape of (8x8) in this case. The structure of the secondary transform itself should also be defined with a complexity constraint according to its own size or the size of the total transform block. For example, a 4x4 secondary transform can be defined by a fully connected transform, but a larger secondary transform should be constrained in its complexity by any of the methods described in Section 3, for example by limiting the number of further transform stages or by reducing its dimensionality by forcing certain coefficients to zero.

[0119] Thus, in determining the transform candidate list (98), the encoder 10 and decoder 20 may be configured to populate the transform candidate list (98) with transforms depending on the block size and shape, where scaling of the block may be used, as exemplified above. As discussed above, for example, a first transform block of size (4×32) has a secondary transform shape of (4×4), a second transform block of size (8×16) has a secondary transform shape of (8×8), etc. That is, the rows of the first and second transform blocks include a 1:2 scaling assignment, and the columns of the first and second transform blocks include a 2:1 scaling assignment. ratio may include.

[0120] Thus, if the height and width of the current residual block 84 correspond to the first predetermined block dimension, the encoder 10 and decoder 20 may select in the transform candidate list 98 a first transform candidate from the subset of transforms whose range corresponds to the height and width of the current residual block 84 and which inverts a multi-stage transform 106 that includes a primary transform 108 applied on a prediction residual 100 in the current residual block 84 and a secondary transform 110 applied on a first subgroup 112 of primary transform coefficients 114 of the primary transform 108. One or more The input may be configured to input a transformation of

[0121] Alternatively, if the height and width of the current residual block 84 correspond to a second predetermined block dimension, the encoder 10 and decoder 20 may select in the transform candidate list 98 a second transform candidate from the subset of transforms whose range corresponds to the height and width of the current residual block 84 and which inverts the multi-stage transform 106 that includes a primary transform 108 applied on the prediction residual 100 in the current residual block 84 and a secondary transform 110 applied on a second subgroup 112 of the primary transform coefficients 114 of the primary transform 108. One or more The input may be configured to input a transformation of

[0122] In either case, the first One or more The first range of the linear transformation 108 of the transformation One or more Scaling the linear transform 108 of the transform onto the second bin does not image the perimeter of the first subgroup in the first bin onto the perimeter of the second subgroup in the second bin.

[0123] With respect to the linear and secondary transformations being applied to the above height / width ratios of the blocks, it may be possible, for example, that the perimeter of a first subgroup 112 within a first range does not image onto the perimeter of a second subgroup, even if the height / width ratio (aspect ratio) of the range of the linear transformation and the height / width ratios of the first and second predetermined dimensions are the same.

[0124] Thus, when determining the transform candidate list, the encoder 10 and decoder 20 select, if the height and width of the current residual block correspond to a first predetermined block dimension, a first transform candidate list from the subset of transforms, the first transform candidate list being a multi-stage transform 106 whose domain corresponds to the height and width of the current residual block 84 and which includes a primary transform 108 applied on a prediction residual 100 in the current residual block and a secondary transform 110 applied on a first subgroup 112 of primary transform coefficients 114 of the primary transform. One or more Alternatively, if the height and width of the current residual block correspond to a second predetermined block dimension, the encoder 10 and decoder 20 may be configured to input to the transform candidate list a second transform from the subset of transforms, the multi-stage transform 106 whose domain corresponds to the height and width of the current residual block 84 and includes a primary transform 108 applied on the prediction residual 100 in the current residual block and a secondary transform 110 applied on a second subgroup 112 of the primary transform coefficients 114 of the primary transform. One or more In this example, the first One or more The first range of the linear transformation of the transformation of One or more Scaling the linear transformation of the transform onto a second range may not image the perimeter of the first subgroup in the first range onto the perimeter of the second subgroup in the second range.

[0125] Given the relative complexity overhead of a transform stage with respect to its total transform block size (e.g. by counting the number of multiplications per pixel in the entire transform block), the structure of the transform stages should be further constrained by their relative size. Here, the worst case corresponds to a first-order only multi-stage transform, where by definition (see Section 3) all coefficients (or all subgroups) of the previous transform stage are input to the next transform stage. Again, all methods from Section 3 are applicable.

[0126] In addition to the shape of the transform block, the prediction method used to obtain the residual block also gives insight into the characteristics of the residual signal and can therefore be used to select appropriate transform candidates.

[0127] The transform candidate list can also be adapted to the color channels by having different transform candidates defined for the chroma residual block and the luma residual block. A set of dedicated chroma transforms would provide more flexibility, but would also increase the amount of memory required to store additional transform cores. Thus, it is desirable to have a unified set of transform structures / cores to select candidates from. If the chroma residual block is associated with the luma transform block (e.g., belonging to the same CU, not dual-tree), it is also desirable to share the same transform index (which only needs to be signaled once) or to derive the selected chroma transform from the selected luma transform. To enable unique identification, the chroma candidate list must be carefully selected and adapted to the luma candidate list. Mainly, chroma subsampling is used in video coding, so the chroma residual block size, and therefore also the chroma transform size, is different from the selected luma transform size. To identify the chroma candidate, the characteristics of the selected luma transform must be matched to the transform class in the candidate list of the chroma block. This is easy, since transforms from the DCT / DST class are defined for different sizes. Thus, we can populate the candidate list of a chroma transform block with the same transform type (DCT / DST) as the associated luma block, but with a smaller size, and we know the selected luma candidate, and the selected transform is uniquely identifiable. For example, if there is only one candidate, Secondary TransformationThis property also allows for unique identification by allowing the candidate to have a dimensionality reduction property. Furthermore, the property of dimensionality reduction can be used to identify the associated chroma transform candidate. If the selected luma transform reduces dimensionality by setting predefined coefficient positions to zero, then the chroma transform can be identified by taking the candidate that forces the same relative positions to zero, where relative positions refer to, for example, the first half of a block, or the first row of a block, etc. If the same property is shared among the same number of candidates, then in the luma and chroma lists, the order in the list and the selected luma transform index can be used to identify the chroma transform.

[0128] It can be useful to add certain transforms to the candidate list only if the quantization parameter is within a certain range. This is especially useful for transforms that involve dimensionality reduction, but specialized transforms can also be trained depending on the quantization parameter.

[0129] Given these characteristics, the transformation candidate list can be derived by construction rules or obtained by table lookup, where having a lookup table of transformation candidate lists for all possible characteristic combinations is the most flexible and therefore preferred variant.

[0130] The following section provides rules for constructing the conversion candidate list.

[0131] <4.1.1 Large residual blocks> Large residual blocks, whose width or height is greater than 64, may be transformed using a DCT-II with high frequency zeroing as described above in section 2.3.

[0132] <4.1.2 Inter-predicted residual blocks> For inter-predicted residual blocks, the AMT conversion and signaling scheme may be used, i.e., the candidate list may include: {[DCT-II,DCT-II],[DST-VII,DST-VII],[DST-VII,DCT-VIII],[DCT-VIII,DST-VII],[DCT-VIII,DCT-VIII]} [Vertical transformation, horizontal transformation].

[0133] <4.1.3 Intra-NN predicted residual block> <Intra-NN conversion> For Intra-NN (Neural Networks), a dedicated set of transforms is provided for use with Intra-NN predicted blocks, hereafter referred to as Intra-NN transforms.

[0134] 7, an encoder 10 (similar to that described above with reference to FIG. 1) and a decoder 20 (similar to that described above with reference to FIG. 2) may be provided for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding. The encoder 10 and the decoder 20 support a set of transforms 94 for block-based residual coding, and within the set 94, One or moreDifferent transforms may be available. The encoder 10 and decoder 20 are configured to predict the intra predicted block 80 by applying the spatial neighborhood 90 of the intra predicted block 80 on a selected neural network from among the plurality of neural networks 92 to obtain a prediction 104 of the intra predicted block 80. The encoder 10 and decoder 20 may determine a set of selectable transforms 98 from the transform set 94 for a residual block 84 covering the intra predicted block 80 based on the selected neural network and a block size 96 of the residual block 84. The encoder 10 may further encode a prediction residual 100 associated with a prediction of the intra predicted block 80 in the residual block 84 using one selected transform 102 from the set of selectable transforms 98 and signal the selected transform in the data stream 14. The decoder 20 may further encode a prediction residual 100 associated with a prediction of the intra predicted block 80 in the residual block 84 using one selected transform 102 from the set of selectable transforms 98 and signal the selected transform in the data stream 14. 102 , the predictive residual 100 associated with a prediction 104 of an intra-predicted block 80 within residual block 84 may be decoded from data stream 14 by deriving

[0135] Thus, a selection may be made from set 94 One or more A transformation set 94 may be provided that includes a selectable set 98 of One or more The selectable set 98 may include a number of transforms 102, and one transform 102 may be selected from the selectable set 98. The encoder 10 may encode the prediction residual 100 using the one selected transform 102. The decoder 20 may decode the prediction residual 100 using the one selected transform 102.

[0136] These transforms are trained on specific properties of the prediction residuals, conditioned on different intra-NN modes. The set of intra-NN transforms 98 includes a first-order only transform for small block sizes (e.g., block sizes below a predetermined threshold) and a first-order + second-order transform for larger block sizes (e.g., block sizes above a predetermined threshold).

[0137] 7-10 show a residual block 84 including a number of samples 105. Each of the residual blocks 84 may include a prediction residual 100 that may be encoded and decoded. One or more 10, for example, a primary-only transform 118 (FIG. 10), or a multi-stage transform 106 (FIG. 8) including a primary transform 108 followed by a secondary transform 110.

[0138] The samples 105 may be subjected to a linear transform 108 such that the samples 105 may be transformed from the spatial domain to the transform domain. As a result, the transform block is One or more A primary subgroup 112 of primary transform coefficients 114 may include a plurality of primary transform coefficients 114 arranged together. For example, as shown in FIG. 9, the primary transform 108 may include a plurality of primary transform coefficients 114 of the residual block 84. One or more While the rows of may be transformed from the spatial domain to the transform domain, the columns of residual block 84 are not transformed and therefore remain in the spatial domain.

[0139] In some examples, the primary transform coefficients 114 may be further transformed by the secondary transform 110 such that the primary transform coefficients 114 may be transformed into a respective secondary transform domain. As a result, the re-transformed (by the secondary transform 110) block 117 may be One or more 9, the secondary transform 110 may include a plurality of secondary transform coefficients 134 of a block. A plurality of secondary transform coefficients 134 may be arranged together in a secondary subgroup 138 of secondary transform coefficients 134. For example, as shown in FIG. 9, the secondary transform 110 may include a plurality of secondary transform coefficients 134 of a block. One or moreThe columns of the block may be transformed from the spatial domain to the transform domain. As mentioned above, the rows of the block may already be transformed by the primary transform 108. As a result, the rows and columns may be transformed by a multi-stage transform, i.e., by applying a primary transform followed by a secondary transform. However, the transform coefficients are not limited to rows and columns, and this is just one non-limiting example.

[0140] As previously mentioned, a predefined set of transforms 94 may be available from which the encoder 10 and decoder 20 may select. For example, with reference to Figures 7 and 8, the encoder 10 and decoder 20 may select, for block sizes above a predefined threshold, a first number of transforms in the set of selectable transforms 98 each including a primary transform 108 applied on a prediction residual 100 and a secondary transform 110 applied on a subgroup 112 of primary transform coefficients 114 of the primary transform 108, the secondary transform 110 being non-separable and dependent on a selected neural network. One or more The subgroup 112 may be configured to determine the selectable set 98 such that the multi-stage transform 106 of FIG.

[0141] For example, for each block size in S={4,8,16,32}×{4,8,16,32} and each intra-NN prediction mode, there is a set of four intra-NN transforms. More generally, the number of multi-stage transforms can be from 2 to 6, inclusive. For block sizes other than the sizes in S, an intra-NN transform of derived size s' is used. Size s' is obtained by taking the size in S that has the smallest vertical and horizontal dimensions in S, respectively, but at least the size of the block.

[0142] 10, in addition or alternatively, the encoder 10 and decoder 20 may add a second number of transforms to the set of selectable transforms 98 for block sizes below a predetermined threshold. One or moreThe multi-stage transform 106 of the first number (FIG. 8) and the non-separable transform 118 of the second number (FIG. 10) may be equal.

[0143] Intra-NN transforms with size less than or equal to 8 for both block height (H) and block width (W) are defined as linear-only transforms. For complexity reasons, large transforms are defined as linear+linear transforms with a low-complexity separable linear transform and a secondary transform applied only to the primary coefficient sub-block containing the first coefficients and having size min(8,W)×min(8,H).

[0144] <Version 1> 9, in addition to or as an alternative to the intra-NN transform, the encoder 10 and decoder 20 may add a third number (e.g., 1 or 5) of selectable transforms to the set 98 of selectable transforms. One or more The selectable set 98 may be configured to determine a separable two-dimensional transform 116 such that the separable two-dimensional transform 116 for the intra-NN predicted residual block is further input, i.e., the transform candidate list for the intra-NN predicted residual block may include five separable (first order only) transforms from the following DCT / DST family: {[DCT-II,DCT-II],[DST-VII,DST-VII],[DST-VII,DCT-VIII],[DCT-VIII,DST-VII],[DCT-VIII,DCT-VIII]} [Vertical transformation, horizontal transformation].

[0145] <Version 2> In addition to or as an alternative to the intra-NN transform, the transform candidate list for the intra-NN predicted residual block may include a separable (first order only) {DCT-II, DCT-II} transform.

[0146] <Version 3> Intra-NN prediction can be defined such that the prediction is given in the transform domain representation rather than the spatial domain representation, and thus the prediction residual is given as the coefficients of a particular transform stage. In this case, the first stage of the effectively applied transform is already explicitly defined by the prediction used.

[0147] For example, the encoder 10 and the decoder 20 may be configured to perform the prediction such that the prediction of the intra predicted block 80 is obtained in the domain 118 of the primary transform 108. The encoder 10 and the decoder 20 may further configure the set of selectable transforms 98 to include a fourth number of transforms each including the primary transform 108 and a secondary transform 110 applied on a subgroup 112 of the primary transform coefficients 114 of the primary transform 108, the secondary transform 110 being non-separable and dependent on the selected neural network. One or more The multi-stage transform 106 may be further configured to determine a selectable set 98 into which the multi-stage transform 106 may be input.

[0148] As an example, the prediction residuals may be given as coefficients in the DCT-II domain, i.e. the primary transform 108 can be a separable two-dimensional transform 116 (FIG. 9). In the case of a primary+secondary transform, the primary transform is considered as the transform defined by the prediction mode (here DCT-II). The secondary transform is thus defined on a subset of the DCT-II coefficients, here on the subset of coefficients with the lowest frequency of size min(8,W)×min(8,H), i.e. the subgroup 112 represents the low-frequency part of the primary transform 108.

[0149] In this version, all transform candidates of an intra-NN predicted residual block have the same first transform stage explicitly given by the intra-mode. In addition to the four (e.g., 2 to 6) trained intra-NN transforms built on a given transform stage, the explicitly given transform itself is also added to the candidate list as a first-order only transform (here DCT-II for all intra-NN modes), i.e., the first-order transform 108 can be further entered into the set of selectable transforms 98.

[0150] Another example of an encoder 10 and a decoder 20 for intra-NN transform is shown in Fig. 11. This example is similar to the example described above with reference to Figs. 7 and 10, which therefore also serve to explain the following examples.

[0151] The encoder 10 of this example performs a neural network NN to obtain a prediction 104 of the intra predicted block 80 in the domain 118 of the linear transform 108 (FIGS. 10 and 11). # The encoder 10 may be configured to predict an intra-predicted block 80 by applying a selected transform 102 onto a residual block 84. The encoder 10 may be configured to encode a prediction residual 100 associated with a prediction 104 of the intra-predicted block 80 in a residual block 84 using a selected transform 102 from a set of selectable transforms 98, and to signal the selected transform 102 in the data stream 14. In this example, the set of selectable transforms 98 is input with at least one multi-stage transform 106 including a primary transform 108 and a secondary transform 110 applied onto a subgroup 112 of primary transform coefficients 114 of the primary transform 108 (FIG. 8).

[0152] The corresponding decoder 20 performs a neural network NN , which computes the spatial neighborhood 90 of the intra-predicted block 80 to obtain a prediction 104 of the intra-predicted block 80 in the domain 118 of the linear transform 108 (FIGS. 10 and 11). #The decoder 20 may be configured to predict the intra predicted block 80 by applying a selected transform 102 on the data stream 14 to predict the intra predicted block 80 in the residual block 84. The decoder 20 may be configured to decode a prediction residual 100 associated with a prediction 104 of the intra predicted block 80 in the residual block 84 from the data stream 14 by deriving a selected transform 102 from the data stream 14 in another domain 119 of a selected transform 102 from a set of selectable transforms 98. The set of selectable transforms 98 has as input at least one multi-stage transform 106 including a primary transform 108 and a secondary transform 110 applied on a subgroup 112 ( FIG. 8 ) of primary transform coefficients 114 of the primary transform 108.

[0153] The encoder 10 and decoder 20 may be configured to input to the set of selectable transforms 98 a number of multi-stage transforms 106, each of which includes a primary transform 108 and a secondary transform 110 applied on a subgroup 112 of primary transform coefficients 114 of the primary transform 108, the secondary transforms 110 being non-separable.

[0154] The number of multi-stage transforms may be from 2 to 6, inclusive. Additionally or alternatively, the primary transform 108 may be a separable two-dimensional transform. Additionally or alternatively, the subgroup 112 may represent a low frequency portion of the primary transform.

[0155] The encoder 10 and decoder 20 may be configured to make a determination of the selectable set 98 such that the primary transform is further input to the set 98 of selectable transforms.

[0156] <4.1.4 Intra predicted residual blocks (DC, Planar, Angular)> For intra-predicted residual blocks, a dedicated set of transforms is defined, trained on the specific properties of the residual signal given its actually applied intra-mode and block size. These transforms include linear-only transforms and also linear+quadratic transforms, where due to their complexity, the linear transform has a particularly low-complexity implementation.

[0157] Referring again to Fig. 7, an encoder 10 (similar to that described above with reference to Fig. 1) and a decoder 20 (similar to that described above with reference to Fig. 2) may be provided for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding. The encoder 10 and the decoder 20 may be configured to predict an intra-predicted block 80 based on a spatial neighborhood 90 of the intra-predicted block 80 using a selected one from a set of intra-prediction modes 120 to obtain a prediction 104 of the intra-predicted block 80. The encoder 10 and the decoder 20 may be further configured to determine a set 98 of selectable transforms for the residual block 84 covering the intra-predicted block 80 from a set of transforms 94 based on the selected intra-prediction mode and a block size 96 of the residual block 84. The encoder 10 may be further configured to encode a prediction residual 100 associated with a prediction 104 of an intra-predicted block 80 in the residual block 84 using a selected transform 102 from the set of selectable transforms 98 and signal the selected transform 102 in the data stream 14. Furthermore, the decoder 20 may be configured to decode a prediction residual 100 associated with a prediction 104 of an intra-predicted block 80 in the residual block 84 from the data stream 14 by deriving the selected transform 102 from the data stream (14) in a domain of the selected transform 102 from the set of selectable transforms 98.

[0158] Among the primary transforms are those of the DCT / DST family defined by AMT, which can be combined with transforms defined by NSST to construct primary-only and primary+secondary transforms, respectively. In other words, the primary transform 108 depends on the selected intra-prediction mode and can be a separable two-dimensional transform 116. Thus, referring again to FIG. 8, the encoder 10 and the decoder 20 determine in the set of selectable transforms 98, for a first predetermined set of block sizes, a number of transforms each including a primary transform 108 applied on the prediction residual 100 and a secondary transform 110 applied on a subgroup 112 of primary transform coefficients 114 of the primary transform 108, the secondary transform 110 being non-separable and depending on the selected intra-prediction mode. One or more The multi-stage transform 106 may be configured to determine the selectable set 98 such that the multi-stage transform 106 is input.

[0159] Additional transforms may be trained, including fully connected transform stages of up to dimension 64, and intra-mode dependent partitioning of the transform stage coefficients, particularly the primary transform coefficients (e.g., subgroup 112 may depend on the selected intra-prediction mode). Which of these transforms is suitable for the residual block depends on its already decoded syntax elements, and is given by a look-up table with construction rules for the candidate list below.

[0160] Thus, referring back to FIG. 10, the encoder 10 and decoder 20 may add a second number of transforms to the set of selectable transforms 98 for a second predetermined set of block sizes that is smaller than the first predetermined set of block sizes. One or more The selectable set 98 may be determined such that the non-separable transform 118 of

[0161] When constructing a multi-stage transform with complexity constraints that require the subgroups to be rectangular, dense and low-dimensional, one is still free to choose the shape of the subgroups. To be precise, the shape can be quadratic or have a dominant dimension in the vertical or horizontal direction. The shape depends on the selected linear transform and the actual intra-prediction mode, if either of them is preferred. For example, if DCT-II is the linear transform and the residual block is predicted with one of the modes in the horizontal direction, the residual linear coefficients are expected to be concentrated on the left side of the block, so the subgroups can be chosen to be taller than they are wide. The opposite can be chosen for the vertical intra direction.

[0162] <Luma component> <Version 1> A look-up table may be used to generate the transform candidate list for the intra-predicted luma residual block. The table holds transform candidate lists for combinations of sizes in S3={64×64}+{4,8,16,32}×{4,8,16,32}+{4×12,12×4} and intra mode indices in I0={0,...,34}. As mentioned above, the encoder 10 and the decoder 20 may be configured to determine a set 98 of selectable transforms for the residual block 84 that cover the intra-predicted block 80 from among the transform sets 94 based on the selected intra-prediction mode and the block size 96 of the residual block 84.

[0163] The actual rules for obtaining the candidate list of residual blocks are as follows: If the intra mode is specified by one of the intra directions in {DC,Planar}+33 directional modes}, then this intra direction is used as the intra mode index into the LUT. If the intra mode is specified by one of the intra directions in {DC,Planar}+65 directional modes}, the intra mode index into the LUT is obtained by the following mapping:

[0164] [Table 1]

[0165] For residual blocks with sizes in S3, this size is used as input to the LUT, otherwise a second lookup table is used, which defines a fixed set of transform candidates, one for each intra-mode index in I0.

[0166] Thus, the encoder 10 and decoder 20 may be configured such that the manner in which the secondary transform 110 depends on the selected intra-prediction mode includes intra-prediction mode aggregation, such as quantization of the intra-prediction mode index and quantization of the intra-prediction direction of the angular intra-prediction modes in the set of intra-prediction modes 120.

[0167] Additionally or alternatively, the encoder 10 and decoder 20 may be configured such that the manner in which the primary transform 108 depends on the selected intra-prediction mode includes intra-prediction mode aggregation, such as quantization of the intra-prediction mode index or quantization of the intra-prediction direction of the angular intra-prediction modes in the set of intra-prediction modes 120.

[0168] <Version 2> The difference from version 1 is that the number of transformation candidates for residual blocks with sizes not in S3 is Derivation Here, the size s' is used to address the lookup table, where s' is the smallest size in S3 whose each dimension is greater than or equal to the corresponding dimension of s.

[0169] <Chroma component (no chroma dual tree)> <Version 1> The transform candidates for the chroma residual blocks are taken from the same transform set as the luma transform candidates. Here, each transform key can define two separate chroma transform candidates. If the prediction mode of the chroma component is equal to the luma mode, the first candidate is taken. Otherwise, the second candidate is taken.

[0170] <Version 2> The transform candidates for the chroma residual blocks are taken from the same set of transforms as the luma transform candidates, where each transform key may define two separate chroma transform candidates. If so, both transforms are added to the chroma candidate list.

[0171] <4.1.5 Addition / replacement and reordering of candidates depending on transformation decisions of neighboring transformation blocks> In addition to transform candidates derived from characteristics of the transform block itself, such as prediction mode, block size, etc., candidates can also be derived from neighboring transform blocks. Some examples are discussed below.

[0172] For example, FIG. 12 shows an image 12 that is partitioned into a current residual block 84 and spatially neighboring residual blocks 84′, 84″ that may be directly adjacent to the current residual block 84 or may be located spatially distant from the current residual block 84. Assuming that the neighboring residual blocks 84′, 84″ are similar in various properties such as residual energy and signal structure, this similarity can be exploited by applying information of the selected neighboring transforms to the candidate residual blocks of the current block 84. Derivation It can be utilized by including it in the process.

[0173] Referring to FIG. 12, an encoder 10 (similar to that described above with reference to FIG. 1) and a decoder 20 (similar to that described above with reference to FIG. 2) may be provided for encoding / decoding an image 12 into / from a data stream 14 using block-based prediction and block-based residual coding.

[0174] The encoder 10 and the decoder 20 may be configured to determine 122 a list of selectable transforms 98 for the current residual block 84 from among the transform set 94 based on the reference transforms selected for the neighboring residual blocks 84′, 84″. The encoder 10 may be configured to encode the prediction residual 100 in the residual block 84 using one selected transform 102 from among the list of selectable transforms 98 and signal the selected transform 102 in the data stream 14. Furthermore, the decoder 20 may be configured to decode the prediction residual 100 in the residual block 84 by deriving the selected transform 102 from the data stream 14 in the domain of the one selected transform 102 from the list of selectable transforms 98.

[0175] In general, if a neighboring transform block 84', 84'' has the same size and prediction mode as the current block 84, then the selected transform of the neighboring transform block 84', 84'' is also a candidate for the current block 84 if the list is constructed as described above. In such a case, the current candidate list may be reordered such that a previously selected candidate is placed in one of the first positions in the candidate list. Thus, the encoder 10 and decoder 20 may be configured to order 122 the selectable list 98 using the reference transform.

[0176] For example, the encoder 10 and decoder 20 may sort the selectable list 98 by the reference transform or a transform corresponding to the reference transform (e.g., one stage signaled by a high transform index 124, and thus a highly specialized transform, analytical transform type for neighboring blocks of different sizes) with a rank ("n #Alternatively, the encoder 10 and decoder 20 may be configured to order 122 the selectable list 98 by placing the reference transform or a transform corresponding to the reference transform in the rank associated with the second most likely selection (# of "n" (e.g., in the first position, or in the position associated with the lowest index value of the index 124, or in the position associated with the index value associated with the shortest VLC). # ")122.

[0177] This allows for improved signaling of candidates and also enables an efficient fast search mode for testing transform candidates in the encoder by skipping testing of subsequent candidates.

[0178] When transformation decisions of multiple neighboring blocks 84', 84" are available, the actual position of each candidate in the current list can be conditioned on how often it was selected in the neighboring blocks 84', 84" by placing the more frequently selected candidates at the top of the list. Since the first transformation is often the most likely, even when conditioned on neighboring decisions, it can be beneficial to leave the first candidate in the list unchanged and place the most frequently selected candidate in the second position.

[0179] Thus, the encoder 10 and decoder 20 may be configured to determine 122 a list 98 of selectable transforms for the current residual block 84 from among the transform set 94 based on the reference transforms selected for the set of neighboring residual blocks 84', 84''.

[0180] Further, the encoder 10 and decoder 20 may be configured to order 122 the selectable list 98 by arranging the reference transforms relative to each other in an order corresponding to the frequency with which the reference transforms are selected for the set of neighboring residual blocks 84', 84''.

[0181] If the prediction modes or block sizes of the neighboring residual blocks 84', 84'' are different, the selected neighboring transform is not necessarily already a candidate for the current block 84. In such a case, the selected transform may be added to the current transform list if it is also defined for the current block size. Thus, the encoder 10 and decoder 20 may be configured to input 122 the reference transform to the selectable list 98.

[0182] If a transform is actually added to the candidate list, it should be conditional on the similarity of the selected prediction modes of both blocks. For example, if both blocks are predicted using a directional mode and their directional indices differ by more than three, and the selected transform of the neighboring block is a transform specifically trained for that mode, then this transform is unlikely to be a promising candidate for the current block. On the other hand, if both prediction modes are similar or a generic transform (such as DCT or DST) is selected, it can also be added to the current candidate list. If both blocks differ only in their size and the known selected transform is from an analytically defined transform, such as one from the DCT / DST family, then the corresponding transform for the current block size can be directly derived and added to the candidate list.

[0183] Thus, the encoder 10 and the decoder 20 are connected to the neighboring blocks 84', 84'' and the current residual block 84. One or more are sufficiently similar in certain characteristics and / or the reference transformations are separable Spectral decomposition transform If so, the input may be configured to be performed. One or more The characteristics of the intra prediction mode are, for example, One or more may include.

[0184] Additionally or alternatively, the reference transform may be an analytically definable separable transform (such as a DCT, DST, FFT, etc.). Spectral decomposition transform, and the encoder 10 and decoder 20 may be configured to input 122 to the selectable list 98 another transform derived by adapting the reference transform to the block size 96 of the current residual block 84.

[0185] Even if the selected transformation of a neighboring block is not a candidate in the current list, knowing that transformation can help predict the best candidate for the current list, and thus the ordering of candidates can be conditioned on that transformation. For example, if the linear transform of a neighboring block is from the DCT / DST family, then all candidates with a corresponding linear transform of the current size can be put at the front of the candidate list, as described above.

[0186] Furthermore, if we assume that the energies of neighboring blocks are correlated, then the number of significant coefficients in the current block is likely to be high when there are a large number of significant coefficients in the neighboring residual blocks. The same correlation can be assumed in the inverse case of only a few coefficients.

[0187] Thus, the encoder 10 and decoder 20 may compile the selectable list 98 as follows: One or more and / or by ranking 122 the transforms in the set of selectable transforms 98 according to their similarity to a reference transform in a particular characteristic of One or more The list of selectable transformations 98 may be configured to exclude transformations that exceed a certain degree of dissimilarity to the reference transformation in a particular characteristic of the One or more The properties of the linear transformation One or more (e.g., equivalence of linear transformations is required).

[0188] This information can be used to reorder the candidates or even remove less likely transformations from the list. For example, if there is a transform candidate with a secondary transformation defined on a subgroup of k coefficients, and there are many fewer than k significant coefficients in the neighboring residual blocks 84', 84'', this candidate can be removed or placed in the last position. Also, if only a few coefficients are expected to be significant, it is unlikely that there will be any benefit from having transform candidates that share the same primary transformation but differ only in their secondary transformations. If there are such transformations among the candidates, then the list can be reduced in such cases by keeping only the first of these transformations, or even by replacing these transformations with the corresponding primary-only transformations.

[0189] According to one example, the encoder 10 One or more The encoder 10 may be configured to determine 122 a list 98 of selectable transforms for the current residual block from among the transform set based on the reference transform selected for the neighboring residual block in a manner that depends on the number, or sum, of significant transform coefficients in the neighboring residual blocks 84′, 84″ of the current residual block. If the number or sum is greater than a certain threshold, the list 98 of selectable transforms for the current residual block is populated with at least one multi-stage transform including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform. If the number or sum is less than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modification of the multi-stage transform in which the primary transform coefficients beyond the subgroup are zeroed in addition to or instead of the at least one multi-stage transform. In either case, the encoder 10 may be configured to code the prediction residual in the residual block using one selected transform from the list of selectable transforms and to signal the selected transform in the data stream.

[0190] The corresponding decoder 20 is One or moreThe decoder 20 may be configured to determine 122 a list 98 of selectable transforms for the current residual block from among the transform set based on the reference transform selected for the neighboring residual block 84′, 84″ in a manner that depends on the number of significant transform coefficients in the neighboring residual block 84′, 84″, or their sum. If the number or sum is greater than a certain threshold, the list 98 of selectable transforms for the current residual block is populated with at least one multi-stage transform including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform. If the number or sum is less than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modification of the multi-stage transform in which the primary transform coefficients beyond the subgroup are zeroed in addition to or instead of the at least one multi-stage transform. In either case, the decoder 20 may be further configured to decode the prediction residual in the residual block by deriving the selected transform from the data stream in the domain of one selected transform from the list of selectable transforms.

[0191] Additionally or alternatively, the encoder 10 and decoder 20 may derive the selectable list 98 from the neighboring blocks 84'. One or more and / or by ranking the transformations in the set of selectable transformations 98 according to certain characteristics of the neighboring blocks 84′. One or more The list of selectable transformations 98 may be determined in a manner that excludes transformations from the selectable list 98 depending on certain characteristics of the One or more The property of One or more may include.

[0192] Additionally or alternatively, One or moreThe characteristic of the method is to rank the less likely transforms among a certain number of significant transform coefficients, or to exclude transforms that are of the multi-stage type having a subgroup 112 of primary transform coefficients that undergo a secondary transform if the number of significant transform coefficients falls below a certain threshold (e.g., depending on the number of coefficients in the subgroup 112). One or more may also include

[0193] <4.1.6 Providing transformation candidates adapted to quantization parameters> The transforms can be optimized for different reconstruction points of the quantization stage, and therefore for different quantization parameters (QPs). When a QP-adaptive transform set is defined for a prediction mode, the candidate list is constructed by one of the following rules:

[0194] If a transformation is defined for the current QP, this transformation is added to the candidate list. The transformations defined for the QP closest to the current QP are added to the candidate list.

[0195] Two transformations are added to the candidate list: first, the transformation defined for the next smaller QP than the current QP, and second, the transformation defined for the next larger QP than the current QP.

[0196] Thus, referring to FIG. 7, the encoder 10 and decoder 20 may be configured such that the quantization parameters of the residual block 84 are One or more If the answer is 'yes', the encoder 10 and decoder 20 may be configured to check whether the selected transform is included in the set of selectable transforms 98, One or more associated with a given set of quantization parameters One or more A selectable set 98 may be determined so that a set of predetermined transformations of the above may be further input.

[0197] Additionally or alternatively, the encoder 10 and decoder 20 may be configured to determine whether the quantization parameters of the intra-predicted blocks are One or moreIf the answer is 'yes', the encoder 10 and decoder 20 may be configured to check whether the selected transform is included in the set of selectable transforms 98, One or more associated with a given set of quantization parameters One or more A selectable set 98 may be determined so that a set of predetermined transformations of the above may be further input.

[0198] Additionally or alternatively, the encoder 10 and decoder 20 may be configured to determine whether the quantization parameter of the residual block 84 is One or more If the answer is 'yes', the encoder 10 and decoder 20 may be configured to check whether the selected transform is included in the set of selectable transforms 98, One or more associated with a given set of quantization parameters One or more A selectable set 98 may be determined so that a set of predetermined transformations of the above may be further input.

[0199] Additionally or alternatively, the encoder 10 and the decoder 20 may be configured to determine whether the quantization parameters of the current block are One or more If the answer is 'yes', the encoder 10 and decoder 20 may be configured to check whether the selected transform is included in the set of selectable transforms 98, One or more associated with a given set of quantization parameters One or more A selectable set 98 may be determined so that a set of predetermined transformations of the above may be further input.

[0200] <Adding / replacing candidates by transforming with dimensionality reduction> Depending on the transform block size and quantization parameter (QP), it may be beneficial to include transform candidates that explicitly reduce the dimensionality of the prediction residual, thus allowing efficient coding of coefficients and reduced complexity implementation. For each dimension, only these transforms lead to an error-free representation for a particular subclass of the residual signal. On the other hand, this is similar to quantizing the coefficients to zero, which may allow efficient coding of non-significant coefficient positions.

[0201] One of the following rules may apply:

[0202] If the candidate list contains at least one linear+quadratic transformation and the current QP is greater than a threshold Tqp, the linear+quadratic transformation is replaced by the corresponding transformation with dimensionality reduction properties.

[0203] These transforms are constructed from the corresponding original candidates in the list by forcing to zero those coefficients that do not belong to the subgroup of coefficients that are further transformed by the secondary transform.

[0204] If the candidate list contains at least one primary+secondary transformation and the current QP is greater than a threshold Tqp, then for each primary+secondary transformation, a transformation with dimensionality reduction is added to the list. These transformations are constructed from the original candidates in the list by forcing to zero those coefficients that do not belong to the subgroup of coefficients that are further transformed by the secondary transformation.

[0205] If the candidate list contains at least one primary+secondary transformation and the sum of significant coefficients of neighboring transform blocks is smaller than a threshold T, the primary+secondary transformation is replaced by a corresponding transformation with dimensionality reduction properties. These transformations are constructed from the corresponding original candidates in the list by forcing to zero those coefficients that do not belong to the subgroup of coefficients that are further transformed by the secondary transform.

[0206] For each primary+secondary transformation, a transformation with dimensionality reduction is added to the list if the candidate list contains at least one primary+secondary transformation and the sum of significant coefficients of neighboring transformation blocks is smaller than a threshold T. These transformations are constructed from the original candidates in the list by forcing to zero those coefficients that do not belong to the subgroup of coefficients that are further transformed by the secondary transformation.

[0207] According to an example, the encoder 10 may be configured to encode the image 12 into a data stream 14 using block-based prediction and block-based residual coding, the encoder supporting a transform set 94 for the block-based residual coding. The encoder 10 may be configured to determine a list of selectable transforms 98 for a current residual block 84 from among the transform set in a manner dependent on the quantization parameter of the current block such that, if the quantization parameter corresponds to a quantization finer than a certain threshold, the list of selectable transforms 98 for the current residual block is populated with at least one multi-stage transform 106 including a primary transform 108 applied on a prediction residual 100 in the current residual block and a secondary transform 110 applied on a subgroup 112 of primary transform coefficients 114 of the primary transform. If the quantization parameter corresponds to a quantization coarser than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modified multi-stage transform in which primary transform coefficients 114 beyond the subgroup 112 are zeroed in addition to or instead of the at least one multi-stage transform 106. In either case, the encoder 10 may be configured to encode the prediction residual 100 in the residual block 84 using a selected transform 102 from the list of selectable transforms 98 and signal the selected transform in the data stream 14.

[0208] Further, a corresponding decoder 20 is provided for decoding the image 12 from the data stream 14 using block-based prediction and block-based residual decoding, the decoder may support a transform set 94 for block-based residual decoding. The decoder 20 may be configured to determine a list of selectable transforms 98 for a current residual block 84 from among the transform set in a manner dependent on the quantization parameter of the current block such that, if the quantization parameter corresponds to a quantization finer than a certain threshold, the list of selectable transforms 98 for the current residual block is populated with at least one multi-stage transform 106 including a primary transform 108 applied on a prediction residual 100 in the current residual block and a secondary transform 110 applied on a subgroup 112 of primary transform coefficients 114 of the primary transform. If the quantization parameter corresponds to a quantization coarser than a certain threshold, the list of selectable transforms for the current residual block is populated with at least one modification of a multi-stage transform in which primary transform coefficients 114 beyond a subgroup 112 are zeroed in addition to or instead of the at least one multi-stage transform 106. In either case, the decoder 20 extracts the selected transform from the data stream 14 in the domain of one selected transform 102 from the list of selectable transforms 98. 102 , may be configured to decode the prediction residual 100 in residual block 84 by deriving

[0209] <4.2 Reducing the list of conversion candidates> During the decoding process of the residual coefficients, the transform candidate list can be subsequently reduced if newly acquired information does not satisfy the constraints of a particular transform in the list. This helps reduce the coding cost of the selected candidate and also allows conditioning of subsequent syntax elements on certain characteristics of the remaining transform candidates, thus increasing coding efficiency. It may be beneficial to increase the number of syntax elements affected by the candidate list reduction by explicitly coding information about the selected transform in advance.

[0210] With reference to FIG. 14, an encoder 10 and a decoder 20 may be provided for encoding / decoding an image 12 into / from a data stream 14 using block-based prediction and block-based residual coding. The encoder 10 and the decoder 20 may support a transform set 94 for block-based residual coding in a manner similar to that described above. However, according to this example, the encoder 10 and the decoder 20 may be configured to determine 130 a list 98 of selectable transforms for a current residual block from among the transform set 94. The encoder 10 may be configured to encode transform coefficient data 126 representing a prediction residual 100 in the residual block 84 into the data stream 14 using one selected transform 102 from among the list 98 of selectable transforms. The encoder 10 may be further configured to signal 128 the selected transform 102 in the data stream 14 and / or to determine 130 the selectable list 98 in a manner dependent on the transform coefficient data 126 or a portion of the transform coefficient data 126. Further, the decoder 20 may be configured to decode transform coefficient data 126 representing the prediction residual 100 in the residual block 84 from the data stream 14 using a selected transform 102 from the list of selectable transforms 98. The decoder 20 may be configured to decode the transform coefficient data 126 representing the prediction residual 100 in the residual block 84 from the data stream 14 using a selected transform 102 from the list of selectable transforms 98. 102 and / or determine 130 the selectable list 98 in a manner that depends on the transform coefficient data 126 or a portion of the transform coefficient data 126.

[0211] For example, the encoder 10 and decoder 20 may be configured to determine 130 the list of selectable transforms 98 by inputting into the selectable list 98 a number of transforms from the transform set 94 that depend on the transform coefficient data 126, or a portion of the transform coefficient data, and to signal 128 the selected transforms 102 in the data stream 14 in such a way that the lower the coding rate consumed in the data stream 14 for signaling the selected transforms 102, the lower their number.

[0212] Additionally or alternatively, the encoder 10 and decoder 20 may be configured to determine 130 the list of selectable transforms 98 by ordering the list of selectable transforms 98 depending on the transform coefficient data 126, or a portion of the transform coefficient data 126, and to signal 128 the selected transform 102 in the data stream 14 using variable length codes whose code words are assigned to transforms of the list of selectable transforms 98 according to the order of the transforms in the list of selectable transforms 98.

[0213] 15 shows another example of an encoder 10 and decoder 20 for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding, both of which support a transform set for block-based residual coding. The encoder 10 and decoder 20 of this example may be configured to determine 130 a list 98 of selectable transforms for a current residual block from among the transform set. The encoder 10 may be configured to signal 128 the selected transform in the data stream 14 and to encode 140 transform coefficients representing the prediction residual in the residual block using one selected transform from the list of selectable transforms in a manner that depends on the selected transform 102 or the list of selectable transforms 98. Furthermore, the decoder 20 may be further configured to derive 128 a selected transform from the data stream 14 and decode 140 transform coefficients representing the prediction residual in the residual block from the data stream 14 using a selected transform from a list of selectable transforms in a manner dependent on the selected transform 102 or the list of selectable transforms 98.

[0214] For example, Figure 14 shows a block 114 having a plurality of values ​​of transform coefficients 134 of a selected transform, whose values ​​of transform coefficients 134 may be traversed in a predetermined scan order 136. is not important One exemplary significant (ie, non-zero-valued) transform coefficient position 132, which may be a transform coefficient position, is marked with an 'x' in FIG.

[0215] A possible constraint on the transform candidate list may include that at least a certain number of significant (i.e., non-zero-valued) coefficients 134 are required to compensate for the additional signaling cost of the transform candidates. This constraint can be met by pruning the candidate list after the exact number of significant coefficients 134 is obtained, but it is also possible to remove the transform candidate indicated by the position of the last significant coefficient 132. This would likely increase the number of syntax elements affected by the list reduction, since the last position is usually pre-coded.

[0216] According to one example (see FIGS. 13 and 14), the encoder 10 and decoder 20 may be arranged such that the transform coefficient data 126 includes indications of significant transform coefficient positions 132 of the selected transform 102 and values ​​of transform coefficients 134 of the selected transform as traversed from the significant transform coefficient positions 132 forward by a predetermined scan order 136 that sequentially traverses the coefficient positions of the selected transform, and the determination 130 of the selectable list 98 depends on the indications, i.e., the indications include "significant" values ​​of the transform coefficient data 126. position ".

[0217] According to another example, the encoder 10 and decoder 20 may be configured to count the number of significant transform coefficients 134 of the selected transform 102 and to make the determination 130 of the selectable list 98 dependent on this count.

[0218] For multi-stage transform candidates that only apply further transform stages to a certain subset of their primary transform coefficients, a constraint can be defined based on that subset, including that this subset must contain at least a certain number of significant positions. This constraint may possibly already be violated by a maximum number of significant positions, possibly known after decoding the last significant position or as soon as the exact number of significant positions is known. As soon as a constraint is violated, the corresponding transform candidate is removed from the list and the subsequent decoding process can benefit from a reduced set of transform candidates.

[0219] Thus, the encoder 10 and decoder 20 may be configured to evaluate the transform coefficient data 126 as to whether the number of significant transform coefficients 134 of the selected transform 102 within a subgroup 138 of transform coefficients 134 of the selected transform 102 (FIG. 13) exceeds a particular threshold, and making a decision 130 of the selectable list 98 may depend on the count, i.e., the number of significant transform coefficients 134, as follows: 1) if the number is greater than a certain threshold, then adding to the list of selectable transforms 98 for the current residual block 84 at least one multi-stage transform including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on the subgroup 112 of the primary transform coefficients 114 of the primary transform 106 to result in a subgroup 138 of transform coefficients of the selected transform 102; 108 is entered. 2) if the number is less than a certain threshold, then the list of selectable transforms for the current residual block does not include any multi-stage transforms 106 that include a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform.

[0220] Furthermore, the reduction of the chroma candidate list can depend on the actual transform decision of the luma component. Different candidate lists (e.g., luma and chroma) can also share one transform index, in which case the second list is reduced to the selected transform given by the transform index already decoded for the first list.

[0221] Exemplary embodiments In the following, some exemplary embodiments are presented to embody the above general description.

[0222] If the number of significant coefficients is less than a threshold T0 (eg, 3), the list of transform candidates is reduced to its first two entries.

[0223] If the number of significant coefficients is less than a threshold T1 (eg, 3), the list of transformation candidates is reduced to its first entry.

[0224] Luma and chroma share the same coding structure, and all T U If the total number of significant coefficients (of all TUs in the CU) is less than the threshold T0, the candidate lists of all TUs in the CU are reduced to their first entries.

[0225] If the total number of significant coefficients (of all TUs in a CU), including luma and chroma components, is less than a threshold T0, the candidate list of all TUs in that CU is reduced to its first two entries. Otherwise, if the total number of significant luma coefficients of a single TU is less than a threshold T1, the candidate list is reduced to its first five entries.

[0226] Given a table that maps the number of significant coefficients to the number of candidates to which a threshold should be retained, the candidate list is appropriately reduced to its first entry. Given a table that maps the number of significant coefficients to the number of candidates for which a threshold and prediction mode should be retained, the candidate list is appropriately reduced to its first entry.

[0227] Given the number of significant coefficients and a table that maps the threshold and QP to the number of candidates for which to retain, the candidate list is appropriately reduced to its first entry.

[0228] If luma and chroma share the same coding structure and no residual signal is coded in the luma component, the candidate list of matched chroma components is reduced to its first entry.

[0229] If luma and chroma share the same coding structure and the sum of significant coefficients including luma and chroma components of a transform unit is less than a threshold T3 (e.g., 3), the candidate list of all TUs in that CU is reduced to its first candidate.

[0230] If the luma and chroma share the same coding structure and the luma transform has already been decoded, the chroma transform candidates are kept only if they are of the same class as the luma transform.

[0231] If the luma and chroma share the same coding structure and the luma transform has already been decoded, the chroma transform candidates are only kept if they are of the same class as the luma transform or if they are first in the list.

[0232] For each candidate multi-stage transform, given the positions of the significant coefficients in the transform block, it is checked whether the input to the secondary transform contains at least K significant coefficients. If it does not contain K significant coefficients, the transform is removed from the candidate list.

[0233] Given the location of the last significant position, we count how many coefficients are probably significant. Given a table that maps a threshold to the number of candidates to hold, the candidate list is appropriately reduced to its first entry.

[0234] For each candidate multi-stage transform, given the location of the last significant position in the transform block and the scan order, it is checked whether it contains at least K positions where the input to the secondary transform is likely to be significant. If the transform does not contain K significant coefficients, it is removed from the candidate list.

[0235] Thus, according to one embodiment, an encoder and a respective decoder are proposed, both configured to encode the transform coefficients (134) of a current residual block (84) by traversing a predefined scan order (136) and to determine the last significant transform coefficient (132) in said scan order. If the determined location of the last coded significant transform coefficient (132) indicates a first number of available significant transform coefficients ("first number" = sufficient significant coefficients available to perform a multi-stage transform), inputting at least one multi-stage transform (106) including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108) into a list (98) of selectable transforms for the current residual block (84) to result in a subgroup (138) of transform coefficients of the selected transform (102); If the determined location of the last coded significant transform coefficient (132) indicates a second number of available significant transform coefficients ("second number" = insufficient significant coefficients available to perform the multi-stage transform), The list (98) of selectable transforms for the current residual block (84) is not populated with any multi-stage transform (106) that includes a primary transform applied on the prediction residual (100) of the current residual block (84) and a secondary transform applied on a subgroup (112) of the primary transform's primary transform coefficients. The method is further configured to perform this based on the location of the last coded significant transform coefficient (132) determined as follows:

[0236] For each secondary transform, the number of possibly significant input coefficients is counted, given the location of the last significant position in the transform block and the scan order.

[0237] The transformation with the maximum value is kept and all other secondary transformations are removed.

[0238] All embodiments can be conditional on prediction mode and QP.

[0239] <4.3 Signaling> As mentioned above, reducing the transform candidate list by utilizing already decoded information about the transform block coefficients reduces the cost required to signal the actual selected transform. On the other hand, this dependency can also be exploited by making the coefficient decoding process a characteristic of the selected transform. Both approaches require a specific coding order of the syntax elements and can be applied together in an interleaved manner. As a special case, the transform index to the list of candidates can be signaled completely at the beginning of the encoding process of the transform block or at the end when all coefficients are known. In the latter case, information about all coefficients can be used to reduce the candidate list, in the former case, a prior reduction of candidates is not possible, but the decoding of each coefficient can benefit from a complete knowledge of the transform used.

[0240] It is clear that reducing the candidate list also reduces the actual signaling cost of the selected candidates, but the constrained coding of the coefficients of the selected transform is not very straightforward and whether benefits can be taken with reasonable complexity depends on the actual selected transform, the candidate list and the residual signal itself.

[0241] If the selected candidate is a transform with dimensionality reduction, there is no need to signal significance information for the entire subgroup of coefficients at all. This also allows for efficient signaling of the last position of a smaller set of remaining positions (set values), possibly even omitting this entirely by setting the last significant position to the last position given by the transform. In such a case, a significance flag may possibly be coded for each significant position as defined by the transform, but here the idea of ​​first signaling a significance group flag for the entire group of positions is also applicable. To exploit this dependency, it may be beneficial to first check whether a transform with dimensionality reduction is among the candidates before coding the coefficients, and if so, signal which of those candidates is selected, if necessary.

[0242] 13 and 15, as an example, the list of selectable transforms may include transforms of different numbers of transform coefficients 134. The encoder 10 and decoder 20 may be configured to encode / decode the different numbers of transform coefficients 134, where encoding / decoding the transform coefficients 134 includes encoding an indication (e.g., significance flag) of the significant transform coefficient positions 132 of the selected transform 102 and values ​​of the transform coefficients 134 of the selected transform traversed from the significant transform coefficient positions forward by a predefined scan order 136 that sequentially traverses the coefficient positions of the selected transform. The encoder 10 and decoder 20 of this example may be configured to encode / decode the transform coefficients 134 in such a way that the lower the code rate spent in the data stream 14 to encode the indication, the lower the number of transform coefficients 134 of the selected transform.

[0243] As another example, the encoder 10 and decoder 20 may be configured to selectively encode / decode the transform coefficients 134 depending on whether a transform with or without dimensionality reduction may be used, i.e., whether the list of selectable transforms includes a transform of a first number of transform coefficients 134 (e.g., a transform with dimensionality reduction) and a transform of a second number of transform coefficients 134 greater than the first number.

[0244] When the number of transform coefficients 134 of the selected transform is a second number, the encoder 10 and decoder 20 may be configured to encode / decode indications of significant transform coefficient positions 132 of the selected transform and values ​​of the transform coefficients 134 of the selected transform traversed forward from the significant transform coefficient positions 132 by a predetermined scan order 136 that sequentially traverses the coefficient positions of the selected transform.

[0245] If the number of transform coefficients 134 of the selected transform is a first number, the encoder 10 and decoder 20 may be configured to encode / decode values ​​of all of the first number of transform coefficients 134 of the selected transform.

[0246] Another example where knowledge of the selected transform can be used more efficiently to code the coefficients is when multiple transform candidates are defined by a primary+secondary transform, but the subgroup of positions to which the secondary transform is applied is different. In such a case, it may be beneficial to first code the selected transform, or at least the partition used. With this knowledge, a set of dedicated context models can be used to code the coefficients at positions that receive only the primary transform and those that are transformed by the secondary transform.

[0247] This knowledge can also be used to encode the last significant position as follows:

[0248] First, a flag is decoded indicating whether the last significant position is within the positions input to the secondary transform, and then the exact position is coded as the difference to the last possibly significant position defined by the transform, or to the position of the first coefficient of a particular subgroup.

[0249] More generally, and still referring to Figures 13 and 15, the selectable transformations are list is the linear transformation applied on the current residual block, and conversion and a secondary transform applied on subgroups 138 of the primary transform coefficients 134, the subgroups 138 differing by the at least one multi-stage transform.

[0250] According to a first alternative, the encoder 10 and decoder 20 may be configured to encode / decode the transform coefficients 134 by encoding / decoding an indication of the significant transform coefficient positions 132 of the selected transform and the values ​​of the transform coefficients 134 of the selected transform traversed from the significant transform coefficient positions 132 forwards by a predefined scan order 136 that sequentially traverses the coefficient positions of the selected transform. The indication may depend on whether the selected transform is a multi-stage transform or not. If the selected transform is a multi-stage transform, the instructions include a flag indicating whether the significant transform coefficient position 132 is within a subgroup 138 of transform coefficients 134, and if the transform coefficient position 132 is within the subgroup 138 of transform coefficients 134, the instructions indicate the significant transform coefficient position 132 relative to the last traversed position among all transform coefficients 134 of the selected transform in a predetermined scan order 136, and if the transform coefficient position 132 is not within the subgroup 138 of transform coefficients 134, the instructions indicate the significant transform coefficient position 132 relative to the last traversed position among all transform coefficients 134 that are not within the subgroup 138 in the predetermined scan order 136, and the predetermined scan order 136 traverses all transform coefficients 134 in the subgroup 138 after traversing all transform coefficients 134 that are not within the subgroup 138.

[0251] According to a second alternative, the encoder 10 and the decoder 20 may be configured to encode / decode the transform coefficients 134 by encoding the values ​​of the transform coefficients 134 of the selected transform using entropy coding that uses a probability estimate that depends on whether the transform coefficient 134 is within a subgroup 138 of transform coefficients 134 of the selected transform if the selected transform is of a multi-stage type.

[0252] According to a third alternative, the encoder 10 and the decoder 20 may filter the transform coefficients 134 in a manner that depends on whether the transform coefficients 134 are within a subgroup 138 of transform coefficients 134 of the selected transform, if the selected transform is of the multi-stage type. One or moreThe encoding / decoding may be configured to encode / decode by encoding the values ​​of the transform coefficients 134 of the selected transform using entropy coding that uses probability estimates that depend on neighboring transform coefficients 134 of the selected transform.

[0253] An index into the list of transform candidates shall be coded such that the signaling cost of the candidates depends on their position in the list, with increasing cost for each position. This can be done by coding the transform index using a truncated unary code.

[0254] In addition, the transform candidate list is constructed in such a way that the candidates are sorted by their specialization. For example, the first transform is the most universal separable transform [DCT-II, DCT-II], followed by [DST-VII, DST-VII], then a transform specifically trained for the particular intra mode of the current residual block. This list is then filled up with additional transforms, each specialized for the particular characteristics of the residual signal that may occur with that intra mode. This specialization can be trained by including a clustering method, such as fitting a Gaussian mixture model to the data examples, and a separate transform can be trained for each cluster obtained. Under this assumption of a sorted candidate list, knowing the selected transform gives insight into the characteristics of the actual residual signal, and thus the expected coefficient distribution. In particular, for small transform indices, low signal energy is expected, since the coding cost of higher transform indices cannot be justified by coding gains using more specialized transforms.

[0255] This can be exploited by deriving a context model for coding the coefficients depending on the transform index itself: if Golomb-Rice codes are used to code the coefficient levels, the actual Rice parameters can also be derived from the selected transform index.

[0256] According to one example, the encoder 10 may be configured to signal the selected transform using an index and to encode the transform coefficients 134 by encoding values ​​of the transform coefficients 134 of the selected transform using binarization and binary entropy coding, where the probability estimates used in the binary entropy coding depend on the index and / or the binarization depends on the index.

[0257] The decoder 20 may be configured to derive a selected transform using an index, decode the transform coefficients 134 by decoding values ​​of the transform coefficients 134 of the selected transform using binarization and binary entropy decoding, where the probability estimates used in the binary entropy coding depend on the index and / or the binarization depends on the index.

[0258] The index may be a variable length coded (VLC) index.

[0259] A more complex example that leverages information from the transform selected for coefficient coding assumes that there is a residual dependency of transform coefficients that can be modeled by the correlation of their variances. Taking two coefficients and assuming that their mean values ​​are zero, the correlation of variances is simply assumed to know that one of the coefficients is large, and the probability model of the other is chosen to have a high variance, thus also increasing the probability that the second coefficient is significant. This assumed variance correlation model can be included in the process of training the transform, so that a specific correlation model is assumed in the decoder given the transform. The actual correlation model can also depend on the actual coefficient position or subgroup of the multi-stage transform. One specific model defines which coefficients are assumed to have variance correlation, and how strong this correlation is. This can be used to adapt the context model selection for a particular coefficient position on already decoded coefficients. More precisely, when decoding a coefficient at a particular position, it is ascertained which correlation model is defined for that position by the selected transform. If already decoded coefficients are within the energy correlation position given by the model, the levels of those coefficients are compared with a given threshold, also defined by the model, and a set of specific context models is derived to decode the significance flag and the level of the current coefficient.

[0260] The correlation may be derived from neighboring transform coefficients. As an example, the encoder 10 and decoder 20 may encode / decode the transform coefficients 134 by encoding the values ​​of the transform coefficients 134 of a selected transform using entropy coding, the entropy coding being in a manner that depends on the selected transform. One or more The transform coefficients 134 may be configured to depend on neighboring transform coefficients 134 of the first transform coefficient 134 .

[0261] As another example, encoder 10 and decoder 20 may map transform coefficients 134 in a manner that depends on the position of the transform coefficient in the selected transform. One or more neighbors Select a local template with a conversion coefficient of 134, and One or more neighbors For the domain of possible values ​​of the conversion factor 134 of One or more to a set of contexts, determining a selected context for the transform coefficients 134, and encoding / decoding the values ​​of the transform coefficients 134 of the selected transform using entropy coding by using probability estimates associated with the selected context for entropy coding the values ​​of the transform coefficients 134, wherein at least one of the selection of the local template and the mapping depends on the selected transform.

[0262] As a concrete example, assume that a linear + secondary transform is used, and that it is known at the decoder which positions undergo the secondary transform. Also assume two different variance correlation models, one for the primary-only transform coefficients and one for the remaining coefficient positions. Both models are given a set of four context models and three thresholds. When decoding a coefficient, it is first ascertained which of both context model groups applies to the current position. Then the average value of the last four already decoded coefficient levels is taken and compared to the thresholds. The result uniquely defines the set of context models used to decode the current coefficient.

[0263] As shown by the provided example, different information about the selected transform can be beneficial to the specific part of the coefficient decoding process. This motivates splitting the information required to identify the selected transform among the candidates into specific transform characteristics and signaling them interleaved separately with the coefficient coding. In doing so, already decoded information about the transform can be used to decode information about the coefficients, while newly obtained information about the coefficients can be used to reduce the transform candidate list, and thus less information needs to be signaled to identify the selected candidate.

[0264] Which transform properties should be coded and in which order can also be selected adaptively, depending on the available candidates in the list and on the already acquired information about the residual signal. For example, the transform index is pre-coded only if there is a transform with dimensionality reduction among the candidates. Also, as soon as at least one coordinate of the last position is decoded, the further decoding of the selected transform can be conditioned on that coordinate. Already, the knowledge of one coordinate gives some evidence about the maximum energy to be expected in the residual signal, for example by counting the maximum number of significant positions. Only if this expected value is above a threshold, the transform index is decoded next and the structure of the selected transform can be conditioned on the decoding of the second coordinate.

[0265] According to this example, the encoder 10 and the decoder 20 may be configured to signal the selected transform in the data stream 14 in such a way that the relative placement of the signaling of the selected transform with respect to the transform coefficient data and / or the decomposition of the signaling into syntax elements depends on the list of selectable transforms.

[0266] If not, once the actual number of significant positions is known and the minimum coefficient constraint is violated, the decoding of the transform can be postponed or skipped.

[0267] In either case, whether the transform index is signaled first, interleaved, or at the end of the transform block, splitting the information about the selected transform into its characteristics and signaling them separately may be beneficial if a dependency of these characteristics exists between neighboring transform blocks. This dependency can then be exploited by using a set of dedicated context models per transform characteristic. The selected context models may then still depend on the selected transforms of neighboring residual blocks that have already been decoded.

[0268] According to this aspect, referring to Fig. 13 and Fig. 14, an encoder 10 and a decoder 20 are provided for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding, the encoder 10 and the decoder 20 supporting a transform set 94 for block-based residual coding. The encoder 10 and the decoder 20 may be configured to determine 122 a list 98 of selectable transforms for a current residual block 84 from among the transform set 94, and to encode / decode a prediction residual in the residual block using one selected transform from the list of selectable transforms. The encoder 10 may be configured to signal 128 the selected transform in the data stream 14 using a transform indicator 124, and the decoder 20 may be configured to derive the selected transform from the data stream 14 using said transform indicator 124. The transform indicator 124 may be signaled in the data stream 14 using a single or multiple digit code, each of a single or multiple digit set uniquely associated with a respective transform characteristic. The code may include syntax elements that can be parsed from the data stream 14 from which the bins or transformation indicators 124 are constructed that are subjected to binary arithmetic encoding.

[0269] In general, knowing only the indices of already selected transform candidates of neighboring blocks does not provide much insight into the selected transform of the current block. One exception is that the candidates are ordered such that high indices indicate high specialization or high residual energy. Also, if the actual selected transforms of neighboring blocks are highly specialized in their prediction mode, a comparison to the current candidate is only meaningful if they share the same mode. On the other hand, if we know, for example, that transforms from the DCT / DST family are predominantly selected in the neighborhood, we can exploit this information to first signal whether the current transform is from the same family and then reduce the candidate list appropriately.

[0270] According to one example, each transform characteristic may be one of whether the selected transform is a multi-stage transform or whether the selected transform is a transform having transform coefficients less than a predetermined number of transform coefficients.

[0271] Another advantage of explicitly reducing the candidate list stepwise by splitting the encoding of the selected candidates and signaling the transformation properties is the possibility of simplifying the exploitation of dependencies among different candidates in the list, e.g., binarization of the transformation index and the context model. Derivation can depend on the transform candidates in the list as follows: Assuming one of the candidates (preferably the first one) is associated with the highest probability, a flag indicating whether this transform should be used is coded first, otherwise this transform is removed from the list. If the remaining transform candidates are of a different transform class, then that transform class can be decoded first. This allows the context model on which the transform class depends for signaling the remaining transform indices. Derivation This will be possible.

[0272] Assume that the (remaining) candidates in the list are ordered with respect to their expected probabilities, with the most probable candidates coming first in the list. Then, the transform index can be binarized using a context-modeled truncated unary code.

[0273] According to one example, the encoder 10 and decoder 20 may be configured to encode / decode at least one of the set of at least one digit using entropy coding that uses probability estimates that depend on the transform indicators 124 of neighboring residual blocks.

[0274] 4.3.1 Exemplary embodiments In the following, One or more Some specific, but non-limiting examples are provided to substantiate the above generalized description of exemplary embodiments of the present invention.

[0275] <Encoding order> If it is known that the residual signal is coded for a block, which is the case when the Coded Block Flag (CBF) is decoded and equal to 1, then an index into the candidate list may be decoded first, followed by decoding of the actual transform coefficients.

[0276] If the residual signal is known to be coded for a block, which is the case if the Coded Block Flag (CBF) has been decoded and is equal to 1, then all coefficients of that block may be decoded first. If there are multiple transforms in the candidate list, then an index into the candidate list may be coded after all candidates that violate the decoded coefficients have been removed.

[0277] If it is known that the residual signal is coded for a block, which is the case if the Coded Block Flag (CBF) has been decoded and is equal to 1, and the candidate list contains different primary transform classes (such as non-separable transforms and transforms from the DCT / DST family), the primary transform may be decoded first and the candidate list may be appropriately reduced. Decoding of all transform coefficients of that block may occur later. If there are multiple transforms in the candidate list, an index into the candidate list may be decoded after all candidates that violate the decoded coefficients have been removed.

[0278] If it is known that the residual signal is coded for a block, which is the case when the Coded Block Flag (CBF) is decoded and equal to 1, and the candidate list includes a primary-only transform and a primary+secondary transform, whether the secondary transform is applied may be signaled first. Decoding of all transform coefficients of that block may occur later. If there are multiple transforms in the candidate list, an index into the candidate list may be decoded after all candidates that violate the decoded coefficients are removed.

[0279] If it is known that the residual signal is coded for a block, which is the case if the coded block flag (CBF) has been decoded and is equal to 1 and the candidate list contains at least one transform with dimensionality reduction, then it may be decoded first whether a transform with dimensionality reduction is used. If this is the case and multiple candidates have dimensionality reduction properties with different positions set to zero, then an index may be signaled next to uniquely define which positions are set to zero. Decoding of all transform coefficients of that block may be done later. If there are multiple transforms in the candidate list, then an index into the candidate list may be decoded after all candidates that violate the decoded coefficients have been removed.

[0280] If it is known that the residual signal is coded for a block, which is the case when the coded block flag (CBF) is decoded and equal to 1, the position of the last significant coefficient can be decoded first. If there is a transform with dimensionality reduction among the candidates, that transform is removed if it conflicts with the last significant position. Depending on the quantization parameter (QP), transforms with secondary transforms, where the number of possibly further transformed coefficients given by the last significant position is less than a threshold, are also removed or further returned to the candidate list. If several transform candidates remain, an index into the list is decoded. The coefficients are decoded later.

[0281] If it is known that the residual signal is coded for a block, which is the case when the coded block flag (CBF) has been decoded and is equal to 1, all significant coefficient positions may be decoded first. If there is a transform with dimensionality reduction among the candidates, that transform may be removed if it conflicts with a significant position. Depending on the quantization parameter (QP), a transform with a secondary transform, where the number of possibly further transformed coefficients given by the last significant position is less than a threshold, may also be removed or further returned to the candidate list. If multiple transform candidates remain, an index into the list may be decoded. The remaining coefficient levels and signs may be decoded later.

[0282] If it is known that the residual signal is coded for a block, which is the case when the candidate list contains transforms of different classes, such that the coded block flag (CBF) is decoded and is equal to 1, and each class provides evidence to expect the last significant position to be in a different region, the transform class may be decoded first, followed by the last significant position. If necessary, the transform index may be signaled next, followed by the coefficients. For example, at a particularly high QP, a transform with a secondary transform provides evidence to expect the last position to be in the region of the coefficients that are further transformed by the secondary transform.

[0283] <Signaling the transformation index> The transform index may be encoded using a truncated unary code, and each bin may use a dedicated context model. Depending on the number of remaining transform candidates, a different set of context models may be used.

[0284] The transform index may be coded using a truncated unary code, and each bin may use a dedicated context model. The context model selection for each bin may depend on the transform decisions of neighboring residual blocks. Depending on the number of remaining transform candidates, a set of context models consisting of k context models may be selected for each bin of the truncated unary code. The transform indexes of neighboring blocks may be summed, and the sum may be compared to k-1 thresholds that define the actual context model to be used.

[0285] The transform index may be encoded by first encoding a flag that may indicate whether the first candidate is selected. The context model selection for this flag may depend on the transform decisions of neighboring residual blocks. Depending on the number of remaining transform candidates, a set of context models consisting of k context models may be selected. The transform indexes of the neighboring blocks may be summed, and the sum may be compared to k-1 thresholds that define the actual context model to be used. The remaining part of the index may be encoded by a fixed-length code.

[0286] The transform index is coded by first coding a flag indicating whether the first candidate is selected or not. A fixed length code may be used for the rest. The flag may be coded using a dedicated context model selected according to the size of the residual block.

[0287] If the division into blocks includes a division into coding units (CUs) that can be further divided into transform blocks (TUs), and at least one residual may be coded such that its coding block flag is 1, one additional flag may be coded at the CU level. If this flag is zero, all residual blocks may take the first candidate in their list. Otherwise, the first candidate in each list may be placed at the end of the list and a transform index for each residual block may be coded.

[0288] The number of transform candidates may be five. A flag may be coded first indicating whether the first candidate is selected. If not, a second flag may be coded indicating that the second or third candidate is selected. The final decision may be coded by a third flag. All flags may be coded using a dedicated context model.

[0289] The transform index may be encoded by first encoding a flag indicating whether the first candidate is selected. If the remaining transform candidates are associated with different transform classes, the transform class may be encoded using a truncated unary code, where primary-only transforms may form one class and primary+secondary transforms may form a second class. Finally, if necessary, the remaining index may be encoded using a truncated unary code by utilizing a dedicated set of context models that depend on the transform class.

[0290] The transform index is encoded by first encoding a flag indicating whether the first candidate is selected. If the remaining transform candidates are associated with different transform classes, the transform class may be encoded using a truncated unary code. Here, primary-only transforms may form one class and primary+secondary transforms may form a second class. The context model may be selected by counting the occurrence of the transform classes of neighboring transforms already decoded. If the first class occurs more, the first context model may be used, if the second class occurs more, the second context model may be used, and if the number of each class is equal, the third context model may be used. Finally, if necessary, the remaining index may be encoded using a truncated unary code by utilizing a dedicated set of context models that depend on the already decoded transform classes.

[0291] The transform index is encoded by first encoding a flag indicating whether the first candidate is selected. If the remaining transform candidates are associated with different transform classes, the transform class may be encoded using a truncated unary code, where transforms from the DCT / DST family form the first class, non-separable first-order only transforms form the second class, and first-order+second-order transforms form the third class. Finally, if necessary, the remaining index may be encoded using a truncated unary code by utilizing a dedicated set of context models that depend on the transform class.

[0292] If one coordinate of the last significant position has already been decoded and is at least 2, the transform index may be decoded next; otherwise, the decoding of the transform index may be postponed until the last step (if necessary) when the coefficients are already known and the candidate list is already reduced. If the transform is already known when decoding the second coordinate of the last position and it is a linear+quadratic transform, the context model selection for the truncated unary code portion may be such that the bins associated with the positions that fall within the domain of the quadratic transform may use a different context model set than the others.

[0293] Context model for coefficient coding Derivation > (See Figures 13, 14, and 15) The transformation index into the list of candidates is directly related to the expected energy of the residual signal and thus shall be used for context model selection for coding the last significant position. Each transformation index value defines its own dedicated set of context models.

[0294] The transform index into the list of candidates is directly related to the expected energy of the residual signal, and thus it shall be used for context model selection for coding the last significant position. Transform index 0 and transform index 1 have their own dedicated set of context models, while the remaining indices share one set of context models.

[0295] If the transform class is already known, the coefficient coding utilizes a dedicated set of context models for each transform class. Possible transform classes with dedicated context model sets are transforms from the DCT / DST family and non-separable transforms.

[0296] The division of coefficient positions is defined by the transformation, SplitIf , is already known, the coefficient coding utilizes a set of dedicated context models per subgroup. One possible partition is given by the coefficients that are further transformed by the secondary transform. Another partition is given by the number of subsequent transform stages.

[0297] Each defined transformation is associated with a set of context models and is known by the decoder.

[0298] If the transform is already uniquely identified, then a mapping from the transform to a set of context models is used to derive a set of context models to be used for coefficient decoding.

[0299] If the coefficient coding involves a context model selection based on already decoded coefficients, the (already known) coefficient levels are weighted and accumulated according to their relative positions. The actual context model set used is selected by comparing the sum with a predefined threshold. The weights and the thresholds used are given by the selected transform.

[0300] <Special encoding mode> It may be coded first whether the first candidate in the list is used, otherwise it may be checked whether at least one of the remaining transform candidates is a transform with dimensionality reduction. In this case, it may be coded whether the selected transform is one of those transforms. If this is the case and all other candidates are removed from the list, it may be checked whether the remaining transforms differ in their positions that are explicitly set to zero. If necessary, an index that uniquely identifies which dimensionality reduction is selected may be signaled. Thus, it is known exactly which coefficient positions are likely to remain significant, even if there are still multiple candidates left. If necessary, the index of the selected transform may be decoded after the coefficients are decoded.

[0301] If a transform with dimensionality reduction is used and the positions that are explicitly set to zero are already known, the signaling of the last significant coefficient position can be skipped. Given a scan order of the coefficients, the last position is likely significant and can be set to that position that is the last in the scan order. In this case, a significance flag should also be signaled for the last position.

[0302] If a transform with dimensionality reduction is used and the positions that are explicitly set to zero by the transform are already known, the significant flags of all remaining coefficient groups may be set to 1 and therefore do not need to be coded.

[0303] If a transform with dimensionality reduction is used, and knowing the positions that are set to zero, a set of dedicated context models can be used to decode the remaining coefficients.

[0304] If a transform with dimensionality reduction is used, a set of dedicated context models can be used to decode the last coefficient position of a block, knowing the positions that are set to zero.

[0305] If a transformation with dimensionality reduction is used, a dedicated Rice parameter for coding the coefficient levels is used. Derivation may be used.

[0306] If a transformation with dimensionality reduction is used, a dedicated template for context selection based on nearby coefficients may be used.

[0307] All embodiments can be conditional on prediction mode and QP.

[0308] <5. Linear transformations set to zero> Section 4.1 provides a general description, but in the following: One or more We present some examples (see Section 5.1 below) where only the first-order coefficients of are zeroed, i.e., set to zero. One or moreare zeroed, i.e. set to zero (see section 5.2 below). Coefficients that may potentially be zeroed may also be called 'candidates' for zeroing.

[0309] Generally, referring to Figure 12, this aspect of the invention provides an encoder 10 and decoder 20 for encoding / decoding an image 12 to / from a data stream 14 using block-based prediction and block-based residual coding, the encoder 10 and decoder 20 supporting a transform set 94 for block-based residual coding. The encoder 10 and decoder 20 may be configured to determine 122 a transform candidate list 98 of transforms for a current residual block 84 from the transform set 94 based on a size 96, such as height and width, of the current residual block 84. The encoder 10 may be configured to encode a prediction residual 100 in the residual block 84 using one selected transform 102 from the list of selectable transforms 98 and selectively signal the selected transform 102 in the data stream 14. Further, the decoder 20 decodes the prediction residual 100 in the residual block 84 by selectively deriving a selected transform 102 from the data stream 14 in the domain of one selected transform 102 from the list of selectable transforms 98.

[0310] <5.1 Zeroing only the first-order coefficients> When there is a multi-stage transform including a linear and a secondary transform, the linear-only transform coefficients (i.e., coefficients that are not further transformed) are set to zero. a. The secondary transformation is the full-size transformation. b. The secondary transformation is a transformation involving dimensionality reduction (the number of possible significant coefficients is further reduced).

[0311] For example, if there is a candidate transform that contains a first-order (separable) transform (such as [DCT-II, DCT-II]) for a residual block whose width and height are 8 or more, the combination of this transform with a second-order transform of input size (8 × 8) may also be a candidate. Applying a second-order transform implies that all coefficients that are not further transformed by the second-order transform are forced to zero.

[0312] For example, if there is a candidate transform that contains a first-order (separable) transform (such as [DCT-II, DCT-II]) for a residual block whose width or height (but not both) is 4, then a combination of this transform with a second-order transform of input size (4 × 4) may also be a candidate. Applying a second-order transform implies that all coefficients that are not further transformed by the second-order transform are forced to zero.

[0313] For example, if there is a candidate transform that includes a first-order (separable) transform (such as [DCT-II,DCT-II]) for a residual block with width and height 8 or more, a candidate could also be a combination of this transform with a second-order transform of input dimension 48 and output dimension 16 applied to the three left / top-most coefficient subgroups of size (4×4). Applying the second-order transform implies that all coefficients not further transformed by the second-order transform are set to zero, so further, the second-order transform is a transform with a dimensionality reduction to 16 coefficients, with the last significant coefficient position being (3,3).

[0314] <5.2 Zeroing coefficients at specific positions> If there is a multi-stage transform including a linear and a quadratic transform, set the coefficients with position X>Tx or position Y>Ty to zero (eg, Tx=Ty=8). a. The secondary transformation is the full-size transformation. b. The secondary transformation is a transformation involving dimensionality reduction (the number of possible significant coefficients is further reduced)

[0315] FIG. 17 shows an example of a residual block having a size, e.g., width and height, greater than 8, that has been subjected to a linear transform. Thus, there may be subgroups 112 with multiple primary coefficients 114. In this example, the subgroups 112 may include a size of primary coefficients 114 of 4×4. Shown here are the three left / top most coefficient subgroups of size 4×4. Thus, the three subgroups 112 together include 48 primary coefficients 114. These may be subjected to a secondary transform 110 of dimension 48×16, i.e., the coefficients are reduced from 48 to 16, and some primary coefficients may be candidates for zeroing.

[0316] For example, if there is a candidate transform that includes a first-order (separable) transform (such as [DCT-II,DCT-II]) for a residual block with width and height 8 or more, a combination of this transform with a second-order transform of dimension (48×16) applied to the three left / top-most coefficient subgroups 112 of size (4×4) may also be a candidate. Applying a second-order transform implies that the last significant coefficient position can be at most (7,7), and therefore the remaining first-order only coefficients (shaded) are set to zero.

[0317] For example, if there is a candidate transform that contains a first-order (separable) transform (such as [DCT-II,DCT-II]) for a residual block with width and height ≥ 8, a combination of this transform with a second-order transform of dimension (48 × 16) applied to the three leftmost / topmost coefficient subgroups of size (4 × 4) may also be a candidate. Applying a second-order transform implies that the last significant coefficient position can be at most (3,3), so the remaining first-order-only coefficients are set to zero.

[0318] <Signaling for 1+2> a) First signal the selected primary transform and coefficients. If the coefficients are significant (non-zero) in a position that would verify the constraints given by the transform candidates with the secondary transform, then it is clear that the secondary transform was not used and therefore can be removed from the candidate list and no additional signaling is required. 。 b) Signal the selected primary transformation if the corresponding primary+secondary transformation is among the candidates, and signal if a secondary transformation was applied. Set the maximum allowed final position appropriately, signal the final position, and subsequently signal the remaining coefficient information if a secondary transformation was applied, potentially by using a dedicated set of context models.

[0319] New compared to the state of the art: c) Signaling of secondary transform flags / indexes before / interleaved with the last position d) Constrain quadratic transformations by last position instead of the number of smallest significant positions (explicit lastPosX+lastPosY ≥ T, e.g. 1 or 2). e) Context modeling of coefficient sig flags depending on secondary transformation flags Thus, encoder 10 and decoder 20 may be configured to determine the transform candidate list such that the transforms in the transform candidate list match in range, ie, have the same number and arrangement of transform coefficients.

[0320] The transforms of the transform candidate list may include a first transform and a second transform that are range-matching, i.e., have the same number and arrangement of transform coefficients. The second transform may correspond to a secondary transform applied on a subgroup 112 of the primary transform coefficients 114 of the first transform 108, with further primary transform coefficients 114 of the first transform 108 outside the subgroup 112 of the primary transform coefficients of the first transform 108 being set to zero.

[0321] In this case, the encoder 10 may be configured to test the encoding of the current residual block for the second transform by setting further primary transform coefficients 114 of the first transform 108 outside the subgroup 112 of primary transform coefficients of the first transform 108 to zero.

[0322] Additionally or alternatively, the encoder 10 and the decoder 20 may be configured to determine whether to input or remove from the transform candidate list at least one multi-stage transform 106 including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on a subgroup 112 of the primary transform coefficients 114 of the primary transform 108 to result in a subgroup 138 of transform coefficients of the selected transform 102, or where in the transform candidate list the at least one multi-stage transform 106 should be located depending on where the significant transform coefficients are located according to the transform coefficient data 126.

[0323] Additionally or alternatively, the encoder 10 and the decoder 20 may be configured to determine whether to input or remove at least one multi-stage transform 106, including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on the subgroup 112 of the primary transform coefficients 114 of the primary transform 108 to result in a subgroup 138 of transform coefficients of the selected transform 102, from the transform candidate list, or where in the transform candidate list the at least one multi-stage transform 106 should be located, depending on whether any transform coefficients outside the subgroup 138 of transform coefficients are significant according to the transform coefficient data 126.

[0324] Additionally or alternatively, the encoder 10 and decoder 20 may be configured to encode / decode the transform coefficients by encoding / decoding values ​​of the transform coefficients of a selected transform using binarization and binary entropy coding, where the probability estimates used in the binary entropy coding depend on the selected transform and / or where the binarization depends on the selected transform.

[0325] The dependency on a selected transform may be a dependency on whether the selected transform is a multi-stage transform or not.

[0326] Additionally or alternatively, the encoder 10 and decoder 20 may be configured to determine whether to input or remove from the transform candidate list 98 at least one multi-stage transform 106, including a primary transform applied on the prediction residual 100 of the current residual block 84 and a secondary transform applied on the subgroup 112 of primary transform coefficients 114 of the primary transform 106 to result in a subgroup 138 of transform coefficients of the selected transform 102, or where in the transform candidate list 98 the at least one multi-stage transform 106 should be located depending on whether any transform coefficients within the subgroup 112 but outside the subgroup 138 of transform coefficients are significant according to the transform coefficient data 126.

[0327] Additionally or alternatively, the encoder 10 and decoder 20 may further include a linear transform applied on the prediction residual 100 of the current residual block 84 to result in transform coefficients of the multi-stage transform 106 that correspond in transform coefficient positions to a subset 138 of transform coefficients of another transform in the transform candidate list 98 that is less in number than the residual samples in the prediction residual 100 of the current residual block 84 and is equal to the linear transform. 108 and a secondary transform applied on a subgroup 112 of the primary transform coefficients 114 of the first ... 108 is equal to the sample 105 in the prediction residual 100 of the current residual block 84, or 108 In addition or alternatively, the encoder 10 and decoder 20 may be configured to determine where at least one multi-stage transform 106 should be placed in the transform candidate list 98 relative to a further transform, the determination being based on: a) where the significant transform coefficients are located according to the transform coefficient data 126; or b) whether the significant transform coefficient is located outside the subset 138 of transform coefficients according to the transform coefficient data 126; or c) whether the significant conversion coefficients are located outside the subgroup 112 of the primary conversion coefficients 114 according to the conversion coefficient data 126; The method may be configured to depend on

[0328] Some examples will be described with reference to FIGS. 8, 18 and 19.

[0329] For example, as mentioned above, the signaling may depend on the significant transform coefficient positions 132. Figures 18 and 19 illustrate the transform coefficients 114, 134 and subgroups 112, 138 of the transform coefficients 114, 134 from the decoder's perspective.

[0330] 18 illustrates one non-limiting example of a multi-stage transform with zeroing, where a primary transform may result in a primary subgroup 112 of primary transform coefficients 114 that may be subjected to a secondary transform that may result in a secondary subgroup 138 of secondary transform coefficients 134. Some of the primary transform coefficients 114 and / or some of the secondary transform coefficients 134 may be zeroed based on, for example, significant transform coefficient positions 132.

[0331] For example, as shown in FIG. 18, where a secondary transform may have been applied, the transform coefficients 134 outside the secondary subgroup 138, and therefore the transform coefficients 114 outside the primary subgroup 138, may be zero.

[0332] FIG. 19 shows another example, also from the decoder's point of view, where secondary transform coefficients 134 may be zeroed. As mentioned above, the outlined 8×8 area represents a primary subgroup 112 of primary transform coefficients 114. The transform coefficients within the area marked with cross-hatched lines (green in the first-filed application) may represent primary-only transform coefficients. The transform coefficients within the area marked with top-left to bottom-right hatched lines (blue in the first-filed application) may represent transform coefficient positions that would have to be zero if a secondary transform were applied. This further means that if there are significant (non-zero-valued) coefficients within the (blue) top-left to bottom-right hatched area, the decoder 20 knows that it does not need to perform a secondary transform, since otherwise these positions would have to be zero for the secondary transform. Thus, no index needs to be signaled.

[0333] Regarding signaling, some examples such as those described above allow that whether signaling (e.g., of an index) is needed or can be omitted can be derived from the significant coefficient positions 132. However, some examples allow that signaling can be derived from the significant coefficient positions 132. Derived 19. Alternatively, it may provide the possibility that the index can always be signaled instead of 19. In the latter case, if a secondary transform was applied to the primary subgroup 112, the primary transform coefficients 114 outside the primary subgroup 112 should be zero (such as in FIG. 19).

[0334] Thus, the decoder 20 may be configured to apply a multi-stage transform having a primary and secondary transform, where the primary transform can be performed with zeroing up to the dimensions of only the primary subgroup 112. That is, both the horizontal and vertical primary transforms may be performed with zeroing, e.g., for 8×8 (primary subgroup 112) and 16×16 transform blocks: If the block has a secondary transformation index set, nonZeroW=8 16 if not If the block has a secondary transformation index set, nonZeroH=8 16 if not

[0335] <5.3 Reducing the list of conversion candidates / restricting the application of secondary conversions> <minimum last position> If the CBF flag is set: a. If the last position is the smallest (e.g., X+Y>T, last position !=(0,0)), signal first (by coarse signaling) b. Signal the first location with the maximum first (i.e., the exact location up to T, so it is either precisely known or is T or >T) If the conditions for a quadratic transformation are met, signal a transformation with quadratic transformation / dimensionality reduction a. A transformation with a reduction in dimension (quadratic with a forced zero dimension reduction linear transformation) and, if necessary, encoding the exact final position while setting the maximum final position according to the forced zero position (= refinement) I. Adapting the context model for encoding of meaningful positions II. Adapting the context model for grX coding III. Adapting the Golomb-Rice parameters If not, the zeroing cannot be inferred and we continue to encode the exact last position if necessary.

[0336] Thus, referring to Figures 13 to 16, the encoder 10 and the decoder 20 may be configured to code / decode a coarse version of the indication of the significant transform coefficient positions 132 of the selected transform 102. If the candidate list 98 does not include a multi-stage transform, the exact last position, i.e., a refinement of the coarse version of the indication, may be coded and all coefficients are coded / decoded (Figure 16, case B). If the candidate list 98 includes a multi-stage transform, it may be further determined whether the actual transform is a multi-stage transform. If so, the exact last position, i.e., a refinement of the coarse version of the indication, may be coded and only the primary transform coefficients 112 are coded / decoded (Figure 16, case A). If not, the exact last position, i.e., a refinement of the coarse version of the indication, may be coded and all coefficients are coded / decoded.

[0337] Thus, the encoder 10 and decoder 20 may be configured to encode / decode a coarse version of the instructions to / from the data stream 14 prior to signaling 128 / derivation of the selected transform 102 in / from the data stream 14, and the determination 130 of the selectable list 98 depends on the coarse version of the instructions. The encoder 10 and decoder 20 may be further configured to encode / decode a refinement of the coarse version of the instructions to / from the data stream 14 following the signaling / derivation in dependence on the selected transform 102.

[0338] Additionally or alternatively, the encoder 10 and the decoder 20 may be configured to encode / decode, as a coarse version, a hint that reveals whether the significant transform coefficient position 132 is within the subgroup 138. As a refinement, the encoder 10 and the decoder 20 may be further configured to encode / decode relative position information that distinguishes the significant transform coefficient position 132 among positions within the subgroup 138 if the significant transform coefficient position 132 is within the subgroup 138, and that distinguishes the significant transform coefficient position 132 among positions outside the subgroup 138 if the significant transform coefficient position 132 is outside the subgroup 138.

[0339] Additionally or alternatively, if the hint reveals that the significant transform coefficient position 132 is within the subgroup 138, the encoder 10 and decoder 20 may be configured to determine the selectable list 98 to include at least one multi-stage transform 106 including a primary transform applied on the prediction residual 100 of the current residual block and a secondary transform applied on the subgroup 112 of the primary transform coefficients 114 of the primary transform 106 to result in the subgroup 138 of transform coefficients of the selected transform 102. If the hint reveals that the significant transform coefficient position 132 is not within the subgroup 138, the encoder 10 and decoder 20 may be configured to determine the selectable list 98 to not include the at least one multi-stage transform 106.

[0340] If the hint reveals that the significant transform coefficient positions 132 are within the subgroup 138, the encoder 10 and decoder 20 may be configured to encode / decode the relative position information using variable length coding, VLC, with a first maximum number of codes corresponding to the number of coefficients within the subgroup 138. If the hint reveals that the significant transform coefficient positions 132 are not within the subgroup 138, the encoder 10 and decoder 20 may be configured to encode / decode a second maximum number of codes corresponding to the number of coefficients outside the subgroup 138.

[0341] If the hint reveals that a significant transform coefficient position 132 is within the subgroup 138, the encoder 10 and decoder 20 may be configured to encode / decode a secondary transform indicator after the coarse version but before refinement that indicates whether the selected transform is among the at least one multi-stage transform 106 and / or which of the at least one multi-stage transform 106 is the selected transform.

[0342] If the hint reveals that a significant transform coefficient position 132 is within subgroup 138, then the encoder 10 and decoder 20 may be configured to use a context that depends on the secondary transform indicator to context-adaptively entropy code the transform coefficients within subgroup 138.

[0343] If the hints reveal that there are no significant transform coefficient positions 132 within the subgroup 138, the encoder 10 and decoder 20 may be configured to context-adaptively entropy code the transform coefficients within and outside the subgroup 138.

[0344] <5.4 (Encoder only) If a secondary transformation is tested, the encoder is conditioned on the last position of the transformation with dimensionality reduction already tested>

[0345] <5.5 Chroma> Signal once for luma (brightness) and both chroma (color) components: a) Signal once for luma + once for both chroma components: Signals in the first component where the condition is met (lastPosX>0 or lastPosY>0) (This is only allowed if both components satisfy the condition (which requires interleaved coding of the last positions of both components) b) Signal for each component (if the condition is met)

[0346] The present invention may be further realized by the following embodiments, which may be combined with any of the examples and embodiments described and claimed in this application.

[0347] <Embodiment 1> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, the encoder supporting a transform set (94) for block-based residual coding; predicting the intra predicted block by applying a spatial neighborhood (90) of the intra predicted block (80) onto a selected neural network from among a plurality of neural networks (92) to obtain a prediction (104) of the intra predicted block (80); determining a set (98) of selectable transforms (96) for the residual block (84) covering the intra predicted block (80) from among the set of transforms (94) based on the selected neural network and a block size (96) of the residual block (84); encoding a prediction residual (100) associated with a prediction of an intra-predicted block (80) within a residual block (84) using a selected transform (102) from a set of selectable transforms (98), and signaling the selected transform in a data stream (14); The encoder configured as described above.

[0348] <Embodiment 2> decision but , For block sizes above a given threshold, A first number of selectable transforms (98) each including a primary transform (108) applied on the prediction residuals (100) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on a selected neural network. One or more The multi-stage transformation (106) of Do as follows 2. The encoder of claim 1, configured as follows:

[0349] <Embodiment 3> decision but , The neural network ( NN # ), which is a separable 2D transformation (116). Do as follows 3. The encoder of embodiment 2, configured as follows.

[0350] <Embodiment 4> 4. The encoder of embodiment 3, wherein the subgroup (112) represents a low-frequency part of the linear transform (108).

[0351] <Embodiment 5> 5. The encoder of any one of embodiments 2-4, wherein the first number is between 2 and 6, inclusive.

[0352] <Embodiment 6> decision but , For block sizes below a given threshold, A set of selectable transformations (98) with a second number One or more As the non-separable transformation (118) of conduct 6. An encoder according to any one of embodiments 2 to 5, configured as follows:

[0353] <Embodiment 7> The encoder of embodiment 6, referring to embodiment 4, wherein the first number and the second number are equal.

[0354] <Embodiment 8> decision but , A set of selectable transformations (98) with a third number One or more The separable 2D transformation (116) of conduct 8. An encoder according to any one of embodiments 2 to 7, configured as follows:

[0355] <Embodiment 9> 9. The encoder of embodiment 8, wherein the third number is 1 or 5.

[0356] <Embodiment 10> The prediction is performed such that the prediction of the intra predicted block (80) is obtained in the domain (118) of the linear transform (108); decision but , a fourth number of transforms, each of which includes a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on the selected neural network; One or more The multi-stage transformation (106) of Do as follows 10. An encoder as claimed in any one of embodiments 1 to 9, configured as follows.

[0357] <Embodiment 11> decision but , The linear transformation (108) is a separable 2D transformation (116). Do as follows 11. The encoder of embodiment 10, configured as follows.

[0358] <Embodiment 12> 12. The encoder of embodiment 10 or 11, wherein the subgroup (112) represents a low-frequency part of the primary transform (108).

[0359] <Embodiment 13> 14. The encoder of any one of embodiments 10-13, wherein the first number is between 2 and 6, inclusive.

[0360] <Embodiment 14> decision but , A primary transformation (108) is further input to the set of selectable transformations (98). Do as follows 15. An encoder according to any one of embodiments 10 to 14, configured as follows:

[0361] <Embodiment 15> The quantization parameter of the residual block (84) is One or more Check whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 15. The encoder of any one of embodiments 1 to 14, configured to

[0362] <Embodiment 21> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms (94) for block-based residual decoding; predicting the intra predicted block by applying a spatial neighborhood (90) of the intra predicted block (80) onto a selected neural network from among a plurality of neural networks (92) to obtain a prediction (104) of the intra predicted block (80); Based on the selected neural network and the block size (96) of the residual block (84), conversion determining a set (98) of selectable transforms for a residual block (84) covering the intra-predicted block (80) from the set (94); Decoding a prediction residual (100) associated with a prediction (104) of an intra-predicted block (80) in a residual block (84) from the data stream (14) by deriving a selected transform (102) from the data stream (14) in a domain of a selected transform (102) from a set of selectable transforms (98). A decoder configured to:

[0363] <Embodiment 22> decision but , For block sizes above a given threshold, A first number of selectable transforms (98) each including a primary transform (108) applied on the prediction residuals (100) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on a selected neural network. One or more The multi-stage transformation (106) of Do as follows 22. A decoder as described in embodiment 21, configured as follows.

[0364] <Embodiment 23> decision but , A linear transformation (108) is performed on the selected neural network ( NN # ), which is a separable 2D transformation (116). 23. A decoder as described in embodiment 22, configured as follows.

[0365] <Embodiment 24> 24. The decoder of embodiment 23, wherein the subgroup (112) represents a low-frequency part of the primary transform (108).

[0366] <Embodiment 25> 25. The decoder of any one of embodiments 22-24, wherein the first number is between 2 and 6, inclusive.

[0367] <Embodiment 26> decision but , For block sizes below a given threshold, A set of selectable transformations (98) with a second number One or more As the non-separable transformation (118) of conduct 26. A decoder as claimed in any one of embodiments 22 to 25, configured to

[0368] <Embodiment 27> The decoder of embodiment 26, referring to embodiment 24, wherein the first number and the second number are equal.

[0369] <Embodiment 28> decision but , A set of selectable transformations (98) with a third number One or more The separable 2D transformation (116) of conduct 28. A decoder as claimed in any one of embodiments 22 to 27, configured to

[0370] <Embodiment 29> 29. The decoder of embodiment 28, wherein the third number is 1 or 5.

[0371] <Embodiment 30> prediction but , the prediction (104) of the intra predicted block (80) is Domain of Linear Transformation (108) (118) So that it can be obtained by decision but , a fourth number of transforms, each of which includes a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on the selected neural network; One or more The multi-stage transformation (106) of Do as follows 30. A decoder as claimed in any one of embodiments 21 to 29, configured to

[0372] <Embodiment 31> decision but , The linear transformation (108) is a separable 2D transformation (116). Do as follows 31. A decoder as described in embodiment 30, configured as follows.

[0373] <Embodiment 32> 32. The decoder of embodiment 30 or 31, wherein the subgroup (112) represents a low-frequency part of the primary transform (108).

[0374] <Embodiment 33> 34. The decoder of any one of embodiments 30 to 33, wherein the first number is between 2 and 6, inclusive.

[0375] <Embodiment 34> decision but , A primary transformation (108) is further input to the set of selectable transformations (98). Do as follows 35. A decoder according to any one of embodiments 30 to 34, configured to

[0376] <Embodiment 35> The quantization parameter of the residual block (84) is One or more Check whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 35. A decoder according to any one of embodiments 21 to 34, configured to

[0377] <Embodiment 41> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, comprising: To obtain a prediction (104) of the intra predicted block (80) in the domain (118) of the linear transformation (108), the spatial neighborhood (90) of the intra predicted block (80) is subjected to a neural network (NN # ) to predict an intra predicted block (80), encoding a prediction residual (100) associated with a prediction (104) of an intra-predicted block (80) within a residual block (84) using a selected transform (102) from a set of selectable transforms (98), and signaling the selected transform (102) in a data stream (14); a set of selectable transforms (98) is input with at least one multi-stage transform (106) including a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108); The encoder configured as described above.

[0378] <Embodiment 42> 42. The encoder of embodiment 41, wherein a number of multi-stage transforms (106) are input to the set of selectable transforms (98), each of the multi-stage transforms (106) including a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable.

[0379] <Embodiment 43> 43. The encoder of embodiment 41 or 42, wherein the number is between 2 and 6, inclusive.

[0380] <Embodiment 44> 44. The encoder of any one of embodiments 41 to 43, wherein the primary transform (108) is a separable 2D transform.

[0381] <Embodiment 45> 45. The encoder of any one of embodiments 41 to 44, wherein the subgroup (112) represents a low-frequency part of a linear transform.

[0382] <Embodiment 46> decision but , The primary transformation is further input to the set of selectable transformations (98). Do as follows 46. ​​An encoder as in any one of embodiments 41 to 45, configured to

[0383] <Embodiment 47> The quantization parameter of the intra predicted block is One or more Check whether the quantization parameter set of If included, determine but , The set of transformations you can choose from includes: One or more associated with a given set of quantization parameters One or more A set of predetermined transformations of Do as follows 47. An encoder as in any one of embodiments 41 to 46, configured to

[0384] <Embodiment 51> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual coding, comprising: To obtain a prediction (104) of the intra predicted block (80) in the domain (118) of the linear transformation (108), the spatial neighborhood (90) of the intra predicted block (80) is subjected to a neural network (NN # ) to predict an intra predicted block (80), Decoding a prediction residual (100) associated with a prediction (104) of an intra-predicted block (80) in a residual block (84) from the data stream (14) by deriving a selected transform (102) from a set of selectable transforms (98) in another domain (119) [which may be the same as the domain 118 as described in embodiment 56] of the selected transform (102) from the data stream (14); a set of selectable transforms (98) is input with at least one multi-stage transform (106) including a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108); A decoder configured to:

[0385] <Embodiment 52> 52. A decoder as described in embodiment 51, in which a set of selectable transforms (98) is input with several multi-stage transforms (106), each including a primary transform (108) and a secondary transform (110) applied on a subgroup (112) of primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable.

[0386] <Embodiment 53> The numbers are between 2 and 6 inclusive, A decoder as described in embodiment 51 or 52.

[0387] <Embodiment 54> 54. The decoder of any one of embodiments 51 to 53, wherein the primary transform (108) is a separable 2D transform.

[0388] <Embodiment 55> 55. The decoder of any one of embodiments 51 to 54, wherein the subgroup (112) represents a low-frequency part of a linear transform.

[0389] <Embodiment 56> decision but , The primary transformation is further input to the set of selectable transformations (98). Do as follows 56. A decoder as described in any one of embodiments 51 to 55, configured so as to

[0390] <Embodiment 57> The quantization parameter of the intra predicted block is One or more Check whether the quantization parameter set of If included, determine but , The set of transformations you can choose from includes: One or more associated with a given set of quantization parameters One or more A set of predetermined transformations of Do as follows 57. A decoder as described in any one of embodiments 51 to 56, configured so as to

[0391] <Embodiment 61> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, the encoder supporting a transform set (94) for block-based residual coding; predicting an intra-predicted block (80) based on spatial neighborhoods (90) of the intra-predicted block (80) using a selected one of a set of intra-prediction modes (120) to obtain a prediction (104) of the intra-predicted block (80); determining a set (98) of selectable transforms (94) for the residual block (84) that cover the intra predicted block (80) from among the set of transforms (94) based on the selected intra prediction mode and a block size (96) of the residual block (84); encoding a prediction residual (100) associated with a prediction (104) of an intra-predicted block (80) within a residual block (84) using a selected transform (102) from a set of selectable transforms (98), and signaling the selected transform (102) in a data stream (14); The encoder configured as described above.

[0392] <Embodiment 62> decision but , For a first predetermined set of block sizes of predetermined block sizes, A number of selectable transforms (98) each including a primary transform (108) applied on the prediction residual (100) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on the selected intra-prediction mode. One or more The multi-stage transformation (106) of Do as follows An encoder as described in embodiment 61, configured as follows.

[0393] <Embodiment 63> decision but , Linear transformation(108) but Depending on the selected intra prediction mode, it is a separable 2D transform (116). Do as follows 63. An encoder as described in embodiment 62, configured as follows.

[0394] <Embodiment 64> 64. The encoder of embodiment 62 or 63, wherein the subgroup (112) depends on a selected intra-prediction mode.

[0395] <Embodiment 65> decision but , for a block size of a second predetermined set of predetermined block sizes that is smaller than the first predetermined set of predetermined block sizes; A set of selectable transformations (98) with a second number One or more As the non-separable transformation (118) of conduct 65. An encoder as in any one of embodiments 62 to 64, configured so as to

[0396] <Embodiment 66> An encoder as described in any one of embodiments 62 to 65, configured such that the manner in which the secondary transform (110) depends on the selected intra prediction mode includes intra prediction mode aggregation (e.g., quantization of the intra prediction mode index or quantization of the intra prediction direction of the angular intra prediction mode in the set of intra prediction modes 120).

[0397] <Embodiment 66a> 63. An embodiment in which the manner in which the primary transform (108) depends on the selected intra-prediction mode includes intra-prediction mode aggregation (e.g., quantization of intra-prediction mode indices and quantization of intra-prediction directions of angular intra-prediction modes in the set of intra-prediction modes 120). To The encoder described herein.

[0398] <Embodiment 67> The quantization parameter of the residual block (84) is One or more Check whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 67. The encoder of any one of embodiments 61 to 66, configured so as to

[0399] <Embodiment 71> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms (94) for block-based residual decoding; predicting an intra-predicted block (80) based on spatial neighborhoods (90) of the intra-predicted block (80) using a selected one of a set of intra-prediction modes (120) to obtain a prediction (104) of the intra-predicted block (80); determining a set (98) of selectable transforms (94) for the residual block (84) that cover the intra predicted block (80) from among the set of transforms (94) based on the selected intra prediction mode and a block size (96) of the residual block (84); Decoding a prediction residual (100) associated with a prediction (104) of an intra-predicted block (80) within a residual block (84) from the data stream (14) by deriving a selected transform (102) from the data stream (14) in a domain of a selected transform (102) from a set of selectable transforms (98). A decoder configured to:

[0400] <Embodiment 72> decision but , For a first predetermined set of block sizes of predetermined block sizes, A number of selectable transforms (98) each including a primary transform (108) applied on the prediction residual (100) and a secondary transform (110) applied on a subgroup (112) of the primary transform coefficients (114) of the primary transform (108), the secondary transform (110) being non-separable and dependent on the selected intra-prediction mode. One or more The multi-stage transformation (106) of Do as follows 72. A decoder as described in embodiment 71, configured as follows.

[0401] <Embodiment 73> decision but , The primary transform (108) depends on the selected intra-prediction mode and is a separable 2D transform (116). Do as follows 73. A decoder as described in embodiment 72, configured as follows.

[0402] <Embodiment 74> 74. The decoder of embodiment 72 or 73, wherein the subgroup (112) depends on a selected intra-prediction mode.

[0403] <Embodiment 75> decision but , for a block size of a second predetermined set of predetermined block sizes that is smaller than the first predetermined set of predetermined block sizes; A set of selectable transformations (98) with a second number One or more A non-separable transformation of ( 118 ) is entered conduct 75. A decoder as described in any one of embodiments 72 to 74, configured to

[0404] <Embodiment 76> A decoder as described in any one of embodiments 72 to 75, configured such that the manner in which the secondary transform (110) depends on the selected intra prediction mode includes intra prediction mode aggregation (e.g., quantization of the intra prediction mode index or quantization of the intra prediction direction of the angular intra prediction mode in the set of intra prediction modes 120).

[0405] <Embodiment 77> The quantization parameter of the residual block (84) is One or more Check whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 77. A decoder as described in any one of embodiments 71 to 76, configured to

[0406] <Embodiment 81> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, the encoder supporting a transform set (94) for block-based residual coding; determining (122) a list (98) of selectable transforms for the current residual block (84) from among the set of transforms (94) based on a reference transform selected for a neighboring residual block (84', 84'') [e.g., also from the set 94 for coding a prediction residual of the neighboring residual block]; encoding the prediction residual (100) in the residual block (84) using a selected transform (102) from a list of selectable transforms (98) and signaling the selected transform (102) in a data stream (14); The encoder configured as described above.

[0407] <Embodiment 82> Ordering a list of options (98) using a Lookup transformation (122) 82. The encoder of embodiment 81, configured as follows:

[0408] <Embodiment 83> The selectable list (98) is a transform corresponding to the reference transform process reference transform [e.g., wrt is a one-stage signaled by a high transform index 124, and thus is very likely a specialized transform, an analytical transform type for neighboring blocks of different sizes], The most probable selection [the first position, or the index value associated with the lowest index value of index 124, or the index value associated with the shortest VLC] position ],or The second most likely choice and the associated rank ("n # (122) 83. An encoder as described in embodiment 81 or 82, configured so as to

[0409] <Embodiment 84> determining (122) a list (98) of selectable transformations for the current residual block (84) from among a set of transformations (94) based on a reference transformation selected for a set of neighboring residual blocks (84', 84''); 84. An encoder as in any one of embodiments 81 to 83, configured so as to

[0410] <Embodiment 85> Ordering (122) the selectable list (98) by placing the reference transformations relative to one another in an order that corresponds to the frequency with which the reference transformations are selected for the set of neighboring residual blocks (84', 84''). 85. The encoder of embodiment 84, configured as follows.

[0411] <Embodiment 86> Add reference transformation to the selectable list (98) addition (122) 86. An encoder as in any one of embodiments 81 to 85, configured so as to

[0412] <Embodiment 87> The reference transform is analytically definable and separable [such as DCT, DST, FFT, etc.] Spectral decomposition transform and the encoder is Enter (122) in the selectable list (98) another transform derived by adapting the reference transform to the block size (96) of the current residual block (84). 87. The encoder of embodiment 86, configured as follows.

[0413] <Embodiment 88> Input, associated with the neighboring blocks (84', 84'') and the current residual block (84) One or more are sufficiently similar in certain characteristics, and / or Reference conversion is separable Spectral decomposition transform is This is done under the condition that 88. An encoder as described in any of embodiments 86 or 87, configured so as to

[0414] <Embodiment 89> One or more The characteristics of Intra Prediction Mode of One or more 89. The encoder of embodiment 88, comprising:

[0415] <Embodiment 90> Available list(98) One or more ordering (122) by ranking the transformations in the set of selectable transformations (98) according to their similarity to a reference transformation in a particular characteristic of From the available list (98), One or more determine a list of selectable transformations (98) in such a way as to exclude transformations that differ more than a certain degree from the reference transformation in a certain characteristic of 90. An encoder as in any one of embodiments 81 to 89, configured so as to

[0416] <Embodiment 91> One or more The characteristics of Linear transformations [e.g., equivalence of linear transformations is required] of One or more An encoder as described in embodiment 90, comprising:

[0417] <Embodiment 92> Selectable list (98) of nearby blocks (84') One or more ordering (122) by ranking the transformations in the set of selectable transformations (98) according to certain characteristics of Neighboring block (84') One or more determining a list of selectable transformations (98) in a manner that excludes transformations from the selectable list (98) depending on certain characteristics of 92. An encoder as in any one of embodiments 81 to 91, configured so as to

[0418] <Embodiment 93> One or more The characteristics of A number of meaningful conversion factors Of One or more 94. An encoder as described in embodiment 93, comprising:

[0419] <Embodiment 94> One or more The characteristics of A certain number of significant transform coefficients by ranking less likely transforms or by eliminating transforms that are of a multi-stage type having subgroups 112 of primary transform coefficients that undergo secondary transforms if the number of significant transform coefficients falls below a certain threshold (e.g., depending on the number of coefficients in the subgroups 112). Of One or more 93. An encoder as in embodiment 93 or 92, comprising:

[0420] <Embodiment 95> The quantization parameter of the current block is One or moreCheck whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 95. An encoder as in any one of embodiments 81 to 94, configured to

[0421] <Embodiment 101> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms (94) for block-based residual decoding; determining (122) a list (98) of selectable transforms for the current residual block (84) from among the set of transforms (94) based on a reference transform selected for a neighboring residual block (84', 84'') [e.g., also from the set 94 for coding a prediction residual of the neighboring residual block]; Decoding a prediction residual (100) in a residual block (84) by deriving a selected transform (102) from the data stream (14) in a domain of a selected transform (102) from a list of selectable transforms (98). A decoder configured to:

[0422] <Embodiment 102> Ordering a list of options (98) using a Lookup transformation (122) A decoder as described in embodiment 101, configured as follows.

[0423] <Embodiment 103> The selectable list (98) is a transform corresponding to the reference transform process reference transform [e.g., wrt is a one-stage signaled by a high transform index 124, and thus is very likely a specialized transform, an analytical transform type for neighboring blocks of different sizes], The most probable selection [the first position, or the index value associated with the lowest index value of index 124, or the index value associated with the shortest VLC] position ],or The second most likely choice and the associated rank ("n # (122) 103. A decoder as described in embodiment 101 or 102, configured to

[0424] <Embodiment 104> determining (122) a list (98) of selectable transformations for the current residual block (84) from among a set of transformations (94) based on a reference transformation selected for a set of neighboring residual blocks (84', 84''); 104. A decoder according to any one of embodiments 101 to 103, configured to

[0425] <Embodiment 105> Ordering (122) the selectable list (98) by placing the reference transformations relative to one another in an order that corresponds to the frequency with which the reference transformations are selected for the set of neighboring residual blocks (84', 84''). A decoder as described in embodiment 104, configured as follows.

[0426] <Embodiment 106> Entering a lookup transformation into a selectable list (98) (122) 106. A decoder according to any one of embodiments 101 to 105, configured to

[0427] <Embodiment 107> The reference transform is analytically definable and separable [such as DCT, DST, FFT, etc.] Spectral decomposition transform and the decoder Enter (122) in the selectable list (98) another transform derived by adapting the reference transform to the block size (96) of the current residual block (84). A decoder as described in embodiment 106, configured to:

[0428] <Embodiment 108> Input, associated with the neighboring blocks (84', 84'') and the current residual block (84) One or more are sufficiently similar in certain characteristics, and / or Reference conversion is separable Spectral decomposition transform is This is done under the condition that 108. A decoder as described in embodiment 106 or 107, configured to

[0429] <Embodiment 109> One or more The characteristics of One or more A decoder as described in embodiment 108, including an intra prediction mode.

[0430] <Embodiment 110> Available list(98) One or more ordering (122) by ranking the transformations in the set of selectable transformations (98) according to their similarity to a reference transformation in a particular characteristic of From the available list (98), One or more determine a list of selectable transformations (98) in such a way as to exclude transformations that differ more than a certain degree from the reference transformation in a certain characteristic of 110. A decoder as in any one of embodiments 101 to 109, configured to

[0431] <Embodiment 111> One or more The characteristics of One or more 101. The decoder of embodiment 100, comprising a linear transform (eg, equality of the linear transform is required).

[0432] <Embodiment 112> Selectable list (98) of nearby blocks (84') One or moreordering (122) by ranking the transformations in the set of selectable transformations (98) according to certain characteristics of Neighboring block (84') One or more determining a list of selectable transformations (98) in a manner that excludes transformations from the selectable list (98) depending on certain characteristics of 112. A decoder according to any one of embodiments 101 to 111, configured to:

[0433] <Embodiment 113> One or more The characteristics of One or more A decoder as described in embodiment 112, comprising a certain number of significant transform coefficients.

[0434] <Embodiment 114> One or more The characteristics of by ranking less likely transforms or excluding transforms that are of the multi-stage type having subgroups 112 of primary transform coefficients that undergo secondary transforms if the number of significant transform coefficients falls below a certain threshold (e.g., depending on the number of coefficients in the subgroups 112); One or more 113. A decoder as described in embodiment 112 or 113, comprising a number of significant transform coefficients.

[0435] <Embodiment 115> The quantization parameter of the current block is One or more Check whether the quantization parameter set of If included, determine but , A set of selectable transformations (98) One or more associated with a given set of quantization parameters One or more , a given set of transformations is further input. 115. A decoder according to any one of embodiments 101 to 114, configured to

[0436] <Embodiment 121> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, the encoder supporting a transform set (94) for block-based residual coding; a list (98) of selectable transformations for the current residual block (84) from among the set of transformations; If the quantization parameter corresponds to a quantization finer than a certain threshold, inputting into a list (98) of selectable transforms for the current residual block at least one multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block and a secondary transform (110) applied on a subgroup (112) of primary transform coefficients (114) of the primary transform; If the quantization parameter corresponds to a quantization coarser than a certain threshold, In addition to or instead of the at least one multi-stage transform (106), a modification of at least one multi-stage transform in which primary transform coefficients (114) beyond a subgroup (112) are zeroed is input to the list of selectable transforms for the current residual block. The quantization parameter of the current block is determined in such a manner that the quantization parameter of the current block is determined as follows: encoding the prediction residual (100) in the residual block (84) using a selected transform (102) from a list of selectable transforms (98) and signaling the selected transform in a data stream (14); The encoder configured as described above.

[0437] <Embodiment 122> 1. An encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding; a list (98) of selectable transformations for the current residual block from among a set of transformations based on the reference transformations selected for the neighboring residual blocks; If the number or sum is greater than a certain threshold, inputting into a list (98) of selectable transforms for the current residual block at least one multi-stage transform including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform; If the number or sum is less than a certain threshold, In addition to or instead of the at least one multi-stage transform, a modification of at least one multi-stage transform in which primary transform coefficients beyond a subgroup are zeroed is input to the list of selectable transforms for the current residual block. like One or more (122) in a manner that depends on the number of significant transform coefficients in the neighboring residual blocks (84', 84'') of Encoding the prediction residual in the residual block using a selected transform from a list of selectable transforms and signaling the selected transform in the data stream. An encoder comprising:

[0438] <Embodiment 123> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms (94) for block-based residual decoding; a list (98) of selectable transformations for the current residual block (84) from among the set of transformations; If the quantization parameter corresponds to a quantization finer than a certain threshold, inputting into a list (98) of selectable transforms for the current residual block at least one multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block and a secondary transform (110) applied on a subgroup (112) of primary transform coefficients (114) of the primary transform; If the quantization parameter corresponds to a quantization coarser than a certain threshold, In addition to or instead of the at least one multi-stage transform (106), a modification of at least one multi-stage transform in which primary transform coefficients (114) beyond a subgroup (112) are zeroed is input to the list of selectable transforms for the current residual block. The quantization parameter of the current block is determined in such a manner that the quantization parameter of the current block is determined as follows: Decoding a prediction residual (100) in a residual block (84) by deriving a selected transform (102) from the data stream (14) in a domain of a selected transform (102) from a list of selectable transforms (98). A decoder configured to:

[0439] <Embodiment 124> 1. A decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms for block-based residual decoding; a list (98) of selectable transformations for the current residual block from among a set of transformations based on the reference transformations selected for the neighboring residual blocks; If the number or sum is greater than a certain threshold, inputting into a list (98) of selectable transforms for the current residual block at least one multi-stage transform including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform; If the number or sum is less than a certain threshold, In addition to or instead of the at least one multi-stage transform, a modification of at least one multi-stage transform in which primary transform coefficients beyond a subgroup are zeroed is input to the list of selectable transforms for the current residual block. like One or more (122) in a manner that depends on the number of significant transform coefficients in the neighboring residual blocks (84', 84'') of Decoding the prediction residual in the residual block by selectively deriving a selected transform from the data stream in a domain of a selected transform from a list of selectable transforms. A decoder configured to:

[0440] <Embodiment 131> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, the encoder supporting a transform set (94) for block-based residual coding; determining (130) a list (98) of selectable transforms for the current residual block from among the set of transforms (94); encoding transform coefficient data (126) representative of the prediction residual (100) in the residual block (84) into a data stream (14) using a selected transform (102) from a list of selectable transforms (98); Signaling (128) the selected transform (102) in the data stream (14) and / or determining (130) the selectable list (98) in a manner dependent on the transform coefficient data (126) or a portion of the data transform coefficient data (126). The encoder configured as described above.

[0441] <Embodiment 132> determining (130) a list of selectable transforms (98) by inputting into the selectable list (98) a number of transforms from the set of transforms (94) that depend on the transform coefficient data (126), or a portion of the transform coefficient data; The signaling (128) of the selected transform (102) in the data stream (14) is performed in such a way that the lower the coding rate in the data stream (14) is consumed for the signaling of the selected transform (102), the lower the number. An encoder as described in embodiment 131, configured as follows.

[0442] <Embodiment 133> determining (130) a list of selectable transforms (98) by ordering (130) the list of selectable transforms (98) in dependence on the transform coefficient data (126) or a portion of the transform coefficient data (126); signaling (128) the selected transform (102) in the data stream using a variable length code whose codeword is assigned to a transform of the list of selectable transforms (98) according to the order of the transforms in the list of selectable transforms (98); An encoder as described in embodiment 131, configured as follows.

[0443] <Embodiment 134> the transform coefficient data includes indications of significant transform coefficient locations (132) of the selected transform (102) and values ​​of transform coefficients (134) of the selected transform traversed forward (132) from the significant transform coefficient locations in a predetermined scan order (136) that sequentially traverses the coefficient locations of the selected transform; Decision (130) depends on instructions An encoder as described in embodiment 132 or 133, configured so as to

[0444] <Embodiment 135> Counting the number of significant transform coefficients (134) of the selected transform (102); Decision (130) depends on count An encoder as described in embodiment 132, 133 or 134, configured so as to

[0445] <Embodiment 136> evaluating the transform coefficient data (126) as to whether a number of significant transform coefficients (134) of the selected transform (102) within a subgroup (138) of transform coefficients (134) of the selected transform (102) exceeds a certain threshold; Decision (130) that depends on counting, If the number is greater than a certain threshold, A list (98) of selectable transforms (90) for the current residual block (84) includes a linear transform to be applied on the prediction residual (100) of the current residual block to result in a subgroup (138) of transform coefficients of the selected transform (102), and a linear transform ( 108 and a secondary transform applied on a subgroup (112) of the primary transform coefficients (114) of If the number is below a certain threshold, In the list of available transformations for the current residual block, any multi-stage transform (106) including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform; is not entered Do as follows An encoder as described in embodiment 134 or 135, configured to

[0446] <Embodiment 201> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms (94) for block-based residual decoding; determining (130) a list (98) of selectable transforms for the current residual block from among the set of transforms (94); A selected transform (102) from a list of selectable transforms (98) is used to generate transform coefficient data (126) representing a prediction residual (100) in a residual block (84) in a data stream (14). from Decrypt, Deriving (128) the selected transform (102) from the data stream (14) and / or determining (130) the selectable list (98) in a manner dependent on the transform coefficient data (126) or a portion of the data transform coefficient data (126). A decoder configured to:

[0447] <Embodiment 202> determining (130) a list of selectable transforms (98) by inputting into the selectable list (98) a number of transforms from the set of transforms (94) that depend on the transform coefficient data (126), or a portion of the transform coefficient data; The signaling (128) of the selected transform (102) in the data stream (14) is performed in such a way that the lower the coding rate in the data stream (14) is consumed for the signaling of the selected transform (102), the lower the number. A decoder as described in embodiment 201, configured as described above.

[0448] <Embodiment 203> determining (130) a list of selectable transforms (98) by ordering (130) the list of selectable transforms (98) in dependence on the transform coefficient data (126) or a portion of the transform coefficient data (126); signaling (128) the selected transform (102) in the data stream using a variable length code whose codeword is assigned to a transform of the list of selectable transforms (98) according to the order of the transforms in the list of selectable transforms (98); A decoder as described in embodiment 201, configured as follows.

[0449] <Embodiment 204> the transform coefficient data includes indications of significant transform coefficient locations (132) of the selected transform (102) and values ​​of transform coefficients (134) of the selected transform traversed forward (132) from the significant transform coefficient locations in a predetermined scan order (136) that sequentially traverses the coefficient locations of the selected transform; Decision (130) depends on instructions 204. A decoder as described in embodiment 202 or 203, configured to

[0450] <Embodiment 205> Counting the number of significant transform coefficients (134) of the selected transform (102); Decision (130) depends on count 205. A decoder according to embodiment 202, 203 or 204, configured to

[0451] <Embodiment 206> evaluating the transform coefficient data (126) as to whether a number of significant transform coefficients (134) of the selected transform (102) within a subgroup (138) of transform coefficients (134) of the selected transform (102) exceeds a certain threshold; Decision (130) that depends on counting, If the number is greater than a certain threshold, A list (98) of selectable transforms (90) for the current residual block (84) includes a linear transform to be applied on the prediction residual (100) of the current residual block to result in a subgroup (138) of transform coefficients of the selected transform (102), and a linear transform ( 108 and a secondary transform applied on a subgroup (112) of the primary transform coefficients (114) of If the number is below a certain threshold, In the list of available transformations for the current residual block, any multi-stage transform (106) including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients of the primary transform; is not entered Do as follows 206. A decoder as described in embodiment 204 or 205, configured to

[0452] <Embodiment 141> 1. An encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding; determining (130) a list of selectable transforms (98) for the current residual block from among the set of transforms; signaling (128) the selected transformation in the data stream; encoding (140) transform coefficients representing the prediction residual in the residual block using a selected transform from the list of selectable transforms in a manner that depends on the selected transform (102) or the set of selectable transforms (98); The encoder configured as described above.

[0453] <Embodiment 142> the list of selectable transforms includes transforms with different numbers of transform coefficients; The transform coefficients are coded in such a way that the lower the code rate consumed in the data stream for coding the transform coefficients, the lower the number of transform coefficients of the selected transform. An encoder as described in embodiment 141.

[0454] <Embodiment 143> the list of selectable transforms includes transforms with different numbers of transform coefficients (134); encoding the transform coefficients includes encoding indications of significant transform coefficient positions (132) of the selected transform (102) and values ​​of transform coefficients (134) of the selected transform traversed forward from the significant transform coefficient positions according to a predetermined scan order (136) that sequentially traverses the coefficient positions of the selected transform; The encoder The lower the code rate used in the data stream to code the instructions, the lower the number of transform coefficients (134) of the selected transform. configured to encode the transform coefficients in such a manner that An encoder as described in embodiment 141 or 142.

[0455] <Embodiment 144> the list of selectable transforms includes a transform with a first number of transform coefficients (134) and a transform with a second number of transform coefficients (134) greater than the first number; encoding the transform coefficients If the number of transform coefficients of the selected transform is a second number, encoding indications of significant transform coefficient positions of the selected transform and values ​​of transform coefficients of the selected transform traversed forward from the significant transform coefficient positions according to a predetermined scan order that sequentially traverses the coefficient positions of the selected transform; If the number of transform coefficients of the selected transform is a first number, Encoding values ​​of all of a first number of transform coefficients of a selected transform Including, An encoder as described in embodiment 141 or 142.

[0456] <Embodiment 145> The list of selectable transformations is the linear transformation applied on the current residual block, and the linear transformation of and at least one multi-stage transform each including a secondary transform applied on subgroups of the primary transform coefficients, the subgroups differing by the multi-stage transform of the at least one multi-stage transform; The encoder converts the transform coefficients into and / or by encoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed from the significant transform coefficient position forward by a predefined scan order that sequentially traverses the coefficient positions of the selected transform, the indication including a flag whether the significant transform coefficient position is within a subgroup of transform coefficients if the selected transform is a multi-stage transform, and if the transform coefficient position is within the subgroup of transform coefficients, the indication indicating the significant transform coefficient position to a position last traversed among all transform coefficients of the selected transform in the predefined scan order, and if the transform coefficient position is not within the subgroup of transform coefficients (138), the indication indicating the significant transform coefficient position to a position last traversed among all transform coefficients not within the subgroup in the predefined scan order, and the predefined scan order traversing all transform coefficients in the subgroup after traversing all transform coefficients not within the subgroup; If the selected transform is of a multi-stage type, by encoding the values ​​of the transform coefficients of the selected transform using entropy coding that uses a probability estimate that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform, If the selected transform is of a multi-stage type, in a manner that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. One or more by encoding values ​​of transform coefficients of a selected transform using entropy coding that uses probability estimates that depend on neighboring transform coefficients; configured to encode An encoder as in any one of embodiments 141 to 144.

[0457] <Embodiment 145a> The list of selectable transformations is the linear transformation applied on the current residual block, and the linear transformation of and at least one multi-stage transform each including a secondary transform applied on a subgroup of the primary transform coefficients, the selectable list including at least one transform of only the primary transform type; The encoder converts the transform coefficients into and / or by encoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed from the significant transform coefficient position forwards according to a predefined scan order that sequentially traverses the coefficient positions of the selected transform, the indication being for the last traversed position among all transform coefficients of the selected transform in the predefined scan order if the selected transform is of the type of linear transform only, or for the last traversed position among all transform coefficients not in a subgroup in the predefined scan order if the selected transform is a multi-stage transform (106); and / or by encoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed from the significant transform coefficient position forward by a predefined scan order that sequentially traverses coefficient positions of the selected transform, the indication including a flag whether the significant transform coefficient position is within a subgroup of transform coefficients if the selected transform is a multi-stage transform, and if the transform coefficient position is within the subgroup of transform coefficients, the indication indicating the significant transform coefficient position to a position last traversed among all transform coefficients of the selected transform in the predefined scan order, and if the transform coefficient position is not within the subgroup of transform coefficients, the indication indicating the significant transform coefficient position to a position last traversed among all transform coefficients not within the subgroup in the predefined scan order, and the predefined scan order traversing all transform coefficients in the subgroup after traversing all transform coefficients not within the subgroup; If the selected transform is of a multi-stage type, by encoding the values ​​of the transform coefficients of the selected transform using entropy coding that uses a probability estimate that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform, If the selected transform is of a multi-stage type, in a manner that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. One or more by encoding the values ​​of the transform coefficients of the selected transform using entropy coding that uses probability estimates that depend on neighboring transform coefficients of configured to encode An encoder as in any one of embodiments 141 to 145.

[0458] <Embodiment 146> configured to signal the selected transformation using a variable length coding index; encoding the transform coefficients by encoding values ​​of the transform coefficients of the selected transform using binarization and binary entropy coding; the probability estimates used in the binary entropy coding depend on the VLC index, and / or Thresholding depends on VLC index It was configured as follows: An encoder as in any one of embodiments 141 to 145.

[0459] <Embodiment 147> encoding the transform coefficients by encoding values ​​of the transform coefficients of the selected transform using entropy coding; In a way that the entropy coding depends on the chosen transform. One or more neighbors Depends on the conversion factor It was configured as follows: An encoder as in any one of embodiments 141 to 146.

[0460] <Embodiment 148> The conversion factor is In a manner that depends on the position of the transform coefficients in the selected transform One or more neighbors Select a local template for the conversion coefficients of One or more For a conversion coefficient in the domain of possible values ​​of nearby conversion coefficients One or more determining a selected context for the transform coefficients using a mapping of Using probability estimates associated with selected contexts for entropy coding the values ​​of transform coefficients encoding the values ​​of transform coefficients of the selected transform using entropy coding by The local template selection and / or mapping depends on the selected transformation, It was configured as follows: An encoder as described in embodiment 147.

[0461] <Embodiment 149> and encoding the values ​​of the transform coefficients of the selected transform using binary entropy coding of the binarization of the values, the probability estimates associated with the selected context being used for binarization bins indicating whether the value is zero or not. An encoder as described in embodiment 148.

[0462] <Embodiment 150> Signaling the selected transformation in the data stream Relative positioning of the signaling of the selected transform with respect to the transform coefficient data, and / or Breaking down signaling into syntactic elements depends on the list of possible transformations 150. An encoder as in any one of embodiments 141 to 149, configured to

[0463] <Embodiment 161> 1. A decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms for block-based residual decoding; determining (130) a list of selectable transforms (98) for the current residual block from among the set of transforms; Deriving (128) the selected transformation from the data stream; Decode (140) transform coefficients representing the prediction residual in the residual block from the data stream using a selected transform from the list of selectable transforms in a manner dependent on the selected transform (102) or the set of selectable transforms (98). A decoder configured to:

[0464] <Embodiment 162> the list of selectable transforms includes transforms with different numbers of transform coefficients; Decoding the transform coefficients in such a way that the lower the code rate consumed in the data stream for the transform coefficients, the lower the number of transform coefficients of the selected transform. A decoder as described in embodiment 161.

[0465] <Embodiment 163> the list of selectable transforms includes transforms with different numbers of transform coefficients (134); Decoding the transform coefficients includes decoding indications of significant transform coefficient positions (132) of the selected transform (102) and values ​​of transform coefficients (134) of the selected transform traversed forward from the significant transform coefficient positions according to a predetermined scan order (136) that sequentially traverses the coefficient positions of the selected transform; The decoder converts the transform coefficients into The lower the coding rate consumed in the data stream for the instruction, the lower the number of transform coefficients (134) of the selected transform. configured to decode in such a manner that A decoder as described in embodiment 161 or 162.

[0466] <Embodiment 164> the list of selectable transforms includes a transform with a first number of transform coefficients (134) and a transform with a second number of transform coefficients (134) greater than the first number; Decoding the transform coefficients If the number of transform coefficients of the selected transform is a second number, decoding indications of significant transform coefficient positions of the selected transform and values ​​of transform coefficients of the selected transform traversed forward from the significant transform coefficient positions according to a predetermined scan order that sequentially traverses the coefficient positions of the selected transform; If the number of transform coefficients of the selected transform is a first number, Decoding values ​​of all of a first number of transform coefficients of a selected transform Including, A decoder as described in embodiment 161 or 162.

[0467] <Embodiment 165> the list of selectable transforms includes at least one multi-stage transform each including a primary transform applied on the current residual block and a secondary transform applied on subgroups of the primary transform coefficients, the subgroups differing by the multi-stage transform of the at least one multi-stage transform; The decoder converts the transform coefficients into and / or by decoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed from the significant transform coefficient position forward by a predefined scan order that sequentially traverses the coefficient positions of the selected transform, the indication including a flag whether the significant transform coefficient position is within a subgroup of transform coefficients if the selected transform is a multi-stage transform, and if the transform coefficient position is within the subgroup of transform coefficients, the indication indicates the significant transform coefficient position to a position last traversed among all transform coefficients of the selected transform in the predefined scan order, and if the transform coefficient position is not within the subgroup of transform coefficients (138), the indication indicates the significant transform coefficient position to a position last traversed among all transform coefficients not within the subgroup in the predefined scan order, and the predefined scan order traverses all transform coefficients in the subgroup after traversing all transform coefficients not within the subgroup; If the selected transform is of a multi-stage type, the entropy using a probability estimate that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. Decryption by decoding the values ​​of the transform coefficients of the selected transform using If the selected transform is of a multi-stage type, in a manner that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. One or more Entropy using probability estimates that depend on the transform coefficients in the neighborhood of Decryption by decoding the values ​​of the transform coefficients of the selected transform using Decryption configured to A decoder according to any one of embodiments 161 to 164.

[0468] <Embodiment 165a> the list of selectable transforms includes at least one multi-stage transform each including a linear transform applied on the current residual block and a secondary transform applied on a subgroup of the primary transform coefficients, the selectable list including at least one transform of only the linear transform type; The decoder converts the transform coefficients into and / or by decoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed forward from the significant transform coefficient position by a predefined scan order that sequentially traverses the coefficient positions of the selected transform, the indication being for the last traversed position among all transform coefficients of the selected transform in the predefined scan order if the selected transform is of a type of linear transform only, or for the last traversed position among all transform coefficients not in a subgroup in the predefined scan order if the selected transform is a multi-stage transform (106); and / or by decoding an indication of a significant transform coefficient position of the selected transform and values ​​of transform coefficients of the selected transform traversed forward from the significant transform coefficient position by a predefined scan order that sequentially traverses coefficient positions of the selected transform, the indication including a flag whether the significant transform coefficient position is within a subgroup of transform coefficients if the selected transform is a multi-stage transform, and if the transform coefficient position is within the subgroup of transform coefficients, the indication indicates the significant transform coefficient position to a position last traversed among all transform coefficients of the selected transform in the predefined scan order, and if the transform coefficient position is not within the subgroup of transform coefficients, the indication indicates the significant transform coefficient position to a position last traversed among all transform coefficients not within the subgroup in the predefined scan order, and the predefined scan order traverses all transform coefficients in the subgroup after traversing all transform coefficients not within the subgroup; If the selected transform is of a multi-stage type, the entropy using a probability estimate that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. Decryption by decoding the values ​​of the transform coefficients of the selected transform using If the selected transform is of a multi-stage type, in a manner that depends on whether the transform coefficient is within a subgroup of transform coefficients of the selected transform. One or more Entropy using probability estimates that depend on the transform coefficients in the neighborhood of Decryptionby decoding the values ​​of the transform coefficients of the selected transform using Decryption configured to A decoder according to any one of embodiments 161 to 165.

[0469] <Embodiment 166> configured to derive the selected transformation using the variable length coded index; Decoding the transform coefficients by decoding values ​​of the transform coefficients of the selected transform using binarization and binary entropy decoding; the probability estimates used in the binary entropy coding depend on the VLC index, and / or Thresholding depends on VLC index It was configured as follows: A decoder according to any one of embodiments 161 to 165.

[0470] <Embodiment 167> Decoding the transform coefficients by decoding values ​​of the transform coefficients of the selected transform using entropy decoding; In entropy decoding, in a manner that depends on the selected transform One or more Depends on nearby conversion coefficients It was configured as follows: A decoder according to any one of embodiments 161 to 166.

[0471] <Embodiment 168> The conversion factor is In a manner that depends on the position of the transform coefficients in the selected transform One or more Select a local template of nearby transformation coefficients, One or more For a conversion coefficient in the domain of possible values ​​of nearby conversion coefficients One or more determining a selected context for the transform coefficients using a mapping of Using probability estimates associated with selected contexts for entropy decoding values ​​of transform coefficients and decoding the values ​​of the transform coefficients of the selected transform using entropy decoding by The local template selection and / or mapping depends on the selected transformation, It was configured as follows: A decoder as described in embodiment 167.

[0472] <Embodiment 169> Decoding values ​​of transform coefficients of the selected transform using binary entropy decoding of the binarization of the values, and a probability estimate associated with the selected context is used for a binarization bin indicating whether the value is zero or not. A decoder as described in embodiment 168.

[0473] <Embodiment 170> Signaling the selected transformation in the data stream Relative positioning of the signaling of the selected transform with respect to the transform coefficient data, and / or Breaking down signaling into syntactic elements depends on the list of available transformations Do it in such a way 170. A decoder as described in any one of embodiments 161 to 169, configured to

[0474] <Embodiment 181> 1. An encoder for encoding an image into a data stream using block-based prediction and block-based residual coding, the encoder supporting a transform set for block-based residual coding; determining a list of selectable transforms for the current residual block from among the set of transforms; encoding the prediction residual in the residual block using a selected transform from a list of selectable transforms; signaling (128) the selected transformation in the data stream using a transformation indicator (124); the conversion indicator is signaled in the data stream using a single or multiple digit code, each set of single or multiple digits being uniquely associated with a respective conversion characteristic; The encoder configured as described above.

[0475] <Embodiment 182> Each conversion characteristic is whether the selected transformation is a multi-stage transformation, and whether the selected transform is a transform having fewer than a predetermined number of transform coefficients; An encoder as described in embodiment 181, which is one of the above.

[0476] <Embodiment 183> Encoding at least one of the set of at least one digit using entropy coding that uses a probability estimate that depends on the transformation indicators of neighboring residual blocks. An encoder as described in embodiment 181 or 182, configured so as to

[0477] <Embodiment 191> 1. A decoder for decoding an image from a data stream using block-based prediction and block-based residual decoding, the decoder supporting a set of transforms for block-based residual decoding; determining a list of selectable transforms for the current residual block from among the set of transforms; Decoding a prediction residual in a residual block from the data stream (14) using a selected transform from a list of selectable transforms; Deriving (128) a selected transformation from the data stream using the transformation indicator (124); A conversion indicator is derived from the data stream using a single or multiple digit code (e.g., a syntax element that can be parsed from 14, where 124 bins or 124 are constructed that undergo binary arithmetic encoding), each of which is uniquely associated with a respective conversion characteristic, A decoder configured to:

[0478] <Embodiment 192> Each conversion characteristic is whether the selected transformation is a multi-stage transformation, and whether the selected transform is a transform having fewer than a predetermined number of transform coefficients; 192. A decoder as described in embodiment 191, wherein the decoder is one of the following:

[0479] <Embodiment 193> Decoding at least one of the set of at least one digit using entropy decoding using a probability estimate that depends on the transformation indicators of neighboring residual blocks. 193. A decoder as described in embodiment 191 or 192, configured to

[0480] A method performed by any of the encoders / decoders listed above, A computer program for carrying out the method, 2 is a data stream produced by the encoding method of the present invention.

[0481] The present invention can be further realized by the following embodiments, which can be combined with any of the examples and embodiments described and claimed herein.

[0482] <Embodiment 1> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, comprising: the encoder supports a set of transforms (94) for the block-based residual coding; determining (122) a candidate list (98) of transforms for the current residual block (84) from the set of transforms (94) based on the height and width of the current residual block (84); encoding a prediction residual (100) in the residual block (84) using a selected transform (102) from the list of selectable transforms (98) and selectively signaling the selected transform (102) in the data stream (14); The encoder configured as described above.

[0483] <Embodiment 2> the encoder supports multiple block sizes; for each block dimension, the set of transformations (94) includes a subset of transformations whose domains correspond to the respective block dimensions; for each transform of the set of transforms (94), the encoder supports an inverse transform whose domain matches the range of the respective transform, and uses the inverse transform to fill a decoded image buffer in those residual blocks for which the respective transform has been selected; the encoder is configured, when determining the transform candidate list (98), to populate the transform candidate list (98) exclusively with transforms from among the subset of transforms whose domain corresponds to the height and width of the current residual block (84). An encoder as described in embodiment 1.

[0484] <Embodiment 3> When determining the conversion candidate list (98), if the height and width of the current residual block satisfy a first criterion, populating the candidate transform list (98) with one or more transforms from the subset of transforms whose domain corresponds to the height and width of the current residual block (84) and whose range is smaller than the domain; If the height and width of the current residual block (84) do not satisfy a first criterion, remove from the list of candidate transforms the one or more transforms from the subset of transforms whose domain corresponds to the height and width of the current residual block (84) and whose range is smaller than the domain. The encoder according to embodiment 1 or 2, configured as described above.

[0485] <Embodiment 4> 4. The encoder of claim 3, wherein the one or more transforms are one or more of DCT-II, DCT-VII, and DCT-III.

[0486] <Embodiment 5> When determining the conversion candidate list (98), if the height and width of the current residual block (84) satisfy a second criterion, populate the list of candidate transforms (98) with an identity transform from among the subset of transforms whose domain corresponds to the height and width of the current residual block (84); or If the height and width of the current residual block (84) do not satisfy a second criterion, exclude from the list of candidate transforms the identity transform from among the subset of transforms whose domain corresponds to the height and width of the current residual block (84). An encoder as described in any one of embodiments 1 to 4, configured as described above.

[0487] <Embodiment 6> An encoder as described in any one of embodiments 1 to 5, configured to check the first and / or second criteria by checking whether the height, the width, both the height and the width, and / or the product of the height and the width exceed a certain threshold.

[0488] <Embodiment 7> When determining the conversion candidate list (98), if the height and width of the current residual block (84) satisfy a third criterion, populating the transform candidate list (98) with a separable spectral decomposition transform from among the subset of transforms whose domain corresponds to the size of the current residual block (84); If the height and width of the current residual block (84) do not satisfy a third criterion, then removing from the candidate transform list (98) the separable spectral decomposition transform from among the subset of transforms whose domain corresponds to the size of the current residual block (84). An encoder as described in any one of embodiments 1 to 6, configured as described above.

[0489] <Embodiment 8> 8. The encoder of embodiment 7, configured to verify the third criterion by verifying whether the ratio of the height to the width exceeds a particular threshold.

[0490] <Embodiment 9> When determining the conversion candidate list (98), if the height and width of the current residual block correspond to a first predetermined block dimension, inputting into the transform candidate list (98) a first one or more transforms from the subset of transforms whose domain corresponds to the height and width of the current residual block (84) and which is a multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block (84) and a secondary transform (110) applied on a first subgroup (112) of primary transform coefficients (114) of the primary transform (108); if the height and width of the current residual block (84) correspond to a second predetermined block dimension, inputting into the transform candidate list (98) a second one or more transforms from the subset of transforms whose domain corresponds to the height and width of the current residual block (84) and which is a multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block and a secondary transform (110) applied on a second subgroup (112) of primary transform coefficients (114) of the primary transform (108); a scaling that images a first range of the linear transform (108) of the first one or more transforms onto the second range of the linear transform (108) of the first one or more transforms does not image a perimeter of the first subgroup within the first range onto a perimeter of the second subgroup within the second range. An encoder as described in any one of embodiments 1 to 8, configured as described above.

[0491] <Embodiment 10> 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, comprising: the decoder supports a set of transforms (94) for the block-based residual decoding; determining (122) a candidate list (98) of transforms for the current residual block (84) from the set of transforms (94) based on the height and width of the current residual block (84); Decoding a prediction residual (100) in the residual block (84) by selectively deriving a selected transform (102) from the data stream (14) in a domain of the selected transform (102) from the list of selectable transforms (98). A decoder configured to:

[0492] <Embodiment 11> the decoder supports multiple block sizes (96); for each block size (96), said set of transformations (94) includes a subset of transformations whose range corresponds to said respective block size (96); the decoder is configured, when determining the transform candidate list (98), to populate the transform candidate list (98) exclusively with transforms from among the subset of transforms whose range corresponds to the height and width of the current residual block (84). A decoder as described in embodiment 10.

[0493] <Embodiment 12> When determining the conversion candidate list (98), if the height and width of the current residual block (84) satisfy a first criterion, populating the candidate transform list (98) with one or more transforms from the subset of transforms whose range corresponds to the height and width of the current residual block (84) and whose domain is smaller than the range; If the height and width of the current residual block (84) do not satisfy a first criterion, remove from the candidate transform list (98) the one or more transforms from the subset of transforms whose range corresponds to the height and width of the current residual block (84) and whose domain is smaller than the domain. 12. A decoder as described in embodiment 10 or 11, configured as described above.

[0494] <Embodiment 13> 13. The decoder of embodiment 12, wherein the one or more transforms are one or more of an inverse DCT-II, DCT-VII and DCT-III.

[0495] <Embodiment 14> When determining the conversion candidate list (98), if the height and width of the current residual block (84) satisfy a second criterion, enter into the list of candidate transforms (98) an identity transform from among the subset of transforms whose range corresponds to the height and width of the current residual block (84); or If the height and width of the current residual block (84) do not satisfy a second criterion, exclude from the candidate transform list (98) the identity transform from among the subset of transforms whose range corresponds to the height and width of the current residual block (84). A decoder as described in any one of embodiments 10 to 13, configured as follows.

[0496] <Embodiment 15> A decoder described in any one of embodiments 10 to 14, configured to check the first and / or second criterion by checking whether the height, the width, both the height and the width, and / or the product of the height and the width exceed a certain threshold.

[0497] <Embodiment 16> When determining the conversion candidate list (98), populate the transform candidate list (98) using a separable spectral decomposition transform from among the subset of transforms whose range corresponds to the size of the current residual block (84) if the height and width of the current residual block (84) satisfy a third criterion; or If the height and width of the current residual block (84) do not satisfy a third criterion, exclude from the list of candidate transforms the separable spectral decomposition transform from among the subset of transforms whose range corresponds to the size of the current residual block (84). A decoder as described in any one of embodiments 10 to 15, configured as follows.

[0498] <Embodiment 17> 17. The decoder of embodiment 16, configured to verify the third criterion by verifying whether the ratio of the height to the width exceeds a particular threshold.

[0499] <Embodiment 18> When determining the conversion candidate list (98), if the height and width of the current residual block (84) correspond to a first predetermined block dimension, inputting into the transform candidate list (98) a first one or more transforms from among the subset of transforms whose range corresponds to the height and width of the current residual block (84) and which invert a multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block (84) and a secondary transform (110) applied on a first subgroup (112) of primary transform coefficients (114) of the primary transform (108); or if the height and width of the current residual block (84) correspond to a second predetermined block dimension, inputting into the transform candidate list (98) a second one or more transforms from the subset of transforms whose range corresponds to the height and width of the current residual block (84) and which invert a multi-stage transform (106) including a primary transform (108) applied on a prediction residual (100) in the current residual block (84) and a secondary transform (110) applied on a second subgroup (112) of primary transform coefficients (114) of the primary transform (108); a scaling that images a first range of the linear transform (108) of the first one or more transforms onto the second range of the linear transform (108) of the first one or more transforms does not image a perimeter of the first subgroup within the first range onto a perimeter of the second subgroup within the second range. A decoder as described in any one of embodiments 10 to 17, configured as follows.

[0500] <Embodiment 19> 1. A method for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, comprising: a transform set (94) for the block-based residual coding, the method comprising: determining (122) a candidate list (98) of transforms for the current residual block (84) from the set of transforms (94) based on the height and width of the current residual block (84); encoding a prediction residual (100) in said residual block (84) using a selected transform (102) from the list of selectable transforms (98) and selectively signaling said selected transform (102) in said data stream (14); A method comprising:

[0501] <Embodiment 20> 1. A method for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, comprising: a transform set (94) for the block-based residual decoding, the method comprising: determining (122) a candidate list (98) of transforms for the current residual block (84) from the set of transforms (94) based on the height and width of the current residual block (84); decoding a prediction residual (100) in said residual block (84) by selectively deriving from said data stream (14) a domain of a selected transform (102) from said list of selectable transforms (98); A method comprising:

[0502] <Embodiment 21> 1. An encoder for encoding an image (12) into a data stream (14) using block-based prediction and block-based residual coding, comprising: the encoder supports a set of transforms (94) for the block-based residual coding; determining (130) a list (98) of selectable transforms (98) for the current residual block (84) from among the set of transforms (94); encoding transform coefficient data (126) representing a prediction residual (100) in said residual block (84) into said data stream (14) using a selected transform (102) from the list of selectable transforms (98); signaling (128) the selected transform (102) in the data stream (14) and / or determining (130) the selectable list (98) in a manner dependent on the transform coefficient data (126) or a portion of the transform coefficient data (126). The encoder configured as described above.

[0503] <Embodiment 22> determining (130) the list of selectable transforms (98) by inputting into the selectable list (98) a number of transforms from the set of transforms (94) that depend on the transform coefficient data (126) or a portion of the transform coefficient data (126); The signaling (128) of the selected transformation (102) in the data stream (14) is performed in such a way that the lower the code rate consumed in the data stream (14) for the signaling of the selected transformation (102), the lower the number. An encoder as described in embodiment 21, configured as follows.

[0504] <Embodiment 23> determining (130) the list of selectable transforms (98) by ordering (130) the list of selectable transforms (98) in dependence on the transform coefficient data (126) or a portion of the transform coefficient data (126); and performing the signaling (128) of the selected transform (102) in the data stream (14) using a variable length code, the codewords being assigned to transforms in the list of selectable transforms (98) according to the order of the transforms in the list of selectable transforms (98). An encoder as described in embodiment 21, configured as above.

[0505] <Embodiment 24> the transform coefficient data (126) includes indications of significant transform coefficient positions (132) of the selected transform (102) and values ​​of transform coefficients (134) of the selected transform (102) traversed forward from the significant transform coefficient positions (132) in a predetermined scan order (136) that sequentially traverses coefficient positions of the selected transform (102); The decision (130) depends on the instruction. 24. An encoder as described in embodiment 22 or 23, configured as described above.

[0506] <Embodiment 25> encoding a coarse version of the indication into the data stream (14) prior to the signaling (128) of the selected transformation (102) in the data stream (14), the decision (130) being dependent on the coarse version of the indication; Following said signalling, encoding a refinement of said coarse version of said indication into said data stream (14) in dependence on said selected transformation (102). An encoder as described in embodiment 24, configured as follows.

[0507] <Embodiment 26> encoding as the coarse version a hint identifying whether the significant transform coefficient position (132) is within a subgroup (138) of transform coefficients (134); As a refinement, encoding relative position information that distinguishes the significant transform coefficient position (132) among positions within the subgroup (138) if the significant transform coefficient position (132) is within the subgroup (138) and that distinguishes the significant transform coefficient position (132) among positions outside the subgroup (138) if the significant transform coefficient position (132) is outside the subgroup (138). An encoder as described in embodiment 25, configured as above.

[0508] <Embodiment 27> The encoder, if the hint reveals that the significant transform coefficient position (132) is within the subgroup (138), determining the selectable list (98) to include at least one multi-stage transform (106) including a primary transform applied on the prediction residual (100) of the current residual block and a secondary transform applied on a subgroup (112) of primary transform coefficients (114) of the primary transform (108) to result in the subgroup (138) of the transform coefficients of the selected transform (102); If the hints reveal that the significant transform coefficient position (132) is not within the subgroup (138), then determining the selectable list (98) to not include the at least one multi-stage transform (106). An encoder as described in embodiment 26, configured as follows.

[0509] <Embodiment 28> The encoder, The relative position information is coded using a variable length coding method, VLC. if the hint revea...

Claims

1. 1. A decoder for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, comprising: the decoder supports a set of transforms (94) for the block-based residual decoding; The decoder comprises: determining a list of transform candidates (98) for a current residual block (84) of transform coefficients from said transform set (94) based on a height and a width of said current residual block (84); The set of transforms includes an inverse transform of a plurality of multi-stage transforms and an inverse transform of a linear transform (108); each of the multi-stage transforms includes a respective non-separable secondary transform (106) applied to the primary transform coefficients of a first subgroup (112) of the current residual block position following the primary transform, thereby obtaining respective secondary coefficients of a second subgroup (138) of the current residual block position; determining the transform candidate list by including the inverse transform of the plurality of multi-stage transforms in the transform candidate list only if transform coefficients of the current residual block outside the second subgroup of coefficient positions (138) are not significant (non-zero) and comprise a linear transform. It is configured as follows: selecting a transform by decoding an index into the transform candidate list from the data stream if the determined transform candidate list includes the inverse transform of the plurality of multi-stage transforms; if the determined transform candidate list does not include the inverse transform of the plurality of multi-stage transforms, selecting the transform as a primary transform without decoding an index from the data stream to the transform candidate list; decoding a prediction residual (100) by applying the selected transform to the current residual block (84) of transform coefficients; A decoder configured to:

2. 1. A method for decoding an image (12) from a data stream (14) using block-based prediction and block-based residual decoding, the method supporting a set of transforms (94) for the block-based residual decoding, determining a list of transform candidates (98) for a current residual block (84) of transform coefficients from said transform set (94) based on a height and a width of said current residual block (84); The set of transforms includes an inverse transform of a plurality of multi-stage transforms and an inverse transform of a linear transform (108); each of the multi-stage transforms includes a respective non-separable secondary transform (106) applied to the primary transform coefficients of a first subgroup (112) of the current residual block position following the primary transform, thereby obtaining respective secondary coefficients of a second subgroup (138) of the current residual block position; determining the transform candidate list by including inverse transforms of the plurality of multi-stage transforms in the transform candidate list only if transform coefficients of the current residual block outside the second subgroup (138) of coefficient positions are not significant (non-zero) and comprise a linear transform; selecting a transform by decoding an index into the transform candidate list from the data stream if the determined transform candidate list includes the inverse transform of the plurality of multi-stage transforms; if the determined transform candidate list does not include the inverse transform of the plurality of multi-stage transforms, selecting the transform as a primary transform without decoding an index from the data stream to the transform candidate list; decoding a prediction residual (100) by applying the selected transform to the current residual block (84) of transform coefficients; method.

3. A computer program for carrying out the method according to claim 2 when executed on a computer or signal processor.

Images