Apparatus and method for decoding transform coefficient blocks, and apparatus and method for encoding transform coefficient blocks - Patents.com

Abstract

To provide apparatuses and the like that efficiently encode and decode a transform coefficient block.SOLUTION: A transform coefficient block decoding apparatus comprises: a decoder 250 to extract, from a data stream, a significance map indicating positions of significant transform coefficients within a transform coefficient block 256, and then sequentially extract values of the significant transform coefficients within the transform coefficient block by context-adaptive entropy decoding; and an associator 252 to associate the values of the significant transform coefficients with positions scanned in predetermined coefficient scan order in units of sub-blocks. In context-adapted entropy decoding of the values of the significant transform coefficients, a set selected from multiple sets of contexts is used. The selection of the set is performed for each sub-block in accordance with the values of the transform coefficients within a sub-block of the transform coefficient block already scanned in sub-block scan order, or the values of the transform coefficients of a co-located sub-block in an equally sized, previously decoded transform coefficient block.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to coding of blocks of transform coefficients, such as may be used, for example, in image and video coding. [Background technology]

[0002] In conventional video coding, images of a video sequence are typically divided into blocks. A block or a color component of a block is predicted by either motion compensated prediction or intra prediction. The blocks may be of different sizes and may be either square or rectangular. All samples of a block or color components of a block are predicted using the same set of prediction parameters. Such prediction parameters include a reference index (identifying a reference image from a set of already coded images), motion parameters (specifying a scale for the block's movement between the reference image and the current image), parameters for specifying an interpolation filter, an intra prediction mode, etc. The motion parameters are represented by a disparity vector with horizontal and vertical components, or by higher-order motion parameters, such as affine motion parameters consisting of six components. It is also possible to associate multiple sets of prediction parameters (such as reference indexes and motion parameters) with a single block. In this case, for each set of prediction parameters, a single intermediate prediction signal is generated for the block or color components of the block, and the final prediction signal is constructed by a weighted sum of the intermediate prediction signals. The weighted parameters, and possibly a constant offset (added to the weighted sum), are fixed to the image, reference image or set of reference images, or are included in the set of prediction parameters for their corresponding block. Similarly, still images may be divided into blocks, which are predicted by intra prediction methods (such as spatial intra prediction methods that predict the DC content of the block, or single intra prediction methods). In rare cases, the prediction signal may be zero.

[0003] The difference between the original block or the color components of the original block and the corresponding prediction signal, also called the residual signal, is usually subjected to a two-dimensional transform, which is transformed and quantized, and the resulting transform coefficients are quantized. In this transform coding, the block or color components of the block for which a particular set of prediction parameters is used may be further divided before applying the transform. The transform block is equal to or smaller than the block used for prediction. It is also possible that the transform block includes two or more of the blocks used for prediction. Different transform blocks in a still image or an image of a video sequence may have different sizes and the transform blocks may represent square or rectangular blocks.

[0004] The resulting quantized transform coefficients, also called transform coefficient levels, are transmitted using entropy coding techniques. Thus, a block of transform coefficient levels is typically mapped to a vector (i.e., an ordered set) of scan-converted coefficient values ​​using a scan. Note that different scans can be used for different blocks. A zigzag scan is often used. For blocks that contain only samples of one field of an interlaced frame (these blocks can be blocks within a coded field or field blocks within a coded frame), it is also common to use a different scan designed specifically for field blocks. The entropy coding algorithm commonly used to code the resulting ordered sequence is run-level coding. Typically, the majority of the transform coefficient levels are zero, and a set of consecutive transform coefficient levels equal to zero is efficiently represented by coding the number of consecutive transform coefficient levels (runs) equal to zero. For the remaining (non-zero) transform coefficients, the actual levels are coded. There are various alternatives to run-level coding. A run before a non-zero coefficient and the level of a non-zero transform coefficient are coded together using a single symbol or codeword. Often an extra symbol for the end of the block is included, transmitted after the last nonzero transform coefficient. It is also possible to code the number of nonzero transform coefficient levels first, and then the levels and runs are coded accordingly.

[0005] A somewhat different approach is used in the very efficient CABAC entropy coding in H.246. Here, the coding of transform coefficient levels is divided into three steps. In a first step, a binary syntax element coded_block_flag is transmitted for each transform block, indicating (called "signaling") whether the transform block contains significant transform coefficient levels (i.e. transform coefficients that are non-zero). If this syntax element indicates that significant transform coefficient levels are present, a binarized significance map is coded, specifying which transform coefficient levels are non-zero values. Then, in reverse scan order, the values ​​of the non-zero transform coefficient levels are coded. The significance map is coded as follows: For each coefficient in the scan order, a binary syntax element significant_coeff_flag is coded, specifying whether the corresponding transform coefficient level is not equal to zero. If a bin of significant_coeff_flag is equal to 1, i.e. if a non-zero transform coefficient level is present at this scan position, a further binary syntax element last_significant_coeff_flag is coded. This bin indicates whether the current significant transform coefficient level is the last significant transform coefficient level in the block or whether further significant transform coefficient levels follow in scan order. If last_significant_coeff_flag indicates that no further significant transform coefficients follow, no further syntax elements are coded to specify a significance map for the block. In a next step, the values ​​of the significant transform coefficient levels whose positions in the block have already been determined by the significance map are coded. The values ​​of the significant transform coefficient levels are coded in reverse scan order by using the following three syntax elements: The binary syntax element coeff_abs_greater_one indicates whether the absolute value of the significant transform coefficient level is greater than 1. If the binary syntax element coeff_abs_greater_one indicates that the absolute value is greater than 1, then a further syntax element coeff_abs_level_minus_one is sent specifying the absolute value of the transform coefficient level minus 1.Finally, a binary syntax element coeff_sign_flag, which specifies the sign of the transform coefficient value, is coded for each significant transform coefficient level. Note again that while the syntax elements related to the significance map are coded in scan order, the syntax elements related to the actual values ​​of the transform coefficient levels are coded in reverse scan order to allow for the use of more appropriate context models.

[0006] In CABAC entropy coding in H.264, all syntax elements for transform coefficient levels are coded using binary probability modeling. The non-binary syntax element coeff_abs_level_minus_one is first binarized, i.e. mapped to a sequence of binary decisions (bins), and these bins are coded sequentially. The binary syntax elements significant_coeff_flag, last_significant_coeff_flag, coeff_abs_greater_one and coeff_sign_flag are coded directly. Each coding bin (including binary syntax elements) is associated with a context. The context represents a probability model for a class of coding bins. A measure related to the probability for one of the two possible bin values ​​is estimated for each context based on the values ​​of the bins already coded together with the corresponding context. For some bins involved in transform coding, the context used for coding is selected based on the syntax elements already transmitted or based on the position within the block.

[0007] The significance map specifies information about significance (transform coefficient levels different from zero) for a scan position. In H.264 CABAC entropy coding, for a block size of 4×4, a separate context is used for each scan position to code the binary syntax elements significant_coeff_flag and last_significant_coeff_flag, where a different context is used for significant_coeff_flag and last_significant_coeff_flag of one scan position. For a block size of 8×8, the same context model is used for four consecutive scan positions, resulting in 16 context models for significant_coeff_flag and an additional 16 context models for last_significant_coeff_flag. This method of context modeling significant_coeff_flag and last_significant_coeff_flag has some disadvantages due to the large block size. On the one hand, if each scanning position is associated with a separate context model, the number of context models will increase significantly when a block larger than 8x8 is coded. Such an increase in the number of context models will lead to a delay in the adaptability of the probability estimation and usually results in an inaccurate probability estimation, which will have a negative impact on the coding efficiency. On the other hand, the assignment of context models to a large number of consecutive scanning positions (as is done for 8x8 blocks in H.264) is also not optimal for large block sizes, because the non-zero transform coefficients are usually concentrated in a certain region of the transform block (this region depends on the main structure in the corresponding block of the residual signal).

[0008] After the coding of the significance map, the block is processed in reverse scan order. If the scan position is significant, i.e. the coefficient is different from zero, the binary syntax element coeff_abs_greater_one is sent. First, the second context model of the corresponding context model set is selected for the coeff_abs_greater_one syntax element. If the coded value of any coeff_abs_greater_one syntax element in the block is equal to 1 (i.e. the absolute coefficient is greater than 2), the context modeling is switched back to the first context model of the set and this context model is used until the end of the block. Otherwise (all coded values ​​of coeff_abs_greater_one in the block are zero and the corresponding absolute coefficient level is equal to 1), the context model is selected depending on the number of coeff_abs_greater_one syntax elements equal to zero that have already been coded or decoded in the reverse scan of the current block. The context model selection for the syntax element coeff_abs_greater_one can be outlined in the following equation: Here, the current context model index Ct+1 is selected based on the previous context model index Ct and the value of the previously coded syntax element coeff_abs_greater_one, represented by bint in the equation: For the first syntax element coeff_abs_greater_one in the block, the context model index is set equal to Ct=1.

number

[0009] Only the second syntax element coeff_abs_level_minus_one for coding the absolute transform coefficient level is coded when the coeff_abs_greater_one syntax element for the same scan position is equal to 1. The non-binary syntax element coeff_abs_level_minus_one is binarized into a sequence of bins, and for the first bin of this binarization, a context model index is selected as described below. The remaining bins of the binarization are coded with a fixed context. The context for the first bin of the binarization is selected as follows: For the first coeff_abs_level_minus_one syntax element, a first context model of a set of context models for the coeff_abs_level_minus_one syntax element is selected, and the corresponding context model index is set equal to Ct=0. For each further first bin of the coeff_abs_level_minus_one syntax element, the context modeling is switched to the next context model in the set, where the number of context models in the set is limited to 5. The context model selection can be expressed by the following equation: where the current context model index Ct+1 is selected based on the value of the previous context model index Ct. As mentioned above, for the first syntax element coeff_abs_level_minus_one in the block, the context model index is set equal to Ct=0. Note that a different set of context models is used for the syntax elements coeff_abs_greater_one and coeff_abs_level_minus_one.

number

[0010] For large blocks, this method has some disadvantages. Since the number of significant coefficients is larger than in small blocks, the selection of the first context model for coeff_abs_greater_one (used when a value of coeff_abs_greater_one equal to 1 is coded for multiple blocks) is usually done too early, and the last context model for coeff_abs_level_minus_one is reached too early. Therefore, most bins of coeff_abs_greater_one and coeff_abs_level_minus_one are coded with a single context model. However, these bins usually have different probabilities, and for this reason, the use of a single context model for a large number of bins has a negative impact on coding efficiency.

[0011] Generally, larger blocks increase the computational overhead for performing a spectral separation transform, but if both small and large blocks can be effectively coded, better coding efficiency can be achieved when coding sample arrays representing images or other spatially sampled information signals such as depth maps. This is due to the dependency between spatial and spectral resolution when transforming the sample array in a block: the larger the block, the higher the spectral resolution of the transform. In general, it is preferable to apply an individual transform locally to the sample array, such that the spectral content of the sample array does not vary significantly within the region of such an individual transform. For small blocks, it is guaranteed that the content within the block is relatively consistent. On the other hand, if the block is too small, the spectral resolution is low and the ratio of non-significant transform coefficients to significant transform coefficients is low.

[0012] It would therefore be desirable to have a coding scheme that allows efficient coding for blocks of transform coefficients, even if the blocks are large, and a significance map for the blocks of transform coefficients. Summary of the Invention [Problem to be solved by the invention]

[0013] It is therefore an object of the present invention to provide a coding scheme for coding a transform coefficient block and a significance map, which respectively indicate the location of significant transform coefficients within the transform coefficient block, thereby increasing coding efficiency. [Means for solving the problem]

[0014] This object is achieved by the subject matter set forth in the independent claims.

[0015] According to a first aspect of the present application, the idea underlying the present application is that high coding efficiency of coding a significance map indicating the positions of significant transform coefficients in a transform coefficient block can be achieved if sequentially extracted syntax elements indicating whether a significant or non-significant transform coefficient is located at each associated position in the transform coefficient block, respectively, are sequentially associated with the positions of the transform coefficient block among the positions of the transform coefficient block depending on the positions of the significant transform coefficients indicated by the previously associated syntax elements. In particular, the inventors have found that in standard sample array content such as image, video or depth map content, the significant transform coefficients mostly form a set at a certain side of the transform coefficient block corresponding to non-zero frequencies in the vertical direction or low frequencies in the horizontal direction or vice versa, so that by taking into account the positions of the significant transform coefficients indicated by the previously associated syntax elements, it is possible to control further factors of the scan such that the probability of reaching the last significant transform coefficient in the transform coefficient block earlier is higher compared to a procedure in which the scan order is predetermined independently of the positions of the significant transform coefficients indicated by the previously associated syntax elements. This is true for small blocks as well, but especially so for larger blocks.

[0016] In one embodiment of the present application, the entropy decoder is configured to extract information from the data stream that allows it to recognize whether the significant transform coefficient currently indicated by the currently associated syntax element is the last significant transform coefficient independent of its exact position in the transform coefficient block, where the entropy decoder is configured not to expect further syntax elements if the current syntax element relates to such a last significant transform coefficient. This information may include the number of significant transform coefficients in the block. Alternatively, a second syntax element is interleaved with the first syntax element, where the second syntax element indicates whether the last transform coefficient in the transform coefficient block is also the same for the associated position where the significant transform coefficient is located.

[0017] In one embodiment, the associator applies, at a given position in the transform coefficient block, a scan order that simply depends on the positions of the significant transform coefficients indicated so far. For example, some sub-paths traversing mutually isolated subsets of positions in the transform coefficient block extend substantially diagonally from a pair of edges of the transform coefficient block corresponding to a minimum frequency along a first direction and a maximum frequency along the other direction, respectively, to an opposite pair of edges of the transform coefficient block corresponding to zero frequency along a second direction and a maximum frequency along the first direction, respectively. In this case, the associator is configured to select the scan order such that the sub-paths are traversed in an order such that the distance to the DC position of the sub-path in the transform coefficient block increases monotonically between the sub-paths, each sub-path is traversed uninterruptedly along the progression direction, and for each sub-path, the direction in which the sub-path is traversed is selected by the associator depending on the position of the significant transform coefficients traversed during the previous sub-path. This approach increases the probability that the last sub-path in which the last significant transform coefficient is located is traversed in a direction such that the last significant transform coefficient is more likely to be in the first half of this last sub-path than in the second half, thereby reducing the number of syntax elements indicating whether a significant or non-significant transform coefficient is located at each position, an effect that is particularly significant for large transform coefficient blocks.

[0018] According to a further aspect of the present application, the present application is based on the insight that a significance map indicating the positions of significant transform coefficients in a transform coefficient block can be coded more efficiently if the aforementioned syntax elements indicating whether a significant or non-significant transform coefficient is located at each position are entropy decoded in a context-adaptive manner for the associated positions in the transform coefficient block, with a context selected individually for each of the syntax elements depending on the number of significant transform coefficients in the neighborhood of the respective syntax element indicated as significant by any of the preceding syntax elements. In particular, the inventors have found that for transform coefficient blocks of increasing size, the significant transform coefficients are more or less aggregated in a predefined area in the transform coefficient block, whereby a context adaptation that is sensitive to the number of significant transform coefficients traversed so far in a predefined scanning order but also considers the neighborhood of the significant transform coefficients leads to a better adaptation of the context and therefore increases the coding efficiency of the entropy coding.

[0019] Of course, both of the aspects outlined above can be combined in any desired manner.

[0020] According to a further aspect of the present application, the present application is based on the insight that the coding efficiency of the coding of the transform coefficient block can be improved if a significance map indicating the positions of the significant transform coefficients in the transform coefficient block precedes the coding of the actual values ​​of the significant transform coefficients in the transform coefficient block, if a predefined scanning order between the positions of the transform coefficients used to sequentially associate the sequence of values ​​of the significant transform coefficients to the positions of the significant transform coefficients is scanned in the transform coefficient block in sub-blocks using a sub-block scanning order between the sub-blocks while scanning the positions of the transform coefficients in the sub-blocks in the coefficient scanning order supplementarily, and if a set selected from a plurality of sets of multiple contexts is used for sequential context-adaptive entropy decoding of the values ​​of the significant transform coefficient values, the selection of the selected set depending on the values ​​of the transform coefficients in the sub-blocks of the transform coefficient block already passed in the sub-block scanning order, or the values ​​of the transform coefficients of the sub-blocks located together in the previously decoded transform coefficient block. In this way, the context adaptation is very suitable for the above-outlined characteristics of the significant transform coefficients that are gathered in a predefined area in the transform coefficient block. In other words, the values ​​are scanned in the sub-blocks with a context selected based on the sub-block statistics.

[0021] Again, this last aspect may be combined with either or both of the previously specified aspects of this application.

[0022] Preferred embodiments of the present application are described below with reference to the drawings. [Brief description of the drawings]

[0023] [Figure 1] 1 shows a block diagram of an encoder according to one embodiment. [Figure 2A] 1 illustrates diagrammatically the subdivision of a sample array, such as an image, into blocks. [Figure 2B] 2 shows a schematic representation of another subdivision into blocks; [Figure 2C] 2 shows a schematic representation of another subdivision into blocks; [Diagram 3] 1 shows a block diagram of a decoder according to one embodiment. [Figure 4] 1 shows a block diagram of an encoder according to one embodiment of the present application in more detail. [Diagram 5] 1 shows a block diagram of a decoder according to one embodiment of the present application in more detail. [Figure 6] 1 illustrates a schematic representation of a transformation of a block from the spatial domain to the spectral domain. [Figure 7] 2 shows a block diagram of an apparatus for decoding a significance map and significant transform coefficients of a transform coefficient block according to one embodiment; [Figure 8] 10 shows a schematic representation of sub-paths of the scanning order and subdivision into different traverse directions. [Figure 9] 1 illustrates a schematic of a neighborhood definition for a scan position within a transform block according to one embodiment; [Figure 10] 10 illustrates diagrammatically possible neighborhood definitions for scan positions within a transform block at a transform block boundary; [Figure 11] 4 illustrates a possible scanning of a transformation block according to a further embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] In the description of the figures, elements which appear in several figures are designated by the same reference numerals in each of those figures, and in order to avoid unnecessary repetition, repeated description of those elements is avoided insofar as their function is of interest, however, the functions and descriptions given with respect to one figure shall apply to the other figures unless expressly stated otherwise.

[0025] FIG. 1 shows an example of an encoder 10 in which aspects of the present application are implemented. The encoder encodes a sequence of information samples 20 into a data stream. By sequence of information samples, we mean any kind of spatially sampled information signal. For example, the sample array 20 can be images constituting a still image or a video. In this case, the information samples are brightness values, color values, luma values, chroma values, etc. The information samples can also be depth values, in case the sample array 20 is a depth map, generated for example by a luminance time sensor, etc.

[0026] The encoder 10 is a block-based encoder, i.e., the encoder 10 encodes the sample array 20 into the data stream 30 in blocks 40. Encoding in blocks 40 does not necessarily mean that the encoder 10 encodes these blocks 40 completely independently of one another. Conversely, the encoder 10 may use reconstructions of previously encoded blocks to extrapolate or intra-predict the remaining blocks, or may use block granularity for setting the encoding parameters, i.e., the manner in which each sample array region corresponding to a respective block is coded.

[0027] Also, the encoder 10 is a transform coder, i.e., the encoder 10 encodes the blocks 40 by using a transform to transfer the information samples in each block 40 from the spatial domain to the spectral domain. A two-dimensional transform such as the DCT of an FFT is used. Preferably, the blocks 40 are square or rectangular.

[0028] The subdivision of the sample array 20 into blocks 40 shown in Figure 1 is for illustrative purposes only. Figure 1 shows the sample array 20 subdivided into a regular two-dimensional arrangement of non-overlapping adjacent square or rectangular blocks 40. The size of the blocks 40 may be predefined. In that case, the encoder 10 does not need to transfer information about the block size of the blocks 40 in the data stream 30 to the decoding side. For example, the decoder may expect the predefined block size.

[0029] However, several alternatives are possible. For example, blocks may overlap each other, but the overlap is restricted so that each block has a part that does not overlap any neighboring block, or each representation of a block overlaps at most one neighboring block that is arranged juxtaposed to the current block along a given direction. The latter means that the left and right neighboring blocks can overlap completely covering the current block, but the neighboring blocks themselves do not overlap each other. The same applies to vertical and diagonal adjacencies.

[0030] As a further alternative, the subdivision of the sample array 20 into blocks 40 may be applied to the contents of the sample array 20 by the encoder 10, and the subdivision information used in the subdivision may be transferred via the bitstream 30 to the decoder side.

[0031] 2A to 2C show different examples of the subdivision of the sample array 20 into blocks 40. FIG. 2A shows a quadtree-based subdivision of the sample array 20 into blocks 40 of different sizes, where the representative blocks are denoted by 40a, 40b, 40c and 40d in order of increasing size. According to the subdivision of FIG. 2A, the sample array 20 is first divided into a regular two-dimensional arrangement of tree blocks 40d. These tree blocks 40d have associated therewith individual subdivision information regarding whether a certain tree block 40d is further subdivided according to the quadtree structure. The tree block to the left of the block 40d is exemplarily subdivided into smaller blocks according to the quadtree structure. The encoder 10 can perform one two-dimensional transform for each of the blocks shown by solid and dashed lines in FIG. 2A. In other words, the encoder 10 can transform the array 20 in units of block subdivision.

[0032] Instead of a quadtree-based subdivision, a more general multitree-based subdivision may be used, and the number of child nodes per hierarchical level may vary between different hierarchical levels.

[0033] FIG. 2B shows another example of subdivision. According to FIG. 2B, the sample array 20 is first divided into non-overlapping adjacent macroblocks 40b in a regular two-dimensional arrangement, where each macroblock 40b has associated thereto subdivision information by which the macroblock is either not sub-divided or, if sub-divided, is sub-divided into equal-sized sub-blocks in a regular two-dimensional arrangement such that different sub-division granularities are achieved for different macroblocks. This results in a subdivision of the sample array 20 into blocks 40 of different sizes, representative examples of which are shown at 40a, 40b and 40a'. As shown in FIG. 2A, the encoder 10 performs a two-dimensional transform on each of the blocks shown in FIG. 2B with solid and dashed lines. FIG. 2C is described below.

[0034] 3 shows a decoder 50 capable of decoding the data stream 30 produced by the encoder 10 to reconstruct a reconstructed version 60 of the sample array 20. The decoder 50 reconstructs the reconstructed version 60 by extracting transform coefficient blocks for each of the blocks 40 from the data stream 30 and performing an inverse transform on each of the transform coefficient blocks.

[0035] The encoder 10 and the decoder 50 are configured to perform entropy coding / decoding, respectively, to insert and extract information about the transform coefficient blocks from the data stream, as will be described in more detail below. It should be noted that the data stream 30 does not necessarily include transform coefficient block information for all blocks 40 of the sample array 20. Conversely, a subset of the blocks 40 may be coded in the bitstream 30 in other ways. For example, the encoder 10 may decide not to insert a transform coefficient block for a block of the blocks 40, and instead insert alternative coding parameters in the bitstream 30 to allow the decoder 50 to predict it or otherwise fill the respective block in the reconstructed version 60. For example, the encoder 10 may perform texture analysis to locate the block in the sample array 20 and indicate this appropriately in the bitstream, so that the decoder can perform texture synthesis at the decoder side to fill the sample array 20.

[0036] As will be explained in the following figures, the transform coefficient blocks do not necessarily represent a spectral domain representation of the original information samples of each block 40 of the sample array 20. Conversely, such transform coefficient blocks may represent a spectral domain representation of a prediction residual of each block 40. FIG. 4 shows an embodiment of such an encoder. The encoder of FIG. 4 comprises a transform stage 100, an entropy coder 102, an inverse transform stage 104, a predictor 106, a subtractor 108 and an adder 110. The subtractor 108, the transform stage 100 and the entropy coder 102 are connected in series, in that order, between an input 112 and an output 114 of the encoder of FIG. 4. The inverse transform stage 104, the adder 110 and the predictor 106 are connected in that order between the output of the transform stage 100 and an inverting input of the subtractor 108, and the output of the predictor 106 is also connected to the other input of the adder 110.

[0037] The coder of Fig. 4 is a predictive transform-based block coder, i.e., blocks of a sample array 20 coming into the input 112 are predicted from previously coded and reconstructed parts of the same sample array 20, or from other previously coded and reconstructed sample arrays that precede or follow the current sample array 20 in time. The prediction is performed by a predictor 106. A subtractor 108 subtracts the predicted value from such original block, and a transform stage 100 performs a two-dimensional transform on the prediction residual. The two-dimensional transform itself or subsequent processing in the transform stage 100 leads to the quantization of the transform coefficients in the transform coefficient block. The quantized transform coefficient block is losslessly coded, for example by entropy coding, in an entropy encoder 102, such that the resulting data stream is output at an output 114. The inverse transform stage 104 reconstructs the quantized residual, which is then combined by the adder 100 with the corresponding prediction to obtain reconstructed information samples on the basis of which the predictor 106 predicts the currently encoded prediction block mentioned above. The predictor 106 can use different prediction modes, such as intra-prediction and inter-prediction modes, to predict the block, and the prediction parameters are forwarded to the entropy encoder 102 for insertion into the data stream.

[0038] That is, according to the embodiment of FIG. 4, the transform coefficient blocks represent a spectral representation of the remainder of the sample array, rather than the actual information samples of the sample array.

[0039] It should be noted that there are several alternatives to the embodiment of Fig. 4, some of which have been described in the introduction of the specification and which are now incorporated into the description of Fig. 4. For example, the prediction generated by the predictor 106 does not have to be entropy coded. Conversely, side information may be transferred to the decoding side via other coding schemes.

[0040] FIG. 5 shows a decoder capable of decoding the data stream generated by the encoder of FIG. 4. The decoder of FIG. 5 comprises an entropy decoder 150, an inverse transform stage 152, an adder 154 and a predictor 156. The entropy decoder 150, the inverse transform stage 152 and the adder 154 are connected in series, in that order, between an input 158 ​​and an output 160 of the decoder of FIG. 5. The other output of the entropy decoder 150 is connected to the predictor 156, which in turn is connected between the output of the adder 154 and the other input. The entropy decoder 150 extracts transform coefficient blocks from the data stream entering the decoder of FIG. 5 at input 158, where an inverse transform is applied to the transform coefficient blocks in stage 152 to obtain a residual signal. The residual signal is combined in adder 154 with a prediction from the predictor 156 to obtain a reconstructed block of a reconstructed version of the sample array at output 160. Based on the reconstructed version, a predictor 156 generates a prediction, thereby reconstructing the prediction performed by predictor 106 on the encoder side. To obtain the same prediction value used on the encoder side, predictor 156 uses prediction parameters, which the entropy decoder 150 obtains from the input 158 ​​data stream.

[0041] It should be noted that in the above embodiment, the spatial granularity at which the residual prediction and transformation are performed may not be equal to each other. This is shown in FIG. 2C, which shows the subdivision of the prediction granularity for the prediction block with solid lines and the residual granularity with dashed lines. As can be seen, the subdivisions are selected by the encoder independent of each other. More precisely, the syntax of the data stream allows the definition of the residual subdivision independent of the prediction subdivision. Alternatively, the residual subdivision may be an extension of the prediction subdivision, with each residual block being equal to the prediction block or a suitable subset thereof. This is shown, for example, in FIG. 2A and FIG. 2B. Again, the prediction granularity is shown with solid lines and the residual granularity with dashed lines. In this regard, in FIG. 2A-2C, the larger solid block encompassing the dashed block 40a is, for example, a prediction block for which prediction parameter settings are individually performed, while all blocks with associated reference codes are residual blocks for which one two-dimensional transformation is performed.

[0042] The above-mentioned embodiments have in common that a block of (residual or original) samples is transformed at the encoder side into a block of transform coefficients, which is then transformed back at the decoder side into a reconstructed block of samples. This is illustrated in Fig. 6. Fig. 6 shows a block of samples 200. In this case, the block 200 is exemplarily a two-dimensional block of samples 202 of size 4x4. The samples 202 are regularly arranged along the horizontal direction x and along the vertical direction y. By the above-mentioned two-dimensional transform T, the block 200 is transformed into the spectral domain, i.e. into a block 204 of transform coefficients 206, which now has the same size as the block 200. That is to say, the transform block 204 has as many transform coefficients 206 in both the horizontal and vertical directions as the block 200 has samples. However, since the transform T is a spectral transform, the positions of the transform coefficients 206 in the transform block 204 correspond to the spectral components, rather than to the spatial positions of the contents of the block 200. In particular, the horizontal axis of the transform block 204 corresponds to an axis along which horizontal spectral frequency increases monotonically, and the vertical axis corresponds to an axis along which vertical spatial frequency increases monotonically, with the DC component transform coefficients located at the corners of the block 204, here illustratively the top left corner, such that the transform coefficients 206 corresponding to the highest frequencies in both the horizontal and vertical directions are located at the bottom right corner. Ignoring spatial orientation, the spatial frequency to which a given transform coefficient 206 belongs generally increases from the top left corner to the bottom right corner. -1 By , the transform block 204 is retransferred from the spectral domain to the spatial domain to reobtain a replica 208 of the block 200. If no quantization / loss is introduced during the transformation, the reconstruction will be perfect.

[0043] As already mentioned above, it can be seen from Fig. 6 that the spectral resolution of the resulting spectral representation 204 increases when the block size of the block 200 is increased. On the other hand, the quantization noise tends to spread over the whole block 208, so that abrupt and very localized objects in the block 200 tend to cause deviations in the retransformed block compared to the original block 200 due to the quantization noise. On the other hand, the main advantageous effect of using larger blocks is that the ratio of the number of significant, i.e. non-zero (quantized) transform coefficients to the number of non-significant transform coefficients is reduced in the larger blocks compared to the smaller blocks, which allows a better coding efficiency. In other words, often the significant transform coefficients, i.e. the transform coefficients that are not quantized to zero, are sparsely distributed over the transform block 204. Due to this, according to an embodiment described in more detail hereinafter, the location of the significant transform coefficients is signaled in the data stream by a significance map. Apart from this, if the transform coefficients are quantized, the values ​​of the significant transform coefficients, i.e. the transform coefficient levels, are transmitted in the data stream.

[0044] Therefore, according to an embodiment of the present application, an apparatus for decoding such a significance map from a data stream, or for decoding a significance map along with corresponding significant transform coefficient values ​​from a data stream, is implemented as shown in FIG. 7, and each of the above-mentioned entropy decoders, i.e. decoder 50 and entropy decoder 150, constitutes the apparatus shown in FIG. 7.

[0045] The apparatus of Fig. 7 comprises a map / coefficient entropy decoder 250 and an associator 252. The map / coefficient entropy decoder 250 is connected to an input 254 to which the syntax elements representing the significance map and the significant transform coefficient values ​​are input. As will be described in more detail below, there are different probabilities regarding the order in which the syntax elements describing the significance map and the significant transform coefficient values ​​are input to the map / coefficient entropy decoder 250. The significance map syntax elements may precede the corresponding levels, or both may be interleaved, provided that the syntax elements representing the significance map precede the values ​​(levels) of the significant transform coefficients, and the map / coefficient entropy decoder 250 first decodes the significance map and then the transform coefficient levels of the significant transform coefficients.

[0046] As the map / coefficient entropy decoder 250 sequentially decodes the syntax elements representing the significance map and the significant transform coefficient values, the associator 252 is configured to associate these sequentially decoded syntax elements / values ​​with positions within the transform block 256. The scanning order in which the associator 252 associates the sequentially decoded syntax elements representing the significance map and the levels of significant transform coefficients with each position of the transform block 256 follows the same one-dimensional scanning order between each position of the transform block 256 as the order used by the encoding side to introduce these elements into the data stream. Also, as will be outlined in more detail below, the scanning order for the significance map syntax elements may or may not be equal to the order used for the significance transform values.

[0047] The map / coefficient entropy decoder 250 can utilize information about currently available transform blocks 256 generated by the associator 252 up to the currently to be decoded syntax element / level to set up a probability estimation context for entropy decoding the currently to be decoded syntax element / level, as indicated by dashed line 258. For example, the associator 252 may store (log) information collected so far from the sequentially associated syntax elements, such as the level itself or information about whether or not the respective positions have significant transform coefficients or do not know anything about the respective positions of the transform blocks 256, and the map / coefficient entropy decoder 250 may access this memory. Although the memory is not shown in FIG. 7, since a memory or log buffer exists to store the prior information obtained so far by the associator 252 and the entropy decoder 250, the reference numeral 256 may also refer to this memory. 7 thus indicates with crosses the significance transform coefficients obtained from previously decoded syntax elements representing the significance map, and a "1" indicates that the significance transform coefficient level of the significance transform coefficient at each position has already been decoded and is 1. If the significance map syntax element precedes the significance values ​​in the data stream, then a cross (significant transform coefficient) is recorded in memory 256 in the "1" position before decoding each value and recording a "1" (this situation would represent the entire significance map).

[0048] The following description focuses on specific embodiments for coding transform coefficient blocks or significance maps, which can be easily transferred to the above-mentioned embodiments. In these embodiments, a binary syntax element coded_block_flag is sent for each transform block to indicate whether the transform block contains significant transform coefficient levels (i.e., transform coefficients that are non-zero). If this syntax element indicates that significant transform coefficient levels are present, the significance map is coded. That is, coding occurs only if significant transform coefficient levels are present. The significance map, as described above, specifies which transform coefficient levels have non-zero values. The significance map coding includes coding of a binary syntax element significant_coeff_flag, each of which specifies, for each associated coefficient position, whether the corresponding transform system coefficient level is not equal to zero. The coding is performed in a scan order, which can be changed depending on the positions of significant coefficients previously identified as significant during the significance map coding. This will be explained in more detail below. Furthermore, the significance map coding includes coding of a binary syntax element last_significant_coeff_flag, which is distributed in the position with a sequence of significant_coeff_flag, which signals a significant coefficient. If the significant_coeff_flag bin is equal to 1, i.e. if a non-zero transform coefficient level is present in this scanning position, a further binary syntax element last_significant_coeff_flag is coded. This bin indicates whether the current significant transform coefficient level is the last significant transform coefficient level in the block or whether further significant transform coefficient levels follow in the scanning order. If last_significant_coeff_flag indicates that no further significant transform coefficients follow, no further syntax elements are coded to specify a significance map for the block.Alternatively, the number of significant coefficient positions may be signaled in the data stream before the coding of the sequence of significant_coeff_flag. In a next step the values ​​of the significant transform coefficient levels are coded. As mentioned above, alternatively the transmission of the levels may be interleaved with the transmission of the significance map. The values ​​of the significant transform coefficient levels are coded in a further scanning order as exemplified below. The following three syntax elements are used: The binary syntax element coeff_abs_greater_one indicates whether the absolute value of the significant transform coefficient level is greater than 1. If the binary syntax element coeff_abs_greater_one indicates that the absolute value is greater than 1, a further syntax element coeff_abs_level_minus_one is sent specifying the absolute value of the transform coefficient level minus 1. Finally, a binary syntax element coeff_sign_flag specifying the sign of the transform coefficient value is coded for each significant transform coefficient level.

[0049] The embodiments described below allow to further reduce the bit rate and thus increase the coding efficiency. To do so, they use a specific approach of context modeling for syntax elements related to transform coefficients. In particular, a new context model selection for the syntax elements significant_coeff_flag, last_significant_coeff_flag, coeff_abs_greater_one and coeff_abs_level_minus_one is used. Furthermore, an adaptive switching of the scanning during encoding / decoding of the significance map (which identifies the locations of the non-zero transform coefficient levels) is described. For the meaning of the syntax elements to be mentioned, reference is made to the above-mentioned introduction of this application.

[0050] The encoding of the significant_coeff_flag and last_significant_coeff_flag syntax elements, which specify the significance map, is improved by a new context modeling based on adaptive scanning and a limited neighborhood of already coded scan positions. These new concepts lead to more efficient encoding of the significance map (i.e., a corresponding bit rate reduction), especially for large block sizes.

[0051] One aspect of the embodiments outlined below is that the scan order (i.e., the mapping of blocks of transform coefficient values ​​to ordered sets (vectors) of transform coefficient values) is adapted during encoding / decoding of the significance map based on the values ​​of syntax elements already encoded / decoded for the significance map.

[0052] In a preferred embodiment, the scan order is adaptively switched between two or more predefined scan patterns. In a preferred embodiment, the switching occurs only at certain predefined scan positions. In a further preferred embodiment of the present invention, the scan order is adaptively switched between two predefined scan patterns. In a preferred embodiment, the switching occurs only at certain predefined scan positions.

[0053] The advantage of switching the scan pattern is a bit rate reduction, which results in a smaller number of coded syntax elements. As an intuitive example, referring to FIG. 6, it is often the case that especially for large transform blocks, the significant transform coefficient values ​​are concentrated at one of the block boundaries 270, 272. This is because the residual block contains mainly horizontal or vertical structures. In the most commonly used zigzag scan 274, there is a probability of about 0.5 that the last diagonal sub-scan of the zigzag scan that encounters the last significant coefficient starts on the side where there is no concentration of significant coefficients. In that case, a large number of syntax elements for transform coefficient levels equal to zero must be coded before the last non-zero transform coefficient value is reached. This can be avoided if the diagonal sub-scan always starts on the side where the significant transform coefficient levels are concentrated.

[0054] Preferred embodiments of the invention are described in further detail below.

[0055] As mentioned above, it is preferable to keep the number of context models reasonably small to allow fast adaptation of the context models even for large block sizes and to provide high coding efficiency. Therefore, a particular context should be used for more than one scan position. However, since significant transform coefficient levels are usually concentrated in a certain area of ​​a transform block (this concentration is usually the result of some dominant structure present, for example, in the residual block), the concept of assigning the same context to many consecutive scan positions, as is done for 8x8 blocks in H.264, is usually not appropriate. To design the context selection, the above consideration that significant transform coefficient levels are often concentrated in a certain area of ​​a transform block can be used. In the following, a concept in which this consideration can be utilized is described.

[0056] In a preferred embodiment, a large transform block (e.g., larger than 8x8) is partitioned into a number of rectangular sub-blocks (e.g., 16 sub-blocks), and each of these sub-blocks is associated with a separate context model for coding significant_coeff_flag and last_significant_coeff_flag (wherein different context models are used for significant_coeff_flag and last_significant_coeff_flag). The partition into sub-blocks may be different for significant_coeff_flag and last_significant_coeff_flag. The same context model may be used for all scan positions located in a particular sub-block.

[0057] In a further preferred embodiment, a large transform block (e.g., larger than 8x8) is partitioned into multiple rectangular and / or non-rectangular sub-regions, each of which is associated with a separate context model for coding significant_coeff_flag and / or last_significant_coeff_flag. The partition into sub-regions may be different for significant_coeff_flag and last_significant_coeff_flag. The same context model is used for all scan positions located in a particular sub-region.

[0058] In a further preferred embodiment, the context model for coding significant_coeff_flag and / or last_significant_coeff_flag is selected based on already coded symbols in a predefined spatial neighborhood of the current scan position. The predefined neighborhood may be different for different scan positions. In a preferred embodiment, the context model is selected based on the number of significant transform coefficient levels in a predefined spatial neighborhood of the current scan position, where only already coded significance indications are counted.

[0059] Further details regarding preferred embodiments of the present invention are provided below.

[0060] As mentioned above, for large block sizes, conventional context modeling codes multiple bins (usually with different probabilities) with a single context model for the syntax elements coeff_abs_greater_one and coeff_abs_level_minus_one. To avoid this drawback for large block sizes, according to an embodiment, a large block is divided into small square or rectangular sub-blocks of a certain size, and a separate context modeling is applied for each sub-block. Furthermore, multiple sets of context models may be used, where one of these sets of context models is selected for each sub-block based on an analysis of the statistics of the previously coded sub-blocks. In a preferred embodiment of the present invention, the number of transform coefficients in previously coded sub-blocks of the same block that are greater than two (i.e., coeff_abs_level_minus_1>1) is used to derive the set of context models for the current sub-block. These extensions of the context modeling of the coeff_abs_greater_one and coeff_abs_level_minus_one syntax elements result in more efficient coding of both syntax elements, especially for large block sizes. In a preferred embodiment, the block size of the sub-blocks is 2x2. In another preferred embodiment, the block size of the sub-blocks is 4x4.

[0061] In a first step, blocks larger than a given size are divided into smaller sub-blocks of a certain size. The coding process of absolute transform coefficient levels maps the sub-blocks, which are square or rectangular blocks, into an ordered set (vector) of sub-blocks using a scan. Different scans can be used for different blocks. In a preferred embodiment, the sub-blocks are processed using a zigzag scan, and the transform coefficient levels within the sub-blocks are processed with an inverse zigzag scan, i.e., scanning reading from the transform coefficients belonging to the highest frequency in the vertical and horizontal directions to the coefficients related to the lowest frequency in both directions. In another preferred embodiment of the invention, an inverse zigzag scan is used for coding the sub-blocks and for coding the transform coefficient levels within the sub-blocks. In another preferred embodiment of the invention, the same adaptive scan (see above) used for coding the significance map is used to process the entire block of transform coefficient levels.

[0062] The division of a large transform block into sub-blocks avoids the problem of using only one context model for most of the bins of a large transform block. Within a sub-block, the latest context modeling (specified in H.264) or a fixed context is used depending on the actual size of the sub-block. Furthermore, the statistics (in terms of probability modeling) for such a sub-block are different from the statistics of a transform block of the same size. This property can be exploited by extending the set of context models to the coeff_abs_greater_one and coeff_abs_level_minus_one syntax elements. Multiple sets of context models may be prepared and for each sub-block, one of these sets of context models may be selected based on the statistics of previously coded sub-blocks in the current transform block or previously coded transform blocks. In a preferred embodiment of the present invention, the selected set of context models is derived based on the statistics of previously coded sub-blocks in the same block. In another preferred embodiment of the present invention, the selected set of context models is derived based on the statistics of the same sub-block of a previously coded block. In a preferred embodiment, the number of the set of context models is set equal to 4, and in another preferred embodiment, the number of the set of context models is set equal to 16. In a preferred embodiment, the statistic used to determine the set of context models is the number of absolute transform coefficient levels greater than 2 in a previously coded sub-block. In another preferred embodiment, the statistic used to determine the set of context models is the difference between the number of significant coefficients and the number of transform coefficient levels, the absolute value of which is greater than 2.

[0063] The encoding of the significance map is performed as outlined below, namely by adaptive switching of the scan order.

[0064] In a preferred embodiment, the scan order for encoding the significance map is adapted by switching between two predefined scan patterns. Switching between scan patterns only occurs at certain predefined scan positions. The decision of whether or not the scan pattern is switched depends on the value of the significance map syntax element already encoded / decoded. In a preferred embodiment, both of the predefined scan patterns are scan patterns with diagonal sub-scans similar to the scan pattern of the zigzag scan. The scan patterns are shown in FIG. 8. Both scan patterns 300 and 302 consist of multiple diagonal sub-scans for a diagonal line from bottom left to top right or vice versa. The scan in the diagonal sub-scan (not shown) occurs from top left to bottom right for both of the predefined scan patterns. However, the scans within the diagonal sub-scans are different (as shown in the figure). For the first scanning pattern 300, the diagonal sub-scanning is from the bottom left to the top right (left diagram of FIG. 8), and for the second scanning pattern 302, the diagonal sub-scanning is from the top right to the bottom left (right diagram of FIG. 8). In one embodiment, the encoding of the significance map begins with the second scanning pattern. During encoding / decoding of syntax elements, the number of significant transform coefficient values ​​is counted by two counters c1 and c2. The first counter c1 counts the number of significant transform coefficients located in the bottom left part of the transform block. That is, the first counter c1 is incremented by one when a significant transform coefficient level in which the horizontal coordinate x is less than the vertical coordinate y in the transform block is encoded / decoded. The second counter c2 counts the number of significant transform coefficients located in the top right part of the transform block. That is, this second counter c2 is incremented by one when a significant transform coefficient level is coded / decoded whose horizontal coordinate x is greater than its vertical coordinate y in the transform block. The adaptation of the counters can be performed by the associator 252 in FIG. 7 and can be described by the following equation: where t represents the scan position index and both counters are initialized at zero.

number

number

[0065] At the end of each diagonal sub-scan, the associator 252 determines which of the first and second predetermined scan patterns 300, 302 will be used for the next diagonal sub-scan. This determination is based on the values ​​of the counters c1 and c2. If the count for the bottom left portion of the transformation block is greater than the count for the bottom left portion, then the scan pattern for a diagonal sub-scan from bottom left to top right is used, otherwise (if the count for the bottom left portion of the transformation block is less than or equal to the count for the bottom left portion), then the scan pattern for a diagonal sub-scan from top right to bottom left is used. This determination is expressed by the following equation:

number

[0066] The above-described embodiment of the present invention can be easily applied to other scan patterns. As an example, the scan pattern used for field macroblocks in H.264 is decomposed into sub-scans. In a further preferred embodiment, any given scan pattern is divided into sub-scans. For each sub-scan, two scan patterns are defined, one from bottom left to top right (as the basic scan direction) and one from top right to bottom left. Furthermore, two counters are introduced to count the number of significant coefficients in a first part (close to the bottom left boundary of the transform block) and a second part (close to the top right boundary of the transform block) in a sub-scan. Finally, at the end of each sub-scan, it is determined (based on the value of the counters) whether the next sub-scan is scanned from bottom left to top right or from top right to bottom left.

[0067] Next, an embodiment of how the entropy decoder 250 models the context will be described.

[0068] In one preferred embodiment, context modeling for significant_coeff_flag is performed as follows: For 4x4 blocks, context modeling is performed as specified in H.264. For 8x8 blocks, the transform block is separated into 16 2x2 sample sub-blocks, and each of these sub-blocks is associated with a separate context. Note that this concept can be extended to larger block sizes, different numbers of sub-blocks, and non-rectangular sub-regions, as described above.

[0069] In a further preferred embodiment, the context model selection for large transform blocks (e.g., for blocks larger than 8x8) is based on the number of significant transform coefficients already coded in a given neighborhood (within the transform block). An example of a neighborhood definition corresponding to a preferred embodiment of the present invention is shown in Figure 9. Those circled with x's are available neighborhoods that are always taken into account for the evaluation, those with x's and △'s are neighborhoods that are evaluated depending on the current scanning position and the current scanning direction: If the current scan position is within the left corner 304 of the 2x2, a separate context model is used for each scan position (Figure 9, left), If the current scanning position is not in the left corner of the 2x2 and is not located in the first row or first column of the transformation block, then the neighborhood shown on the right side of Figure 9 is used to estimate the number of significant transformation coefficients in the neighborhood of the current scanning position "x" that has nothing around it; If the current scan position "x" with nothing around it is the first row of a transformation block, the neighborhood identified in the right diagram of Figure 10 is used, If the current scan position "x" is in the first column of a block, the neighborhood identified in the left diagram of Figure 10 is used.

[0070] In other words, the decoder 250 determines a number of positions where the significant transform coefficients are placed according to the previously extracted and associated significance map syntax elements, and the current significance of each of the positions. The significance map syntax elements are configured to be extracted sequentially by performing entropy decoding in a context-adaptive manner by using an individually selected context for each of the significance map syntax elements according to positions restricted to those in the neighborhood of the position with which the map syntax element is associated (either the "x" on the right side of FIG. 9 and on both sides of FIG. 10, and the positions marked on the left side of FIG. 9). As shown, the neighborhood of the position with which each current syntax element is associated consists of positions that are either merely directly adjacent positions or positions separated from the position with which each significance map syntax element is associated, at most in positions in the vertical direction and / or positions in the horizontal direction. Alternatively, only positions directly adjacent to each current syntax element are considered. As a result, the size of the transform coefficient block is 8×8 positions or more.

[0071] In a preferred embodiment, the context model used to code a particular significant_coeff_flag is selected depending on the number of significant transform coefficient levels already coded in a defined neighborhood, where the number of available context models can be smaller than the possible values ​​of the number of significant transform coefficient levels in the defined neighborhood. The encoder and decoder may include a table (or a different mapping mechanism) for mapping the number of significant transform coefficient levels in the defined neighborhood to an index of a context model.

[0072] In further preferred embodiments, the index of the selected context model depends on the number of significant transform coefficient levels in the defined neighborhood, or on one or more additional parameters, such as the type of neighborhood used or the scan position or the quantized value of the scan position.

[0073] For the coding of last_significant_coeff_flag, a similar context modeling can be used as for significant_coeff_flag. However, the probability measure for last_significant_coeff_flag depends primarily on the distance of the current scan position to the top left corner of the transform block. In a preferred embodiment, the context model for coding last_significant_coeff_flag is selected based on the scan diagonal in which the current scan position lies (i.e., for the above embodiment of FIG. 8, it is selected based on x+y, where x and y are the horizontal and vertical positions of the scan position within the transform block, respectively, or based on how many sub-scans (such as sub-scan index minus 1) lie between the current sub-scan and the top left DC). In a preferred embodiment of the present invention, the same context is used for different values ​​of x°+°y. The distance measure, i.e., x+y or sub-scan index, is mapped onto a set of context models in a predetermined manner (e.g., by quantizing x°+°y or sub-scan index), where the number of possible values ​​for the distance measure is greater than the number of available context models for encoding last_significant_coeff_flag.

[0074] In the preferred embodiment, different context modeling techniques are used for transform blocks of different sizes.

[0075] Below, the coding of absolute transform coefficient levels is described.

[0076] In one preferred embodiment, the size of the sub-blocks is 2x2 and context modelling within the sub-blocks is disabled, i.e. one single context model is used for all transform coefficients within a 2x2 sub-block. Only blocks larger than 2x2 are affected by the subdivision process. In a further preferred embodiment of the invention, the size of the sub-blocks is 4x4, context modelling within the sub-blocks is performed as in H.264 and only blocks larger than 4x4 are affected by the subdivision process.

[0077] Regarding the scanning order, in a preferred embodiment, a zigzag scan 320 is used to scan the sub-blocks 322 of the transform block 256, i.e., along the direction of substantially increasing frequency, and the transform coefficients within the sub-blocks are scanned with an inverse zigzag scan 326 (FIG. 11). In a further preferred embodiment of the present invention, both the sub-blocks 322 and the transform coefficient levels within the sub-blocks 322 are scanned using an inverse zigzag scan (as shown in FIG. 11 where the arrow 320 is inverted). In another preferred embodiment, the same adaptive scan that encodes the significance map is used to process the transform coefficient levels, where the same adaptive decision is made, so that the exact same scan is used for both encoding the significance map and encoding the transform coefficient level values. Note that the scan itself is typically not dependent on the selected statistics or number of context model sets or on the decision to enable or disable context modeling within the sub-blocks.

[0078] Next, an embodiment for context modeling on the coefficient level will be described.

[0079] In preferred embodiments, the context modeling for the sub-blocks is similar to the context modeling for 4x4 blocks in H.264 described above. The number of context models used to code the coeff_abs_greater_one syntax element and the first bin of the coeff_abs_level_minus_one syntax element is, for example, equal to 5, with different sets of context models for the two syntax elements. In further preferred embodiments, context modeling within the sub-blocks is disabled and only one predefined context model is used within each sub-block. For these embodiments, the context model set for the sub-block 322 is selected from among a predefined number of context model sets. The selection of the context model for the sub-block 322 is based on certain statistics of one or more sub-blocks already coded. In preferred embodiments, the statistics used to select the context model set for the sub-block are taken from one or more sub-blocks already coded in the same block 256. How the statistics are used to determine the selected context model set is described below. In a further preferred embodiment, the statistics are taken from the same sub-blocks in a previously coded block having the same block size, such as blocks 40a and 40a' in Figure 2B. In another preferred embodiment of the present invention, the statistics are taken from defined neighboring sub-blocks in the same block depending on the selective scanning for the sub-block. It is also important to note that the source of the statistics should be independent of the scanning order and how the statistics are created to determine the context model set.

[0080] In a preferred embodiment, the number of context model sets is equal to 4, while in another preferred embodiment, the number of context model sets is equal to 16. In common, the number of context model sets is not fixed but should be adapted according to selected statistics. In a preferred embodiment, the context model set for the sub-block 322 is determined based on the number of absolute transform coefficient levels greater than 2 in one or more sub-blocks already coded. An index for the context model set is determined by mapping the number of absolute transform coefficient levels greater than 2 in the reference sub-block or reference sub-blocks to a set of predefined context model indices. This mapping can be performed by quantizing the number of absolute transform coefficient levels greater than 2 or by a predefined table. In a further preferred embodiment, the context model set for the sub-block is determined based on the difference between the number of significant transform coefficient levels and the number of absolute transform coefficient levels greater than 2 in one or more sub-blocks already coded. An index for the context model set is determined by mapping this difference to a set of predefined context model indices. This mapping can be performed by quantizing the difference between the number of significant transform coefficient levels and the number of absolute transform coefficient levels greater than 2 or by a predefined table.

[0081] In another preferred embodiment, when the same adaptive scan is used to process the absolute transform coefficient levels and the significance map, the partial statistics of the sub-blocks in the same block are used to determine the context model set for the current sub-block. Or, if available, the statistics of the previously coded sub-blocks in the previously coded transform block may be used. This means, for example, that instead of using the absolute number of absolute transform coefficient levels greater than 2 in the sub-block to determine the context model, the number of already coded absolute transform coefficient levels greater than 2 is used multiplied by the ratio of the number of transform coefficients in the sub-block to the number of already coded transform coefficients in the sub-block, or instead of using the difference between the number of significant transform coefficient levels and the number of absolute transform coefficient levels greater than 2 in the sub-block, the difference between the number of already coded significant transform coefficient levels and the number of already coded absolute transform coefficient levels greater than 2 is used multiplied by the ratio of the number of transform coefficients in the sub-block to the number of already coded transform coefficients in the sub-block.

[0082] For the context modeling within a sub-block, essentially the inverse of the latest context modeling for H.264 is adopted. This means that, if the same adaptive scanning is used to process the absolute transform coefficient levels and the significance map, the transform coefficient levels are essentially coded in a forward scan order instead of a backward scan order as for H.264. Therefore, the context model switching must be adapted accordingly. According to one embodiment, the coding of the transform coefficient levels starts with the first context model for the coeff_abs_greater_one and coeff_abs_level_minus_one syntax elements, and switches to the next context model in the set when two coeff_abs_greater_one syntax elements equal to zero have been coded since the last context model switch. In other words, the context selection depends on the number of already coded coeff_abs_greater_one syntax elements greater than zero in the scan order. The number of context models for coeff_abs_greater_one and coeff_abs_level_minus_one may be the same as that for H.264.

[0083] Thus, the above-described embodiments are applicable in the field of digital signal processing, in particular in image and video decoders and encoders. In particular, the above-described embodiments enable the coding of syntax elements related to transform coefficients in block-based image and video codecs with improved context modeling for syntax elements related to transform coefficients coded with an entropy coder employing probability modeling. In comparison with the state of the art, improved coding efficiency is achieved, especially for large transform blocks.

[0084] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus.

[0085] The inventive encoded signal representing the transform block or significance map, respectively, can be stored in a digital storage medium or transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as the Internet.

[0086] Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or software. Implementation can be performed using digital storage media, such as, for example, floppy disks, DVDs, Blu-ray, CDs, ROMs, PROMs, EPROMs, EEPROMs or flash memories, on which electronically readable control signals are stored, and which cooperate (or are capable of cooperating) with a programmable computer system such that the respective methods are executed. These digital storage media are therefore computer readable.

[0087] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which can cooperate with a programmable computer system to perform one of the methods described herein.

[0088] Generally, embodiments of the present invention may be implemented as a computer program product having program code operable to perform one of the methods when the computer program product runs on a computer, the program code being, for example, stored on a machine readable carrier.

[0089] Another embodiment comprises the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0090] In other words, an embodiment of the inventive method is therefore a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0091] A further embodiment of the inventive method is, for example, a data carrier (or a digital storage medium or a computer readable medium) having recorded thereon the computer program for performing one of the methods described herein.

[0092] A further embodiment of the inventive method is a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, the data stream or the sequence of signals being adapted to be transmitted over a data communication connection, e.g. via the Internet.

[0093] A further embodiment comprises a processing means, for example a computer, or a programmable logic device configured to or adapted to perform one of the methods described herein.

[0094] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0095] In some embodiments, a programmable logic device (e.g., a field programmable gate array) is used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0096] The above-described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended to be limited only by the claims set forth immediately below and not by the specific details presented in the description and illustration of the embodiments set forth herein.

Claims

1. 1. An apparatus for decoding a significance map, the apparatus comprising: a decoder configured to extract from a data stream the significance map indicating locations of significant transform coefficients within a transform coefficient block, the decoder configured to extract a first type syntax element and a second type syntax element based on context adaptive entropy decoding, the first type syntax element indicating, for an associated position within the transform coefficient block, whether a significant transform coefficient is present at each associated position, and the second type syntax elements each relating to a value of the significant transform coefficient at each associated position within the transform coefficient block; an associator configured to associate the extracted syntax elements of the first type with the positions of the transform coefficient blocks in scan order; Equipped with the decoder is configured to use a selected context for each of the first-type syntax elements in context-adaptive entropy decoding the first-type syntax elements, the context for a current first-type syntax element being decoded being selected according to a combination of the scan order and a number of positions of the significant transform coefficients within the transform coefficient block associated with previously extracted first-type syntax elements in a neighborhood of the current first-type syntax element. Device.

2. 2. The apparatus of claim 1, wherein the vicinity includes at least one location adjacent to the location of the current syntax element of the first type or spaced vertically one location and / or horizontally one location from the location of the current syntax element of the first type.

3. The apparatus of claim 1 , wherein the decoder is further configured to map the number of positions to a context index from a set of possible context indexes.

4. 1. An apparatus for encoding a significance map, the apparatus comprising:

11. An encoder configured to encode the significance map indicating locations of significant transform coefficients within a transform coefficient block into a data stream, the encoder configured to encode a first type of syntax element and a second type of syntax element into the data stream based on context adaptive entropy coding, the first type of syntax element indicating, for an associated position within the transform coefficient block, whether a significant transform coefficient is present at each associated position, and each of the second type of syntax elements associated with a value of the significant transform coefficient at each associated position within the transform coefficient block, the encoder is configured to encode the first type syntax elements into the data stream in scan order and to use a context selected for the first type syntax elements in context-adaptive entropy encoding each of the first type syntax elements, the context for a current first type syntax element being encoded being selected according to a combination of the scan order and a number of positions of the significant transform coefficients within the transform coefficient block associated with previously extracted first type syntax elements in a vicinity of the current first type syntax element. Device.

5. 5. The apparatus of claim 4, wherein the vicinity includes at least one location adjacent to the location of the current syntax element of the first type or spaced one vertical position and / or one horizontal position from the location of the current syntax element of the first type.

6. The apparatus of claim 4 , wherein the encoder is further configured to map the number of positions to a context index from a set of possible context indexes.

7. 1. A method for decoding a significance map, the method comprising: extracting from the data stream the significance map indicating locations of significant transform coefficients within a transform coefficient block, the significance map extracting comprising extracting from the data stream a first type syntax element and a second type syntax element based on context adaptive entropy decoding, the first type syntax element indicating, for an associated position within the transform coefficient block, whether a significant transform coefficient is present at each associated position, and the second type syntax elements each relating to a value of the significant transform coefficient at each associated position within the transform coefficient block; associating the extracted syntax elements of the first type with the positions of the transform coefficient blocks in scan order; Including, a selected context for each of the first type syntax elements is used in context-adaptive entropy decoding of the first type syntax elements, wherein the context for a current first type syntax element being decoded is selected according to a combination of the scan order and a number of positions of the significant transform coefficients within the transform coefficient block associated with previously extracted first type syntax elements in a neighborhood of the current first type syntax element. method.

8. 8. The method of claim 7, wherein the vicinity includes at least one location adjacent to the location of the current syntax element of the first type or spaced one vertical position and / or one horizontal position from the location of the current syntax element of the first type.

9. The method of claim 7 , wherein the number of positions is mapped to a context index from a set of possible context indexes.

10. The method of claim 7 , wherein the data stream comprises at least a portion associated with a color sample.

11. The method of claim 7 , wherein the data stream comprises at least a portion associated with depth values ​​associated with a depth map.

12. 1. A method for encoding into a data stream a significance map indicating locations of significant transform coefficients within a transform coefficient block, the method comprising: encoding syntax elements of a first type and a second type into the data stream based on context adaptive entropy coding, wherein the syntax elements of the first type indicate, for an associated position in the transform coefficient block, whether a significant transform coefficient is located at each associated position, and the syntax elements of the second type are each associated with a value of the significant transform coefficient at each associated position in the transform coefficient block; the step of encoding the first-type syntax elements is performed in scan order and using a context selected for the first-type syntax elements, the context for a current first-type syntax element being encoded being selected according to a combination of the scan order and a number of positions of the significant transform coefficients within the transform coefficient block associated with previously extracted first-type syntax elements in a vicinity of the current first-type syntax element. method.

13. 13. The method of claim 12, wherein the vicinity includes at least one location adjacent to the location of the current syntax element of the first type or spaced one vertical position and / or one horizontal position from the location of the current syntax element of the first type.

14. The method of claim 12 , wherein the number of positions is mapped to a context index from a set of possible context indexes.

15. The method of claim 12 , wherein the data stream comprises at least a portion associated with a color sample.

16. The method of claim 12 , wherein the data stream comprises at least a portion associated with depth values ​​associated with a depth map.

17. 1. A non-transitory computer-readable digital storage medium having stored thereon a computer program comprising program code for performing, when executed on a computer, a method for decoding a significance map, the method comprising: extracting from the data stream the significance map indicating locations of significant transform coefficients within a transform coefficient block, the significance map extracting comprising extracting from the data stream a first type syntax element and a second type syntax element based on context adaptive entropy decoding, the first type syntax element indicating, for an associated position within the transform coefficient block, whether a significant transform coefficient is present at each associated position, and the second type syntax elements each relating to a value of the significant transform coefficient at each associated position within the transform coefficient block; associating the extracted syntax elements of the first type with the positions of the transform coefficient blocks in scan order; Including, a selected context for each of the first type syntax elements is used in context-adaptive entropy decoding of the first type syntax elements, wherein the context for a current first type syntax element being decoded is selected according to a combination of the scan order and a number of positions of the significant transform coefficients within the transform coefficient block associated with previously extracted first type syntax elements in a neighborhood of the current first type syntax element. A non-transitory computer-readable digital storage medium.

18. 20. The non-transitory computer-readable medium of claim 17, wherein the vicinity includes at least one location adjacent to the location of the current syntax element of the first type or spaced vertically one location and / or horizontally one location from the location of the current syntax element of the first type.

19. 20. The non-transitory computer-readable medium of claim 17, wherein the number of positions is mapped to a context index from a set of possible context indexes.

Images