Conversion coefficient block decoding apparatus and method, and conversion coefficient block coding apparatus and method

Abstract

To provide a method for effectively performing a coding and a decoding of a conversion coefficient block, and provide a device of them.SOLUTION: A conversion coefficient block decoding device comprises: a decoder 250 that extracts a significant map indicating a position of a significant conversion coefficient in a conversion coefficient block from a data stream, sequentially, extracts a value of the significant conversion coefficient in the conversion coefficient block in a sequential by a context adaptive entropy decoding; and an associator 252 that associates the value of the significant conversion coefficient to a position scanned in a predetermined coefficient scanning order in a sub-block unit. When the value of the significant conversion coefficient is decoded by the context adaptive entropy decoding, a collection selected from a collection of a plurality of contexts is used. The selection of the collection is executed in accordance with the value of the conversion coefficient of the sub-block arranged in the conversion coefficient block decoded to the value of the conversion coefficient in the sub-block that has to be scanned in the sub-block scanning order of the conversion coefficient block to each sub-block or an equivalent size.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to the encoding of conversion coefficient blocks. Such encoding can be used, for example, in the encoding of images and videos.

Background Art

[0002] In conventional video encoding, usually, the images of a video sequence are divided into blocks. The blocks or the color components of the blocks are predicted either by motion compensation prediction or intra prediction. The blocks may be of different sizes and may be either square or rectangular. All samples of a block or the color components of a block are predicted using the same set of prediction parameters. Such prediction parameters include a reference index (identifying a reference image among a set of already encoded images), a motion parameter (specifying a measure for the movement of a block between the reference image and the current image), parameters for specifying an interpolation filter, an intra prediction mode, etc. The motion parameter is represented by a displacement vector having horizontal and vertical components or by a higher-order motion parameter such as an affine motion parameter consisting of six components. Also, it is possible to associate multiple sets of prediction parameters (such as a reference index and a motion parameter) with a single block. In this case, for each set of prediction parameters, a single intermediate prediction signal is generated for the block or the color components of the block, and the final prediction signal is constructed by the weighted sum of the intermediate prediction signals. The weighted parameters and, optionally, a constant offset (added to the weighted sum) are fixed to an image, a reference image or a set of reference images, or they are included in the set of prediction parameters for the corresponding block. Similarly, still images may be divided into blocks, and the blocks are predicted by an intra prediction method (such as a spatial intra prediction method or a single intra prediction method for predicting the DC component of the block). Although it rarely occurs, the prediction signal may become zero.

[0003] The difference between the original block or its color components and the corresponding predicted signal is also called the residual signal. Typically, a two-dimensional transformation is applied to this residual signal, and the resulting transformation coefficients are quantized. In this transformation coding, the blocks or color components of a block used for a particular set of prediction parameters may be further subdivided before the transformation is applied. The transformed block is equal to or smaller than the block used for prediction. It is also possible for the transformed block to contain two or more of the blocks used for prediction. Different transformed blocks in a single image of a still image or video sequence can have different sizes, and transformed blocks can represent square or rectangular blocks.

[0004] The resulting quantized transformation coefficients, also called transformation coefficient levels, are transmitted using entropy coding techniques. Therefore, blocks of transformation coefficient levels are typically mapped to a vector (i.e., an ordered set) of scanned transformation coefficient values ​​using scanning. Note that different scanning methods can be used for different blocks. Scanning zigzag scanning is commonly used. For blocks containing only a sample of one field of an interlaced frame (these blocks may be blocks within the encoded field or field blocks within the encoded frame), it is also common to use a different scanning method specifically designed for the field block. The entropy coding algorithm commonly used to encode the resulting ordered sequence is run-level coding. Typically, the majority of transformation coefficient levels are zero, and a set of consecutive transformation coefficient levels equal to zero can be coded by encoding the number of consecutive transformation coefficient levels (runs) equal to zero. It is expressed proportionally. For the remaining (non-zero) conversion coefficients, the actual level is encoded. There are various alternatives to run-level coding. The run and non-zero conversion coefficient levels before the non-zero coefficient are encoded together using a single symbol or codeword. Often, a special symbol for the end of the block is included, which is transmitted after the last non-zero conversion coefficient. It is also possible to encode the number of non-zero conversion coefficient levels first, and the level and run are encoded according to this number.

[0005] A somewhat different approach is used in the highly efficient CABAC entropy coding in H.246. Here, the coding of transformation coefficient levels is divided into three steps. In the first step, a binary syntax element, coded_block_flag, is sent for each transformation block, indicating (this is called "signaling") whether the transformation block contains significant transformation coefficient levels (i.e., transformation coefficients that are not zero). If this syntax element indicates the presence of significant transformation coefficient levels, a binarized significance map is coded to identify which transformation coefficient levels are non-zero. Then, in reverse scan order, the values ​​of the non-zero transformation coefficient levels are coded. The significance map is coded as follows: For each coefficient in scan order, a binary syntax element, significant_coeff_flag, is coded to identify whether the corresponding transformation coefficient level is not equal to zero. If the bin of significant_coeff_flag is equal to 1, i.e., if a non-zero transformation coefficient level exists at this scan position, a further binary syntax element, last_significant_coeff_flag, is coded. This bin indicates whether the current significance coefficient level is the last significance coefficient level in the block, or whether further significance coefficient levels follow in scan order. If last_significant_coeff_flag indicates that no further significance coefficients follow, no further syntax elements to identify the significance map for the block are encoded. In the next step, the values ​​of significance coefficient levels whose positions within the block have already been determined by the significance map are encoded. The values ​​of significance coefficient levels are encoded in reverse scan order using the following three syntax elements: The binary syntax element coeff_abs_greater_one indicates whether the absolute value of the significance coefficient level is greater than 1. If the binary syntax element coeff_abs_greater_one indicates that the absolute value is greater than 1, a further syntax element coeff_abs_level_minus_one is sent, which identifies the absolute value of the transformation coefficient level minus 1.Finally, the binary syntax element coeff_sign_flag, which identifies the sign of the transformation coefficient value, is encoded for each significance transformation coefficient level. Again, syntax elements related to the significance map are encoded in traversal order, while syntax elements related to the actual values ​​of the transformation coefficient levels are encoded in reverse traversal order, allowing for the use of a more appropriate contextual model.

[0006] In CABAC entropy coding in H.264, all syntax elements for the transformation coefficient levels are coded using binary probabilistic modeling. The non-binary syntax element coeff_abs_level_minus_one is first mapped to a binary-evolution sequence, i.e., a sequence of binary decisions (bins), and these bins are coded sequentially. The binary syntax elements significant_coeff_flag, last_significant_coeff_flag, coeff_abs_greater_one, and coeff_sign_flag are coded directly. Each coded bin (including binary syntax elements) is associated with a context. The context represents a probabilistic model for a class of coded bins. A measure related to the probability of one of two possible bin values ​​is estimated for each context, along with the corresponding context, based on the values ​​of the already coded bins. For some bins involved in transformation coding, the context used for coding is selected based on syntax elements already sent or based on their position within a block. It can be done.

[0007] The significance map identifies information about significance (that the transformation coefficient level is not zero) for a given scan position. In H.264 CABAC entropy coding, for a 4x4 block size, a separate context is used for each scan position to encode the binary syntax elements significant_coeff_flag and last_significant_coeff_flag, where different contexts are used for significant_coeff_flag and last_significant_coeff_flag for a single scan position. For an 8x8 block size, the same context model is used for four consecutive scan positions, reducing to 16 context models for significant_coeff_flag and an additional 16 context models for last_significant_coeff_flag. This method of contextualizing significant_coeff_flag and last_significant_coeff_flag has some disadvantages due to the large block size. On the one hand, if each scan position is associated with a separate context model, the number of context models increases significantly when encoding blocks larger than 8x8. This increase in the number of context models leads to a lag in the adaptability of probability estimation, usually resulting in inaccuracies in the probability estimation. These negatively impact encoding efficiency. On the other hand, assigning context models to a large number of consecutive scan positions (as is done for 8x8 blocks in H.264) is also not optimal for large block sizes because non-zero conversion coefficients are usually concentrated in specific regions of the conversion block (which depend on the main structure within the corresponding block of the residual signal).

[0008] After encoding the significance map, the blocks are processed in reverse scan order. If the scan position is significant, i.e., the coefficient is not zero, the binary syntax element coeff_abs_greater_one is sent. First, the second context model from the corresponding set of context models is selected for the coeff_abs_greater_one syntax element. If the encoded value of any coeff_abs_greater_one syntax element in the block is equal to 1 (i.e., the absolute coefficient is greater than 2), the context modeling is switched back to the first context model from that set and this context model is used until the end of the block. Otherwise (if all coeff_abs_greater_one in the block have encoded values ​​of zero and the corresponding absolute coefficient level is equal to 1), the context model is selected depending on the number of coeff_abs_greater_one syntax elements that are already encoded or decoded to be equal to zero during the reverse scan of the target block. The context model selection for the syntax element coeff_abs_greater_one can be outlined by the following formula. Here, the current context model index Ct+1 is selected based on the previous context model index Ct and the value of the previously encoded syntax element coeff_abs_greater_one represented by bint in the expression. For the first syntax element coeff_abs_greater_one in the block, the context model index is set equal to Ct=1.

number

[0009] When the coeff_abs_greater_one syntax element for the same scan position is equal to 1, a second syntax element c is used to encode the absolute conversion coefficient level. Only coeff_abs_level_minus_one is encoded. The non-binary syntax element coeff_abs_level_minus_one is binary-coded into a sequence of bins, and for the first bin of this binary, the context model index is selected as described below. The remaining bins of the binary are encoded with a fixed context. The context for the first bin of the binary is selected as follows: For the first coeff_abs_level_minus_one syntax element, the first context model from the set of context models for coeff_abs_level_minus_one syntax elements is selected, and the corresponding context model index is set equal to Ct=0. For each further first bin of the coeff_abs_level_minus_one syntax element, the context modeling is switched to the next context model in its set, where the number of context models in the set is limited to 5. Context model selection can be expressed by the following formula, where the current context model index Ct+1 is selected based on the value of the previous context model index Ct. As described above, for the first syntax element coeff_abs_level_minus_one within the block, the context model index is set to equal Ct=0. Note that different sets of context models are used for the syntax elements coeff_abs_greater_one and coeff_abs_level_minus_one.

number

[0010] For large blocks, this method has several disadvantages. Because the number of significance coefficients is larger than in small blocks, the selection of the first context model for coeff_abs_greater_one (used when a value of coeff_abs_greater_one equal to 1 is encoded for multiple blocks) is usually done too early, and the last context model for coeff_abs_level_minus_one is reached too early. As a result, most bins of coeff_abs_greater_one and coeff_abs_level_minus_one are encoded with a single context model. However, these bins usually have different probabilities, and for this reason, using a single context model for a large number of bins negatively impacts encoding efficiency.

[0011] Generally, larger blocks increase the computational overhead for performing spectral separation transformations. However, if both small and large blocks can be effectively encoded, better encoding efficiency can be achieved when encoding sample arrays representing other spatially sampled information signals, such as images or depth maps. This is due to the dependence between spatial and spectral resolution during the transformation of the sample array within a block; the larger the block, the higher the spectral resolution of the transformation. Generally, it is preferable to apply locally individual transformations to the sample array such that the spectral components of the sample array do not fluctuate significantly within the domain of such individual transformations. For small blocks, relatively consistent content within the block is guaranteed. On the other hand, if the block is too small, the spectral resolution is low, and the ratio of non-significant transformation coefficients to significant transformation coefficients is low.

[0012] Therefore, it is preferable to have an encoding scheme that enables efficient encoding of the conversion coefficient block even when the conversion coefficient block is large, and to have a significant map of the conversion coefficient block. [Overview of the Initiative] [Problems that the invention aims to solve]

[0013] Therefore, the object of the present invention is to provide an encoding scheme for encoding a block of transformation coefficients and a significance map that shows the positions of significant transformation coefficients within the block of transformation coefficients, thereby improving encoding efficiency. [Means for solving the problem]

[0014] This task is accomplished by the subject matter described in the independent claim.

[0015] According to the first aspect of this application, the underlying concept is that high coding efficiency in encoding a significance map showing the locations of significant transformation coefficients within a transformation coefficient block can be achieved when sequentially extracted syntax elements indicating whether significant or non-significant transformation coefficients are placed at each relevant location within the transformation coefficient block, and the scanning order sequentially associated with the locations of the transformation coefficient block, corresponds to the locations of significant transformation coefficients indicated by previously associated syntax elements. In particular, the inventors have found that in standard sample array content such as image, video, or depth map content, significant transformation coefficients mostly form sets on specific sides of the transformation coefficient block corresponding to non-zero frequencies in the longitudinal direction or low frequencies in the transverse direction, or vice versa, thereby allowing for control over further factors in the scanning to increase the probability of reaching the last significant transformation coefficient within the transformation coefficient block earlier, compared to a procedure in which the scanning order is predetermined independently of the locations of significant transformation coefficients indicated by previously associated syntax elements, by considering the locations of significant transformation coefficients indicated by previously associated syntax elements. This applies to smaller blocks as well, but it is especially true for larger blocks.

[0016] In one embodiment of the present application, the entropy decoder is configured to extract information from a data stream that enables it to recognize whether the significance transformation coefficient currently indicated by the currently associated syntax element is the last significance transformation coefficient, independent of its exact position within the transformation coefficient block, where the entropy decoder is configured not to anticipate any further syntax elements if the current syntax element relates to such a last significance transformation coefficient. This information may include the number of significance transformation coefficients within that block. Alternatively, a second syntax element is interleaved with the first syntax element, where the second syntax element indicates whether the last transformation coefficient in the transformation coefficient block is also the same as the associated position where the significance transformation coefficient is located.

[0017] In one embodiment, the associater simply applies a scan order corresponding to the position of a previously indicated significant transformation coefficient at a given location within the transformation coefficient block. For example, several subpaths traversing a mutually isolated subset of locations within the transformation coefficient block extend substantially diagonally from a pair of sides of the transformation coefficient block corresponding to the minimum frequency along a first direction and the maximum frequency along the other direction, respectively, to an opposite pair of sides of the transformation coefficient block corresponding to the zero frequency along a second direction and the maximum frequency along the first direction, respectively. In this case, the associater is configured to select a scan order such that the subpaths are traversed in an order in which the distance to the DC position of the subpath within the transformation coefficient block increases monotonically between subpaths, each subpath is traversed without interruption along the direction of travel, and for each subpath, the direction in which the subpath is traversed is selected by the associater in accordance with the position of a significant transformation coefficient traversed in the previous subpath. This strategy increases the probability that the last subpathway where the last significance transformation coefficient is located is traversed in a direction that makes it more likely that the last significance transformation coefficient is located in the first half of this last subpathway than in the second half, thereby reducing the number of syntax elements that indicate whether a significant or non-significant transformation coefficient is located at each position. This can be achieved. This effect is particularly significant in the case of large conversion factor blocks.

[0018] A further aspect of this application is based on the insight that a significance map indicating the location of significant transformation coefficients within a transformation coefficient block can be encoded more efficiently when the aforementioned syntax elements, which indicate whether significant or non-significant transformation coefficients are located at each position, are context-adaptively entropy-decoded to their associated positions within the transformation coefficient block, using a context individually selected for each syntax element, depending on a number of significant transformation coefficients in the neighborhood of each syntax element indicated as significant by any of the preceding syntax elements. In particular, the inventors have found that for transformation coefficient blocks of increasing size, significant transformation coefficients are clustered to some extent in a given area within the transformation coefficient block, and thereafter, context adaptation that considers the neighborhood of significant transformation coefficients as well as being sensitive to the number of significant transformation coefficients traversed in a given scan order leads to a better fit of the context and thus increases the coding efficiency of entropy coding.

[0019] Of course, both aspects outlined above can be combined in a desirable manner.

[0020] According to a further aspect of the present application, the present application is such that a significance map indicating the positions of significant transform coefficients within a transform coefficient block precedes the encoding of the actual values of the significant transform coefficients within the transform coefficient block, and a predetermined scanning order between the positions of the transform coefficients used to sequentially associate the sequence of values of the significant transform coefficients with the positions of the significant transform coefficients, when scanning the positions of the transform coefficients within sub-blocks in a coefficient scanning order and scanning the transform coefficient blocks in sub-blocks using a sub-block scanning order between sub-blocks, and when a set selected from a plurality of sets consisting of a number of contexts is used for sequential context-adaptive entropy decoding of the selection of the set selected depending on the values of the significant transform coefficient values, the values of the transform coefficients within the sub-blocks of the transform coefficient blocks already passed in the sub-block scanning order, or the values of the transform coefficients of the sub-blocks located together in a previously decoded transform coefficient block, based on the insight that the encoding efficiency for the encoding of the transform coefficient blocks can be improved. In this way, context adaptation becomes very suitable for the above-described characteristics of the significant transform coefficients concentrated in a predetermined area within the transform coefficient block. In other words, the values are scanned within the sub-blocks in a context selected based on sub-block statistics.

[0021] Again, the last aspect can also be combined with any or both of the aspects previously identified in the present application.

[0022] Preferred embodiments of the present application will be described below with reference to the drawings.

Brief Description of the Drawings

[0023] [Figure 1] A block diagram of an encoder according to one embodiment is shown. [Figure 2A] Schematically shows the subdivision of a sample array such as an image into blocks. [Figure 2B] Schematically shows another subdivision into blocks. [Figure 2C] Schematically shows another subdivision into blocks. [Figure 3] A block diagram of a decoder according to one embodiment is shown. [Figure 4] A more detailed block diagram of an encoder according to one embodiment of this application is shown below. [Figure 5] A more detailed block diagram of a decoder according to one embodiment of this application is shown below. [Figure 6] The transformation from the spatial domain to the spectral domain of a block is schematically shown. [Figure 7] A block diagram of a device for decoding a significance map of a transformation coefficient block and significance transformation coefficients according to one embodiment is shown. [Figure 8] This diagram schematically shows the subpaths of the scanning sequence and the subdivisions into different transverse directions. [Figure 9] A schematic representation of the neighbor definition for a certain scan position within a conversion block according to one embodiment is shown below. [Figure 10] A schematic representation of possible neighbor definitions for traversal positions within a transformation block at its boundary is shown. [Figure 11] This invention demonstrates the possible scanning of a conversion block according to a further embodiment of the present application. [Modes for carrying out the invention]

[0024] In the descriptions of the figures, elements appearing in several figures are indicated by the same reference numeral in each of those figures, and repeated descriptions of those elements are avoided as long as their function is of interest, in order to avoid unnecessary repetition. However, the function and description given for one drawing shall apply to other drawings unless otherwise explicitly stated.

[0025] Figure 1 shows an example of an encoder 10 in which various aspects of the present application are implemented. The encoder encodes a series of information samples 20 into a data stream. A series of information samples means any kind of information signal that has been spatially sampled. For example, the sample array 20 may be an image that constitutes a still image or a video. In this case, the information samples may be brightness values, color values, luma values, chroma values, etc. The information samples may also be depth values ​​if the sample array 20 is a depth map generated by, for example, an emission time sensor.

[0026] Encoder 10 is a block-based encoder. That is, encoder 10 encodes the sample array 20 into a data stream 30 in units of 40 blocks. Encoding in units of 40 blocks does not necessarily mean that encoder 10 encodes these blocks 40 completely independently of each other. Conversely, encoder 10 may use reconstructed previously encoded blocks to extrapolate or intra-predict the remaining blocks, or it may use a block granularity to set encoding parameters, i.e., to set the method by which each sample array region corresponding to each block is encoded.

[0027] Furthermore, the encoder 10 is a transform encoder. That is, the encoder 10 encodes the blocks 40 by using a transform to move the information samples within each block 40 from the spatial domain to the spectral domain. A two-dimensional transform such as the DCT of the FFT is used. Preferably, the blocks 40 are square or rectangular.

[0028] The subdivision of the sample array 20 into blocks 40 shown in Figure 1 is for illustrative purposes only. Figure 1 shows the sample array 20 subdivided into a regular two-dimensional arrangement of adjacent square or rectangular blocks 40 that do not overlap. The size of the blocks 40 can be predetermined. In that case, the encoder 10 does not need to transfer information about the block size of the blocks 40 in the data stream 30 to the decoder. For example, the decoder can anticipate its predetermined block size.

[0029] On the other hand, several alternatives are possible. For example, blocks may overlap each other. However, the overlap is limited such that each block has a portion that does not overlap any adjacent block, or that each sample of a block overlaps with at most one adjacent block among those arranged side by side with the current block along a given direction. The latter is such that the adjacent blocks on the left and right overlap so that they completely cover the current block. This means that adjacent blocks can be placed next to each other, but the blocks themselves cannot overlap. The same applies to adjacent blocks in the vertical and diagonal directions.

[0030] As a further alternative, the subdivision of the sample array 20 into blocks 40 can be applied to the contents of the sample array 20 by the ender 10, and the subdivision information used in the subdivision can be transferred to the decoder side via the bitstream 30.

[0031] Figures 2A to 2C show different examples of the subdivision of the sample array 20 into blocks 40. Figure 2A shows a quadtree-based subdivision of the sample array 20 into blocks 40 of different sizes, where representative blocks are shown by 40a, 40b, 40c, and 40d in increasing order of size. According to the subdivision in Figure 2A, the sample array 20 is first divided into a regular two-dimensional arrangement of tree blocks 40d. These tree blocks 40d have associated individual subdivision information regarding whether a given tree block 40d is further subdivided according to the quadtree structure. The tree blocks to the left of block 40d are, exemplariously, subdivided into smaller blocks according to the quadtree structure. The encoder 10 can perform one two-dimensional transformation for each of the blocks shown by solid and dashed lines in Figure 2A. In other words, the encoder 10 can transform the array 20 in units of block subdivision.

[0032] Instead of quadtree-based subdivision, a more general double-tree-based subdivision may be used, and the number of child nodes per hierarchical level may differ between different hierarchical levels.

[0033] Figure 2B shows another example of subdivision. According to Figure 2B, the sample array 20 is first divided into macroblocks 40b in a regular two-dimensional arrangement, adjacent to each other without overlap. Here, each microblock 40b is associated with subdivision information, which subdivides the microblock into subblocks of the same size in a regular two-dimensional arrangement, either without subdivision or, if subdivided, achieving different subdivision granularities for different microblocks. This results in the subdivision of the sample array 20 into blocks 40 of different sizes, representative examples of different sizes are shown in 40a, 40b, and 40a'. As shown in Figure 2A, the encoder 10 performs a two-dimensional transformation on each of the blocks shown by solid and dashed lines in Figure 2B. Figure 2C will be discussed later.

[0034] Figure 3 shows a decoder 50 that can decode the data stream 30 generated by the encoder 10 to reconstruct a reconstructed version 60 of the sample array 20. The decoder 50 reconstructs the reconstructed version 60 by extracting a transformation coefficient block for each of the blocks 40 from the data stream 30 and performing an inverse transformation on each of the transformation coefficient blocks.

[0035] The encoder 10 and decoder 50 are configured to perform entropy coding / decoding, respectively, to insert information about the transformation coefficient blocks and to extract this information from the data stream. Details regarding this will be described later. Note that the data stream 30 does not necessarily have to contain information about the transformation coefficient blocks for all blocks 40 of the sample array 20. Conversely, a subset of blocks 40 may be coded into the bitstream 30 by other methods. For example, the encoder 10 may decide not to insert a transformation coefficient block for a certain block of block 40, and instead insert alternative coding parameters into the bitstream 30 so that the decoder 50 can predict them, or otherwise fill each block in the reconstructed version 60. For example, the encoder 10 may put blocks into the sample array 20 so that the decoder can perform texture synthesis to fill the sample array 20. Alternatively, texture analysis can be performed to determine the placement of elements, and these results can be appropriately displayed within the bitstream.

[0036] As will be explained in the following drawings, the transformation coefficient blocks do not necessarily represent the spectral domain representation of the original information sample of each block 40 of the sample array 20. Conversely, such transformation coefficient blocks may represent the spectral domain representation of the prediction residue of each block 40. Figure 4 shows an embodiment of such an encoder. The encoder in Figure 4 comprises a transformation stage 100, an entropy encoder 102, an inverse transformation stage 104, a predictor 106, a subtractor 108, and an adder 110. The subtractor 108, the transformation stage 100, and the entropy encoder 102 are connected in series between the input 112 and the output 114 of the encoder in Figure 4 in this order. The inverse transformation stage 104, the adder 110, and the predictor 106 are connected in this order between the output of the transformation stage 100 and the inverting input of the subtractor 108, and the output of the predictor 106 is also connected to the other input of the adder 110.

[0037] The coder in Figure 4 is a predictive transform-based block encoder. That is, a block of the sample array 20 entering input 112 is predicted from previously encoded and reconstructed portions of the same sample array 20, or from other previously encoded and reconstructed sample arrays that precede or follow the current sample array 20 in time. The prediction is performed by the predictor 106. The subtractor 108 subtracts the predicted value from such an original block, and the transform stage 100 performs a two-dimensional transform on the prediction residue. The two-dimensional transform itself or subsequent processing within the transform stage 100 leads to the quantization of the transform coefficients within the transform coefficient block. The quantized transform coefficient block is losslessly encoded by entropy coding in the entropy encoder 102, for example, so that the resulting data stream is output at output 114. The inverse transform stage 104 reconstructs the quantized residual, and the adder 100 then combines the reconstructed residual with the corresponding prediction to obtain a reconstructed information sample on which the predictor 106 will predict the current encoded prediction block described above. The predictor 106 can use different prediction modes, such as intra-prediction mode and inter-prediction mode, to predict the block, and the prediction parameters are transferred to the entropy encoder 102 for insertion into the data stream.

[0038] In other words, according to the embodiment shown in Figure 4, the conversion coefficient block represents the spectral representation of the remainder of the sample array, rather than the actual information sample of the sample array.

[0039] Note that several alternative embodiments exist for the embodiment shown in Figure 4, some of which are described in the introduction of the specification and are incorporated here into the description of Figure 4. For example, the predictions generated by the predictor 106 do not need to be entropy encoded. Conversely, subinformation may be transferred to the decoder via other encoding schemes.

[0040] Figure 5 shows a decoder capable of decoding the data stream generated by the encoder in Figure 4. The decoder in Figure 5 comprises an entropy decoder 150, an inverse transform stage 152, an adder 154, and a predictor 156. The entropy decoder 150, the inverse transform stage 152, and the adder 154 are connected in this order in series between the input 158 ​​and the output 160 of the decoder in Figure 5. The other output of the entropy decoder 150 is connected to the predictor 156, which is then connected between the output of the adder 154 and its other inputs. The entropy decoder 150 extracts a transform coefficient block from the data stream entering the decoder in Figure 5 at input 158, where an inverse transform is applied to the transform coefficient block in stage 152 to obtain the residual signal. The residual signal is combined in adder 154 with the prediction from predictor 156 to obtain a reconstructed block of the reconstructed version of the sample array at output 160. Based on the reconstructed version, predictor 156 generates a prediction, thereby reconstructing the prediction made by predictor 106 on the encoder side. To obtain the same predicted value, the predictor 156 uses prediction parameters, which the entropy decoder 150 obtains from the data stream of input 158.

[0041] In the above embodiment, the spatial granularity at which residual prediction and transformation are performed does not have to be equal to each other. This is shown in Figure 2C, where the subdivision of the prediction granularity relative to the prediction block is shown by solid lines and the residual granularity by dashed lines. As can be seen from the figure, the subdivisions are selected by mutually independent encoders. More precisely, the syntax of the data stream allows for the definition of residual subdivisions independent of prediction subdivisions. Alternatively, the residual subdivisions may be an extension of the prediction subdivisions, where each residual block is equal to or an appropriate subset of the prediction block. This is shown, for example, in Figures 2A and 2B, where the prediction granularity is shown by solid lines and the residual granularity by dashed lines. In this regard, in Figures 2A-2C, the large solid block encompassing the dashed block 40a becomes, for example, a prediction block at which prediction parameter settings are performed individually, while all blocks with associated reference codes become residual blocks at which a single two-dimensional transformation is performed.

[0042] The embodiments described above share the common feature that a block of (residual or original) samples is converted into a conversion coefficient block on the encoder side, and that conversion coefficient block is inversely converted into a reconstructed sample block on the decoder side. This is illustrated in Figure 6. Figure 6 shows a block of sample 200. In Figure 6, this block 200 is, exemplarily, a two-dimensional sample 202 with a size of 4x4. The sample 202 is regularly arranged along the horizontal x and vertical y directions. The two-dimensional transformation T described above converts block 200 into a block 204 of spectral domains, i.e., conversion coefficients 206. Here, the transformation block 204 is the same size as block 200. That is, the transformation block 204 has the same number of conversion coefficients 206 as the number of samples in block 200, both horizontally and vertically. However, since the transformation T is a spectral transformation, the positions of the conversion coefficients 206 within the transformation block 204 correspond to spectral components, not the spatial positions of the contents of block 200. In particular, the horizontal axis of the transformation block 204 corresponds to an axis along which the horizontal spectral frequency increases monotonically, and the vertical axis corresponds to an axis along which the vertical spatial frequency increases monotonically. The DC component transformation coefficients are located at the corners of block 204, here exemplarily at the top left corner, such that the transformation coefficient 206 corresponding to the highest frequency in both the horizontal and vertical directions is located at the bottom right corner. If the spatial direction is ignored, the spatial frequency to which a given transformation coefficient 206 belongs generally increases from the top left corner to the bottom right corner. Inverse transformation T -1 As a result, the transformed block 204 is retransferred from the spectral domain to the spatial domain to reacquire a copy 208 of block 200. If no quantization / loss occurs during the transformation, the reconstruction is complete.

[0043] As already mentioned above, Figure 6 shows that as the block size of block 200 increases, the spectral resolution of the resulting spectral display 204 increases. On the other hand, quantization noise tends to spread throughout block 208, and therefore, abrupt and highly localized objects within block 200 tend to cause deviations in the re-transformed block compared to the original block 200 due to quantization noise. On the other hand, the main advantage of using larger blocks is that, compared to smaller blocks, the ratio of significant, i.e., non-zero (quantized) transformation coefficients to the number of non-significant transformation coefficients is reduced within larger blocks, thereby enabling better encoding efficiency. In other words, significant transformation coefficients, i.e., transformation coefficients that have not been quantized to zero, are often sparsely distributed across the transformation block 204. Consequently, according to embodiments described in more detail later, the location of significant transformation coefficients is signaled in the data stream by a significance map. Separately, if the transformation coefficients are quantized, the value of the significant transformation coefficient, i.e., the transformation coefficient level, is transmitted in the data stream.

[0044] Accordingly, according to the embodiments of this application, an apparatus for decoding such a significance map from a data stream, or for decoding a significance map along corresponding significance transformation coefficient values ​​from a data stream, is implemented as shown in Figure 7, and each of the above-mentioned entropy decoders, namely decoder 50 and entropy decoder 150, constitutes the apparatus shown in Figure 7.

[0045] The apparatus in Figure 7 comprises a map / coefficient entropy decoder 250 and an associater 252. The map / coefficient entropy decoder 250 is connected to an input 254, to which syntax elements representing significance maps and significance transformation coefficient values ​​are input. Different probabilities exist regarding the order in which the syntax elements describing significance maps and significance transformation coefficient values ​​are input to the map / coefficient entropy decoder 250, as will be described in more detail below. The significance map syntax elements may precede the corresponding levels, and the two may be interleaved. However, the syntax elements representing the significance map may precede the significance transformation coefficient values ​​(levels), and the map / coefficient entropy decoder 250 will first decode the significance map, and then the transformation coefficient levels of the significance transformation coefficients.

[0046] As the map / coefficient entropy decoder 250 sequentially decodes syntax elements representing significance maps and significance transformation coefficient values, the associater 252 is configured to associate these sequentially decoded syntax elements / values ​​with positions within the transformation block 256. The scan order in which the associater 252 associates the sequentially decoded syntax elements representing the levels of significance maps and significance transformation coefficients with each position in the transformation block 256 follows the same one-dimensional scan order between each position in the transformation block 256 as the order used by the encoding side to introduce these elements into the data stream. Furthermore, as outlined in more detail below, the scan order for the syntax elements of the significance map may or may not be the same as the order used for the significance transformation values.

[0047] The map / coefficient entropy decoder 250 can utilize information about the available transformation blocks 256 generated by the associater 252 up to the syntax element / level to be decoded, in order to set up a probability estimation context for entropy decoding the syntax element / level to be decoded, as shown by the dashed line 258. For example, the associater 252 may store (log) information collected up to that point from sequentially associated syntax elements, such as information about the level itself, or whether significant transformation coefficients are placed at each position, or whether it knows nothing about each position of the transformation block 256, and allow the map / coefficient entropy decoder 250 to access this memory. Although the memory is not shown in Figure 7, since there is a memory or log buffer to store the prior information acquired up to that point by the associater 252 and the entropy decoder 250, it can be assumed that reference numeral 256 also indicates this memory. Therefore, Figure 7 shows that the "x" marks indicate the significance transformation coefficients obtained from the syntax elements that were decoded before representing the significance map, and "1" indicates that the significance transformation coefficient level for the significance transformation coefficient at each location has already been decoded and is 1. If the significance map syntax elements precede the significance values ​​in the data stream, an "x" (significance transformation coefficient) is recorded at the "1" position in memory 256 before each value is decoded and a "1" is recorded (this situation represents the entire significance map).

[0048] The following description focuses on specific embodiments for encoding transformation coefficient blocks or significance maps, but these embodiments can be easily transitioned to the embodiments described above. In these embodiments, the binary syntax element coded_block_flag is sent to each transformation block, and that transformation block is the significance transformation coefficient level (i.e., non-zero). This indicates whether or not it contains the transformation coefficient (which is 'b'). If this syntax element indicates the existence of a significant transformation coefficient level, the significance map is encoded. That is, encoding is performed only if a significant transformation coefficient level exists. The significance map identifies which transformation coefficient levels have non-zero values, as described above. Significance map encoding involves encoding the binary syntax element significant_coeff_flag, where each binary syntax element significant_coeff_flag identifies whether the corresponding transformation coefficient level is not equal to zero for its associated coefficient position. The encoding is performed in a certain scan order, which can change during significance map encoding depending on the positions of significance coefficients previously identified as significant. This will be explained in more detail below. Furthermore, significance map encoding involves encoding the binary syntax element last_significant_coeff_flag. This binary syntax element is distributed at its positions along with a sequence of significant_coeff_flag, where significant_coeff_flag signals the significance coefficient. If the significant_coeff_flag bin is equal to 1, i.e., if a non-zero transformation coefficient level exists at this scan location, a further binary syntax element last_significant_coeff_flag is encoded. This bin indicates whether the current significance transformation coefficient level is the last significance transformation coefficient level in the block, or whether further significance transformation coefficient levels follow the scan order. If last_significant_coeff_flag indicates that no further significance transformation coefficients follow, no further syntax elements are encoded to identify the significance map for that block. Alternatively, the number of significance coefficient locations may be signaled in the data stream before encoding the sequence of significant_coeff_flags. In the next step, the values ​​of the significance transformation coefficient levels are encoded. As mentioned above, alternatively, the transmission of levels may be interleaved with the transmission of significance maps. The values ​​of the significance transformation coefficient levels are encoded in the further scan order illustrated below.The following three syntax elements are used: The binary syntax element coeff_abs_greater_one indicates whether the absolute value of the significance conversion coefficient level is greater than 1. If the binary syntax element coeff_abs_greater_one indicates that the absolute value is greater than 1, a further syntax element coeff_abs_level_minus_one is sent, which specifies the absolute value of the conversion coefficient level minus 1. Finally, the binary syntax element coeff_sign_flag, which specifies the sign of the conversion coefficient value, is encoded for each significance conversion coefficient level.

[0049] The embodiments described below enable further reduction of the bitrate, thereby increasing encoding efficiency. To do so, these embodiments employ a specific approach to contextual modeling for syntax elements related to transformation coefficients. In particular, novel contextual model selections are used for the syntax elements significant_coeff_flag, last_significant_coeff_flag, coeff_abs_greater_one, and coeff_abs_level_minus_one. Furthermore, adaptive switching of scanning during encoding / decoding of the significance map (which identifies the location of non-zero transformation coefficient levels) is described. For the meaning of the syntax elements to be mentioned, refer to the above introduction of this application.

[0050] The encoding of the significant_coeff_flag and last_significant_coeff_flag syntax elements, which identify significant maps, is improved by a novel context modeling based on adaptive scanning and limited neighbors of already encoded scanning locations. These new concepts result in more efficient encoding of significant maps (i.e., a reduction in the corresponding bitrate), especially for large block sizes.

[0051] One aspect of the embodiments outlined below is that the scan order (i.e., the mapping of blocks of transformation coefficient values ​​to an ordered set (vector) of transformation coefficient values) is fitted during the encoding / decoding of the significance map based on the values ​​of syntax elements that have already been encoded / decoded for the significance map.

[0052] In a preferred embodiment, the scanning order is adaptively switched between two or more predetermined scanning patterns. In a preferred embodiment, the switching occurs only at a predetermined scanning position. In a further preferred embodiment of the present invention, the scanning order is adaptively switched between two predetermined scanning patterns. In a preferred embodiment, the switching between the two predetermined scanning patterns occurs only at a predetermined scanning position.

[0053] The advantage of switching scan patterns is a reduction in bitrate, which results in a smaller number of encoded syntax elements. As an intuitive example, referring to Figure 6, particularly for large transformation blocks, significant transformation coefficient values ​​often converge on one of the block boundaries 270,272. This is because the remaining blocks primarily contain horizontal or vertical structures. In the most commonly used zigzag scan 274, there is approximately a 0.5 probability that the last diagonal subscan of the zigzag scan encountering the last significant coefficient begins on the side where the significance coefficients are not concentrated. In that case, a large number of syntax elements for transformation coefficient levels equal to zero must be encoded before reaching the last non-zero transformation coefficient value. This can be avoided if the diagonal subscan always begins on the side where the significant transformation coefficient levels are concentrated.

[0054] Preferred embodiments of the present invention will be described in more detail below.

[0055] As mentioned above, it is preferable to keep the number of context models moderately small in order to enable rapid adaptation of context models even for large block sizes and to achieve high coding efficiency. Therefore, a particular context should be used for two or more scan positions. However, since significance transformation coefficient levels are usually concentrated in a certain area of ​​the transformation block (this concentration is usually the result of a dominant structure present, for example, in the remaining blocks), the concept of assigning the same context to many consecutive scan positions, as is done for 8x8 blocks in H.264, is usually not appropriate. The above consideration that significance transformation coefficient levels are often concentrated in a given area of ​​the transformation block can be used to design context selection. Concepts in which this consideration can be used will be explained below.

[0056] In one preferred embodiment, a large transformation block (e.g., larger than 8x8) is divided into a number of rectangular subblocks (e.g., 16 subblocks), each of which is associated with a separate context model for encoding significant_coeff_flag and last_significant_coeff_flag (here, different context models are used for significant_coeff_flag and last_significant_coeff_flag). The division into subblocks may differ for significant_coeff_flag and last_significant_coeff_flag. The same context model may be used for all scan locations located within a particular subblock.

[0057] In a further preferred embodiment, a large transformation block (e.g., larger than 8x8) is divided into numerous rectangular and / or non-rectangular subregions, each of which is associated with a separate context model for encoding significant_coeff_flag and / or last_significant_coeff_flag. The division into subregions is determined by significant_coe ff_flag and last_significant_coeff_flag may be different. The same context model is used for all scan locations located within a particular sub-region.

[0058] In a further preferred embodiment, a context model for encoding significant_coeff_flag and / or last_significant_coeff_flag is selected based on already encoded symbols in a given spatial neighborhood of the current scan location. The given neighborhood may differ for different scan locations. In a preferred embodiment, the context model is selected based on the number of significance transformation coefficient levels in a given spatial neighborhood of the current scan location, where only already encoded significance representations are counted.

[0059] Further details of preferred embodiments of the present invention are described below.

[0060] As described above, for large block sizes, conventional context modeling encodes a large number of bins with a single context model (usually with different probabilities) for the syntactic elements coeff_abs_greater_one and coeff_abs_level_minus_one. To circumvent this drawback for large block sizes, according to the embodiment, the large block is divided into small, specific-sized square or rectangular subblocks, and a separate context model is applied to each subblock. Furthermore, multiple sets of context models may be used, where one of these sets of context models is selected for each subblock based on an analysis of the statistics of previously encoded subblocks. In a preferred embodiment of the present invention, the number of transformation coefficients greater than 2 (i.e., coeff_abs_level_minus_1 > 1) in previously encoded subblocks within the same block is used to derive a set of context models for the current subblock. These extensions to the contextual modeling of the syntax elements in coeff_abs_greater_one and coeff_abs_level_minus_one result in more efficient encoding of both syntax elements, especially for larger block sizes. In a preferred embodiment, the subblock size is 2x2. In another preferred embodiment, the subblock size is 4x4.

[0061] In the first step, blocks larger than a given size are divided into smaller subblocks of a specific size. The encoding process for absolute transformation coefficient levels maps the subblocks, which are square or rectangular blocks, to an ordered set (vector) of subblocks using scanning. Here, different scans can be used for different blocks. In a preferred embodiment, subblocks are processed using zigzag scanning, and the transformation coefficient levels within the subblocks are processed with inverse zigzag scanning, i.e., scanning readout from the transformation coefficients belonging to the highest frequencies in the vertical and horizontal directions to the coefficients related to the lowest frequencies in both directions. In another preferred embodiment of the present invention, inverse zigzag scanning is used for encoding subblocks and for encoding transformation coefficient levels within subblocks. In another preferred embodiment of the present invention, the same adaptive scan (see above) used to encode the significance map is used to process the entire block of transformation coefficient levels.

[0062] Dividing large transformation blocks into subblocks avoids the problem of using only one context model for most of the bins in a large transformation block. Within the subblocks, the latest context modeling or fixed context (as defined in H.264) is used depending on the actual size of the subblock. Furthermore, statistics (in terms of probabilistic modeling) for such subblocks are used for transformations of the same size. This differs from the statistics of locks. This property can be utilized in the syntax elements of coeff_abs_greater_one and coeff_abs_level_minus_one by extending the set of context models. Multiple sets of context models may be prepared, and for each subblock, one of these sets of context models may be selected based on the statistics of previously encoded subblocks in the current transform block or a previously encoded transform block. In a preferred embodiment of the present invention, the selected set of context models is derived based on the statistics of previously encoded subblocks in the same block. In another preferred embodiment of the present invention, the selected set of context models is derived based on the statistics of the same subblocks in a previously encoded block. In a preferred embodiment, the number of sets of context models is set to be equal to 4, and in another preferred embodiment, the number of sets of context models is set to be equal to 16. In a preferred embodiment, the statistic used to determine the set of context models is the number of absolute transform coefficient levels greater than 2 in the previously encoded subblock. In another preferred embodiment, the statistic used to determine the set of context models is the difference between the number of significance coefficients and the number of transform coefficient levels, with an absolute value greater than 2.

[0063] Significant map coding is performed as outlined below, i.e., by adaptive switching of the scan order.

[0064] In a preferred embodiment, the scan order for encoding the significance map is adapted by switching between two predetermined scan patterns. The switching between scan patterns occurs only at a predetermined scan position. The decision of whether or not to switch scan patterns depends on the values ​​of already encoded / decoded significance map syntax elements. In a preferred embodiment, both predetermined scan patterns are scan patterns with diagonal subscans similar to those of a zigzag scan. The scan patterns are shown in Figure 8. Both scan patterns 300 and 302 consist of numerous diagonal subscans along the diagonal from the bottom left to the top right, or vice versa. The scans in the diagonal subscans (not shown) are performed from the top left to the bottom right for both predetermined scan patterns. However, the scans within the diagonal subscans are different (as shown in the figure). For the first scanning pattern 300, the diagonal subscan is performed from the bottom left to the top right (left panel of Figure 8), and for the second scanning pattern 302, the diagonal subscan is performed from the top right to the bottom left (right panel of Figure 8). In one embodiment, encoding of the significance map begins with the second scanning pattern. While encoding / decoding the syntax elements, the number of significance transformation coefficient values ​​is counted by two counters c1 and c2. The first counter c1 is The number of significant transformation coefficients located in the bottom left part of the transformation block is counted. That is, this first counter c1 is the number of significant transformation coefficients where the horizontal coordinate x in the transformation block is smaller than the vertical coordinate y. The conversion coefficient level is incremented by 1 when it is encoded / decoded. The second counter c2 is This counts the number of significant transformation coefficients located in the upper right section of the transformation block. In other words, this second counter c2 counts the number of significant transformation coefficients where the horizontal coordinate x in the transformation block is greater than the vertical coordinate y. The conversion coefficient level is incremented by 1 when it is encoded / decoded. The counter fitting can be performed by associater 252 in Figure 7 and can be explained by the following equation, where t represents the scan position index, and both counters are initialized to zero.

number

number

[0065] At the end of each diagonal subscan, the associater 252 determines which of the first and second predetermined scan patterns 300, 302 will be used for the next diagonal subscan. This determination is based on the values ​​of counters c1 and c2. If the count value for the bottom left portion of the conversion block is greater than the count value of the bottom left portion, the scan pattern that performs a diagonal subscan from the bottom left to the top right is used. Otherwise (if the count value for the bottom left portion of the conversion block is less than or equal to the count value of the bottom left portion), the scan pattern that performs a diagonal subscan from the top right to the bottom left is used. This determination is expressed by the following formula.

number

[0066] The embodiments of the present invention described above can be readily applied to other scanning patterns. For example, a scanning pattern used in a field macroblock in H.264 can be broken down into subscans. In a more preferred embodiment, any given scanning pattern can be divided into subscans. For each subscan, two scanning patterns are defined (as the basic scanning direction): one from bottom left to top right, and the other from top right to bottom left. Furthermore, two counters are introduced to count the number of significance coefficients in the first part of the subscan (closer to the bottom left boundary of the transform block) and the second part (closer to the top right boundary of the transform block). Finally, at the end of each subscan (based on the counter values), it is determined whether the next subscan will scan from bottom left to top right or from top right to bottom left.

[0067] Next, we will describe an embodiment of how the entropy decoder 250 models the context.

[0068] In one preferred embodiment, context modeling for significant_coeff_flag is performed as follows: For a 4x4 block, context modeling is performed as specified in H.264. For an 8x8 block, the transformation block is separated into 16 2x2 sample subblocks, each of which is associated with a separate context. This concept can be extended to larger block sizes, different numbers of subblocks, and non-rectangular subregions, as described above.

[0069] In a more preferred embodiment, context model selection for larger transformation blocks (e.g., for blocks larger than 8x8) is based on the number of already encoded significant transformation coefficients in a given neighborhood (within the transformation block). An example of a neighborhood definition corresponding to a preferred embodiment of the present invention is shown in Figure 9. Neighbors circled in × are available neighborhoods that are always taken into consideration for evaluation, while those with × and △ are neighborhoods that are evaluated depending on the current scan position and current scan direction): • If the current scan location is within the left corner 304 of a 2x2 grid, a separate context model is used for each scan location (Figure 9, left diagram). If the current scan position is not within the left corner of the 2x2 grid and is not located in the first row or first column of the transformation block, the neighborhood shown on the right side of Figure 9 is used to evaluate the number of significant transformation coefficients in the neighborhood of the current scan position "x" where there is nothing around it. If the current scan position "x", which has nothing around it, is the first row of the transformation block, the neighbor identified in the right diagram of Figure 10 is used. • If the current scan position "x" is in the first column of the block, the neighbors identified in the left diagram of Figure 10 are used.

[0070] In other words, the decoder 250 is configured to sequentially extract significance map syntax elements by performing context-adaptive entropy decoding for each significance map syntax element, using a context individually selected, depending on a number of positions where significance transformation coefficients are placed according to previously extracted and associated significance map syntax elements, and positions limited to those in the neighborhood of the position to which each current significance map syntax element is associated (either the "x" on the right side of Figure 9 and both sides of Figure 10, and the marked position on the left side of Figure 9). As illustrated, the neighborhood of the position to which each current syntax element is associated consists at most of either directly adjacent positions or positions separated from the position to which each significance map syntax element is associated, in both vertical and / or horizontal directions. Alternatively, only positions directly adjacent to each current syntax element are considered. As a result, the size of the transformation coefficient block will be 8 × 8 or larger positions.

[0071] In a preferred embodiment, the context model used to encode a particular significant_coeff_flag is selected based on the number of significant transformation coefficient levels already encoded in a defined neighborhood. Here, the number of available context models can be less than the possible number of significant transformation coefficient levels in the defined neighborhood. The encoder and decoder may include a table (or a different mapping mechanism) for mapping the number of significant transformation coefficient levels in the defined neighborhood to an index of the context model.

[0072] In a further preferred embodiment, the index of the selected context model depends on the number of significance transformation coefficient levels in the defined neighborhood, or on one or more additional parameters such as the type of neighborhood used or the scan location or the quantized value of the scan location.

[0073] For encoding last_significant_coeff_flag, a similar context modeling can be used as for significant_coeff_flag. On the other hand, the probability measurement for last_significant_coeff_flag depends primarily on the distance of the current scan position to the top left corner of the transformation block. In a preferred embodiment, the context model for encoding last_significant_coeff_flag is selected based on a scan diagonal where the current scan position is located (i.e., in the above embodiment of Figure 8, based on x+y, where x and y are the horizontal and vertical positions of the scan position within the transformation block, respectively, or based on how many subscans are located between the current subscan and the upper left DC (such as subscan index minus 1)). In a preferred embodiment of the present invention, the same context is used for different values ​​of x°+°y. The distance measurement, i.e., x+y or subscan index, is mapped onto a set of context models in a predetermined manner (for example, by quantizing x°+°y or subscan index), where the number of possible values ​​for the distance measurement is greater than the number of available context models for encoding last_significant_coeff_flag.

[0074] In a preferred embodiment, different context modeling techniques are used for transformation blocks of different sizes.

[0075] The coding of absolute conversion coefficients is explained below.

[0076] In one preferred embodiment, the subblock size is 2x2, and the subblock Context modeling within a subblock is disabled. That is, one single context model is used for all transformation coefficients within a 2x2 subblock. Only blocks larger than 2x2 are affected by the subdivision process. In a further preferred embodiment of the present invention, the subblock size is 4x4, and context modeling within the subblock is performed as in H.264, with only blocks larger than 4x4 being affected by the subdivision process.

[0077] Regarding the scanning order, in a preferred embodiment, the zigzag scan 320 is used to scan the subblock 322 of the transformation block 256, i.e., to scan along the substantially increasing frequency direction, and the transformation coefficients within the subblock are scanned with the inverse zigzag scan 326 (Figure 11). In a further preferred embodiment of the present invention, both the subblock 322 and the transformation coefficient levels within the subblock 322 are scanned using the inverse zigzag scan (as shown in Figure 11 where the arrow 320 is reversed). In another preferred embodiment, the same adaptive scan used to encode the significance map is used to process the transformation coefficient levels, where, since the adaptive judgment is the same, the exact same scan is used for both encoding the significance map and encoding the transformation coefficient level values. Note that the scan itself does not typically depend on selected statistics or numbers of the context model set, or on judgments to enable or disable context modeling within the subblock.

[0078] Next, we will describe an embodiment of context modeling for the coefficient level.

[0079] In a preferred embodiment, context modeling for a subblock is similar to the context modeling for a 4x4 block in H.264 described above. The number of context models used to encode a coeff_abs_greater_one syntax element, and the first bin of a coeff_abs_level_minus_one syntax element, are equal to 5, for example, using different sets of context models for two syntax elements. In a further preferred embodiment, context modeling within subblocks is disabled, and only one predetermined context model is used within each subblock. For these embodiments, the set of context models for subblock 322 is selected from a predetermined number of context model sets. The selection of context models for subblock 322 is based on some statistics of one or more subblocks that have already been encoded. In a preferred embodiment, the statistics used to select the set of context models for a subblock are taken from one or more subblocks that have already been encoded in the same block 256. How the statistics are used to determine the selected set of context models is described below. In a further preferred embodiment, statistics are taken from the same subblocks in a previously encoded block having the same block size, such as blocks 40a and 40a' in Figure 2B. In another preferred embodiment of the present invention, statistics are taken from defined neighboring subblocks in the same block, depending on a selective scan of the subblocks. It is also important to note that the sources of statistics should be independent of the scan order and how the statistics are constructed to determine the context model set.

[0080] In a preferred embodiment, the number of context model sets is equal to 4, while in another preferred embodiment, the number of context model sets is equal to 16. In common, the number of context model sets is not fixed and should be adapted according to selected statistics. In a preferred embodiment, the context model set for subblock 322 is determined based on the number of absolute transformation coefficient levels greater than 2 in one or more already encoded subblocks. The index for the context model set is determined by mapping the number of absolute transformation coefficient levels greater than 2 in a reference subblock or multiple reference subblocks to a given set of context model indices. This mapping... The mapping can be performed by quantizing the number of absolute transformation coefficient levels greater than 2, or by a predetermined table. In a more preferred embodiment, the set of context models for a subblock is determined based on the difference between the number of significant transformation coefficient levels and the number of absolute transformation coefficient levels greater than 2 in one or more already encoded subblocks. The index for the set of context models is determined by mapping this difference to a predetermined set of context model indices. This mapping can be performed by quantizing the difference between the number of significant transformation coefficient levels and the number of absolute transformation coefficient levels greater than 2, or by a predetermined table.

[0081] In other preferred embodiments, if the same adaptive scan is used to process absolute transformation coefficient levels and significance maps, partial statistics of subblocks in the same block are used to obtain the set of context models for the current subblock. Alternatively, if available, statistics of previously encoded subblocks in previously encoded transformation blocks may be used. This means, for example, that instead of using the absolute number of absolute transformation coefficient levels greater than 2 in the subblock to obtain the context model, the number of already encoded absolute transformation coefficient levels greater than 2 multiplied by the ratio of the number of transformation coefficients in the subblock to the number of already encoded transformation coefficients in the subblock is used, or instead of using the difference between the number of significance transformation coefficient levels and the number of absolute transformation coefficient levels greater than 2 in the subblock, the difference between the number of already encoded significance transformation coefficient levels and the number of already encoded absolute transformation coefficient levels greater than 2 is multiplied by the ratio of the number of transformation coefficients in the subblock to the number of already encoded transformation coefficients in the subblock is used.

[0082] Regarding context modeling within subblocks, essentially, an inversion of the latest context modeling for H.264 is employed. This means that, when the same adaptive scan is used to process absolute transformation coefficient levels and significance maps, the transformation coefficient levels are encoded in essentially forward scan order, rather than in the reverse scan order of H.264. Therefore, context model switching must be adapted accordingly. According to one embodiment, the encoding of transformation coefficient levels begins with a first context model for coeff_abs_greater_one and coeff_abs_level_minus_one syntax elements, and two coeff_abs_greater_one syntax elements equal to zero are switched to the next context model in the set of elements encoded since the last context model switch. In other words, context selection depends on the number of already encoded coeff_abs_greater_one syntax elements greater than zero in scan order. The number of context models for coeff_abs_greater_one and coeff_abs_level_minus_one should be the same as those in H.264.

[0083] Therefore, the above-described embodiments are applicable to the field of digital signal processing, particularly image and video decoders and encoders. In particular, the above-described embodiments enable the encoding of syntax elements related to transformation coefficients in block-based image and video codecs using improved contextual modeling for syntax elements related to transforming coefficients encoded in an entropy coder employing probabilistic modeling. Compared with the latest technology, improved encoding efficiency is achieved, especially for large transformation blocks.

[0084] While some aspects have been described in the context of the apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method step correspond to It also describes the block or item, or the characteristics of the corresponding device.

[0085] The ingenious encoded signals for representing the transformation blocks or significance maps, respectively, can be stored in a digital storage medium or transmitted over a transmission medium such as a wireless transmission medium like the Internet or a wired transmission medium.

[0086] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be carried out using a digital storage medium, such as a flexible disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM®, or flash memory, which stores electronically readable control signals and which cooperates (or is capable of cooperating) with a programmable computer system to perform each method. Thus, these digital storage media are computer-readable.

[0087] Some embodiments of the present invention include data carriers having electronically readable control signals, which can cooperate with a programmable computer system so that one of the methods described herein is performed.

[0088] Generally, embodiments of the present invention can be implemented as a computer program product having program code, the program code being operable to perform one of the methods when the computer program product is running on a computer. The program code is stored, for example, in a machine-readable carrier.

[0089] Other embodiments consist of a computer program for performing one of the methods described herein and stored in a machine-readable carrier.

[0090] In other words, an embodiment of the method of the present invention is a computer program having program code for executing one of the methods described herein when the computer program is running on a computer.

[0091] A further embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded.

[0092] A further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection such as the Internet.

[0093] Further embodiments include, for example, processing means such as a computer or programmable logic device configured or adapted to perform one of the methods described herein.

[0094] Further embodiments consist of a computer on which a computer program for performing one of the methods described herein is installed.

[0095] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) is used to perform some or all of the functions of the methods described herein. In some embodiments, the field-programmable gate array cooperates with a microprocessor to perform one of the methods described herein. This can be done. Generally, the method is preferably carried out by any hardware device.

[0096] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and variations of the configurations and details described herein will be apparent to those skilled in the art. Therefore, the invention is intended to be limited only by the claims set forth immediately below, and not by the specific details presented in the descriptions and explanations of the embodiments herein.

Claims

1. A device for decoding a significance map from a data stream that shows the location of significant transformation coefficients within a transformation coefficient block, A significance map indicating the positions of significant transformation coefficients within the transformation coefficient block, and a decoder configured to extract the values ​​of the significant transformation coefficients within the transformation coefficient block from a data stream, and to sequentially extract a first type of syntax element from the data stream by context-adaptive entropy decoding when extracting the significance map, wherein the first type of syntax element indicates, for each relevant position within the transformation coefficient block, whether the element located at that position is a significant transformation coefficient or a non-significant transformation coefficient, An associater configured to sequentially associate the sequentially extracted syntax elements of the first type with the positions of the conversion coefficient block in a scan order determined from among a plurality of scan orders. Equipped with, The decoder is configured to use a context individually selected for each of the first type of syntax elements when performing context-fit entropy decoding of the first type of syntax elements, based at least the scan order and the number of positions where significant transformation coefficients are located according to previously extracted and associated first type of syntax elements and which are in the vicinity of the position to which the current first type of syntax element is associated. Device.

2. The apparatus according to claim 1, wherein the decoder is configured such that the vicinity of the position to which each of the first type of syntax elements is associated includes at least one of the following: a position directly adjacent to the position to which each of the first type of syntax elements is associated, or a position directly adjacent to or separated from the position to which each of the first type of syntax elements is associated, and the size of the conversion coefficient block is 8 × 8 blocks or more.

3. A device for encoding a significance map, which indicates the location of significance transformation coefficients within a transformation coefficient block, into a data stream, wherein the device comprises a memory and a processor, the memory storing a computer program, and the computer program, when executed by the processor, Along with encoding a significance map showing the positions of significant transformation coefficients within the transformation coefficient block, and the values ​​of the significant transformation coefficients within the transformation coefficient block into the data stream, when encoding the significance map, sequential encoding of a first type of syntax element into the data stream is performed by context-adaptive entropy coding, where the first type of syntax element indicates whether the relevant position within the transformation coefficient block is a significant transformation coefficient or a non-significant transformation coefficient. Perform sequential encoding of the first type of syntax elements into the data stream in a scan order determined from among multiple scan orders. When context-fitting entropy encoding each of the first type of syntax elements, the use of a context individually selected for each of the first type of syntax elements is performed based on the scan order and the number of positions where significant transformation coefficients are placed and previously encoded first type of syntax elements are associated, and which are in the vicinity of the position to which the current first type of syntax element is associated. Device.

4. A method for encoding a significance map in a data stream that shows the location of significant transformation coefficients within a transformation coefficient block, A significance map indicating the positions of significant transformation coefficients within the transformation coefficient block, and a step of encoding the values ​​of the significant transformation coefficients within the transformation coefficient block into the data stream, wherein, when encoding the significance map, a first type of syntax element is sequentially encoded into the data stream by context-adaptive entropy coding, wherein the first type of syntax element indicates, for each relevant position within the transformation coefficient block, whether the position contains a significant transformation coefficient or a non-significant transformation coefficient. The step of sequentially encoding the first type of syntax elements into the data stream is performed in a scan order determined from among a plurality of scan orders. When each of the first type of syntax elements is context-adaptive entropy encoded, a context is used that is individually selected for each of the first type of syntax elements based on the scan order and the number of positions where significant transformation coefficients are placed and previously encoded first type of syntax elements are associated, and which are in the vicinity of the position to which the current first type of syntax element is associated. method.

Images