Inheritance in sample array multitree subdivision

Abstract

A better trade-off between coding complexity and achievable rate-distortion ratio, and / or a better rate-distortion ratio, is achieved by using multi-tree subdivision. This multi-tree subdivision is used not only to divide a continuous region, i.e., the sample array, into multiple leaf regions, but also to use intermediate regions to share coding parameters among corresponding common-location leaf blocks. This approach allows the coding process performed in locally tiled leaf regions to be associated with individual coding parameters, without needing to explicitly transmit the entire coding parameters separately to each leaf region. Instead, multi-tree subdivision effectively explores similarity.

Description

Technical Field

[0001] This invention relates to an encoding scheme for representing information sample arrays of spatially sampled information signals, such as video or still images. Prior Technology

[0002] In image and video coding, an image, or a specific set of sample arrays for such an image, is typically decomposed into blocks, which are associated with specific coding parameters. An image is usually composed of multiple sample arrays. Furthermore, an image may be associated with additional auxiliary sample arrays, such as those carrying transparency information or depth mapping tables. The sample arrays of an image (including auxiliary sample arrays) may also be grouped into one or more so-called planar groups, where each planar group consists of one or more sample arrays. Planar groups of an image can be coded independently, or if the image is associated with more than one planar group, they can be coded using predictions from other planar groups of the same image. Each planar group is typically decomposed into multiple blocks. These blocks (or corresponding blocks of sample arrays) are predicted by cross-image prediction or intra-image prediction. Each block can have a different size and can be square or rectangular. The segmentation of an image into multiple blocks can be fixed by syntax or can be (at least partially) communicated within the bitstream. Frequently sent syntax elements communicate the subdivision of blocks of a predetermined size. These syntax elements can specify whether and how a block is subdivided into smaller blocks and the associated coding parameters, for example, for prediction purposes. The decoding of the associated coding parameters for all samples of a block (or a corresponding block in a sample array) is specified in a certain way. In this example, all samples of a block are predicted using the same set of prediction parameters, such as a reference index (identifying a reference image in the set of encoded images), a movement parameter (specifying the measurement of block movement between a reference image and the current image), parameters of the interpolation filter, an intra-prediction mode, etc. The movement parameter can be represented by a displacement vector with a horizontal and a vertical component, or by a higher-order movement parameter, such as a movement parameter composed of affine six components. It is also possible that more than one specific set of prediction parameters (such as the reference index and movement parameters) is associated with a single block. In this case, for each set of specific prediction parameters, a single intermediate prediction signal is generated for that block (or the corresponding block of the sample array), and the final prediction signal is constructed from a combination including overlaid intermediate prediction signals. The corresponding weighting parameters, and possibly a constant bias (added to the weighting sum), may be fixed for an image, a reference image, or a set of reference images, or included in the set of prediction parameters for the corresponding block. The difference between the original block (or the corresponding block of the sample array) and its prediction signal, also called the residual signal, is typically transformed and quantized. Often, a two-dimensional transformation is applied to the residual signal (or the corresponding sample array for the residual block). For transform coding, the block (or the corresponding block of the sample array) already using a specific set of prediction parameters may be further split before the transformation is applied. The transformed block may be equal to or smaller than the block used for prediction. It is also possible that a transformed block includes more than one block used for prediction. Different transformation blocks can have different sizes, and transformation blocks can represent square or rectangular blocks.After the transformation, the resulting transform coefficients are quantized to obtain the so-called transform coefficient level. The transform coefficient level, prediction parameters, and, if present, subdivision information are entropy encoded.

[0003] In image and video coding standards, the possibilities for subdividing an image (or a group of planes) into blocks provided by the syntax are extremely limited. Typically, it can only be specified whether (and possibly how) a block of a predefined size can be subdivided into smaller blocks. For example, the maximum block size in H.264 is 16x16. A 16x16 block is also called a macro block, and in the first step, each image is segmented into macro blocks. For each 16x16 macro block, it can be signaled whether it is encoded as a 16x16 block, or two 16x8 blocks, or two 8x16 blocks, or four 8x8 blocks. If a 16x16 block is subdivided into four 8x8 blocks, then each 8x8 block can be encoded as one 8x8 block, or two 8x4 blocks, or four 4x4 blocks. State-of-the-art image and video coding standards specify that the possibility of dividing into small sets of blocks has the advantage of maintaining a low side information rate for transmitting segmentation information, but has the disadvantage of a low bit rate required to transmit prediction parameters for such blocks, as will be discussed in detail later. The side information rate for transmitting segmentation information typically represents a significantly high total bit rate for a block. When such side information is achieved, coding efficiency increases, for example, by using larger block sizes. The actual image or picture of a video sequence is composed of objects of arbitrary shapes with specific properties. For example, such objects or object components are characterized by unique textures or unique movements. Typically, the same set of prediction parameters can be added to such objects or object components. However, object boundaries often do not match the possible block boundaries of large prediction blocks (e.g., 16x16 macro blocks in H.264).

[0004] Encoders typically determine the minimization of the cost measurement for a specific rate distortion (within a finite set of possibilities) resulting from segmentation. For objects of arbitrary shapes, this can lead to a large number of small blocks. Since these small blocks are associated with a set of prediction parameters to be transmitted, the side information rate becomes a significant portion of the total bit rate. However, because several of these small blocks still represent the same object or part of an object, the prediction parameters for multiple resulting blocks are identical or extremely similar.

[0005] In other words, the subdivision or tiling of an image into smaller parts, tiles, or blocks substantially impacts coding efficiency and complexity. As summarized above, subdividing an image into multiple smaller blocks allows for finer spatial settings of coding parameters, thereby allowing these parameters to be better adapted to the image / video material. On the other hand, setting coding parameters with finer granularity places a greater burden on the amount of side information required to inform the decoder about the necessary settings. Furthermore, it must be noted that any degree of freedom the encoder has in (further) spatially subdividing the image / video into blocks dramatically increases the number of possible coding parameter settings, and thus usually makes the search for coding parameter settings that lead to the optimal rate / distortion tradeoff more difficult. Summary of the Invention

[0006] One objective is to provide an encoding scheme for encoding an array of information samples representing spatially sampled information signals, such as, but not limited to, video images or still images, which allows for a better trade-off between encoding complexity and achievable rate-distortion ratio, and / or achieves a better rate-distortion ratio.

[0007] This project can be achieved using a decoder as in Request 1, an encoder as in Request 10, a method as in Request 11, a computer program as in Request 12, and a data stream as in Request 13.

[0008] A potential concept of this invention is that when multi-tree subdivision is used not only to subdivide a continuous region, i.e., a sample array, into multiple leaf regions, but also when the intermediate regions are used to share coding parameters among corresponding common-location blocks, a better trade-off can be achieved between coding complexity and achievable rate-distortion ratio, and / or a better rate-distortion ratio can be achieved. With this approach, the coding process performed in locally tiled leaf regions can be associated with individual coding parameters, but it is not necessary to explicitly transmit the entire coding parameters separately to each leaf region. Instead, using multi-tree subdivision allows for efficient exploration of similarities.

[0009] According to one embodiment, firstly, an information sample array representing a spatially sampled information signal is spatially placed in the root region. Then, based on multi-level subdivision information extracted from a data stream, the root region is recursively subdivided into subsets multiple times, thereby dividing at least one subset of the root region into smaller simple connection regions of different sizes. In order to allow finding a good trade-off between excessively fine subdivision and excessively coarse subdivision in the sense of rate distortion, the minimum region size of the root region into which the information sample array is spatially subdivided is included in the data stream and extracted from the data stream at the decoding end. Accordingly, the decoder may include an extractor configured to extract maximum region size and multi-tree subdivision information from the data stream; a subdivision unit configured to spatially divide an information sample array representing spatial sampling information signals into maximum region sizes, and, based on the multi-tree subdivision information, subdivide at least one subset of the root regions into smaller simple linked regions of different sizes by recursively multi-distinguishing the subset of the root regions; and a reconstructor configured to use the subdivision to reconstruct the information sample array from the data stream into a plurality of smaller simple linked regions.

[0010] According to one embodiment, the data stream also contains a top-level hierarchy that recursively differentiates up to a subset of the root region. This method makes the transmission of multi-tree subdivision information easier and requires fewer coding bits.

[0011] In addition, the reconstructor can be configured to perform one or more of the following actions at a granularity depending on the intermediate subdivision: determining which prediction mode to use, at least in the intra- and cross-prediction modes; performing and / or setting cross-prediction parameters from the frequency domain to the spatial domain; and performing and / or setting parameters for intra-prediction.

[0012] Furthermore, the extractor can be assembled to extract syntax elements associated with leaf regions of differentiated tree blocks from the data stream in a depth-first traversal order. In this way, the extractor can explore the statistics of syntax elements already encoded in neighboring leaf regions, with a higher probability than using a breadth-first traversal order.

[0013] According to another embodiment, a further subdivision is used to subdivide at least one subset of the smaller simple linked regions into even smaller simple linked regions based on further multi-tree subdivision information. The first-stage subdivision can be used by the reconstructor to perform prediction of the information sample region, while the second-stage subdivision can be used by the reconstructor to perform a re-transformation from the frequency domain to the spatial domain. Defining residual subdivision as subordinate to prediction subdivision reduces the bit consumption of encoding the total subdivision; on the other hand, the constraints and degrees of freedom of the residual subdivision obtained from the subordinate subdivision have only a small negative impact on coding efficiency because most image regions with similar motion compensation parameters are larger than image regions with similar spectral properties.

[0014] According to yet another embodiment, another maximum region size is included within the data stream, and this maximum region size defines the size of the root subregion, which is segmented before at least a subset of these root subregions is further subdivided into smaller simple link regions based on more metatree subdivision information. This, in turn, allows for the independent setting of the maximum region size of the predicted subregions on the one hand, and allows for residual subdivision on the other hand, thus enabling the identification of a better rate / distortion tradeoff.

[0015] According to another embodiment of the invention, the data stream includes a first subset of syntax elements that does not intersect with a second subset of syntax elements, forming the multi-tree subdivision information. A merger at the decoding end allows combining small simple link regions of spatially adjacent multi-tree subdivisions based on the first subset of syntax elements to obtain an intermediate subdivision of the sample array. A reconstructor can then use these intermediate subdivisions to reconstruct the sample array. In this way, the encoder can more easily adapt the spatial distribution of the properties of the effective subdivisions into the information sample array, finding the optimal rate / distortion tradeoff. For example, if the maximum region size is large, the multi-tree subdivision information may become more complex due to the larger root region. On the other hand, if the maximum region size is small, it is more likely that adjacent root regions are related to information content with similar properties, allowing these root regions to be processed together. Merging fills this gap between the aforementioned extremes, thereby allowing near-optimal granular subdivision. From the encoder's perspective, merging syntax elements allows for a more relaxed or computationally less complex coding process because if the encoder mistakenly uses too fine a subdivision, this error can be compensated for by the encoder subsequently setting whether or not to merge syntax elements, adjusting only a small portion of the syntax elements that were set before merging syntax elements were set.

[0016] According to yet another embodiment, the maximum region size and multi-tree subdivision information are used for residual subdivision rather than predictive subdivision.

[0017] The depth-first traversal order, rather than the width-first traversal order, used to process the simple linked regions of a multi-tree subdivision representing a spatially sampled information signal is employed according to one embodiment. By using this depth-first traversal order, each simple linked region has a higher probability of having neighboring simple linked regions that have already been traversed, so that when reconstructing individual current simple linked regions, information about these neighboring simple linked regions can be positively explored.

[0018] When the information sample array is first divided into root regions of zero-level hierarchical size arranged regularly, and then further subdivided into at least one subset of these root regions into smaller simple-connected regions of unequal size, the reconstructor can use a zigzag scan sequence to scan the root regions. For each root region to be distinguished, the leaf regions of these simple connections are processed in a depth-first traversal order, and then the zigzag scan sequence is used to move on to the next root region. Furthermore, according to the depth-first traversal order, leaf regions of simple connections with the same hierarchical level can also be traversed in a zigzag scan sequence. This increases the probability of maintaining adjacent simple-connected leaf regions.

[0019] According to one embodiment, although the flags associated with the nodes of the multi-tree structure are arranged sequentially in depth-first traversal order, the sequential encoding of the flags uses a probability estimation network. This network is the same for flags associated with nodes within the same hierarchical level of the multi-tree structure, but different for flags associated with nodes within different hierarchical levels of the multi-tree structure. This allows for a good trade-off between the number of networks to be provided, and on the other hand, adjusts to adapt to the actual symbol statistics of the flags.

[0020] According to one embodiment, the probability estimation framework for the predetermined flag also depends on the flags preceding the predetermined flag according to the depth-first traversal order, and corresponds to the regions of the root region that have a predetermined relative positional relationship with the region corresponding to the predetermined flag. Similar to the underlying concept below the aforementioned surface, using the depth-first traversal order ensures a high probability that the encoded flags also include the corresponding flags of the regions adjacent to the predetermined flag region, so that this knowledge can be used to better adapt the framework for the predetermined flag.

[0021] The flags that can be used to set a predetermined flag can be flags that correspond to the upper and / or left areas of the area corresponding to the predetermined flag. In addition, the flags used to select the network can be limited to flags at the same hierarchical level as the nodes associated with the predetermined flag.

[0022] According to one embodiment, the encoded message includes an indication of the highest hierarchical level and a sequence of flags associated with nodes that are not equal to the highest hierarchical level. Each flag indicates whether the associated node is an intermediate node or a child node. The flag sequence obtained by sequentially decoding the data stream in a depth-first or breadth-first traversal order skips the nodes of the highest hierarchical level and automatically points to the same leaf node, thereby reducing the coding rate.

[0023] According to yet another embodiment, the encoded message of the multi-level tree structure may include an indication of the highest-level hierarchical level. In this way, the existence of the flag may be limited to hierarchical levels other than the highest-level hierarchical level, because further distinctions of blocks with the highest-level hierarchical level are excluded in general.

[0024] In cases where spatial multi-tree subdivision is part of the leaf nodes of first-order multi-tree subdivision information and a second subdivision without distinguishing the root region, the network used to encode the second subdivision flags can be selected such that the network is the same for flags associated with regions of equal size.

[0025] According to an additional embodiment, the simple link regions subdivided into an information sample array are advantageously merged or grouped to reduce the amount of data encoding. To achieve this, for each simple link region, a predetermined relative positional relationship is defined to allow the identification of simple link regions within a plurality of simple link regions that have a predetermined relative positional relationship with the predetermined simple link region. In other words, if the number is zero, then there may be no merging index for the predetermined simple link region within the data stream. Furthermore, if the number of simple link regions with a predetermined relative positional relationship with the predetermined simple link region is 1, then the encoding parameters of the predetermined simple link region can be used or used to predict the encoding parameters of the predetermined simple link region without any other syntax elements. Otherwise, in other words, if the number of simple link regions with a predetermined relative positional relationship with the predetermined simple link region is greater than 1, then the introduction of other syntax elements can be prevented, even if the encoding parameters associated with these identified simple link regions are identical.

[0026] According to one embodiment, if the coding parameters of adjacent simple link regions are not equal to each other, a reference neighbor identifier can identify an appropriate subset of the number of simple link regions that have a predetermined relative positional relationship with the predetermined simple link region; and this appropriate subset is used when using the coding parameters or predicting the coding parameters of the predetermined simple link region.

[0027] According to other embodiments, a sample region of a two-dimensional information signal is spatially subdivided into multiple simply connected regions of different sizes by recursively multi-segmenting. This is performed based on a first subset of syntax elements contained in the data stream. Then, based on a second subset of syntax elements within the data stream, a set of unconnected simply connected regions that are not connected in space from the first subset is obtained, which are then combined to form multiple simply connected regions. This intermediate subdivision is used when reconstructing the sample array from the data stream. This allows for optimization in terms of the less critical aspects of subdivision, because overly fine subdivision can be compensated for by subsequent merging. Furthermore, the combination of subdivision and merging allows for intermediate subdivision that is impossible to achieve simply by recursively multi-segmenting. Therefore, by using the unconnected sets of syntax elements for subdivision and merging concatenation, an effective or superior intermediate subdivision can be adapted to the actual content of the two-dimensional information signal. Compared to its advantages, the amount of additional management data required to indicate merge details due to the additional subset of syntax elements is negligible. Simple Explanation of the Diagram

[0028] The preferred embodiments of the present invention will be described below with reference to the following figures, in which...

[0029] Figure 1 shows a block diagram of an encoder according to one embodiment of this invention;

[0030] Figure 2 shows a block diagram of a decoder according to one embodiment of this case;

[0031] Figures 3a-c schematically illustrate one specific embodiment of quadtree subdivision, wherein Figure 3a shows a first-level hierarchy, Figure 3b shows a second-level hierarchy, and Figure 3c shows a third-level hierarchy;

[0032] Figure 4 schematically shows a tree structure of an illustrative quadtree subdivision according to an embodiment of Figures 3a to 3c;

[0033] Figures 5a and 5b schematically illustrate the quadtree subdivisions of Figures 3a to 3c and the tree structure with indicators pointing to individual leaf blocks;

[0034] Figures 6a and 6b schematically illustrate binary strings or flag sequences representing the tree structure of Figure 4 and the quadtree subdivisions of Figures 3a to 3c, according to different embodiments;

[0035] Figure 7 shows a flowchart illustrating the steps performed by a data stream extractor according to an embodiment;

[0036] Figure 8 shows a flowchart illustrating the function of a data stream extractor according to yet another embodiment;

[0037] Figures 9a and 9b show schematic diagrams illustrating an illustrative quadtree subdivision according to one embodiment, highlighting neighboring candidate blocks of a predetermined block;

[0038] Figure 10 shows a flowchart of one function of a data stream extractor according to yet another embodiment;

[0039] Figure 11 schematically shows the composition of an image from a plane and a group of planes according to an embodiment and illustrates the use of cross-plane adaptation / prediction coding;

[0040] Figures 12a and 12b schematically illustrate an inheritance scheme based on a subtree structure and corresponding subdivisions of one embodiment.

[0041] Figures 12c and 12d schematically illustrate the use of adopted and predicted inheritance schemes based on a subtree structure of one embodiment;

[0042] Figure 13 shows a flowchart illustrating the steps performed by implementing an inheritance scheme using an encoder according to an embodiment;

[0043] Figures 14a and 14b illustrate a subdivision and a subordinate subdivision to demonstrate the possibility of implementing a cross-prediction inheritance scheme according to one embodiment;

[0044] Figure 15 shows a block diagram illustrating a decoding method associated with this inheritance scheme according to an embodiment;

[0045] Figure 16 shows a schematic diagram illustrating the scanning order of sub-regions in a multi-tree subdivision according to an embodiment, wherein these sub-regions are subject to internal prediction;

[0046] Figures 17a and 17b show a schematic diagram illustrating different subdivision possibilities based on other embodiments. Implementation

[0047] In the detailed explanation of the diagrams below, elements appearing in several diagrams are indicated by common element symbols to avoid repetitive descriptions of these elements. Conversely, the explanation of elements presented within one diagram also applies to other diagrams in which individual elements appear, provided that any discrepancies are pointed out in the explanations presented in those other diagrams.

[0048] Furthermore, the following description begins with the encoder and decoder embodiments explained with reference to Figures 1 to 11. The embodiments presented in these figures combine multiple aspects of this invention, but if each is also excellent for implementation within the encoding scheme, then, with reference to the following figures, the embodiments will briefly discuss the aforementioned individual aspects. These embodiments are summaries of the embodiments described in Figures 1 and 11 in different senses.

[0049] Figure 1 shows an encoder according to an embodiment of the present invention. The encoder 10 of Figure 1 includes a predictor 12, a residual pre-encoder 14, a residual reconstructor 16, a data stream inserter 18, and a block divider 20. The encoder 10 is used to encode a spatiotemporal sampling information signal into a data stream 22. The spatiotemporal sampling information signal may be, for example, video, i.e., a sequence of images. Each image represents an array of image samples. Other examples of spatiotemporal information signals include, for example, depth images captured by a time-of-light camera. It should also be noted that a spatial sampling information signal may contain more than one array for each frame or timestamp, such as in the case of color video, where the color video may contain, for example, a luminance sample array along with two chroma sample arrays for each frame. The time sampling rate may also be different for different components of the information signal, i.e., luminance and chroma. Similarly, this applies to spatial resolution. The video may also be accompanied by additional spatial sampling information, such as depth or transparency information. However, the focus of the following description will be on the processing of one of these arrays to first better understand the gist of the invention, and then on the processing of more than one plane. The encoder 10 of Figure 1 is configured to form an information stream 22 such that the syntax elements in the information stream 22 describe images at a granularity between the whole image and individual image samples. To achieve this, the segmenter 20 is configured to subdivide each image 24 into simple connected regions 26 of different sizes. Hereinafter, these regions will be simply referred to as blocks or sub-regions 26.

[0050] As will be detailed later, segmenter 20 uses multi-element subdivision to divide image 24 into blocks 26 of different sizes. For greater precision, quadtree subdivision is mostly used in the specific embodiments described below for Figures 1 through 11. As will be detailed later, segmenter 20 may internally include a cascade of subdivision units 28 to subdivide image 24 into the aforementioned blocks 26, followed by a merger 30 that allows these blocks 26 to be grouped to obtain an effective subdivision or granularity between the unsubdivided portion of image 24 and the subdivisions defined by subdivision units 28.

[0051] As illustrated by the dashed lines in Figure 1, the predictor 12, residual pre-encoder 14, residual reconstructor 16, and data stream inserter 18 operate on the image subdivision defined by the segmenter 20. For example, as detailed later, the predictor 12 uses the predictive subdivision defined by the segmenter 20 to determine whether an individual sub-region of the predictive subdivision should use the corresponding prediction parameters of that individual sub-region, based on the settings of the selected prediction mode, accepting intra-image prediction or cross-image prediction.

[0052] The residual preencoder 14 then uses image-similar individual sub-regions to encode the predicted residuals of image 24 provided by predictor 12. The residual reconstructor 16 reconstructs the residuals from the syntax elements output by the residual preencoder 14, and also operates on the aforementioned residual subdivisions. The data stream inserter 18 can explore the aforementioned subdivisions, i.e., the prediction subdivisions and residual subdivisions, to use, for example, entropy coding to determine the insertion order and proximity relationships between syntax elements for inserting the syntax elements output by the residual preencoder 14 and predictor 12 into the data stream 22.

[0053] As shown in Figure 1, encoder 10 includes an input terminal 32, where the original information signal enters encoder 10. A subtractor 34, a residual pre-encoder 14, and a data stream inserter 18 are connected in series in the aforementioned order between the input terminal 32 of data stream inserter 18 and the output terminal of encoded data stream 22. Subtractor 34 and residual pre-encoder 14 are part of a prediction loop, which is shut down by residual reconstructor 16, adder, and predictor 12, which are connected in series in the aforementioned order between the output terminal of residual pre-encoder 14 and the inverting input terminal of subtractor 34. The output terminal of predictor 12 is also connected to another input terminal of adder 36. In addition, predictor 12 includes an input terminal directly connected to input terminal 32 and may include another input terminal, which is also connected to the output terminal of adder 36 through selective loop filter 38. Furthermore, predictor 12 generates side information during operation, therefore the output of predictor 12 is also coupled to data stream inserter 18. Similarly, splitter 20 includes an output that is connected to another input of data stream inserter 18.

[0054] The structure of encoder 10 has been described, and further details of its operating modes are described below.

[0055] As mentioned above, segmenter 20 determines how to subdivide each image 24 into cells 26. Based on the subdivision of the image 24 to be used for prediction, predictor 12 determines how to predict individual cells corresponding to such subdivision. The predicted cell output of predictor 12 is fed to the inverting input of subtractor 34 and another input of adder 36, and prediction information reflecting how predictor 12 obtained this prediction from the previously encoded portion of the video is output to data stream inserter 18.

[0056] At the output of subtractor 34, the prediction residual is thus obtained, wherein the residual pre-encoder 14 processes this prediction residual according to the residual subdivision also specified by segmenter 20. As further explained in detail below with reference to Figures 3 to 10, the residual subdivision of image 24 used by residual pre-encoder 14 can be related to the prediction subdivision used by predictor 12, so that each prediction sub-region is used as a residual sub-region or further subdivided into smaller residual sub-regions. However, there may also be completely independent prediction subdivision and residual subdivision.

[0057] The residual precoder 14 transforms each residual sub-region from the spatial to the frequency domain using a two-dimensional transform, and then quantizes the resulting transform coefficients, which are specifically related to the resulting transformed blocks. Therefore, the distortion result comes from quantization noise. For example, the data stream inserter 18 can use entropy coding to losslessly encode the syntax elements describing the aforementioned transform coefficients into a data stream 22.

[0058] The residual reconstructor 16 then uses requantization followed by retransformation to reconvert the transform coefficients back into a residual signal. This residual signal is combined within adder 36 with the prediction obtained by subtractor 34 to obtain the prediction residual, thereby obtaining a reconstructed portion or sub-region of the current image at the output of adder 36. Predictor 12 can directly use this reconstructed image sub-region for intra-prediction; in other words, it can be used to predict a prediction sub-region by extrapolating from previously reconstructed prediction sub-regions in the vicinity. However, it is theoretically possible to perform intra-prediction directly in the frequency domain by predicting the spectrum of the current sub-region from the vicinity spectrum.

[0059] For interactive prediction, predictor 12 can use a previously encoded and reconstructed image version, which has been filtered by selective in-loop filter 38. Filter 38 may include, for example, a deblocking filter or an adaptive filter, with a transfer function suitable for effectively generating the aforementioned quantization noise.

[0060] Predictor 12 selects prediction parameters, indicating how a prediction sub-region is predicted by comparing it with the original samples within image 24. As detailed later, the prediction parameters may include indications of the prediction mode for each prediction sub-region, such as intra-image prediction and cross-image prediction. In the case of intra-image prediction, the prediction parameters also include an indication of the angle of the main extension of the inner edge of the prediction sub-region to be intra-image predicted; and in the case of cross-image prediction, the motion vector, the motion image exponent, and the final higher-order motion transform parameters; and in the case of both intra-image and / or cross-image prediction, selective filtering information used to predict the reconstructed image samples, based on which the current prediction sub-region can be predicted.

[0061] As will be explained in detail later, the subdivision defined by segmenter 20 substantially affects the highest rate-to-distortion ratio achievable by the residual pre-encoder 14, predictor 12, and data stream inserter 18. If the subdivision is too fine, the prediction parameter 40 output by predictor 12 to be inserted into data stream 22 requires an excessively high encoding rate, but the prediction obtained by predictor 12 may be better, and the residual signal to be encoded by residual pre-encoder 14 may be smaller, allowing for encoding with fewer bits. The opposite applies if the subdivision is too coarse. Furthermore, the aforementioned considerations also apply to residual subdivision in a similar manner: using finer-grained transformations of individual transform blocks results in reduced complexity in calculating the transform and increased spatial resolution of the resulting transform. In other words, fewer residual sub-regions allow for a more consistent spectral distribution of content within individual residual sub-regions. In other words, spectral resolution is reduced, and the ratio of significant coefficients to insignificant coefficients (i.e., quantized to zero) deteriorates. In other words, the transform granularity must be adjusted to suit the local image content. Furthermore, independent of the positive effects of finer granularity, finer granularity regularly increases the amount of side information required to indicate the subdivision selected for the decoder. As will be detailed later, the embodiments described thereafter provide encoder 10 with a highly efficient adjustment and adaptation of the subdivision to the information signal content to be encoded, which communicates the subdivision to be used at the decoding end by instructing data stream inserter 18 to insert subdivision information into data stream 22. Details are shown below.

[0062] However, before defining the subdivisions of the segmenter 20 in further detail, the decoder according to an embodiment of the present invention will be described in further detail with reference to Figure 2.

[0063] The decoder in Figure 2, indicated by reference symbol 100, includes an extractor 102, an image segmenter 104, a residual reconstructor 106, an adder 108, a predictor 110, a selective in-loop filter 112, and a selective post-filter 114. The extractor 102 receives the encoded data stream at input 116 of the decoder 100 and extracts subdivision information 118, prediction parameters 120, and residual data 122 from the encoded data stream. The extractor 102 outputs this information to the image segmenter 104, the predictor 110, and the residual reconstructor 106, respectively. The residual reconstructor 106 has an output connected to the first input of the adder 108. Another input and its output of adder 108 are connected to a prediction loop in which the selective in-loop filter 112 and predictor 110 are connected in series in the aforementioned order via a bypass path from the output of adder 108 to predictor 110, directly similar to the connection between adder 36 and predictor 12 described earlier in Figure 1. In other words, one is used for intra-image prediction and the other for cross-image prediction. The output of adder 108 or the output of selective in-loop filter 112 can be connected to the output of decoder 100, where the reconstructed information signal is output, for example, to a copying device. The selective post-filter 114 can be connected to the path leading to output 124 to improve the visual quality of the reconstructed signal at output 124.

[0064] In summary, the residual reconstructor 106, adder 108, and predictor 110 function similarly to components 16, 36, and 12 in Figure 1. In other words, they simulate the operation of the components in Figure 1. To achieve this, the residual reconstructor 106 and predictor 110 use prediction parameters 120 and subdivision control provided by the subdivision information 118 obtained from the extractor 102 to predict the predicted sub-region in the same manner as the predictor 12 performs or decides to perform, and re-transform the received transform coefficients at the same granularity as the residual precoder 14. The image segmenter 104 also relies on the subdivision information 118 to synchronously reconstruct the subdivision selected by the segmenter 20. The extractor can use the subdivision information to control data extraction, such as context selection, proximity determination, probability estimation, and parsing of data stream syntax.

[0065] Several deviations can be made to the foregoing embodiments. Some deviations will be described in detail later in the section on subdivision performed by subdivision unit 28 and merging performed by merger 30, while others will be explained in Figures 12 through 16. All such deviations can be applied individually or in subsets to the detailed descriptions of Figures 1 and 2 above, provided there are no obstacles. For example, subdivision units 20 and 104 do not determine predictive subdivision but only residual subdivision for each image. Instead, they may determine filtered subdivision for selective loop filters 38 and 112. Other predictive subdivisions or other residual encoded subdivisions are independent or dependent. Furthermore, subdivision determination by these elements may not be based on a frame-by-frame basis. Instead, subdivision performed on a frame can be reused or applied to a number of frames, with new subdivisions simply transferred subsequently.

[0066] In providing further details regarding image segmentation into sub-regions, the following description first focuses on the segmentation portions presumably handled by subdividers 28 and 104a. Then, the merging processes performed by mergers 30 and 104b are described. Finally, cross-plane adaptation / prediction is described.

[0067] The method by which subdividers 28 and 104a segment an image allows it to be further divided into multiple blocks of potentially different sizes for predictive and residual coding of image or video data. As mentioned earlier, image 24 can be used as one or more arrays of image sample values. In the case of the YUV / YCbCr color space, for example, the first array can represent the luminance channel, while another two arrays can represent the chroma channel. These arrays can have different dimensions. All arrays can be grouped into one or more planar groups, each planar group consisting of one or more contiguous blocks, such that each plane is contained within a single and unique planar group. The following applies to each planar group. The first array of a particular planar group can be called the primary array of that planar group. Possible subsequent arrays are subordinate arrays. The block segmentation of the primary array can be performed based on a quadtree approach, as detailed later. The block segmentation of the subordinate array can be derived from the segmentation of the primary array.

[0068] According to the embodiments described later, subdivision units 28 and 104a are combined to divide a primary array into multiple square blocks of equal size, referred to hereafter as tree blocks. When using a quadtree, the side length of the tree block is typically a multiple of 2, such as 16, 32, or 64. However, for completeness, it should be noted that other types of trees, as well as binary trees or trees with any number of leaves, are possible. Furthermore, the number of children of a tree can depend on the tree's level and the signal that the tree represents.

[0069] In addition, as explained above, sample arrays can also represent information other than video sequences, such as depth maps or light spots. For simplicity, the following explanation focuses on quadtrees as a representative example of multi-level trees. A quadtree is a tree in which each internal node has exactly four children. Each tree block forms a linear quadtree together with the subordinate quadtrees of each leaf in that linear quadtree. The linear quadtree determines the subdivision of a given tree block for prediction, while the subordinate quadtrees determine the subdivision of a given prediction tree block for residual coding.

[0070] The root node of a quadtree corresponds to a complete tree block. For example, Figure 3a shows a tree block 150. The images must be divided into columns and rows of this tree block 150 using a regular grid, thus seamlessly covering the sample array. However, it should be noted that for all block subdivisions shown below, seamless subdivision without overlap is not of critical importance. Instead, adjacent blocks can overlap each other, as long as no leaf block is an appropriate sub-part of a neighboring leaf block.

[0071] Along with the quadtree structure of tree block 150, each node can be further divided into four child nodes. In the case of a single quadtree, this means that tree block 150 can be split into four sub-blocks, each having half the width and half the height of tree block 150. In Figure 3a, these sub-blocks are indicated by element symbols 152a to 152d. In the same way, each of these sub-blocks is further divided into four smaller sub-blocks, each having half the width and half the height of the original sub-block. In Figure 3b, sub-block 152c is shown as an example, and it is subdivided into four smaller sub-blocks 154a to 154d. So far, Figures 3a to 3c show how tree block 150 is first divided into four sub-blocks 152a to 152d, then the lower left sub-block 152c is further divided into four smaller sub-blocks 154a to 154d; and finally, as shown in Figure 3c, the upper right block 154b of these smaller sub-blocks is again divided into four blocks, each having one-eighth the width and one-eighth the length of the original tree block 150, and these even smaller sub-blocks are labeled 156a to 156d.

[0072] Figure 4 shows the potential tree structure based on quadtree partitioning instances as shown in Figures 3a to 3c. The numbers next to the tree nodes are the so-called subdivision flag values, which will be explained in more detail later when discussing quadtree structure communication. The root node of the quadtree is shown at the top of the figure (labeled as level "0"). The four branches of this root node at level 1 correspond to the four sub-blocks shown in Figure 3a. Therefore, the third of these sub-blocks is further subdivided into its four sub-blocks in Figure 3b, and the third node at level 1 in Figure 4 also has four branches. Again, corresponding to the subdivision of the second (top right) child node in Figure 3c, there are four sub-branches connected to the second node at level 2 of the quadtree hierarchy. The nodes at level 3 are not further subdivided.

[0073] Each leaf of a quadtree corresponds to a variable-size block (i.e., internal or span, prediction mode, movement parameters, etc.) that can be specified for individual prediction parameters. Hereafter, these blocks are referred to as prediction blocks. Specifically, these leaf blocks are those shown in Figure 3c. Briefly referring back to Figures 1 and 2, the segmenter 20 or subdivision 28 determines the quadtree subdivision as explained above. Subdivisions 152a-d perform decisions on further subdivision or further division of tree blocks 150, sub-blocks 152a-d, small sub-blocks 154a-d, etc., aiming to obtain the optimal trade-off between overly fine and overly coarse prediction subdivisions as indicated above. Predictor 12 then uses the specified prediction subdivisions to determine the aforementioned prediction parameters based on the granularity of the prediction subdivisions or for each prediction sub-region represented, for example, the block shown in Figure 3c.

[0074] The prediction block shown in Figure 3c can be further subdivided into smaller blocks for residual coding. For each prediction block, i.e., for each leaf node of a linear quadtree, the corresponding subdivision is determined by one or more subordinate quadtrees used for residual coding. For example, when a maximum residual block size of 16x16 is allowed, a given 32x32 prediction block will be divided into four 16x16 blocks, each determined by a subordinate quadtree used for residual coding. In this example, each 16x16 block corresponds to the root node of the subordinate quadtree.

[0075] As described in the case of a given tree block being subdivided into prediction blocks, each prediction block can be divided into multiple residual blocks using a subordinate quadtree decomposition. Each leaf of a subordinate quadtree corresponds to a residual block, and individual residual coding parameters (i.e., transform mode, transform coefficients, etc.) can be specified for that residual block by the residual precoder 14. These residual coding parameters, in turn, control the residual reconstructors 16 and 106.

[0076] In other words, the subdivision unit 28 can be configured to determine a predictive subdivision for each image or for each group of images. The predictive subdivision method can first divide the image into a regular arrangement of tree blocks 150, and then recursively subdivide a subset of these tree blocks using quadtree subdivision to obtain predictive subdivisions into predictive blocks. If individual tree blocks are not distinguished, the predictive block can be a tree block, or further subdividing a subset of these predictive blocks results in a leaf block of the quadtree subdivision; similarly... If a prediction block is larger than the maximum size of the subordinate residual subdivision, the residual block is obtained by first dividing the individual prediction blocks into subtree blocks according to a rule, and then subdividing a subset of these subtree blocks according to the quadtree subdivision procedure. If the individual prediction blocks are not divided into subtree blocks, the residual block can be a prediction block, or if the individual subtree blocks are not divided into smaller subtree blocks, the residual block can be a subtree block or a leaf block of the residual quadtree subdivision.

[0077] As summarized above, the subdivisions selected for the primary array can be mapped to the subordinate arrays. This is quite straightforward when considering subordinate arrays of the same size as the primary array, but special measures must be taken when the dimensions of the subordinate arrays differ from those of the primary array. In general, mapping the subdivisions of the primary array to the subordinate arrays for different sizes can be done through spatial mapping, that is, by mapping the boundary space of the subdivided blocks of the primary array to the subordinate arrays. Specifically, for each subordinate array, there can be scaling factors in the horizontal and vertical directions, which determine the dimensionality ratio of the primary array to the subordinate array. The subdivision of the subordinate array into sub-blocks for prediction and residual coding can be determined by the primary quadtree, the common location tree of the primary array, and the respective subordinate quadtrees of the sub-blocks. The resulting tree blocks of the subordinate array are scaled by a relative scaling factor. When the scaling factors in the horizontal and vertical directions are different (e.g., in 4:2:2 chroma subsampling), the resulting prediction and residual blocks of the subordinate array will no longer be square. In this case, it can be predetermined or adaptively selected (for the entire sequence, for a single image in the sequence, or for individual prediction blocks or residual blocks) whether a non-square residual block should be split into a square block. For example, in the first case, the encoder and decoder will agree to split the mapped block into a square block each time it is not square. In the second case, the subdivision unit 28 will communicate the selection to the subdivision unit 104a via the data stream inserter 18 and the data stream 22. For example, in the case of 4:2:2 chroma subsampling, the slave array has half the width but equal height of the primary array, and the height of the residual block is twice its width. By splitting this block vertically, two square blocks can be obtained again.

[0078] As previously mentioned, subdivision 28 or segmenter 20 respectively transmits quadtree-based subdivisions to subdivision 104a via data stream 22. To achieve this, subdivision 28 informs data stream inserter 18 about the subdivisions selected for image 24. Data stream inserter also transmits the structure of primary and secondary quadtrees, thus transmitting image array segmentation into variable-size blocks for prediction or residual blocks within data stream or bit stream 22 to the decoder.

[0079] The minimum and maximum permissible block sizes are transmitted as side information and can vary depending on the image. Alternatively, the minimum or maximum permissible block size can be fixed in the encoder and decoder. These minimum and maximum block sizes can differ for prediction blocks and residual blocks. For communication using a quadtree structure, the quadtree must be traversed, and for each node, it must be specified whether this particular node is a leaf node of the quadtree (i.e., the corresponding block is not further subdivided), or whether this particular node branches into its four child nodes (i.e., the corresponding block is divided into four sub-blocks of half size).

[0080] Communication within an image is performed block by block in a raster scan order, such as from left to right and from top to bottom, as shown in Figure 5a at 140. This scan order can also be different, for example, in a checkerboard pattern from bottom right to top left. In a preferred embodiment, each tree block and thus each quadtree is traversed in a depth-first manner to communicate the subdivision information.

[0081] In a preferred embodiment, not only the subdivided information, i.e., the tree structure, but also the predicted data, i.e., the payload associated with the leaf nodes of the tree, are transmitted / processed in depth-first order. This is done because depth-first traversal has advantages over width-first order. In Figure 5b, the quadtree structure is represented by leaf nodes labeled a, b, ..., j. Figure 5a shows the resulting block segmentation. If the blocks / leaf nodes are traversed in width-first order, the following order is obtained: abjchidefg. However, in depth-first order, this order is abc...ij. As can be seen from Figure 5a, in depth-first order, the left and top neighbor blocks are often transmitted / processed before the current block. Thus, movement vector prediction and context models often use the parameters specified for the left and top neighbor blocks to achieve improved coding efficiency. This is not the case for width-first order because block j is, for example, transmitted before blocks e, g, and i.

[0082] As a result, the communication for each tree block is recursively performed along the quadtree structure of a linear quadtree, such that a flag is transmitted for each node indicating whether the corresponding block has been split into four sub-blocks. If this flag has a value of "1" (for "true"), this communication procedure is recursively repeated for all four child nodes, that is, the sub-blocks are scanned in raster order (top left, top right, bottom left, bottom right) until the leaf node of the linear quadtree is reached. Note that the characteristic of the leaf node is that it has a subdivision flag value of "0". For nodes residing at the lowest level of the linear quadtree and thus corresponding to the minimum allowable predicted block size, no subdivision flag needs to be transmitted. For the example of diagrams 3a-c, as shown in 190 of diagram 6a, first a value of "1" is transmitted indicating that tree block 150 has been split into its four sub-blocks 152a-d. Then, the subdivision information of all four sub-blocks 152a-d is recursively encoded in raster scan order 200. For the first two sub-blocks 152a and 152b, "0" will be transmitted, indicating that they are not subdivided (see Figure 202 in Figure 6a). For the third sub-block 152c (bottom left), "1" will be transmitted, indicating that this block is subdivided (see Figure 204 in Figure 6a). Now, according to the recursive method, the four sub-blocks 154a-d of this block will be processed. Here, "0" will be transmitted for the first sub-block (206) and "1" will be transmitted for the second (top right) sub-block (208). Now, the four blocks 156a-d of the smallest sub-block size in Figure 3c will be processed. If the minimum allowable block size of this example has been reached, no further data needs to be transmitted because further subdivision is not possible. Otherwise, "0000" indicating that these blocks will not be further subdivided will be transmitted, as indicated at 210 in Figure 6a. Subsequently, "00" will be transmitted to the two lower blocks of Figure 3b (refer to Figure 6a, 212), and finally, "0" will be transmitted to the lower right block of Figure 3a (refer to 214). Therefore, the complete binary string representing the quadtree structure will be as shown in Figure 6a.

[0083] In Figure 6a, the different background shadows of this binary string representation correspond to different levels in the hierarchical relationship based on quadtree subdivision. Shadow 216 represents level 0 (corresponding to a block size equal to the original tree block size), shadow 218 represents level 1 (corresponding to a block size equal to half the original tree block size), shadow 220 represents level 2 (corresponding to a block size equal to one-quarter of the original tree block size), and shadow 222 represents level 3 (corresponding to a block size equal to one-eighth of the original tree block size). All subdivision flags of the same hierarchical level (corresponding to the same block size and the same color in the binary string representation) can be entropy encoded using the same probability model, for example, through inserter 18.

[0084] Note that for the breadth-first traversal case, the subdivision information will be transmitted in a different order, as shown in Figure 6b.

[0085] Similar to the subdivision of tree blocks used for prediction, the resulting prediction blocks, divided into residual blocks, must be transmitted in the bitstream. Furthermore, for residual coding, which is transmitted as side information and may change depending on the image, there can be maximum and minimum block sizes. Alternatively, the maximum and minimum block sizes for residual coding can be fixed in the encoder and decoder. At each leaf node of a quadtree, as shown in Figure 3c, the corresponding prediction block can be divided into residual blocks of the maximum permissible size. These blocks are the root nodes of the subordinate quadtree structure used for residual coding. For example, if the maximum residual block size of the image is 64x64 and the prediction block size is 32x32, then the entire prediction block will correspond to a subordinate (residual) quadtree root node of size 32x32. On the other hand, if the maximum residual block size for the image is 16x16, then the 32x32 prediction block will consist of four residual quadtree root nodes, each with a size of 16x16. Within each prediction block, the communication of the subordinate quadtree structure is performed sequentially from the root node (left to right, top to bottom) using a raster scan order. Similar to the case of a first-order (predictive) quadtree structure, for each node, a flag is encoded indicating whether this particular node has split into four child nodes. Then, if this flag has a value of "1", the raster scan order (top left, top right, bottom left, bottom right) is recursively repeated for all four corresponding child nodes and their corresponding sub-blocks until the leaf node of the subordinate quadtree is reached. As with the case of a first-order quadtree, no communication is required for nodes at the lowest level of the subordinate quadtree because these nodes cannot be further divided relative to the smallest possible residual block size.

[0086] For entropy coding, residual block subdivision flags belonging to residual blocks of the same block size can be encoded using one and the same probability model.

[0087] Thus, based on the examples presented in Figures 3a to 6a above, subdivision 28 defines subdivisions of different sizes for a primary subdivision used for prediction and for a primary subdivision used for residual coding. Data stream inserter 18 encodes a primary subdivision by transmitting information in a zigzag scan order for each tree block, the bit order being established according to Figure 6a, along with the maximum primary block size and maximum hierarchical level for encoding the primary subdivision. For each prediction block thus defined, the associated prediction parameters are already included in the bit stream. Furthermore, similar information, namely the encoding of the maximum size, maximum hierarchical level, and bit order according to Figure 6a, can be applied to each prediction block whose size is equal to or smaller than the maximum size of the residual subdivision; and to each residual root block, where the prediction block has been pre-divided into sizes exceeding the maximum size defined for the residual block. For each residual block thus defined, residual data is inserted into the data stream.

[0088] Extractor 102 extracts individual bits from the data stream at input 116 and notifies image segmenter 104 of the resulting segmentation information. Furthermore, data stream inserter 18 and extractor 102 can use the aforementioned sequence to transmit additional syntax elements, such as residual data output by residual precoder 14 and prediction parameters output by predictor 12, between prediction and residual blocks. The advantage of using this sequence is that by examining the syntax elements already encoded / decoded in neighboring blocks, an appropriate context for encoding individual syntax elements for a given block can be selected. Similarly, residual precoder 14 and predictor 12, as well as residual reconstructor 106 and predictor 110, can process individual prediction and residual blocks in the sequence described above.

[0089] Figure 7 shows a flowchart of the steps that can be performed by extractor 102 to extract subdivision information from data stream 22 in a manner previously summarized during encoding. In the first step, extractor 102 divides image 24 into root blocks 150. This step is indicated as step 300 in Figure 7. Step 300 involves extractor 102 extracting the maximum prediction block size from data stream 22. Additionally, step 300 may involve extractor 102 extracting the maximum hierarchical level from data stream 22.

[0090] Next, in step 302, extractor 102 decodes a flag or a bit from the data stream. In the first step 302, extractor 102 knows that an individual flag is the first flag in the bit order of the first root block 150 in the root block scan sequence 140. Therefore, a flag is a flag with a hierarchical level 0. In step 302, extractor 102 can use the context model associated with this hierarchical level 0 to determine a context. Each context has an individual probability estimate for the entropy encoding of its associated flag. The probability estimate of a context can be individually adapted to the individual context symbol statistics. For example, in order to determine the appropriate context for decoding the flag of hierarchical level 0 in step 302, extractor 102 may select a context from a set of contexts that is associated with hierarchical level 0, depends on the hierarchical level 0 flag of the neighboring tree block, and even more so on the information contained in the bit string of the quadtree subdivision that defines the neighboring tree blocks such as the top and left neighboring tree blocks of the currently processed tree block.

[0091] In the next step, namely step 304, extractor 102 checks whether the current flag indicates a distinction. If so, extractor 102 distinguishes the current block as a tree block, or in step 306 indicates this distinction to subdivider 104a, which in step 308 checks whether the current hierarchical level is equal to the maximum hierarchical level minus 1. For example, extractor 102 may also have the maximum hierarchical level extracted from the data stream in step 300. If the current hierarchical level is not equal to the maximum hierarchical level minus 1, then in step 310, extractor 102 increments the current hierarchical level by 1 and returns to step 302 to decode the next flag from the data stream. In this case, the flag to be decoded in step 302 belongs to another hierarchical level, therefore, according to one embodiment, extractor 102 may select one of different sets of contexts that belong to the current hierarchical level. This selection can also be based on the already decoded neighbor tree blocks according to the subdivided bit sequence of Figure 6a.

[0092] If a flag is decoded and the check for leakage in step 304 does not indicate the current block distinction, then extractor 102 proceeds to step 312 to check if the current hierarchical level is 0. If this is the case, extractor 102 processes the next root block in scan sequence 140 in step 314, or if no root block to be processed is left, then processing to extract subdivision information stops.

[0093] It should be noted that the description in Figure 7 focuses on decoding only the segmentation indicator flags for predicted segmentation. Therefore, step 314 actually involves decoding other bins or syntax elements associated with, for example, the current tree block. In this case, if there is another or a next tree root block, the extractor 102 proceeds from step 314 to step 302 to decode the next flag from the segmentation information, that is, the first flag in the flag sequence for the new tree block.

[0094] In step 312, if the hierarchical level is not equal to 0, the operation proceeds to step 316 to check if there are any other child nodes related to the current node. In other words, when extractor 102 performs the check in step 316, it has already checked hierarchical levels other than the current hierarchical level of 0 in step 312. This then indicates the existence of a parent node, which belongs to one of the root block 150, smaller blocks 152a-d, or even smaller blocks 152a-d. The tree structure node to which the current decoding flag belongs has a parent node that is shared by the other three nodes in the current tree structure. The scanning order among these child nodes with a shared parent node is illustrated in Figure 3a, with element symbol 200 for hierarchical level 0. Thus, in step 316, extractor 102 checks whether all four child nodes have been visited in the processing procedure of Figure 7. If this is not the case, i.e., the current parent node has additional child nodes, then the processing of Figure 7 proceeds to step 318, where the next child node within the current hierarchical level is visited according to the zigzag scan order 200. Therefore, its corresponding sub-block now represents the current block of Figure 7, and subsequently, in step 302, a flag is decoded from the data stream of the relevant block or the current node. However, at step 316, if there are no additional child nodes for the current parent node, then the method of Figure 7 proceeds to step 320, where the current hierarchical level is decremented by 1, and the method is then performed in step 312.

[0095] By performing the steps shown in Figure 7, decimator 102 and subdivision 104a cooperate to retrieve the selected subdivision from the data stream at the encoder end. The method in Figure 7 focuses on the aforementioned predicted subdivision. Combining the flowchart of Figure 7, Figure 8 shows how decimator 102 and subdivision 104a cooperate to retrieve the residual subdivision from the data stream.

[0096] Specifically, Figure 8 illustrates the steps performed by extractor 102 and subdivider 104a for each predicted block derived from the predicted subdivision. As previously mentioned, these predicted blocks are traversed between the tree blocks 150 of the predicted subdivision according to a zigzag scan order 140, and, for example, Figure 3c shows, the tree blocks are traversed using a depth-first traversal currently visited within each tree block 150. According to the depth-first traversal order, the leaf blocks of the subdivided tree block are visited in a depth-first traversal order, the sub-blocks sharing a certain hierarchical level with a common current node are visited in a zigzag scan order 200, and the subdivisions of these sub-blocks are primarily scanned before proceeding to the next sub-block in this zigzag scan order 200.

[0097] For the example in diagram 3c, the scan order between the leaf nodes of tree block 150 is shown by element symbol 350.

[0098] For the currently visited prediction block, the processing procedure in Figure 8 begins at step 400. In step 400, the internal parameter indicating the current size of the current block is set to be equal to the size of the hierarchical level 0 of the residual subdivision, i.e., the maximum block size of the residual subdivision. It should be recalled that the maximum residual block size can be less than the smallest block size of the prediction subdivision, or it can be equal to or greater than the latter. In other words, according to one embodiment, the encoder is free to choose either of the aforementioned possibilities.

[0099] In the next step, namely step 402, a check is performed to determine whether the predicted block size of the currently visited block is greater than the internal parameter indicated as the current size. If this is the case, the currently visited predicted block, which may be a leaf block or a tree block of the predicted subdivision without any further differentiation, is larger than the maximum residual block size. In this case, the processing in Figure 8 proceeds to step 300 in Figure 7. In other words, the currently visited predicted block is divided into residual tree root blocks, and the first order of the flag sequence of the first residual tree block within this currently visited predicted block is determined in step 302, etc.

[0100] However, if the currently visited predicted block has an internal parameter that is equal to or smaller than the current size, the process in Figure 8 proceeds to step 404, where the predicted block size is checked to determine if it is equal to the internal parameter indicating the current size. If so, the segmentation step 300 can be skipped, and the process proceeds directly to step 302 in Figure 7.

[0101] However, if the predicted block size of the currently visited predicted block is smaller than the internal parameter indicating the current size, the processing in Figure 8 proceeds to step 406, where the hierarchical level is incremented by 1, and the current size is set to the size of the new hierarchical level, such as a 2-partition (in the example of quadtree subdivision, in the two-axis direction). Subsequently, the check in step 404 is performed again. The loop formed by steps 404 and 406 ensures that the hierarchical level consistently corresponds to the size of the block to be distinguished, independent of individual predicted blocks having a size less than or equal to / greater than the maximum residual block size. Thus, when decoding the flag in step 302, the context model performed depends simultaneously on both the hierarchical level indicated by the flag and the block size. The advantage of using different veins for flags of different hierarchical levels or block sizes is that the probability estimation is highly suitable for the actual probability distribution of flag values. On the other hand, it has a medium number of veins to be processed, thus reducing the amount of additional management data for vein management and increasing the vein adjustment to adapt to actual symbol statistics.

[0102] As previously described, there may be more than one sample array, which can be grouped into one or more planar groups. For example, the input signal to be encoded at input 32 may be an image of a video sequence or still image. Thus, the image is in the form of one or more sample arrays. In the encoding context of an image of a video sequence or still image, a sample array refers to three color planes, such as red, green, and blue, or to a luminance plane and a chroma plane, such as in the color representation of YUV or YCbCr. In addition, sample arrays representing α, i.e., transparency and / or depth information of 3-D video material, may also be presented. Multiple such sample arrays may be grouped together into so-called planar groups. For example, luminance (Y) may be a planar group with only one sample array, and chroma such as CbCr may be another planar group with two sample arrays; or in another instance, YUV may be a planar group with three matrices and the depth information of 3-D video material may be different planar groups with only one sample array. For each plane group, a first-order quadtree structure can be encoded within the data stream 22 to represent segmentation into prediction blocks; and for each prediction block, a second-order quadtree structure represents segmentation into residual blocks. Thus, according to the first example above, the luminance component forms one plane group, and the chroma component forms another plane group. One quadtree structure is for the prediction blocks of the luminance plane, one quadtree structure is for the residual blocks of the luminance plane, one quadtree structure is for the prediction blocks of the chroma plane, and one quadtree structure is for the residual blocks of the chroma plane. However, in the second example above, there may be one quadtree structure for the prediction blocks (YUV) of luminance and chroma together, one quadtree structure for the residual blocks (YUV) of luminance and chroma together, one quadtree structure for the prediction blocks of the depth information of the 3-D video material, and one quadtree structure for the residual blocks of the depth information of the 3-D video material.

[0103] Furthermore, as described above, the input signal was divided into multiple prediction blocks using a primary quadtree structure. Now, we describe how these prediction blocks are further subdivided into residual blocks using a subordinate quadtree structure. According to another embodiment, the subdivision does not end at the subordinate quadtree stage. In other words, the blocks obtained from the subdivision using a subordinate quadtree structure may be further subdivided using a ternary quadtree structure. This subdivision is then used for the purpose of using additional encoding tools, which may assist in the encoding of the residual signal.

[0104] The preceding description focuses on subdivision performed by subdivision unit 28 and subdivision unit 104a, respectively. As mentioned earlier, subdivision performed by subdivision unit 28 and subdivision unit 104a controls the processing granularity of the aforementioned encoder 10 and decoder 100 modules. However, according to the embodiment described later, subdivision unit 228 and subdivision unit 104a are subsequently followed by merger 30 and merger 104b, respectively. It should be noted, however, that merger 30 and merger 104b are optional and can be omitted.

[0105] However, as will be detailed later, the combiner provides the encoder with the opportunity to group or cluster some of the prediction blocks or residual blocks, allowing other modules or at least a portion of them to process these blocks together. For example, predictor 12 can sacrifice the deviation between the prediction parameters of some predicted blocks by using subdivision optimization of subdivision 28, and instead use prediction parameters shared by all such prediction blocks, provided that the transmission of the prediction block grouping along with the shared parameters of all blocks belonging to that group is more forward-looking in terms of rate / distortion ratio than the individual transmission of the prediction parameters of all such prediction blocks. Based on these shared prediction parameters, the process of predictors 12 and 110 retrieving predictions is still performed block by block. However, it is also possible that predictors 12 and 110 may even perform the prediction process on the entire group of prediction blocks at once.

[0106] As will be detailed later, it is also possible that the prediction block group not only uses the same or shared prediction parameters for a group of prediction blocks, but also, additionally, allows the encoder 10 to send a prediction parameter for this group along with the prediction residuals for the prediction blocks belonging to this group, thus reducing the amount of additional management data used to transmit prediction parameters for this group. In the latter case, the merging process only affects the data stream inserter 18 and not the decisions made by the residual pre-encoder 14 and the predictor 12. Further details will be provided later. However, for completeness, it should be noted that the aforementioned profile also applies to other subdivisions, such as the aforementioned residual subdivision or filtering subdivision.

[0107] First, the merging of sample sets such as the aforementioned prediction blocks and residual blocks is motivated in a more general sense, that is, it is not limited to the aforementioned multi-tree subdivision. However, the following description will focus on the merging of blocks obtained from multi-tree subdivision in the aforementioned embodiments.

[0108] In general, merging syntax elements associated with a specific set of samples for the purpose of transmitting associated coding parameters allows for a reduction in side information rate in image and video coding applications. For example, the array of samples to be encoded is typically divided into specific sets of samples or sample collections, which can represent rectangular or square blocks, or any other set of samples, including arbitrary shaped regions, triangles, or other shapes. In the foregoing embodiments, the simple linking region is the prediction block and residual block obtained from multi-tree subdivision. The subdivision of the sample array can be fixed by the syntax; or, as mentioned above, the subdivision can also be communicated at least partially within the bitstream. To keep the communication of subdivision information and side information rate low, the syntax typically allows only a limited number of choices to result in simple subdivisions, such as subdividing blocks into smaller blocks. The sample set is associated with specific coding parameters, which may specify prediction information or residual coding patterns, etc. Details on this topic are as described above. For each sample set, individual coding parameters, such as those specifying prediction coding and / or residual coding, can be transmitted. To improve coding efficiency, the merging surface described below, which combines two or more sample sets into a so-called sample set group, allows for several advantages, detailed later. For example, sample sets can be merged so that all sample sets in this group share the same coding parameters, which can be transmitted along with one of the sample sets within the group. In this way, the coding parameters do not need to be transmitted individually for each sample set in the sample set group; instead, the coding parameters are transmitted only once for the entire sample set group. As a result, the side information used to transmit the coding parameters is reduced, and the overall coding efficiency is improved. Alternatively, additional refinement of one or more coding parameters can be transmitted for one or more sample sets within a sample set group. The refinement can be applied to all sample sets in a group or only to the sample set for which it is transmitted.

[0109] The merging profile, further described below, also provides the encoder with greater freedom in forming the bitstream 22 because the merging method significantly increases the number of possibilities for selecting and distinguishing an array of image samples. Because the encoder can choose from a wider range of options, such as reducing specific rate / distortion measurements, coding efficiency can be improved. Several methods exist for operating the encoder. In a simple approach, the encoder can first determine the optimal subdivision of the sample array. Briefly referring to Figure 1, the subdivision unit 28 will determine the optimal subdivision in the first stage. Subsequently, for each sample set, it is checked whether it should be merged with another sample set or another group of sample sets to reduce the specific rate / distortion cost measurement. In this respect, the prediction parameters associated with a merged sample set group can be re-estimated, such as by performing a new move search and estimation; or prediction parameters already determined for a shared sample set and candidate sample sets or a group of sample sets used for merging can be evaluated for the considered sample set group. In a more comprehensive approach, the specific rate / distortion cost measurement can be evaluated for additional candidate sample set groups.

[0110] It should be noted that the merging method described below does not change the processing order of the sample sets. In other words, the merging concept can be implemented in a way that prevents further delays, meaning that each sample set can still be decoded instantaneously at the same time without using the merging method.

[0111] For example, the bit rate saved by reducing the number of encoding prediction parameters is greater than the bit rate consumed in encoding merging information to indicate merging to the decoder. The merging method (described in detail later) leads to increased encoding efficiency. It should be further mentioned that the syntax extension used for merging provides the encoder with additional degrees of freedom to select image or planar groups to be divided into multiple blocks. In other words, the encoder is not limited to first performing subdivision and then checking whether some of the resulting blocks have the same or similar set of prediction parameters. A simple alternative is that, based on the rate-distortion cost metric, the encoder first determines the subdivision, and then for each block, the encoder can check whether merging with its neighboring blocks or associated already measured block groups reduces the rate-distortion cost metric. Thus, the prediction parameters associated with the new block group can be re-estimated, such as by performing a new move search; or these prediction parameters already measured for the current block and neighboring blocks or block groups can be evaluated for the new block group. Merging information provides block-based communication. Effectively, merging can also be interpreted as the result of inference based on the prediction parameters of the current block, where the inferred prediction parameters are set to be equal to the prediction parameters of one of the neighboring blocks. Additionally, residuals can be transferred between blocks within a block group.

[0112] Thus, the underlying basic concept behind the merging idea described later is to reduce the bit rate required to transmit prediction parameters or other coding parameters by merging neighboring blocks into a block group. Each block group is associated with a unique set of coding parameters, such as prediction parameters or residual coding parameters. In addition to subdivision information (if present), merging information is also transmitted within the bitstream. The advantage of the merging idea is that the reduction in side information from coding parameters leads to increased coding efficiency. It should be noted that the merging method described here can also be extended to dimensions other than spatial dimensions. For example, a sample or block set within several different video images can be merged into a block group. Merging can also be applied to 4-compression and bit-count coding.

[0113] Thus, briefly referring back to the explanations in Figures 1 through 8 above, note that the merging process after subdivision is superior, independent of the specific way the subdividers 28 and 104a subdivide the image. More specifically, the latter can also subdivide the image in a manner similar to H.264, that is, subdividing each image into a regular arrangement of rectangular or square clusters with predetermined sizes, such as 16x16 luminance samples or the transmission size within the data stream. Each cluster has several associated coding parameters, including, for example, regular subgrids defining each cluster as 1, 2, 4, or several other numbers of distinctions, used as prediction granularity or corresponding prediction parameters in the bit stream, and distinction parameters used to define the distinctions of residuals and corresponding residual transformation granularity.

[0114] In summary, merging offers the advantages briefly discussed above, such as reducing the side information rate bits in image and video coding applications. A specific set of samples, representing a rectangular or square block, an arbitrary shape, or any other sample set such as any simply linked region or sample, is typically linked to a specific set of coding parameters; for each sample set, the coding parameters are contained within the bitstream, and these parameters, for example, represent prediction parameters that specify how the corresponding sample set is predicted using the encoded samples. An image sample array can be divided into sample sets by syntax fixation or by corresponding subdivision information within the bitstream. The coding parameters for that sample set can be transmitted in a predetermined order, i.e., the order given by the syntax. Depending on the merging function, the merger 30 can transmit a group of sample sets for a shared sample set or a current block, such as a prediction block or residual block merged with one or more other sample sets. The coding parameters for a group of sample sets therefore only need to be transmitted once. In a particular embodiment, if the current sample set is merged with a sample set whose encoding parameters have already been transmitted or an existing group of sample sets, then the encoding parameters of the current sample set are not transmitted. Instead, the encoding parameters of the current sample set are set to be equal to the encoding parameters of the current sample set and the sample set or group of sample sets it merged with. Alternatively, additional refinement of one or more of the encoding parameters can be transmitted to the current sample set. The refinement can be applied to the entire sample set of a group or only to the sample set to which it is transmitted.

[0115] According to one embodiment, for each sample set such as the aforementioned prediction block, the aforementioned residual block, or the aforementioned leaf block of a multi-tree subdivision, the set of all previously encoded / decoded sample sets is referred to as the "set of causal sample sets." See, for example, Figure 3c. All blocks shown in this figure are some kind of subdivision result, such as prediction subdivision, residual subdivision, or any multi-tree subdivision, and the encoding / decoding order defined between these blocks is defined by arrow 350. Considering a block between these blocks as the current sample set or the current simple link region, its set of causal sample sets consists of all blocks preceding the current block along order 350. However, it should be noted that other subdivisions without multi-tree subdivision are also possible, provided that the discussion of the merging principle is considered later.

[0116] The set of samples that can be merged with the current set of samples is referred to below as the "set of candidate sample sets," which is often a subset of the "set of causal sample sets." How this subset is formed is known to the decoder, or may be specified from the encoder to the decoder within the data stream or bit stream. If a particular current set of samples is encoded / decoded, this set of candidate sample sets is not empty; it is communicated by the encoder within the data stream, or by the decoder deducing from the data stream whether the shared set is merged with one of the sample sets in this set of candidate sample sets, and if so, which of those sample sets it is merged with. Otherwise, merging cannot be used in this block because the set of candidate sample sets is often empty.

[0117] There are different ways to determine the subset of the set of candidate samples that will be represented in the set of causal sample sets. For example, the determination of the candidate sample set can be based on samples within the current sample set that have a unique geometric definition, such as the upper left image sample of a rectangular or square block. Starting with this unique geometric definition sample, a specific non-zero number of samples is determined, representing the direct spatial neighbors of this unique geometric definition sample. For example, this specific non-zero number of samples includes the upper neighbor and left neighbor samples of the unique geometric definition sample of the current sample set, so the non-zero number of neighbors is at most 2, or 1 if one of the upper or left neighbors is unavailable or located outside the image; or zero, in the case of missing two neighbors.

[0118] The set of candidate samples can be determined to encompass the set of samples containing at least one of the aforementioned non-zero numbers of neighboring samples. See Figure 9 for example. Currently, the sample set is considered as a merged object, which must be block X, and its uniquely defined geometric sample is exemplified as the top-left sample, indicated by 400. The top and left neighboring samples of sample 400 are indicated as 402 and 404, respectively. The set of causal sample geometries and the set of causal blocks are emphasized by overlay. Therefore, among these blocks, blocks A and B contain one of the neighboring samples 402 and 404, and these samples form the set of candidate block sets and the set of candidate sample sets.

[0119] According to another embodiment, the set of candidate samples determined for merging purposes may additionally or exclusively include a set of samples containing a specific non-zero number of samples, which may be 1 or 2, having the same spatial location but contained in different images, i.e., previously encoded / decoded images. For example, in addition to blocks A and B in Figure 9a, blocks from previously encoded images containing samples at the same location as sample 400 can be used. In this way, note that only the upper neighbor sample 404 or only the left neighbor sample 402 can be used to define the aforementioned non-zero number of neighbor samples. Generally, the set of candidate samples can be derived from previously processed data within the current image or other images. The derivation may include spatial orientation information, such as transform coefficients associated with a specific orientation and image gradient of the current image; or may include temporal orientation information, such as neighbor movement representations. Thus, the set of candidate samples can be derived from such data available to the receiver / decoder and other data and side information (if present) within the data stream.

[0120] It should be noted that the derivation of the candidate sample set is performed in parallel by the combiner 30 at the encoder end and the combiner 104b at the decoder end. As mentioned above, the two can determine the set of independent candidate sample sets based on a predefined method known to them; or the encoder can send hints within the bit stream, which brings the combiner 104b to a position to perform the derivation of these candidate sample sets in the same way as the combiner 30 at the encoder end, which determines the set of candidate sample sets.

[0121] As will be detailed later, the merger 30 and the data stream inserter 18 cooperate to transmit one or more syntax elements for each sample set, indicating whether the sample set is merged with another sample set, which may be part of a group of already merged sample sets, and which of the candidate sample sets is used for merging. The extractor 102 then extracts these syntax elements and notifies the merger 104b accordingly. Specifically, according to a particular embodiment described later, one or two syntax elements are transmitted for a specific sample set to indicate merging information. The first syntax element indicates whether the current sample set is merged with another sample set. Only if the first syntax element indicates that the current sample set is merged with another sample set, and only in this case, is a second syntax element transmitted indicating which of the candidate sample sets is used for merging. If the calculated set of candidate sample sets is empty, the transmission of the first syntax element can be stopped. In other words, only if the calculated set of candidate sample sets is not empty, and only in this case can the first syntax element be transmitted. The second syntax element can only be transmitted if the derived set of candidate samples contains more than one sample set. This is because if the set of candidate samples contains only one sample set, further selection is impossible. Furthermore, if the set of candidate samples contains more than one sample set, the transmission of the second syntax element can be prevented; however, this is not the case if all sample sets in the set of candidate samples are associated with the same coding parameter. In other words, the second syntax element can only be transmitted if at least two sample sets in a derived set of candidate samples are associated with different coding parameters.

[0122] Within this bitstream, merge information for a sample set can be encoded before prediction parameters or other specific encoding parameters associated with that sample set. Prediction parameters or encoding parameters are only transmitted when the merge information is being transmitted and the current sample set has not yet been merged with any other sample set.

[0123] Information about a sample set, such as the merged information of a block, can be encoded after an appropriate subset of prediction parameters; or, more generally, the encoded parameters associated with that individual sample set have already been transmitted. The subset of prediction / encoding parameters may consist of one or more reference image indices, or one or more motion parameter vectors or reference index components, and one or more motion parameter vector components, etc. The transmitted subset of prediction or encoding parameters can be used to derive a set of candidate sample sets from a temporary set of larger candidate sample sets, as described above. For example, the encoded prediction parameters of the current sample set and the difference between the encoded parameters and the corresponding prediction or encoding parameters of the previous candidate sample sets, or the distance based on a predetermined distance measurement, can be calculated. Then, only sample sets whose calculated difference measurement or distance is less than or equal to a predetermined threshold or a derived threshold are included in the final set, i.e., the reduced set of candidate sample sets. See Figure 9a for example. The current sample set must be block X. A subset of the encoding parameters for this block must have been inserted into bitstream 22. Suppose, for example, block X is a prediction block. In this case, a subset of appropriate encoding parameters can be a subset of the prediction parameters for block X, such as a subset of a set containing image reference indices and motion mapping information such as motion vectors. If block X is a residual block, then the subset of encoding parameters is a subset of residual information, such as transform coefficients or mapping tables indicating the location of significant transform coefficients within block X. Based on this information, both the data stream inserter 18 and the extractor 102 can use this information to determine a subset of blocks A and B, which in this particular embodiment constitutes the preliminary set of the aforementioned candidate sample set. Specifically, since blocks A and B belong to the set of causal sample sets, and the encoding parameters of block X are currently available to both the encoder and decoder during encoding / decoding, the aforementioned comparison can be used to eliminate any number of blocks in the preliminary sets of candidate sample sets A and B using different methods. The resulting reduced set of candidate samples can then be used as described above, in other words, to determine whether the merge indicator indicates that a merge should be transmitted from or extracted from the data stream, depending on the number of sample sets within the reduced set of candidate samples and whether the second syntax element must be transmitted therein; or whether it has already been extracted from the data stream, with the second syntax element indicating which sample set within the reduced set of candidate samples must be the merge companion block.

[0124] The aforementioned distance relative to the critical value used for comparison can be fixed and known to both the encoder and decoder, or it can be derived from the calculated distance, such as the median of different values ​​or some other median tendency. In this case, inevitably, the narrowed set of candidate samples must be an appropriate subset of the initial set of candidate samples. Furthermore, only the sample sets with the minimum distance measurement value are selected from the initial set of candidate samples. Additionally, using the aforementioned distance measurement value, exactly one sample set is selected from the initial set of candidate samples. In the latter case, the merging information only needs to specify which current sample set is to be merged with a single candidate sample set.

[0125] Thus, the candidate block set can be formed or derived as shown later in Figure 9a. Starting from the top-left sample position 400 of the current block X in Figure 9a, its left neighbor sample position 402 and its top neighbor sample position 404 are derived at the encoder and decoder ends. Thus, the candidate block set has at most two elements, namely, blocks B and A in the causal set of the shaded lines in Figure 9a that contain one of the two sample positions (in the case of Figure 9a). Thus, the candidate block set has only the two directly adjacent blocks of the top-left sample position of the current block as its elements. According to another embodiment, the candidate block set can be given by all blocks that have been encoded before the current block and contains one or more samples representing the directly spatially adjacent samples of any sample in the current block. Direct spatial proximity is limited to the directly left neighbor sample and / or directly top neighbor sample and / or directly right neighbor sample and / or directly bottom neighbor sample of any sample in the current block. For example, another block subdivision is shown in Figure 9b. In this case, the candidate blocks include four blocks, namely blocks A, B, C and D.

[0126] Additionally, the candidate block set may additionally or exclusively include blocks containing one or more samples located at the same position as any sample in the current block, but contained in different images, i.e., encoded / decoded images.

[0127] Furthermore, the candidate block set represents a subset of the aforementioned block set, determined by spatial or temporal proximity. The candidate block subset can be fixed, transmitted, or derived. The derivation of the candidate block subset can consider decisions made about other blocks in the same or other images. For example, blocks associated with the same or very similar coding parameters to other candidate blocks cannot be included in the candidate block set.

[0128] The following description of the embodiments applies to the case where only two blocks of the left and top neighboring samples of the top left sample containing the current block are considered as at most possible candidates.

[0129] If the candidate block set is not empty, a flag called merge_flag indicates whether the current block is to be merged with any candidate block. If merge_flag is 0 (for "No"), this block will not be merged with any of its candidate blocks, and all encoding parameters are typically transmitted. If merge_flag is 1 (for "Yes"), the following applies. If the candidate block set contains one and only one block, this candidate block is used for merging. Otherwise, the candidate block set contains exactly two blocks. If the prediction parameters of these two blocks are the same, these prediction parameters are used for the current block. Otherwise (the two blocks have different prediction parameters), a flag called merge_left_flag is transmitted. If merge_left_flag is 1 (for "Yes"), the block containing the left neighbor of the top-left sample position of the current block is selected from the candidate block set. If merge_left_flag is equal to 0 (for "No"), then another (i.e., top neighbor) block is selected from the candidate block set. The prediction parameters of the selected block are used for the current block.

[0130] Referring to the summaries of the foregoing embodiments, Figure 10 shows the steps performed by extractor 102 to extract merged information from the data stream 22 entering input terminal 116.

[0131] Processing begins at 450, identifying candidate blocks or sample sets for the current sample set or block. It must be recalled that the block encoding parameters are transmitted within data stream 22 in a certain one-dimensional order, and accordingly, Figure 10 indicates the method for retrieving merged information for the currently visited sample set or block.

[0132] As mentioned above, identification and step 450 include identification based on proximity features within the previously decoded blocks, i.e., the set of causal blocks. For example, these proximity blocks may point to candidates, which contain a neighboring sample that is spatially or temporally adjacent to a predetermined sample of the geometry of one or more current blocks X. Furthermore, the identification step may include two phases: a first phase involves identifying a preliminary set of candidate blocks based on proximity as described above; and a second phase whereby these blocks are simply pointing blocks that have been decoded from the data stream prior to step 450, and the encoded parameters transmitted by these blocks satisfy a certain relation to an appropriate subset of the encoded parameters of the current block X.

[0133] Next, the method proceeds to step 452, where it is determined whether the number of candidate blocks is greater than zero. If so, the merge_flag is extracted from the data stream in step 454. Extraction step 454 may involve entropy decoding. The context used for entropy decoding of the merge_flag in step 454 may be based on syntax elements belonging to, for example, a set of candidate blocks or a preliminary set of candidate blocks, where the dependency on syntax elements may be limited to information such as whether blocks belonging to the set of interest are eligible for merging. The probability estimate for selecting the context may be adjusted accordingly.

[0134] However, the number of candidate blocks is determined to be zero 452. The method in Figure 10 proceeds to step 456, where the current block encoding parameters are extracted from the bit stream, or in the case of the aforementioned two-stage identification alternative, where the extractor 102 processes the next block in a block scanning order such as the order shown in Figure 3c 350, and the remaining encoding parameters are processed.

[0135] Referring to step 454, the method proceeds to step 458, which checks whether the extracted merge_flag indicates the presence or absence of a current block merge. If no merge is performed, the method proceeds to step 456. Otherwise, the method proceeds to step 460, which includes checking whether the number of candidate blocks is equal to 1. If this is the case, the transmission of a candidate block indication is not necessary, so the method in Figure 10 proceeds to step 462, whereby the current block's merge companion is set as the only candidate block. The encoding parameters of the merge companion block after step 464 are used to adjust or predict the encoding parameters of the current block or the remaining encoding parameters of the current block. For example, in the case of adjustment, the missing encoding parameters of the current block are simply copied from the merge companion block. In another case, namely the case of prediction, step 464 may involve further extracting residual data from the data stream, residual data of the prediction residuals of the missing coding parameters of the current block, and a combination of such residual data and predictions of such missing coding parameters obtained from the merged companion block.

[0136] However, if the number of candidate blocks is determined to be greater than 1 in step 460, the method in Figure 10 proceeds to step 466. Here, the encoding parameters or the parts of interest in the encoding parameters are checked, that is, whether the related sub-parts of the parts that have not yet been transferred within the data stream of the current block are mutually recognized. If this is the case, these shared encoding parameters are set as merge references, or the candidate blocks are set as merge partners in step 468, or the individual encoding parameters of interest are used for adjustment or prediction in step 464. It should be noted that the merge partner itself can be a block that has already been merged. In this example, the encoding parameters obtained by adjustment or prediction of the merge parameters are used in step 464.

[0137] Otherwise, with different encoding parameters, the method in Figure 10 proceeds to step 470, where the additional syntax element is extracted from the data stream, i.e., this merge_left_flag. A separate set of contexts can be used for entropy decoding of this flag. The set of contexts used for entropy decoding of merge_left_flag can also contain a single context. After step 470, the candidate block indicated by merge_left_flag is set as a merge partner in step 472 and used for adaptation or prediction in step 464. After step 464, the extractor 102 processes the next block in block order.

[0138] Of course, there may be other alternatives. For example, the combined syntax element can be transmitted within the data stream, instead of the separate syntax elements merge_flag and merge_left_flag mentioned above, which are used to communicate the merge processing procedure. Furthermore, the aforementioned merge_left_flag can be transmitted within the data stream, independent of whether the two candidate blocks have the same prediction parameters, thereby reducing the amount of additional data management required for executing the 10th figure processing procedure.

[0139] As illustrated in Figure 9b above, more than two blocks can be included in the candidate block set. Furthermore, merge information conveys whether a block should be merged; if so, the candidate blocks to be merged can be communicated using one or more syntax elements. A syntax element can indicate whether the block should be merged with any of the aforementioned candidate blocks, such as the aforementioned `merge_flag`. If the candidate block set is not empty, it can indicate the transmission flag. A second syntax element can communicate which candidate block is to be merged, such as the aforementioned `merge_left_flag`, but typically indicates a choice between two or more candidate blocks. A second syntax element can only be transmitted if the first syntax element communicates that the current block should be merged with one of the candidate blocks. The second syntax element is further transmitted only when the candidate block set contains more than one candidate block, and / or when any of the candidate blocks has a different prediction parameter than any of the other candidate blocks. The syntax may depend on how many candidate blocks are given and / or how the different prediction parameters are associated with the candidate blocks.

[0140] The syntax for determining which candidate blocks to use can be set simultaneously and / or side-by-side at both the encoder and decoder. For example, if three candidate block selections are identified in step 450, the syntax is selected such that only three selections are available, for instance, for entropy encoding in step 470. In other words, syntax elements are selected such that their symbolic alphabet has the same number of elements as the existing candidate block selections. All other selection probabilities can be considered zero, and entropy encoding / decoding can be adjusted simultaneously at both the encoder and decoder.

[0141] Furthermore, as described above in step 464, the prediction parameters referred to as the result of the merging method can represent the complete set of prediction parameters associated with the current block, or can represent a subset of such prediction parameters, such as the prediction parameters of one hypothesis for a block that uses multiple hypotheses for prediction.

[0142] As mentioned above, the syntax elements for merging information can be entropy-encoded using a context model. The syntax elements can consist of the aforementioned `merge_flag` and `merge_left_flag` (or similar syntax elements). In a specific instance, one of the three context models or one of the context models can be used in step 454 to encode / decode `merge_flag`. The context model index `merge_flag_ctx` used can be derived as follows: if the candidate block set contains two elements, the value of `merge_flag_ctx` is equal to the sum of the `merge_flag` values ​​of the two candidate blocks. However, if the candidate block set contains one element, the value of `merge_flag_ctx` is equal to twice the `merge_flag` value of this candidate block. Since the `merge_flag` of neighboring candidate blocks can be 1 or 0, three context models are available for `merge_flag`. `merge_left_flag` can be encoded using only a single probability model.

[0143] However, according to alternative embodiments, different context models can be used. For example, non-binary syntax elements can be mapped to a sequence of binary symbols, known as bins. The context model defining certain syntax elements or syntax element bins of merged information can be derived based on the number of transmitted syntax elements or neighboring or candidate blocks or other measurements, while other syntax elements or syntax element bins can be encoded using a fixed context model.

[0144] Regarding the description of block merging above, it should be noted that the candidate block set can also be derived in the same manner as described in any of the foregoing embodiments, with the following modifications: candidate blocks are limited to blocks predicted or interpreted using motion compensation. Only such elements can be elements of the candidate block set. The communication and contextualization of merging information can be performed in the manner described above.

[0145] Turning to the combination of the aforementioned multi-tree subdivision embodiments and the now-described merging surface. If the image is segmented into unequal-sized square blocks using a quadtree-based subdivision structure, for example, merge_flag and merge_left_flag or other syntax elements indicating merging can be interleaved with the prediction parameters transmitted for each leaf node of the quadtree structure. Consider again, for example, Figure 9a. Figure 9a shows an example of an image subdivided into variable-sized prediction blocks based on a quadtree. The two blocks above the largest size are the so-called tree blocks, i.e., the prediction blocks of the largest possible size. The other blocks in the figure are subdivisions of their corresponding tree blocks. The current block is labeled "X". All shaded blocks are encoded and decoded before the current block, thus forming a causal block set. As described in the derivative description of one of the candidate block sets in the embodiments, only the direct (i.e., top or left) neighbor sample containing the top-left sample position of the current block can become a member of the candidate block set. Thus, the current block is to merge either block "A" or block "B". If merge_flag is equal to zero (for "No"), block "X" has not merged with either of the two blocks. If blocks "A" and "B" have the same prediction parameters, no distinction is needed because merging with either block will result in the same outcome. Therefore, in this case, merge_left_flag is not transmitted. Otherwise, if blocks "A" and "B" have different prediction parameters, merge_left_flag=1 (for "Yes") will merge blocks "X" and "B", while merge_left_flag=0 (for "No") will merge blocks "X" and "A". In another preferred embodiment, additional neighboring (already transmitted) blocks represent merge candidates.

[0146] Figure 9b shows another example. Here, block "X" and its left neighbor "B" are tree blocks, meaning they have the maximum allowed block size. The top neighbor block "A" is one-quarter the size of the tree block. These blocks, which are elements of the causal block set, are highlighted. Note that according to one of the preferred embodiments, block "X" can only be merged with either block "A" or "B", and not with any other top neighbor block. In another preferred embodiment, additional adjacent (already transmitted) blocks represent merge candidates.

[0147] Before proceeding with the description of how different sample arrays of an image are processed according to the embodiments of this case, it should be noted that the previously discussed multi-tree subdivision and the communication and merging configurations clearly offer the advantage of being independently operable. In other words, as explained above, the combination of multi-tree subdivision and merging has specific advantages, but these advantages also arise from alternative examples where the merging feature is implemented, for example, by subdivision using subdivision modifiers 30 and 104a, rather than a quadtree based on multi-tree subdivision, but rather corresponding to the subdivision of clustered blocks that are distinguished into smaller subdivisions by macro block rules. On the other hand, the combination of multi-tree subdivision with the transmission of the maximum tree block size within the bitstream, and the use of corresponding coding parameters for multi-tree subdivision with the transmission of blocks in depth-first traversal order, have advantages independent of whether or not the merging feature is used simultaneously. Generally, it is intuitive to understand the advantages of merging when the sample array coding syntax is expanded to allow not only the subdivision of one block but also the merging of two or more subdivisions, resulting in improved coding efficiency. As a result, a set of blocks is obtained that are encoded with the same prediction parameters. The prediction parameters for this set of blocks only need to be encoded once. Furthermore, regarding the merging of sample sets, it is necessary to understand that the sample set under consideration can be rectangular or square blocks. In this case, the merged sample set represents a set of rectangular and / or square blocks. Additionally, if the sample set under consideration is an image region of arbitrary shape, the merged sample set represents a set of image regions of arbitrary shape.

[0148] The focus of the following description is the processing of different sample arrays of an image when each image has more than one sample array. Several aspects summarized in this description have the advantage of being independent of the type of subdivision used, that is, independent of whether the subdivision is based on multi-tree subdivision and independent of whether merging is used. Before describing a specific embodiment of the processing of different sample arrays of an image, the main topic of this embodiment is a brief introduction to the processing of different sample arrays of each image.

[0149] The following discussion focuses on the use of coding parameters between blocks of different sample arrays of an image in image or video coding applications, particularly the adaptive prediction of coding parameters between different sample arrays of an image in encoders and decoders such as those in Figures 1 and 2, or other image or video coding environments. As mentioned earlier, a sample array refers to an array of samples associated with different color components, or an image associated with additional information such as transparency information or depth mapping. The sample array associated with the color components of an image is also called a color plane. The technique described later, also known as cross-plane adoption / prediction, can be used in block-based image and video encoders and decoders, thereby allowing for arbitrary processing order of sample array blocks of an image.

[0150] Image and video encoders are typically designed to encode color images (still images or video sequences). A color image contains multiple color planes, which represent sample arrays of different color components. Often, a color image is encoded using a set of sample arrays consisting of a luminance plane and two chroma planes, the latter indicating chromatic aberration components. In some applications, it is also common to use three color planes representing sample arrays of the three primary colors: red, green, and blue. Furthermore, to improve color representation, a color image may consist of more than three color planes. Additionally, an image can be associated with auxiliary sample arrays that carry additional information about the image. For example, such auxiliary sample arrays may be arrays indicating the transparency of related color samples (suitable for display purposes) or arrays indicating depth mapping (suitable for representing multiple viewpoints, such as in 3D displays).

[0151] In conventional image and video coding standards (such as H.264), color planes are typically encoded together, and specific coding parameters such as macro and sub-macro block prediction modes, reference indices, and shift vectors are used for all color components of a block. The luma plane can be considered a primary color plane, with specific coding parameters specified in the bitstream; while the chroma plane can be considered a secondary plane, with corresponding coding parameters deduced from the primary luma plane. Each luma block is associated with two chroma blocks representing the same area in the image. Depending on the chroma sampling format used, the chroma sample array can be smaller than the luma sample array for a block. For macro blocks consisting of one luma component and two chroma components, further subdivision into smaller blocks is used (if the macro block is subdivided). For blocks consisting of one luma sample block and two chroma sample blocks (which can be the macro block itself or a sub-block of the macro block), the same set of prediction parameters, such as reference indices, shift parameters, and occasional intra-prediction modes, are used. Within a specific contour of a conventional video coding standard (such as the 4:4:4 contour in H.264), different color planes of an image can be encoded independently. In this configuration, macro block segmentation, prediction mode, reference index, and shift parameters can be selected separately for the color components of a macro block. Conventional coding standards either encode all color planes together using the same set of specific coding parameters (such as subdivision information and prediction parameters), or encode each color plane completely independently.

[0152] If color planes are encoded together, a set of subdivision and prediction parameters must be used for all color components of a block. This ensures that side information remains small, but compared to independent encoding, it may lead to a decrease in encoding efficiency. This is because using different block decomposition and prediction parameters for different color components may reduce rate-distortion costs. For example, using different motion vectors or reference frames for chroma components can significantly reduce the residual signal energy of chroma components and increase their overall encoding efficiency. If color planes are encoded separately, encoding parameters such as block decomposition, reference index, and motion parameters can be selected separately for each color component to optimize encoding efficiency for each component, but redundancy between color components is not possible. The multi-transmission of specific encoding parameters does lead to an increase in side information (compared to combined encoding), and this increase in side information rate may negatively impact overall encoding efficiency. Furthermore, in state-of-the-art video coding standards (such as H.264), support for auxiliary sample arrays is limited to the auxiliary sample array using its own set of encoding parameters.

[0153] Thus, in all embodiments described so far, the image plane can be processed as described above. However, as discussed earlier, the overall coding efficiency of encoding multiple sample arrays (which may be related to different color planes and / or auxiliary sample arrays) may increase. In this case, it can be determined based on block references, for example, whether all sample arrays in a block should be encoded with the same coding parameters or whether different coding parameters should be used. The basic concept of cross-plane prediction described later allows for such adaptive decisions based on blocks, for example. Based on rate-distortion criteria, the encoder can choose whether to use the same coding parameters for all or part of the sample arrays in a specific block, or whether to use different coding parameters for different sample arrays. This selection can also be achieved by communicating whether specific coding parameters are inferred from the encoded common localization blocks of different sample arrays for a specific sample array block. Different sample arrays may be configured for one of the images in a group, which is also called a sample array group or a plane group. Another plane group may contain one or more sample arrays of an image. Then, within a plane group, sample array blocks share the same selected coding parameters, such as subdivision information, prediction mode, and residual coding mode; while other coding parameters, such as transform coefficient levels, are transmitted separately for each sample array within that plane group. A plane group is coded as a first-order plane group, meaning that coding parameters are not inferred or predicted from other plane groups. For each block of a second-order plane group, an adaptive choice is made as to whether to transmit a new set of selected coding parameters, or whether the selected coding parameters are inferred or predicted from a first-order plane set or another second-order plane set. The decision on whether the selected coding parameters for a particular block are inferred or predicted is included in the bitstream. Cross-plane prediction allows for a trade-off between side information rate and prediction quality, offering greater flexibility compared to state-of-the-art image coding composed of multiple sample arrays. The advantage is improved coding efficiency compared to conventional image coding composed of multiple sample arrays.

[0154] An in-plane, predictive, expandable image or video encoder, such as the image or video encoder of the foregoing embodiments, allows for adaptive selection of whether a selected set of encoding parameters is inferred or predicted from other commonly located sample array blocks already encoded within the same image, or whether the selected set of encoding parameters for that block is encoded separately without referencing other commonly located sample array blocks within the same image. The decision of whether the selected set of encoding parameters is inferred or predicted for one or more sample array blocks can be incorporated into the bitstream. Different sample arrays associated with an image do not need to have the same size.

[0155] As mentioned earlier, sample arrays associated with an image (which may represent color components and / or auxiliary sample arrays) can be arranged into two or more so-called planar groups, where each planar group consists of one or more sample arrays. The sample arrays contained within a particular planar group do not need to be of equal size. Note that this arrangement into planar groups includes cases where the sample arrays are encoded separately.

[0156] More specifically, according to one embodiment, for each block of a plane group, adaptive selection is made to determine whether the coded block indicates how a block is inferred or predicted from already coded common localization blocks of different plane groups for the same image, or whether such coded parameters are coded separately for the block. The coding parameters specifying how a block is predicted include one or more of the following coding parameters: the block prediction mode specifies which prediction method is used for the block (intra-prediction, cross-prediction using a single motion vector and a reference image, cross-prediction using two motion vectors and a reference image, cross-prediction using a higher-order cross-prediction, i.e., a non-translation motion model and a single reference image, or cross-prediction using multiple motion models and reference images); the intra-prediction mode specifies how the intra-prediction signal is generated; an identifier specifies how many prediction signal combinations are used to generate the final prediction signal for the block; the reference index specifies which reference image(s) are used for motion-compensated prediction; the motion parameters (such as displacement vectors or affine motion parameters) specify how the prediction signal is generated using the reference images; and an identifier specifies how the reference images are filtered to generate the motion-compensated prediction signal. Note that generally, a block may be associated with only a subset of the aforementioned coding parameters. For example, if the block prediction mode specifies that a block is intra-prediction, the coding parameters of a block may additionally include the intra-prediction mode, but the coding parameters for generating the cross-prediction signal, such as the reference index and the shift parameter, are not specified; or if the block prediction mode specifies cross-prediction, the associated coding parameters may additionally include the reference index and the shift parameter, but the intra-prediction mode is not specified.

[0157] One of two or more plane groups can be encoded or indicated as a primary plane group within the bitstream. For all blocks of this primary plane group, the encoded parameters indicating how the prediction signal is generated are transmitted without referencing other plane groups of the same image. The remaining plane groups are encoded as secondary plane groups. For each block of the secondary plane group, one or more syntax elements are transmitted, which communicate whether the encoded parameters for how the block is predicted are inferred or predicted from common location blocks of other plane groups, or whether a new set of such encoded parameters is transmitted for the block. One or more syntax elements may be referred to as a cross-plane prediction flag or cross-plane prediction parameter. If the syntax element does not infer or predict the corresponding encoded parameters, a new set of corresponding encoded parameters for the block is transmitted within the bitstream. If the syntax element indicates that the corresponding encoded parameters are inferred or predicted, then the common location block in the so-called reference plane group is determined. The designation of the block and the reference plane group can be combined in various ways. In one embodiment, a specific reference group is specified for each secondary plane group; this specification may be fixed or may be communicated in higher-order syntax structures such as parameter sets, access unit headers, image headers, or slice headers.

[0158] In the second embodiment, the designation of the reference plane group is encoded within the bit stream, and the selected encoding parameters are indicated by one or more syntax elements for encoding a block, indicating whether they are presumed or predicted or separately encoded.

[0159] To facilitate understanding of the associated cross-plane prediction and the previously described possibilities in the detailed embodiments described below, refer to Figure 11, which schematically shows an image composed of three sample arrays 502, 504, and 506. For ease of understanding, Figure 11 only shows a subset of sample arrays 502-506. The sample arrays are shown as if they were aligned with each other in space, such that sample arrays 502-506 overlap each other along direction 508, and the samples of sample arrays 502-506 protrude along direction 508, resulting in the correct spatial positioning of all samples in sample arrays 502-506. In other words, planes 502 and 506 are unfolded in the horizontal and vertical directions to adjust to each other's spatial resolution and alignment.

[0160] According to one embodiment, all sample arrays of an image belong to the same part of a spatial scene, wherein the resolution along the vertical and horizontal directions may differ between individual sample arrays 502-506. Furthermore, for illustrative purposes, sample arrays 502 and 504 are considered to belong to one planar group 510, while sample array 506 is considered to belong to another planar group 512. Also, Figure 11 shows an example where the spatial resolution along the horizontal axis of sample array 504 is twice the resolution in the horizontal direction of sample array 502. Moreover, sample array 504 is considered to form a primary array relative to sample array 502, and sample array 502 forms a subordinate array related to primary array 504. As explained above, in this case, as determined by the subdivision unit 30 in Figure 1, the sample array 504 is subdivided into multiple blocks by the subordinate array 502. In the example in Figure 11, since the vertical resolution of the sample array 502 is half of the vertical resolution of the primary array 504, each block has been divided into two horizontally parallel blocks. When measured in units of sample positions within the sample array 502, each block becomes a square block again due to being halved.

[0161] As illustrated in Figure 11, the subdivision system selected for sample array 506 differs from that of other sample groups 510. As previously mentioned, subdivision 30 can select the subdivision of pixel array 506 separately from or independently of the subdivision of plane group 510. Of course, the resolution of sample array 506 can differ from the resolution of planes 502 and 504 of plane group 510.

[0162] Now, when encoding individual sample arrays 502-506, encoder 10 begins encoding, for example, the primary array 504 of plane group 510 in the manner described above. The block shown in Figure 11 can be, for example, the aforementioned prediction block. Alternatively, the block can be a residual block or other blocks that define the granularity used to define certain coding parameters. Cross-plane prediction is not limited to quadtree or multi-tree subdivision, but quadtree or multi-tree subdivision is illustrated in Figure 11.

[0163] After the syntax elements of the primary array 504 are transmitted, the encoder 10 can decide to declare the primary array 504 as the reference plane of the subordinate plane 502. The encoder 10 and the extractor 102 can respectively transmit this decision through the bit stream 22. At the same time, the fact that the primary array of the plane group 510 is formed from the sample array 504 clearly shows the correlation, and this information can also be part of the bit stream 22. In summary, for each block within sample array 502, inserter 18 or any other module of encoder 10, together with inserter 18, can determine whether to prevent the transfer of encoding parameters for this block within the bit stream, and whether to communicate within the bit stream and instead use the encoding parameters of the common location block within primary array 504 for communication of this block within the bit stream; or determine whether the encoding parameters of the common location block within primary array 504 will be used as a prediction of the encoding parameters for the current block of sample array 502, and only transfer the residual data of the current block of sample array 502 within the bit stream. In the case of a negative decision, the encoding parameters are transferred within the data stream as usual. Decisions for each block are communicated in data stream 22. At the decoder end, the extractor 102 uses this cross-plane prediction information for each block to obtain the coding parameters of individual blocks of the sample array 502. In other words, if the cross-plane adoption / prediction information indicates cross-plane adoption / prediction, the coding parameters of the common location blocks of the primary array 504 are inferred, or the residual data of the block is extracted from the data stream and the residual data is combined with the prediction of the coding parameters of the common location blocks obtained from the primary array 504; or, as usual, the coding parameters of the current block of the sample array 502 are extracted independently of the primary array 504.

[0164] As mentioned earlier, the reference plane is not limited to the same bit plane where the block of interest currently resides in the cross-plane prediction. Therefore, as explained above, plane group 510 represents the reference plane group of a primary plane group or a secondary plane group 510. In this case, the bitstream may contain a syntax element that indicates for each block of the sample array 506 whether the adoption / prediction of encoding parameters for any co-location macro block of planes 502 and 504 of the aforementioned primary plane group or reference plane group 510 should be performed. In the latter case, the encoding parameters of the current block of the sample array 506 are transmitted as usual.

[0165] It should be noted that the subdivision / prediction parameters for multiple planes within a plane group can be the same. That is, since a plane group is only encoded once (all secondary planes of a plane group are derived from the primary planes within the same plane group to obtain subdivision information and / or prediction parameters), adaptive prediction or perturbation of subdivision information and / or prediction parameters is performed across multiple plane groups.

[0166] It should be noted that the reference plane group can be a primary plane group or a secondary plane group.

[0167] Co-location among different planar blocks within a group of planes is readily understood. The subdivision of the primary sample array 504 is adopted spatially by the subordinate sample array 502, except for the aforementioned block subdivision to transform the adopted leaf blocks into square blocks. In the case of cross-plane adoption / prediction between different groups of planes, co-location can be defined in a way that allows for greater freedom between the subdivisions of these groups of planes. Given a reference group of planes, the co-location blocks within that reference group of planes are determined. The derivation of the co-location blocks and the reference group of planes can be performed using a method similar to that described later. A specific sample 514 within the current block 516 is selected from one of the sample arrays 506 of the secondary group of planes 512. The same applies to the upper left sample of the current block 516, as shown in Figure 11 at 514 for illustrative purposes; or any other sample in the current block 516 that is close to the center of the current block 516 or within the current block, whose geometry is uniquely defined. The position of this selected sample 515 within sample arrays 502 and 504 of reference plane group 510 is calculated. The positions of sample 514 within sample arrays 502 and 504 are indicated at 518 and 520 respectively in Figure 11. Which planes 502 and 504 are actually used within reference plane group 510 can be predetermined or communicated within the bit stream. The sample at the closest position 518 and 520 within the corresponding sample arrays 502 or 504 of reference plane group 510 is determined, and the block containing this sample is selected as the common location block within the individual sample arrays 502 and 504. In the case of Figure 11, these are blocks 522 and 524 respectively. Alternative methods for determining common location blocks in other planes will be detailed later.

[0168] In one embodiment, the encoding parameters for the prediction of the current block 516 are fully estimated using the corresponding prediction parameters of common-location blocks 522 / 524 within different plane groups 510 of the same image 500 without transmitting additional side information. The estimation may include simply copying the corresponding encoding parameters, or adjusting the encoding parameters to take into account the differences between the current plane group 512 and the reference plane group 510. For example, such adjustment may include adding motion parameter correction (e.g., displacement vector correction) to account for the phase difference between the luminance and chroma sample arrays; or adjustment may include modifying the precision of the motion parameters (e.g., modifying the precision of the displacement vector) to account for the different resolutions of the luminance and chroma sample arrays. In an additional embodiment, one or more estimated encoding parameters used to generate the prediction signal are not directly used for the current block 516, but rather used as predictions of the corresponding encoding parameters for the current block 516, and the refinement of these encoding parameters for the current block 516 is transmitted in bitstream 22. For example, instead of directly using the assumed movement parameters, the movement parameter difference (such as the displacement vector difference) that specifies the deviation between movement parameters is used in the current block 516. The assumed movement parameters are encoded in the bit stream. At the decoder end, the actual movement parameters used are obtained by combining the assumed movement parameters and the transmitted movement parameter difference.

[0169] In another embodiment, the subdivision of a block, such as the aforementioned predicted subdivision into tree blocks of predicted blocks (i.e., sample blocks using the same set of prediction parameters), is adaptively inferred or predicted from common localization blocks already encoded according to different planar groups of the same image, i.e., bit sequences, as shown in Figures 6a or 6b. In one embodiment, one of two or more planar groups is encoded as a primary planar group. For all blocks of this primary planar group, subdivision parameters are transmitted without inferring other planar groups within the same image. The remaining planar groups are encoded as secondary planar groups. For blocks of the secondary planar groups, one or more syntax elements are transmitted, indicating whether subdivision information is inferred or predicted from common localization blocks of other planar groups, or whether subdivision information is transmitted in the bitstream. One or more syntax elements may be referred to as a cross-planar prediction flag or a cross-planar prediction parameter. If a syntax element indicates that subdivision information is not inferred or predicted, then the subdivision information for that block is transmitted in the bitstream without mentioning other planar groups of the same image. If the syntax elements indicate that the subdivision information is presumed or predicted, then a common location block within a so-called reference plane group is determined. The configuration of this reference plane group can be arranged in various ways. In one embodiment, a specific reference plane group is assigned to each secondary plane group; this assignment can be fixed or communicated in a higher-order syntax structure as a parameter set, access unit header, image header, or slice header. In a second embodiment, the reference plane assignment is encoded within the bitstream and communicated via one or more syntax elements that specify whether the subdivision information is presumed, predicted, or separately encoded for a block. The reference plane group can be a primary plane group or other secondary plane groups. Given a reference plane group, a common location block within that reference plane group is determined. A common location block is a reference plane group corresponding to the same image region of the current block, or represents a block within a reference plane group that shares the largest portion of the image region with the current block. The common location block can be subdivided into smaller prediction blocks.

[0170] In an additional embodiment, the subdivision information of the current block, such as the quadtree-based subdivision information according to Figures 6a or 6b, is estimated using the subdivision information of colocation blocks in different groups on the same plane, without transmitting additional side information. For a specific example, if a colocation block is divided into 2 or 4 prediction blocks, the current block is also divided into 2 or 4 sub-blocks for prediction purposes. As for another specific example, if a colocation block is divided into four sub-blocks, and one of these sub-blocks is further divided into four smaller sub-blocks, the current block is also divided into four sub-blocks, and one of these sub-blocks (corresponding to the further decomposition of the colocation block's sub-block) is also divided into four smaller sub-blocks. In yet another preferred embodiment, the estimated subdivision information is not directly used for the current block, but rather as a prediction of the actual subdivision information for the current block; the corresponding refinement information is transmitted in a bitstream. For example, the subdivision information inferred from the colocation block can be further refined. For each sub-block corresponding to a sub-block within the colocation block that has not been further subdivided into smaller blocks, the syntax element can be encoded in the bitstream, indicating whether the sub-block is further subdivided in the current plane group. The transmission of this syntax element can be conditional on the size of the sub-block. Alternatively, it can be transmitted in the bitstream that a sub-block further subdivided in the reference plane group has not been further subdivided into smaller blocks in the current plane group.

[0171] In another embodiment, a block is subdivided into prediction blocks and encoding parameters specifying how the sub-blocks are predicted. Both predictions are adaptively inferred or predicted from already encoded common localization blocks of different plane groups for the same image. In a preferred embodiment of the invention, one of the two or more plane groups is encoded as a primary plane group. For all blocks of such a primary plane group, the prediction parameters for subdivision information are transmitted without reference to other plane groups of the same image. The remaining plane groups are encoded as secondary plane groups. For blocks of secondary plane groups, one or more syntax elements are transmitted, indicating whether the subdivision information and prediction parameters are inferred or predicted from common localization blocks of other plane groups; or whether the subdivision information and prediction parameters are transmitted in the bitstream. One or more syntax elements may be referred to as a cross-plane prediction flag or cross-plane prediction parameter. If the syntax element conveys subdivision information and prediction parameters without inference or prediction, then the subdivision information of that block and the prediction parameters of the resulting sub-blocks are transmitted in the bitstream without reference to other plane groups of the same image. If the subdivision information and prediction parameters for the subblock are estimated or predicted, then the common location blocks in the so-called reference plane group are determined. The designation of the reference plane group for the block can be configured in various ways. In one embodiment, a specific reference plane group is assigned to various secondary plane groups; this assignment can be fixed or can be communicated in higher-order syntax structures such as parameter sets, access unit headers, image headers, or slice headers. In a second embodiment, the assignment of reference plane groups is encoded within the bitstream and indicates whether the subdivision information and prediction parameters are estimated, predicted, or separately encoded by communicating one or more syntax elements for a block. The reference plane group can be a primary plane group or other secondary plane groups. Given a reference plane group, the common location blocks within that reference plane group are determined. The common location block can be a block in the reference plane group corresponding to the same image region of the current block, or it can represent a block within the reference plane group that shares the largest portion of the image region with the current block. The colocality block can be divided into smaller prediction blocks. In a preferred embodiment, the subdivision information of the current block and the prediction parameters of the resulting sub-blocks are obtained using the subdivision information of the colocality block and the prediction parameters of the corresponding sub-blocks in different plane groups of the same image, without transmitting additional side information. In a specific instance, if the colocality block is divided into 2 or 4 prediction blocks, the current block is also divided into 2 or 4 sub-blocks for prediction purposes, and the prediction parameters for the sub-blocks of the current block are derived as described above.To give another specific example, if a colocation block is divided into four sub-blocks, and one of these sub-blocks is further divided into four smaller sub-blocks, then the current block is also divided into four sub-blocks, and one of these sub-blocks (corresponding to the sub-block in which the colocation block is further subdivided) is also divided into four smaller sub-blocks. The prediction parameters for all but not further subdivided sub-blocks are estimated as described above. In yet another preferred embodiment, the subdivision information is estimated entirely based on the subdivision information of the colocation block in the reference plane group, but the estimated prediction parameters for these sub-blocks are only used as predictions of the actual prediction parameters for the sub-blocks. The differentiation between the actual prediction parameters and the estimated prediction parameters is performed in bitstream encoding. In yet another embodiment, the estimated subdivision information is used as a prediction of the actual subdivision information for the current block, and the difference is transmitted in the bitstream (as described above), but the prediction parameters are entirely estimated. In another embodiment, both the estimated segmentation information and the estimated prediction parameters are used for prediction, while the difference between the actual segmentation information and the prediction parameters, and their estimated values, are transmitted in a bitstream.

[0172] In another embodiment, for a block within a plane group, an adaptive selection is made of whether the residual coding mode (such as transform type) is used to infer or predict the same image from already encoded common localized blocks in different plane groups, or whether the residual coding mode is used to encode the block separately. This embodiment is similar to the aforementioned embodiment of adaptive inference / prediction for prediction parameters.

[0173] In another embodiment, the subdivision of a block (e.g., a prediction block) into transform blocks (i.e., sample blocks to which a two-dimensional transformation is applied) is adaptively inferred or predicted from already encoded common localization blocks for different plane groups of the same image. This embodiment is similar to the aforementioned embodiment of adaptive inference / prediction of subdivision into prediction blocks.

[0174] In another embodiment, a block is subdivided into transform blocks, and the residual coding pattern (e.g., transform type) of the resulting transform blocks is inferred or predicted from already coded common location blocks for different plane groups of the same block. This embodiment is similar to the adaptive inference / prediction of subdivided predictive blocks and the prediction parameters for the resulting predictive blocks described above.

[0175] In another embodiment, a block is subdivided into prediction blocks, associated prediction parameters, subdivision information of the prediction blocks, and a residual coding pattern for the transformed block, which is adaptively inferred or predicted from common localization blocks already encoded for different plane groups of the same image. This embodiment represents a combination of the foregoing embodiments. It is also possible that only a portion of the coding parameters is inferred or predicted.

[0176] Thus, cross-plane adoption / prediction can improve the aforementioned coding efficiency. However, the coding efficiency gain obtained through cross-plane adoption / prediction can also be achieved in other block subdivisions used based on multi-tree subdivision, and is independent of whether block merging is implemented.

[0177] The aforementioned embodiments regarding cross-plane adaptation / prediction can be applied to image and video encoders and decoders, which segment an image's color plane and (if any) an auxiliary sample array associated with the image into blocks and combine these blocks with coding parameters. For each block, a set of coding parameters may be included in the bitstream. For example, these coding parameters may be parameters describing how a block is predicted and decoded at the decoder. In specific instances, coding parameters may represent macro blocks or block prediction modes, subdivision information, intra-prediction modes, reference exponents for motion-compensated prediction, motion parameters such as displacement vectors, residual coding modes, transform coefficients, etc. Different sample arrays associated with an image may have different sizes.

[0178] Next, we describe one scheme for enhancing communication of encoded parameters within the tree-based segmentation scheme, as described in Figures 1 to 8 above. The effects and advantages of other schemes, namely merging and cross-plane adoption / prediction, and the enhanced communication scheme (often referred to as inheritance in the following text), are described independently of the foregoing embodiments, but these schemes can be combined with any of the foregoing embodiments, either alone or in combination.

[0179] In general, an improved encoding scheme (called inheritance, as explained below) used to encode side information within a tree-based partitioning scheme allows for comparison with conventional encoding parameter processing to obtain the following advantages.

[0180] In conventional image and video coding, an image, or a specific set of sample arrays for an image, is typically decomposed into multiple blocks, which are associated with specific coding parameters. An image is typically composed of multiple sample arrays. Furthermore, an image may be associated with additional auxiliary sample arrays, which may, for example, contain transparency information or depth maps. An image's sample arrays (including auxiliary sample arrays) can be divided into one or more so-called planar groups, where each planar group is composed of one or more sample arrays. A planar group of an image can be coded independently, or if the image is associated with more than one planar group, a planar group of an image can be predicted from other planar groups of the same image. Each planar group is typically decomposed into multiple blocks. These blocks (or corresponding blocks of sample arrays) are predicted by cross-image prediction or intra-image prediction. Blocks can have different sizes and can be square or rectangular. The segmentation of an image into multiple blocks can be achieved by syntax fixing or (at least partially) within a bitstream. Frequently transmitted syntax elements convey the subdivision of blocks of predetermined sizes. These syntax elements specify whether and how a block is subdivided into smaller blocks, and are associated with coding parameters for purposes such as prediction. For all samples of a block (or blocks in a corresponding sample array), the decoding of the associated coding parameters is specified in a predetermined manner. In this example, all samples of a block are predicted using the same set of prediction parameters, such as a reference index (identifying a reference image in the already encoded image set), motion parameters (specifying the measurement of block movement between a reference image and the current image), parameters of the interpolation filter, and the intra-prediction mode. Motion parameters can be represented as displacement vectors with horizontal and vertical components, or as higher-order motion parameters such as affine motion parameters composed of six components. There may be more than one specific set of prediction parameters (such as reference indices and motion parameters) associated with a single block. In this case, for each set of specific prediction parameters, a single intermediate prediction signal is generated for that block (or corresponding block in the sample array), and the final prediction signal is constructed by combining overlapping intermediate prediction signals. The corresponding weighting parameters and possibly constant offsets (added to the weighting sum) can be fixed for an image, a reference image, or a set of reference images; or they can be included in the set of prediction parameters for the corresponding block. The difference between the original block (or the corresponding sample array block) and its predicted signal, also known as the residual signal, is usually transformed and quantized. A two-dimensional transform is often applied to the residual signal (or the corresponding sample array of the residual block). For transform coding, the block (or the corresponding sample array block) that has already used a specific set of prediction parameters can be further segmented before applying the transform. The transformed block can be equal to or smaller than the block used for prediction. It is also possible that a transformed block includes more than one of the blocks used for prediction. Different transformed blocks can have different sizes; transformed blocks can represent square blocks or rectangular blocks. After the transform, the resulting transform coefficients are quantized, and the so-called transform coefficient hierarchy is obtained.The transformation coefficient hierarchy, prediction parameters, and (if any) subdivision information are entropy-encoded.

[0181] In many image and video coding standards, the syntax for providing the possibility of subdividing an image (or a group of planes) into multiple blocks is extremely limited. It typically only indicates whether (and how) a block of a predetermined size can be subdivided into smaller blocks. For example, the largest block in H.264 is 16x16. A 16x16 block is also called a macro block, which is formed in the first step of image segmentation. For each 16x16 macro block, it can be signaled whether it is encoded as a 16x16 block, or as two 16x8 blocks, or two 8x16 blocks, or four 8x8 blocks. If a 16x16 block is subdivided into four 8x8 blocks, then each of these 8x8 blocks can be encoded as one 8x8 block, or two 8x4 blocks, or two 4x8 blocks, or four 4x4 blocks. State-of-the-art image and video coding standards specify a minimum set of possibilities for subdividing into multiple blocks. This has the advantage of maintaining a small side information rate for conveying subdivision information, but it has the disadvantage of significantly increasing the bit rate required to transmit prediction parameters for those blocks, as will be discussed later. The side information rate used to convey prediction parameters typically represents the significant total bit rate of a block. And when this side information is reduced, for example, by using a larger block size, coding efficiency can be improved. The actual image or picture of a video sequence is composed of objects of arbitrary shape with specific properties. For example, such objects or parts of objects are characterized by unique textures or unique movements. Typically, the same set of prediction parameters can be applied to such objects or parts of objects. However, the object boundaries usually do not coincide with the possible block boundaries of large prediction blocks (e.g., in a 16x16 macro block in H.264). The encoder typically determines the subdivision (within a finite set of possibilities) that results in the smallest specificity rate-distortion cost measurement. For objects of arbitrary shape, this can result in a large number of small blocks. Since these small blocks are associated with a set of prediction parameters to be transmitted, the side information rate becomes a significant part of the total bit rate. However, because several small blocks still represent areas of the same object or parts of an object, the prediction parameters of multiple resulting blocks are the same or very similar. Intuitively, coding efficiency can be improved when the syntax is extended in a way that not only allows for the subdivision of a block but also for sharing coding parameters among multiple subdivided blocks. In tree-based subdivision, the sharing of coding parameters for a given set of blocks can be achieved by assigning coding parameters or portions thereof to one or more parent nodes using a tree-based hierarchical relationship. As a result, shared parameters or portions thereof can be used to reduce the side information required to actually select coding parameters for subdivided blocks. This reduction can be achieved by deleting parameters from subsequent blocks, or by sharing parameters in the prediction modeling and / or context modeling of parameters for subsequent blocks.

[0182] The basic idea behind the inheritance scheme described below is to reduce the bit rate required to transmit encoded information by sharing information along the tree-based hierarchical relationships of these blocks. Shared information is transmitted within the bitstream (excluding subdivision information). The advantage of the inheritance scheme is that reducing the side information rate results in increased coding efficiency for the coding parameters.

[0183] To reduce side information rate, according to the embodiments described later, individual encoding parameters for a specific sample set, i.e., simple link regions, can represent rectangular or square blocks, arbitrary shapes, or any other sample sets of multi-tree subdivisions that are efficiently communicated within the data stream. The inheritance scheme described later allows encoding parameters not to be explicitly included in the bitstream for each sample set within the sample set. Encoding parameters can represent prediction parameters that specify that the corresponding sample set is predicted using encoded samples. Many possibilities and examples, as described above, also apply here. As previously stated and detailed later, with respect to the following schemes, an image sample array divided into multiple sample sets based on a tree structure can be fixed by syntax or communicated by corresponding subdivision information within the bitstream. As mentioned earlier, encoding parameters for sample sets can be transmitted in a predefined order, which is given by syntax.

[0184] According to the inheritance scheme, the decoder or its extractor 102 is configured to derive the encoding parameters of individual simple linked regions or sample sets in a specific manner. Specifically, the encoding parameters, or portions thereof such as parameters used for prediction purposes, are shared among blocks along the given tree-based segmentation scheme, and the shared groups along the tree structure are determined by the encoder or inserter 18, respectively. In one particular embodiment, the sharing of encoding parameters of all child nodes of a given internal node in the segmentation tree is indicated using a specific binary value sharing flag. Alternatively, the encoding parameters can be refined for each node so that, along the hierarchical relationship of the tree-based blocks, the cumulative refinement of the parameters can be applied to the entire sample set of that block at a given page node. In another embodiment, the portion of the encoding parameters for internal nodes transmitted along the hierarchical relationship of the tree-based blocks can be used for contextual adaptive entropy encoding and decoding of the encoding parameters or portions thereof for that block at a given leaf node.

[0185] Figures 12a and 12b illustrate the basic concept of inheritance using a special case of quadtree partitioning. However, as indicated several times above, other multi-level tree subdivision schemes can also be used. This tree structure is shown in Figure 12a, and the spatial partitioning corresponding to the tree structure in Figure 12a is shown in Figure 12b. The partitioning shown is similar to that shown in Figures 3a to 3c. In short, the inheritance scheme allows side information to be assigned to nodes at different non-leaf levels within the tree structure. Depending on the side information assigned to nodes at different levels of the tree, such as internal nodes or the root node of the tree in Figure 12a, different degrees of shared side information can be achieved in the hierarchical tree relationship of the blocks shown in Figure 12b. If it is decided that all leaf nodes in layer 4, in the case of diagram 12a, have the same parent node, and side information is virtually shared, this virtually means that the smallest blocks indicated by 156a to 156d in diagram 12b share this side information, and it is no longer necessary to transmit the side information completely for all such small blocks 156a to 156d, i.e., transmit it four times, but this remains an option for the encoder. It is also possible to decide that the entire area of ​​hierarchical layer 1 (layer 2) in diagram 12a, that is, the upper right quarter of tree block 150 including sub-blocks 154a, 154b, and 154d and the aforementioned even smaller sub-blocks 156a to 156d, is used as the area for sharing coding parameters. In this way, the area for sharing side information is increased. The next layer to be added is to sum all the sub-blocks of layer 1, that is, sub-blocks 152a, 152c, and 152d and the aforementioned even smaller blocks. In other words, in this case, the entire tree block has the side information assigned to it, and all child blocks of this tree block 150 share the side information.

[0186] In the following inheritance description, the following notes are used to describe these embodiments: a. Current reconstructed sample of leaf node: r b. Reconstructed samples of adjacent leaves: r' c. Current leaf node predictor: p d. Current residuals of leaf nodes: Res e. Current leaf node reconstruction residual: RecRes f. Scaling and inverse transformation: SIT g. Shared flag: f

[0187] The first instance of inheritance can be described as predictive communication within internal nodes. More precisely, it describes how communication is performed within the tree-block-based internal node partitioning using a predictive pattern for predictive purposes. Internal nodes (including the root node) can transmit some side information from the root node to the leaf nodes, which will be explored by their corresponding child nodes. More specifically, the shared flag f is emitted for internal nodes and has the following meanings: If f has a value of 1 ("true"), then all child nodes of that given internal node share the same internal prediction mode. In addition to sharing the flag f with a value of 1, the internal node also transmits internal prediction mode parameters for use by all child nodes. As a result, all subsequent child nodes do not carry any prediction mode information or any shared flag. To reconstruct all relevant leaf nodes, the decoder applies the internal prediction mode from the corresponding internal node. If f has a value of 0 ("pseudo"), then the child nodes of the corresponding internal node do not share the same internal prediction pattern, and each child node of the internal node carries a separate shared flag.

[0188] Figure 12c illustrates the aforementioned intra-prediction communication within internal nodes. In Layer 1, internal nodes transmit shared flags and side information given by the intra-prediction pattern information, while child nodes do not carry any side information.

[0189] Regarding the second inheritance instance, cross-prediction refinement can be described. More specifically, it describes how, based on the block segmentation of the tree, side information of the cross-prediction pattern is communicated for refinement purposes, such as by using movement parameters given by a movement vector. Internal nodes (including the root node) can transmit some side information from the root node to the leaf nodes, and this information will be explored by their corresponding child nodes. More specifically, the shared flag f is emitted towards internal nodes and has the following meanings: If f has a value of 1 ("true"), then all child nodes of the given internal node share the same motion vector reference. In addition to sharing the flag f with a value of 1, the internal node also transmits the motion vector and reference index. As a result, all subsequent child nodes do not carry an additional shared flag, but instead carry a refinement of this inherited motion vector reference. For the purpose of reconstructing all relevant leaf nodes, the decoder refines the motion vector at the given leaf node to the inherited motion vector reference value of 1 with the shared flag f belonging to its corresponding internal parent node. This means that the motion vector refinement at a given leaf node is the difference between the actual motion vector to be applied to this leaf node for motion compensation prediction and the motion vector reference value of its corresponding internal parent node. If f has a value of 0 ("pseudo"), then the child nodes of the corresponding internal node do not necessarily share the same cross-prediction pattern, and since these child nodes do not refine the movement parameters by using the movement parameters obtained from the corresponding internal node, each child node belonging to the internal node carries a separate shared flag.

[0190] Figure 12d illustrates the aforementioned refinement of the movement parameters. Internal nodes in Layer 1 transmit shared flags and side information. Child nodes belonging to leaf nodes only carry the refinement of movement parameters; for example, internal child nodes in Layer 2 do not carry side information.

[0191] Now refer to Figure 13. Figure 13 shows a flowchart illustrating the operation of a decoder, such as the one in Figure 2, for reconstructing an array of information samples representing spatial instance information signals from a data stream (which is subdivided into leaf regions of different sizes by multi-tree subdivision). As mentioned earlier, each leaf region has a hierarchical level selected from a series of hierarchical levels in the multi-tree subdivision. For example, all blocks shown in Figure 12b are leaf regions. Leaf region 156c is associated, for example, with hierarchical level 4 (or level 3). Each leaf region has encoding parameters associated with it. Examples of these encoding parameters have been described previously. For each leaf region, the encoding parameters are represented by a separate set of syntax elements. Each syntax element is an individual syntax element type selected from a set of syntax element types. Each syntax element type is, for example, a prediction mode, a movement vector component, an indication of the inner-prediction mode, etc. According to Figure 13, the decoder performs the following steps.

[0192] In step 550, the inheritance information is extracted from the data stream. In the case of Figure 2, extractor 102 is responsible for step 550. The inheritance information indicates whether inheritance is used for the current information sample array. The following description will show that the inheritance information can be of several kinds, such as sharing a flag f and transmitting information in a multi-tree structure divided into primary and secondary parts.

[0193] The information sample array already constitutes a sub-part of an image, such as a tree block, for example, tree block 150 in Figure 12b. Such inheritance information indicates whether inheritance is used for a specific tree block 150. This inheritance information can, for example, be used to insert data streams for all predicted subdivisions of the tree blocks.

[0194] Furthermore, if inheritance is indicated, the inheritance information indicates that at least one inherited region of the information sample array, consisting of a set of leaf regions and corresponding to one of the hierarchical levels in the hierarchical sequence of the multi-level tree subdivision, is lower than the respective hierarchical levels associated with the set of leaf regions. In other words, the inheritance information indicates whether inheritance is used for the current sample array, such as tree block 150. If yes, it indicates that leaf regions within at least one inherited region or sub-region of this tree block 150 share encoding parameters. Thus, the inherited region may not be a leaf region. In the example of Figure 12b, the inherited region may, for example, be a region formed by sub-blocks 156a to 156b. In addition, the inherited region may be larger, also covering sub-blocks 154a, b, and d, and even further, the inherited region may be tree block 150 itself, whose entire leaf region shares the encoding parameters associated with the inherited region.

[0195] However, it should be noted that more than one inherited region can be defined within a sample array or tree block 150. For example, suppose the lower left sub-block 152c is also divided into smaller blocks. In this case, sub-block 152c can form an inherited region.

[0196] In step 552, the inheritance information is checked to determine if inheritance is used. If yes, the processing in Figure 13 proceeds to step 554, where, for each cross-inheritance region, an inherited subset of at least one syntax element of a predetermined syntax element type is extracted from the data stream. Subsequently, in step 556, this inherited subset is copied into one of the corresponding syntax element inherited subsets within the syntax element set, or used as a prediction of the inherited subset, which represents the encoding parameters associated with the leaf region set of at least one inheritance region. In other words, for each inheritance region indicated within the inheritance information, the data stream contains an inherited subset of syntax elements. Furthermore, inheritance relates to at least one syntax element type or syntax element category that can be used for inheritance. For example, prediction mode or cross-prediction mode or intra-prediction mode syntax elements can be inherited. For instance, the inherited subset contained within the data stream for an inheritance region may include cross-prediction mode syntax elements. The inherited subset also contains additional syntax elements whose syntax element type depends on the value of the aforementioned fixed syntax element type associated with the inheritance scheme. For example, when the cross-prediction pattern is a fixed component of an inherited subset, the syntax elements for defining movement compensation, such as the movement vector component, can be included or excluded from the inherited subset by the syntax. For instance, assuming the upper right quarter of tree block 150, i.e., sub-block 152b, is an inherited region, the cross-prediction pattern alone can be used for this inherited region, or together with the movement vector and movement vector index, it can be used for the cross-prediction pattern.

[0197] All syntax elements contained in the inherited subset are copied into the leaf blocks within that inherited region, namely leaf blocks 154a, b, d and 156a to 156d, and their corresponding encoding parameters, or used for prediction. When prediction is used, residuals are transmitted for individual leaf blocks.

[0198] One possibility for transmitting inheritance information for tree block 150 is the transmission of the aforementioned shared flag f. In step 550, the extraction of inheritance information is described later in this example. More specifically, the decoder can be configured to use a hierarchical order from lower to higher hierarchical levels to extract and examine the shared flag f obtained from the data stream for any non-leaf region corresponding to any inheritance set of at least one hierarchical level of the multi-tree subdivision, regarding whether an individual inheritance flag or a shared flag indicates inheritance. For example, the inheritance set of hierarchical levels can be formed using hierarchical layers 1 to 3 of Figure 12a. Thus, any non-leaf node located in any subtree structure of layer 1 to layer 3 can have a shared flag associated with it within the data stream. The decoder extracts these shared flags in an order from layer 1 to layer 3, such as in a depth-first or breadth-first traversal order. Once one of the shared flags equals 1, the decoder knows that the leaf blocks contained in the corresponding inheritance region share that inheritance subset, and then proceeds to extraction in step 554. For the child nodes of the current node, there is no longer a need to check the inheritance flags. In other words, the inheritance flags of these child nodes are not transmitted within the data stream because it is clear that these node regions already belong to the inheritance subset of their syntax elements that is shared by that inheritance region.

[0199] The shared flag f can be obtained by interleaving the bits of the aforementioned quadtree subdivision. For example, the bit sequence of the intersection of the subdivision flag and the shared flag can be: 1 [0]001 [1]01(0000)000,

[0200] It is the same subdivision information shown in Figure 6a, with two scattered shared flags, which are highlighted by the underline to indicate the shared coding parameters of the sub-block in the lower left quarter of tree block 150 in Figure 3c.

[0201] Another way to define the inheritance information of an inheritance region is to use two subdivisions that are defined in a subordinate manner, as explained above in the descriptions of predictive and residual subdivisions. In short, a leaf block of a primary subdivision forms the inheritance region, which defines a shared subset of inherited syntactic elements within it; while a subordinate subdivision defines blocks within these inheritance regions for which inherited subsets of syntactic elements are copied or used for prediction.

[0202] For example, consider the residual tree as an extension of the prediction tree. Further, consider that prediction blocks can be further subdivided into smaller blocks for residual coding purposes. For each prediction block corresponding to a leaf node of the prediction-related quadtree, the corresponding subdivision for residual coding is determined by one or more subordinate quadtrees.

[0203] In this case, instead of using any prediction parameters in the internal nodes, the inventors considered interpreting the residual tree system in the following manner, where the residual tree also contains a refined representation of the prediction tree using a constant prediction pattern (communication via the corresponding leaf nodes of the prediction correlation tree) but with a refined reference sample. Examples of this will be given later to illustrate this situation.

[0204] For example, Figures 14a and 14b show quadtree segmentation for intra-prediction, where neighboring reference samples are emphasized for a specific leaf node in a single segmentation, and Figure 14b shows residual quadtree segmentation with refined reference samples for the same predicted leaf node. All sub-blocks shown in Figure 14b share the same intra-prediction parameters contained within the data stream for the individual leaf blocks emphasized in Figure 14a. Thus, Figure 14a shows a conventional example of quadtree segmentation for intra-prediction, showing a reference sample for a specific leaf node. However, in the inventors' preferred embodiment, a separate intra-prediction signal is calculated for each leaf node in the residual tree using neighboring samples of leaf nodes already reconstructed in the residual tree, such as the gray shading text in Figure 14b. A reconstructed signal for a given residual leaf node is then obtained in the conventional manner by adding the quantized residual encoded signal to this prediction signal. This reconstructed signal is then used as a reference signal for subsequent prediction procedures. Note that the decoding order used for prediction is the same as the residual decoding order.

[0205] As shown in Figure 15, in the decoding process, for each residual leaf node, the prediction signal p is calculated using the reference sample r' based on the actual internal-prediction mode (indicated by the leaf node indication of the prediction related quadrilateral).

[0206] After the SIT process, RecRes=SIT(Res) calculate The reconstructed signal r and its storage are used in the next prediction calculation program: r= [RecRes]+p For The decoding procedure for [prediction] is the same as the residual decoding sequence shown in Figure 16.

[0207] Each residual leaf node is decoded as described above. The reconstructed signal r is stored in a buffer, as shown in Figure 16. In this buffer, the reference sample r' will be used in the next prediction and decoding process.

[0208] Having described specific embodiments in combination with the aforementioned subsets of the facets summarized in Figures 1 to 16, additional embodiments of this invention will be described. The focus is on certain of the aforementioned facets, but the embodiments represent generalizations of the aforementioned embodiments. Specifically, the aforementioned embodiments of the architecture of Figures 1 and 2, which primarily combine multiple facets of this invention, can also be excellently employed in other applications or other coding domains. As frequently mentioned above, for example, multi-tree subdivisions can be used without merging and / or without cross-plane adoption / prediction and / or inheritance. For example, the transmission of maximum block size, the use of depth-first traversal order, the contextual adaptation of hierarchical levels based on individual subdivision flags, and the transmission of the maximum hierarchical level within the bitstream to save side information bitrate—all these facets are excellent but independent of each other. This is also true when considering inheritance schemes. The advantage of inheriting encoding parameters is that it is independent of the exact multi-tree subdivision used to subdivide an image into simply connected regions, and that it is independent of the presence of more than one sample array or the use of cross-plane application / prediction. Similarly, the advantages of inheritance also apply to cross-plane application / prediction.

[0209] Accordingly, by popularizing these embodiments, the encoding scheme using the inheritance scheme described above is not limited to hybrid encoding environments. In other words, reconstruction can be performed without prediction. The inherited encoding parameters can be related to other encoding parameters such as filter detail indicators. As mentioned above, the simple linked regions into which the information sample array is subdivided can be based on quadtree subdivision and can be square or rectangular in shape. Furthermore, the specific embodiment used to subdivide the sample array is only a specific embodiment, and other subdivisions can also be used. Several possibilities are shown in Figures 17a and 17b. For example, Figure 17a shows a sample array 606 subdivided into a regular two-dimensional arrangement of adjacent non-overlapping tree blocks 606, some of which are subdivided into sub-blocks 610 of different sizes according to a multi-tree structure. As mentioned above, although quadtree subdivision is illustrated in Figure 17a, each parent node may also be divided into any other number of child nodes. Figure 17b shows an embodiment in which the sample array 606 is subdivided into sub-blocks of different sizes by directly applying multi-tree subdivision to the entire pixel array 606. In other words, the entire pixel array 606 is treated as a block of the tree. The subdivisions in Figures 17a and 17b result in the sample array 606 being subdivided into simply connected regions, which are non-overlapping according to the embodiments in Figures 17a and 17b. However, several alternatives are possible. For example, blocks may overlap each other. However, the overlap may be limited to the following extent: each block partially does not overlap with any neighboring block, or each block sample overlaps at most with one of the neighboring blocks arranged side by side with the current block along a predetermined direction. The latter case will indicate that the left and right neighboring blocks may overlap the current block, thus completely covering the current block, but they do not overlap with each other; similarly, this applies to neighboring blocks in the vertical and diagonal directions.

[0210] As described above with reference to Figures 1 to 16, the information sample array does not necessarily represent a video image or a still image. The sample array can also represent a depth map or transparency map of certain objects.

[0211] The determination of encoding parameters and inherited information can be an iterative process. For example, if the initial preparation is performed, the optimization of the encoding parameters for adjacent simple linked regions in terms of rate / distortion belongs to the initial preparation. Similarly, in terms of rate / distortion, the optimization of the parent region should consider whether inheritance allows for the complete suppression of the explicit transmission of encoding parameters for all such simple linked regions, or the complete replacement of the transmission of such encoding parameters by only submitting residuals or by simply transmitting shared encoding parameters.

[0212] Although several facets have been described in relation to the apparatus, these facets clearly also represent descriptions of corresponding methods, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, the facets described in the method step context also represent descriptions of features of corresponding blocks or items or corresponding apparatuses. Some or all of the method steps may be executed by means of (or using) hardware devices, such as microprocessors, programmable computers, or electronic circuits. In several embodiments, one or more of the most important method steps may be executed by such a device.

[0213] The encoded / compressed signal of this invention can be stored on a digital storage medium or transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0214] Depending on certain specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using digital storage media, such as floppy disks, DVDs, Blu-ray discs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory, storing electronically readable control signals that cooperate (or can cooperate with) a programmable computer system to execute individual methods. Therefore, the digital storage media can be computer-readable.

[0215] According to several embodiments of the present invention, a data carrier having electronically readable control signals is included, which can cooperate with a programmable computer system to perform one of the methods described herein.

[0216] Generally, embodiments of the present invention can be implemented as a computer program product having code operable to execute one of the methods when the computer program product runs on a computer. The code may, for example, be stored on a machine-readable medium.

[0217] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0218] In other words, therefore, an embodiment of the method of the present invention is a computer program having code for executing one of the methods described herein when the computer program is run on a computer.

[0219] Therefore, another embodiment of the present invention is a data carrier (or digital storage medium or computer-readable medium) containing thereon a computer program for performing one of the methods described herein.

[0220] Therefore, another embodiment of the method of the present invention represents a data stream or signal sequence for a computer program used to perform one of the methods described herein. This data stream or signal sequence is, for example, configured to be transmitted via a data communication link, such as via the Internet.

[0221] Another embodiment includes, for example, a processing means, such as a computer or programmable logic device, configured or adapted to perform one of the methods described herein.

[0222] Another embodiment includes a computer on which a computer program is installed to perform one of the methods described herein.

[0223] In several embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In several embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally speaking, these methods are preferably performed by any hardware device.

[0224] From the above discussion, it will be understood that the present invention can be embodied in various embodiments, including but not limited to the following:

[0225] 1. A decoder for reconstructing an array of information samples representing a spatially sampled information signal from a data stream, the array of information samples being subdivided into leaf regions of different sizes by multi-tree subdivision, wherein each leaf region is associated with a hierarchical level in a serial hierarchical hierarchy of the multi-tree subdivision, each leaf region having associated coding parameters, wherein for each leaf region, the coding parameters are represented by a separate set of syntax elements, each syntax element belonging to a specific syntax element type in a set of syntax element types, and the decoder being assembled from: An inheritance information is extracted from the data stream, indicating whether inheritance is used. If inheritance is indicated, at least one inheritance region in the information sample array is composed of a set of leaf regions and corresponds to one of the hierarchical levels in the sequence hierarchy of the multi-tree subdivision. The inheritance region is lower than each of the hierarchical levels associated with the set of leaf regions. If the instruction intends to use inheritance Extract an inherited subset from the data stream of each inheritance region, including at least one syntax element of a predetermined syntax element type. The inherited subset is copied into, or the inherited subset is used to predict the inherited subset corresponding to one of the syntax elements that represents the encoding parameters associated with the leaf region set within the syntax element set, wherein at least one inherited region is composed of one of the individual leaf regions.

[0226] 2. The decoder as in Example 1, wherein the decoder is assembled from: For any region in the inheritance set corresponding to at least one hierarchical level of the multi-tree subdivision, such regions are composed of a corresponding set of leaf regions, and the information sample array is recursively segmented along a hierarchical order from a lower hierarchical level to a higher hierarchical level, an associated inheritance flag is extracted from the data stream, and the individual inheritance flag is checked to see if it indicates inheritance or not; For each district whose inheritance is specified in the relevant inheritance flag, Extract an inherited subset from the data stream, including at least one set of syntax elements of a predetermined syntax element type. The inherited subset is copied into the syntax element set that represents the corresponding inherited subset of one of the syntax elements associated with the encoded parameters of the leaf region, the associated inheritance flag indicating the inheritance of the region, or the inherited subset is used as a prediction of the corresponding inherited subset; and In examining these succession flags using a hierarchical order from lower to higher hierarchical levels, further extraction and examination of succession flags in the leaf area, which consists of associated succession flags indicating succession, is prevented.

[0227] 3. The decoder as in Example 2, wherein the decoder is a hybrid decoder.

[0228] 4. A decoder as described in any of Examples 1 to 3, wherein the decoder is configured to decode a residual for each leaf block.

[0229] 5. A decoder as described in any of Embodiments 1 to 4, wherein the decoder is configured to extract from the data stream the corresponding inherited subset of the set of syntax elements representing the encoded parameters associated with the leaf regions within the set of syntax elements, the leaf regions not belonging to any set of leaf regions composed of any inherited regions indicated by the inheritance information.

[0230] 6. A decoder as described in any of Examples 1 to 5, wherein when inheritance is not used, the decoder is assembled to extract the encoded parameters within the set of syntax elements that represent the entire leaf region associated with the information sample array.

[0231] 7. A decoder as described in any of Embodiments 1 to 6, wherein the decoder is configured to use a prediction pattern that is locally changed on a leaf-area basis to reconstruct the array information sample by prediction, wherein the decoder is configured to cause the predetermined syntax element type to affect the prediction pattern.

[0232] 8. A decoder as described in any of Embodiments 1 to 7, wherein at least one syntax element of the predetermined syntax element type is an internal syntax element of the prediction mode, and the decoder is assembled from: For each of these leaf regions, which consists of associated inheritance flags specifying inheritance, a residual signal is decoded in a residual decoding sequence, and... For each of these leaf regions, which is composed of associated inheritance flags that specify inheritance, the individual leaf region is reconstructed by adding the inner-prediction signal and the residual signal using the neighboring samples of the reconstructed signal of one of the reconstructed leaf regions as a reference signal, according to the inner-prediction mode indicated by the inner-prediction mode syntax element, and a separate inner-prediction signal is calculated.

[0233] 9. The decoder as in Example 8, wherein the decoder is configured such that the residual decoding order is a depth-first traversal order.

[0234] 10. A method for reconstructing an information sample array representing a spatially sampled information signal from a data stream, the information sample array being subdivided into leaf regions of different sizes by multi-tree subdivision, wherein each leaf region is associated with a hierarchical level in a serial hierarchical level of the multi-tree subdivision, each leaf region having associated encoding parameters, wherein for each leaf region, the encoding parameters are represented by a separate set of syntax elements, each syntax element belonging to a separate syntax element type in a set of syntax element types, and the method comprising: An inheritance information is extracted from the data stream, indicating whether inheritance is used. If inheritance is indicated, at least one inheritance region in the information sample array is composed of a set of leaf regions and corresponds to one of the hierarchical levels in the sequence hierarchy of the multi-tree subdivision. The inheritance region is lower than each of the hierarchical levels associated with the set of leaf regions. If the instruction intends to use inheritance Extract an inherited subset from the data stream of each inheritance region, including at least one syntax element of a predetermined syntax element type. The inherited subset is copied into, or the inherited subset is used to predict the inherited subset corresponding to one of the syntax elements that represents the encoding parameters associated with the leaf region set within the syntax element set, wherein at least one inherited region is composed of one of the individual leaf regions.

[0235] 11. An encoder for encoding an array of information samples representing a spatially sampled information signal into a data stream, the array of information samples being subdivided into leaf regions of different sizes by multi-tree subdivision, wherein each leaf region is associated with a hierarchical level in a serial hierarchical level of the multi-tree subdivision, each leaf region having associated encoding parameters, wherein for each leaf region, the encoding parameters are represented by a separate set of syntax elements, each syntax element belonging to a specific syntax element type in a set of syntax element types, and the encoder being configured to: Inheritance information is inserted into the data stream. This inheritance information indicates whether inheritance is used. If inheritance is indicated, at least one inheritance region in the information sample array is composed of a set of leaf regions and corresponds to one of the hierarchical levels in the sequence hierarchy of the multi-tree subdivision. This inheritance region is lower than each of the hierarchical levels associated with the set of leaf regions. If the instruction intends to use inheritance Insert a subset of the data stream from each inheritance region, including at least one syntactic element of a predetermined syntactic element type, into the data stream. The suppression encoding represents a corresponding inherited subset of a syntax element that represents the encoding parameters associated with the leaf region set within the set of syntax elements, wherein at least one inherited region is composed of an individual of such leaf regions; or the inherited subset inserted into the data stream is used as a prediction of a corresponding inherited subset of a syntax element that represents the encoding parameters associated with the leaf region set within the set of syntax elements, wherein at least one inherited region is composed of an individual of such leaf regions.

[0236] 12. A method for encoding an array of information samples representing a spatially sampled information signal into a data stream, the array of information samples being subdivided into leaf regions of different sizes by multi-tree subdivision, wherein each leaf region is associated with a hierarchical level in a serial hierarchical level of the multi-tree subdivision, each leaf region having an associated encoding parameter, wherein for each leaf region, the encoding parameter is represented by a separate set of syntax elements, each syntax element belonging to a separate syntax element type in a set of syntax element types, and the method comprising: Inheritance information is inserted into the data stream. This inheritance information indicates whether inheritance is used. If inheritance is indicated, at least one inheritance region in the information sample array is composed of a set of leaf regions and corresponds to one of the hierarchical levels in the sequence hierarchy of the multi-tree subdivision. This inheritance region is lower than each of the hierarchical levels associated with the set of leaf regions. If the instruction intends to use inheritance Insert a subset of the data stream from each inheritance region, including at least one syntactic element of a predetermined syntactic element type, into the data stream. The suppression encoding represents a corresponding inherited subset of a syntax element that represents the encoding parameters associated with the leaf region set within the set of syntax elements, wherein at least one inherited region is composed of an individual of such leaf regions; or the inherited subset inserted into the data stream is used as a prediction of a corresponding inherited subset of a syntax element that represents the encoding parameters associated with the leaf region set within the set of syntax elements, wherein at least one inherited region is composed of an individual of such leaf regions.

[0237] 13. A computer-readable digital storage medium having stored thereon a computer program having a code thereon, which, when run on a computer, is used to perform the method as described in any of Examples 10 and 12.

[0238] 14. A data stream in which an array of information samples representing a spatially sampled information signal has been reconstructed from a data stream, the array of information samples being subdivided into leaf regions of different sizes by multi-tree subdivision, wherein each leaf region is associated with a hierarchical level in a serial hierarchical level of the multi-tree subdivision, each leaf region having associated encoding parameters, wherein for each leaf region, the encoding parameters are represented by a separate set of syntax elements, each syntax element belonging to a separate syntax element type in a set of syntax element types, wherein the data stream comprises: An inheritance information indicator indicates whether inheritance is used, and if the indicator indicates that inheritance is to be used, it means that at least one inheritance region of the information sample array consisting of a set of leaf regions and for one of the hierarchical levels of the hierarchical sequence of the multi-tree subdivision is lower than each of the hierarchical levels associated with the set of leaf regions; If the instruction intends to use inheritance The data stream from each inheritance region, at least one syntax element of a predetermined syntax element type is included in one inheritance subset of the data stream, and Represents the coding parameters associated with the leaf region set, wherein at least one inherited region is composed of the individual region, and there is no inherited subset of a corresponding grammar element within the grammar element set; or, related to the inserted inherited subset, represents the coding parameters associated with the leaf region set, wherein at least one inherited region is composed of the individual region, and the residual of the inherited subset of a corresponding grammar element within the grammar element set is used as a prediction.

[0239] The foregoing embodiments are merely illustrative of the principles of the invention. It should be understood that modifications and variations to the details of the configurations described herein are readily apparent to those skilled in the art. Therefore, the intent is limited only to the scope of the appended claims and not to the specific details used to illustrate the embodiments herein.

[0240] 10: Encoder 12, 110: Predictor 14: Residual Pre-encoder 16, 106: Residual Reconstructor 18: Data Stream Inserter 20: Block divider, divider 22: Data Streaming 24, 500: Images 26: Block or sub-block 28, 104a: Subdivision 30, 104b: Merger 32, 116: Input terminals 34: Subtractor 36, 108: Adders 38: In-loop filter 40, 120: Prediction parameters 100: Decoder 102: Extractor 104: Image segmenter, segmenter 112: Selective In-Loop Filter 114: Selective Post-Filter 118: Encoded Data Streaming Sub-Information 122: Residual Data 124: Output terminal 140, 200: Raster Scanning Sequence 150: Tree Block 152a-d: Sub-blocks, Leaf regions 154a-d: Small sub-blocks, leaf regions 156a-d: Even smaller sub-blocks and leaf regions 190, 204, 208: Transmission 1 202, 206, 214: Transmission 0 210: Transmission 0000 212: Transmission 00 216: Level 0 218: Level 1 220: Level 2 222: Level 3 300-320, 400-406, 450-472, 550-556: Steps 350: Arrow 400: Top left sample 402: Left Neighbor Sample 404: Top Neighbor Sample 502, 504, 506: Sample arrays 508: Direction 510, 512: Plane Groups 514: Selected Sample 516: Current block 518, 520: Location 522 / 524: Commonly Locating Blocks 606: Sample Array 608: Non-overlapping tree block 610: Sub-block

Claims

1. A decoder for reconstructing an array of information samples encoded in a data stream and representing video information, the decoder comprising: an extractor configured to: extract multi-tree subdivision information from the data stream and inheritance information associated with one of the inherited encoded blocks of the information sample array, the inheritance information indicating whether inheritance is used, wherein, The inherited coding block corresponds to a first-level hierarchical level in a series of hierarchical levels that conform to the multi-tree subdivision information and is composed of a set of coded sub-blocks. Each coded sub-block in this set of coded sub-blocks corresponds to a second-level hierarchical level in the series of hierarchical levels that conform to the multi-tree subdivision information. If inheritance is used for the inherited coding block, an inherited subset associated with the inherited coding block is extracted from the data stream. The inherited subset includes at least one syntax element of a predetermined syntax element type, and individual residual information associated with each coded sub-block in the set of coded sub-blocks is extracted from the data stream. A predictor is configured to: copy the at least one syntax element in the inherited subset into a set of syntax elements that represent the coding parameters used in the inter-coder processing corresponding to each coded sub-block in the set of coded sub-blocks. For each coded subblock in the group of coded subblocks, a coding parameter used in the inter-code processing procedure associated with the corresponding coded subblock is determined based on the at least one syntax element. For each coded subblock in the group of coded subblocks, a separate prediction signal is predicted based on information associated with a previously reconstructed coded subblock that corresponds to the inter-code processing procedure. Each coded subblock in the group of coded subblocks is reconstructed based on the separate coding parameter, the separate prediction signal, and the separate residual information.

2. As in the decoder of request item 1, where, The inheritance information includes an inheritance flag that indicates whether inheritance is used for the inheritance-coded block defined by the multi-tree subdivision based on the multi-tree subdivision information, and the extractor is configured to extract the inheritance flag from the data stream.

3. As in request item 1, the decoder, where, This decoder is a hybrid decoder.

4. As in request item 1, the decoder, where, The decoder is configured to decode residual information for each coded subblock in the group of coded subblocks.

5. As in request item 1, the decoder, where, The decoder is configured to: extract from the data stream at least one syntax element of the predetermined syntax element type that represents a coding parameter associated with a leaf area of ​​a coding sub-block that is not a inherited coding block.

6. As in request item 1, the decoder, where, The decoder is configured to: if it is indicated that inheritance is not used, extract at least one syntax element of the predetermined syntax element type that represents an encoded parameter associated with a leaf region of the information sample array that does not use inheritance.

7. The decoder for request item 1, wherein, The decoder is configured to reconstruct the information sample array by means of a prediction in a prediction mode based on at least one syntax element of the predetermined syntax element type associated with the leaf region of the information sample array.

8. The decoder as in request item 1, wherein, The data stream contains at least a portion associated with color samples.

9. The decoder as requested in item 1, wherein, The data stream contains at least a portion of depth values ​​associated with a depth mapping table.

10. A method for reconstructing an array of information samples encoded in a data stream and representing video information, the method comprising: extracting multi-tree subdivision information from the data stream and inheritance information associated with an inherited encoded block of one of the information sample arrays, the inheritance information indicating whether inheritance is used, wherein, The inherited coding block corresponds to a first-level hierarchical level in a series of hierarchical levels that conform to the multi-tree subdivision information and is composed of a set of coding sub-blocks. Each coding sub-block in the set of coding sub-blocks corresponds to a second-level hierarchical level in the series of hierarchical levels that conform to the multi-tree subdivision information. If inheritance is used for the inherited coding block, an inherited subset associated with the inherited coding block is extracted from the data stream. The inherited subset includes at least one syntax element of a predetermined syntax element type. Individual residual information associated with each coding sub-block in the set of coding sub-blocks is extracted from the data stream. The at least one syntax element in the inherited subset is copied into a set of syntax elements, which represent the coding parameters used in the inter-code processing procedure corresponding to each coding sub-block in the set of coding sub-blocks. For each coding sub-block in the set of coding sub-blocks, a coding parameter used in the inter-code processing procedure associated with the corresponding coding sub-block is determined based on the at least one syntax element. For each coded subblock in the group of coded subblocks, a prediction signal is predicted based on information associated with a previously reconstructed coded subblock that is consistent with the inter-coding processing procedure; and each coded subblock in the group of coded subblocks is reconstructed based on the respective coding parameters, the respective prediction signals and the respective residual information.

11. As in request item 10, wherein, The inheritance information includes an inheritance flag that indicates whether inheritance is used for the inheritance-coded block defined by the multi-tree subdivision that conforms to the multi-tree subdivision information. The method further includes: extracting the inheritance flag from the data stream.

12. The method of request item 10 further includes: decoding residual information for each coded subblock in the group of coded subblocks.

13. The method of claim 10 further comprises: extracting from the data stream at least one syntax element of the predetermined syntax element type representing a coding parameter associated with a leaf area that is not a coded sub-block of an inherited coding block.

14. The method of request 10 further includes: if it is indicated that inheritance is not used, extracting at least one syntax element of the predetermined syntax element type that represents an encoded parameter associated with a leaf region of the information sample array that does not use inheritance.

15. The method of claim 10 further comprises: reconstructing the information sample array by means of a prediction in a prediction mode determined based on at least one syntax element of the predetermined syntax element type associated with a leaf region of the information sample array.

16. As in request item 10, wherein, The data stream contains at least a portion associated with color samples.

17. As in request item 10, wherein, The data stream contains at least a portion of depth values ​​associated with a depth mapping table.

18. A non-transitory computer-readable digital storage medium storing a computer program therein, which, when read by a computer, causes the computer to perform the method of claim 10.

19. A method for encoding an array of information samples representing a video information signal into a data stream, the method comprising: inserting multi-tree subdivision information and inheritance information associated with an inherited coding block of one of the information sample arrays into the data stream, the inheritance information indicating whether inheritance is used, wherein, The inherited coding block corresponds to one of the first-level hierarchical levels in a series of hierarchical levels that conform to the multi-tree subdivision information and is composed of a set of coding sub-blocks. Each coding sub-block in the set of coding sub-blocks corresponds to one of the second-level hierarchical levels in the series of hierarchical levels that conform to the multi-tree subdivision information. If it is indicated that inheritance should be used for the inherited coded block, an inherited subset is inserted into the data stream. The inherited subset includes at least one syntax element of a predetermined syntax element type associated with the inherited coded block. During decoding, at least one syntax element in the inherited subset is copied into a set of syntax elements representing the encoding parameters used in the inter-code processing corresponding to each coded sub-block in the set of coded sub-blocks to reconstruct the coded sub-block. Individual residual information for each coded sub-block in the set of coded sub-blocks is inserted into the data stream so that each coded sub-block in the set of coded sub-blocks in the inherited coded block is reconstructed based on one or more of the following: the encoding parameters used in the inter-code processing, the individual residual information specific to the coded sub-block, and individual prediction signals generated based on information associated with a previously reconstructed coded sub-block consistent with the inter-code processing.

20. The method as described in request item 19, wherein, The inheritance information includes an inheritance flag that indicates whether inheritance is used for the inheritance-coded block defined by the multi-tree subdivision that conforms to the multi-tree subdivision information. The method further includes inserting the inheritance flag into the data stream.

21. The method of claim 19 further comprises: inserting at least one syntax element of the predetermined syntax element type, representing a coding parameter associated with a leaf area that is not a inherited coding block, into the data stream.

22. The method of claim 19 further includes: if it is indicated that inheritance is not used, inserting at least one syntax element of the predetermined syntax element type, representing an encoded parameter associated with a leaf area of ​​the information sample array where inheritance is not used, into the data stream.

23. The method of claim 19 further comprises: inserting at least one syntax element of the predetermined syntax element type, which indicates a prediction pattern associated with a leaf region of the information sample array, into the data stream.

24. As in request item 19, wherein, The data stream contains at least a portion associated with color samples.

25. As in request item 19, wherein, The data stream contains at least a portion of depth values ​​associated with a depth mapping table.

26. A non-transitory computer-readable digital storage medium storing a computer program having code that, when read by a computer, causes the computer to perform the method described in claim 19.

27. A machine-readable non-transitory medium for storing data associated with a video signal, comprising a data stream stored on the non-transitory machine-readable medium, the data stream including an information sample array representing video information, multi-tree subdivision information, and inheritance information associated with a inheritance coding block of the information sample array, the inheritance information indicating whether inheritance is used, wherein, The inherited coded block corresponds to one of the first-level hierarchies in a series of hierarchical levels that conform to the multi-tree subdivision information, and is composed of a set of coded sub-blocks. Each coded sub-block in this set of coded sub-blocks corresponds to one of the second-level hierarchies in the series of hierarchical levels that conform to the multi-tree subdivision information. The data stream further includes individual residual information associated with each coded subblock in the group of coded subblocks, and if it is indicated that inheritance is to be used for the inherited coded subblock, it also includes an inherited subset including at least one syntax element of a predetermined syntax element type associated with the inherited coded subblock, such that the at least one syntax element associated with the inherited coded subblock is copied as a corresponding syntax element representing a coding parameter used in an inter-coder process associated with the corresponding coded subblock, and each coded subblock in the group of coded subblocks is reconstructed based on the following: the coding parameter used in the inter-coder process, the individual residual information specific to the coded subblock, and individual prediction signals generated based on information associated with a previously reconstructed coded subblock consistent with the inter-coder process.

28. The non-temporary media as described in claim 27, wherein, The inheritance information includes an inheritance flag that indicates whether inheritance is used for the inheritance-coded block defined by the multi-tree subdivision, and the data stream contains the inheritance flag.

29. The non-temporary media as described in claim 27, wherein, The data stream contains at least a portion associated with color samples.

30. Non-temporary media as in claim 27, wherein, The data stream contains at least a portion of depth values ​​associated with a depth mapping table.

Images