Picture coding supporting block merging and skip mode, and related apparatus and method

Abstract

An improvement in coding efficiency is achieved by using generalized signaling within the bitstream regarding merge actuation and skip mode actuation. That is, one of the possible states of one or more syntax elements within the bitstream can signal the current sample set of an image regarding whether individual sample sets will be merged and encoded without predictive residuals and inserted into the bitstream. Additionally, a general flag can generally signal whether coding parameters associated with a current sample set will be set according to a merge candidate or obtained from the bitstream, and whether the current sample set of the image will be reconstructed solely based on a predictive signal according to one of the coding parameters associated with the current sample set, without any residual data, or will be reconstructed using residual data within the bitstream by refining the predictive signal according to those coding parameters associated with the current sample set.

Description

Technical Field

[0001] This application relates to image and / or video encoding, and more particularly to codecs that support block separation and skipping modes. Prior Technology

[0002] Many image and / or video codecs process images in blocks. For example, predictive codecs use a block spacing size to achieve a good compromise between, on the one hand, spending too much side information on prediction parameters—very precisely set prediction parameters with high spatial resolution—and on the other hand, too coarsely set prediction parameters that result in an increased number of bits required to encode prediction residuals due to lower spatial resolution prediction parameters. In practice, the optimal setting of these prediction parameters lies somewhere between the two extremes.

[0003] Many attempts have been made to find the best solution to the above problems. For example, instead of using regular image segmentation that arranges images into blocks of columns and rows, multi-tree segmentation seeks to increase the degree of freedom in segmenting an image into blocks, given reasonable requirements for segmentation information. However, even though multi-tree segmentation requires a considerable amount of data, the degree of freedom in segmenting an image is still quite limited, even when using this multi-tree segmentation.

[0004] To achieve a better trade-off between the amount of side information required for image segmentation and the degrees of freedom in segmenting the image, block merging can be used to increase the possible number of image segments with a reasonable amount of additional data required for merging information. For the merged blocks, the encoding parameters need to be completely transmitted only once within the bitstream, as if the resulting merged block clusters were direct segments of the image.

[0005] To further enhance the efficiency of encoded image content, skipping modes have been introduced into some block-based image codecs. These skipping modes allow the encoder to suppress the transmission of residual data from certain blocks to the decoder. In other words, skipping modes can suppress the transmission of residual data for certain blocks. This ability to suppress the transmission of residual data for certain blocks allows for a wider interval range for encoding / prediction parameters, where an optimal trade-off between encoding quality and total bit rate can be expected: of course, increasing the spatial resolution of the encoded / prediction parameters leads to an increase in information rate, but also reduces the amount of residual data, thus lowering the bit rate at which residual data must be encoded. However, due to the availability of skipping modes, their availability only moderately increases the interval size at which the encoded / prediction parameters are transmitted, while the residual data is so small that the individual transmission of the residual data can be omitted, resulting in an unexpected saving in coding rate.

[0006] However, due to the redundancy recently caused by block merging and the use of skipping modes, there is still a need to achieve better coding performance.

[0007] Therefore, the object of this invention is to provide an encoding concept that increases encoding efficiency. This object will be achieved using the pending independent patent application scope. Summary of the Invention

[0008] The present invention envisions that further increases in coding performance can be achieved if general communication regarding merge actuation and skip mode actuation is used within the bitstream. That is, one of the possible states of one or more syntax elements within the bitstream can be communicated, for a current sample set of an image, where each sample set is to be merged and no prediction residual is encoded and embedded within the bitstream. Furthermore, a general flag can be communicated regarding whether the coding parameters associated with a current sample set will be set based on a merge candidate or obtained from the bitstream, and whether the current sample set of the image will be reconstructed solely based on a prediction signal according to the coding parameters associated with the current sample set without any residual data, or will be reconstructed using residual data within the bitstream by refining the prediction signal according to the coding parameters associated with the current sample set.

[0009] The inventors of this invention have discovered that a universal communication introduction for both merge-mode actuation and skip-mode actuation saves bit rate because the additional recurring overhead of communication for merge-mode and / or skip-mode actuation, which are separate from each other, can be reduced or may only be incurred when merge-mode and skip-mode actuation are not simultaneously performed.

[0010] Advantageous practices of the present invention are subject to the scope of the patent application. Simple Explanation of the Diagram

[0011] The preferred embodiments of this application will be described in more detail below with reference to relevant figures, wherein: Figure 1 shows a block diagram of an encoding device according to one embodiment; Figure 2 shows a block diagram of the encoding device according to a more detailed embodiment; Figure 3 shows a block diagram of a decoding apparatus according to one embodiment; Figure 4 shows a block diagram of a device used for decoding according to a more detailed embodiment; Figure 5 shows a block diagram of the possible internal structure of the encoder in Figure 1 or 2; Figure 6 shows a block diagram of the possible internal structure of the decoder in Figure 3 or 4; Figure 7a decomposes an image into possible fine segments: root block, coding unit (block), and prediction unit (segment). Figure 7b illustrates a detailed tree structure based on an explanatory example, showing the root block of the tree in Figure 7a, extending downwards to the dividing line. Figure 8 illustrates an embodiment of a set of potentially supported partition patterns; Figure 9 illustrates how block splitting, when used according to Figure 8, effectively results in self-combining block merging and the possible splitting patterns generated during block splitting. Figure 10 shows, in an exploded view, a candidate block for a skip / direct mode according to one embodiment; Figures 11-13 illustrate the grammatical portion according to one embodiment of the grammar; and Figure 14 illustrates, in detail, the definition of a neighboring partition for a partition according to one embodiment. Implementation

[0012] Regarding the following explanation, it should be noted that whenever the same reference symbol is used in different figures, the explanation of the individual elements relative to one of these figures will be equally applicable to the other figures, provided that the explanation transferred from one figure to another does not conflict with the rest of the explanation of the other figure.

[0013] Figure 1 shows a device 10 for encoding an image 20 into a bitstream 30. Of course, the image 20 can be part of a video, in which case the encoder 10 would be a video encoder.

[0014] Although not explicitly shown in Figure 1, image 20 is represented as a sample array. The sample array of image 20 is divided into sample sets 40, which can be any sample set, for example, a sample set encompassing non-overlapping, single-connected regions of image 20. For ease of understanding, sample set 40 is shown as such and is referred to hereinafter as block 40; however, the following description is not to be considered a limitation on any particular type of sample set 40. According to one specific embodiment, sample set 40 is a rectangular and / or square block.

[0015] For example, image 20 can be subdivided into a regular arrangement of blocks 40, thus blocks 40 are typically arranged in columns and rows as shown in Figure 1. However, any other subdivision of image 20 into blocks 40 is also possible. In particular, the subdivisions that subdivide image 20 into blocks 40 can be fixed, i.e., predetermined values ​​known to the decoder or can be transmitted to the decoder within bitstream 30. In particular, blocks 40 of image 20 can have varying interval sizes. For example, multi-tree subdivision, such as quarter-tree subdivision, can be applied to image 20 or a regular pre-subdivision applied to image 20 to form regularly arranged root blocks to obtain block 40, in which case block 40 forms leaf blocks of the multi-tree subdivision of the root blocks.

[0016] In any case, encoder 10 is configured to encode a flag into bit stream 30 for the current sample set 40 and generally communicate whether the encoding parameters associated with the current sample set 40 are set based on a merge candidate or obtained from bit stream 30, and whether the current sample set of image 20 is based solely on a prediction signal according to the encoding parameters associated with the current sample set without any residual data being reconstructed, or reconstructed using residual data in bit stream 30 by refining the prediction signal according to the encoding parameters associated with the current sample set 40. For example, encoder 10 is configured to generally transmit a flag as bitstream 30 for the current sample set 40, if we assume a first state where the encoding parameters associated with the current sample set 40 are set based on a merge candidate rather than being obtained from bitstream 30, and the current sample set of image 20 is reconstructed solely based on a prediction signal according to the encoding parameters associated with the current sample set, without any residual data; and if we assume any other state where the encoding parameters associated with the current sample set 40 are obtained from bitstream 30, or the current sample set of image 20 is reconstructed using residual data within bitstream 30 by refining the prediction signal according to the encoding parameters associated with the current sample set 40. This means the following: Encoder 10 supports merging of blocks 40. This merging is authorized. That is, not every block 40 is merged. For some blocks 40, for example, in the sense of bitrate / distortion optimization, it is suitable to merge the current block 40 with a merge candidate, but for others, the opposite is true. To determine whether a block 40 should be merged, encoder 10 determines a set or list of merge candidates, and for each of these merge candidates, whether merging the current block 40 and the merge candidates will form a better encoding choice, for example, in the sense of bit rate / distortion optimization. Encoder 10 is configured to determine the set or list of merge candidates for a current block 40 based on the previously encoded portions of bitstream 30. For example, encoder 10 obtains at least a portion of the set or list of merge candidates by accepting encoding parameters associated with regional and / or temporally adjacent blocks 40 that have been previously encoded according to the encoding order applied by encoder 10. Temporal proximity means, for example, that a block of previously encoded video image to which image 20 belongs has temporally adjacent blocks that are spatially arranged to spatially overlap the current block 40 of the current image 20. Therefore, for this portion of the set or list of merge candidates, there is a one-to-one association between each merge candidate and the spatial and / or temporally adjacent blocks. Each merge candidate has associated encoding parameters. If the current block 40 is merged with any merge candidate, the encoder 10 sets the encoding parameters of the current block 40 according to the merge candidate.For example, encoder 10 can set the encoding parameters of the current block 40 to be equal to those of each individual merging candidate; that is, encoder 10 can copy the encoding parameters of the current block 40 from each individual merging candidate. Therefore, for this description of the set or list of merging candidates, the encoding parameters of a merging candidate are directly adopted from a spatially and / or temporally neighboring block, or by similarly adopting the same, that is, setting the merging candidates equally, so that the encoding parameters of each merging candidate are obtained from the encoding parameters of this spatially and / or temporally neighboring block, but while also considering regional changes, for example, by scaling the adopted encoding parameters according to the regional changes. For example, the encoding parameters of at least a portion undergoing merging may include movement parameters. However, the movement parameters may indicate different reference image indices. More precisely, the motion parameters used can indicate a time interval between the current image and the reference image, and in merging the current block and individual merge candidates with their respective motion parameters, the encoder 10 can be configured to adjust the motion parameter scales of the individual merge candidates to adapt their time intervals to the time interval selected for the current block.

[0017] In all cases, the aforementioned merging candidates share their associated encoding parameters, and there is a one-to-one correlation between these merging candidates and their neighboring blocks. Therefore, merging block 40 with the just-described merging candidates can be considered as merging these blocks into one or more groups of blocks 40, thus the encoding parameters do not change across the image 20 within these groups of blocks 40, except for scale adaptation or similar. In fact, merging with any just-described merging candidate reduces the scale of encoding parameter changes across the image 20. Furthermore, merging with any just-described merging candidate results in additional degrees of freedom for finely segmenting the image 20 into blocks 40 and groups of blocks 40. Therefore, in this respect, merging blocks 40 into these groups of blocks can be considered as causing the encoder 10 to encode the image 20 using encoding parameters that change across the image 20 in units of these groups of blocks 40.

[0018] In addition to the merge candidates mentioned above, encoder 10 may also add merge candidates to the set / list of merge candidates, which is the result of a combination of the coding parameters of two or more neighboring blocks, such as their arithmetic mean, geometric mean, or the median of the coding parameters of neighboring blocks and similar.

[0019] Therefore, by comparing the interval size defined by which image 20 is finely segmented into blocks 40, encoder 10 effectively reduces the interval size at which coding parameters are explicitly transmitted within bitstream 30. Some of these blocks 40 are grouped into blocks by using one of the merging options outlined above, along with its coding parameters. Some blocks are coupled to each other by merging, but use different coding parameters associated with each other through individual scaling and / or combination functions. Some blocks 40 are not merged, and therefore encoder 10 directly encodes these coding parameters into bitstream 30.

[0020] Encoder 10 uses the encoding parameters of the thus defined block 40 to determine the prediction signal for image 20. Encoder 10 determines the prediction signal in a block-by-block manner, wherein the prediction signal is based on the encoding parameters associated with each individual block 40.

[0021] Another decision made by encoder 10 concerns the residual amount, i.e., the difference between the prediction signal and the original image content in each local region of the current block 40, and whether it will be transmitted within the bitstream 30. In other words, encoder 10 determines whether a skip mode will be applied to each block of 40. If a skip mode is applied, encoder 10 encodes the image 20 within the current portion of block 40 only in the form of a prediction signal derived from the encoding parameters associated with each block 40, or according to the encoding parameters associated with each block 40. In the case where the skip mode is canceled, encoder 10 encodes the image 20 within block 40 as bitstream 30 using the prediction signal and residual data.

[0022] To conserve bit rate in communicating both the merge decision and the skip mode, encoder 10 generally uses a single flag to communicate both decisions for block 40. More precisely, this general communication is implemented such that the activation of both the merge and skip modes is generally indicated within bitstream 30 by a flag of each block 40 assuming a first possible flag state; however, other flag states only indicate to the decoder that either the merge or skip mode is not activated. For example, encoder 10 may decide to activate a merge for a block 40 but cancel the skip mode. In this case, encoder 10 uses other flag states to communicate the cancellation of at least one of the merge and skip modes within bitstream 30, while sequentially, for example, communicating the activation of the merge within bitstream 30 using another flag. Therefore, encoder 10 must send this further flag only when block 40 is activated in different merge and skip modes. In the embodiments further described below, the first flag is referred to as mrg_cbf or skip_flag, while the auxiliary merge indicator flag is referred to as mrg or merge_flag. The inventors of this application have discovered the common use of this signaling state so that the actuation of signaling merging and skip mode can reduce the overall bit rate of bit stream 30.

[0023] Regarding the aforementioned signaling state, it should be noted that this signaling state can be determined using one bit state of the bit stream 30. However, the encoder 10 can be configured to encode the bit stream 30 using entropy coding techniques, and thus the correspondence between the flag signaling state and the bit stream 30 may become more complex. In this case, the state can correspond to one bit of the bit stream 30 in the entropy decoding domain. Furthermore, the signaling state can correspond to one of the two flag states specified for the coded word group according to the variable-length coding mechanism. In the case of arithmetic coding, the signaling state for the actuation of the signal merging and skipping modes can generally correspond to one of the symbols of the symbolic text under the arithmetic coding mechanism.

[0024] As described above, encoder 10 uses a flag within bitstream 30 to signal simultaneous actuation of merging and skipping modes. As will be described in more detail below, this flag can be sent within a syntax element having more than two possible states. This syntax element can also, for example, signal other encoding options. The details will be explained in more detail below. However, in that case, one of the possible states of one or more syntax elements will signal simultaneous actuation. That is, whenever the syntax element of the current block 40 described above determines this predetermined possible state, encoder 10 thus signals actuation of both merging and skipping modes. Decoder therefore does not need to separately signal actuation of merging and skipping modes.

[0025] Regarding the above explanation, it should be noted that dividing image 20 into blocks 40 may not represent the optimal resolution for the encoding parameters determined for image 20. Instead, encoder 10 can append further segmentation information to each block 40 to transmit within bitstream 30 for separately segmenting the current block 40 into sub-blocks 50 and 60, i.e., sample subsets, of one of the supported segmentation patterns. In this case, simultaneous merging / skipping decisions are performed by encoder 10 on a block 40 basis, and therefore the encoding parameters, and, for example, separate auxiliary merging and / or skipping pattern decisions, are defined for image 20 on a sub-segmentation basis, i.e., on sub-blocks 50 and 60 of the example shown in Figure 1 block 40. Of course, a non-segmentation pattern can represent one of the supported segmentation patterns, thus causing encoder 10 to determine only one set of encoding parameters for block 40. Regardless of the number of sub-blocks 50 and 60 of each individual delimiter pattern, the merge decision can be applied to all sub-blocks, i.e., one or more sub-blocks. That is, if a merge for block 40 is activated, this activation can be effective for all sub-blocks. According to an embodiment further described below, the general signaling merge and skip mode activation of the above general state can additionally and simultaneously signal non-delimiter patterns between the supported delimiter patterns of the current block 40, thus eliminating the need to further send delimiter information for the current block when the flag or syntax element assumes this state. Of course, in addition to merge and skip mode activation, any other delimiter patterns between supported delimiter patterns can be additionally indicated simultaneously.

[0026] According to some embodiments of this application, encoder 10 avoids bit performance loss arising from the shared use of block segmentation of block 40 on the one hand, and from the merging of sub-blocks 50 and 60 on the other. For greater precision, for example in the sense of bit rate / distortion optimization, encoder 10 can determine whether further segmentation of block 40 is preferable, and which supporting segmentation pattern should be used in the current block 40, in order to adjust the interval size of certain coding parameters set or defined within the current block 40 of image 20. As will be described in more detail below, coding parameters, for example, may represent prediction parameters, such as inter-frame prediction parameters. These inter-frame prediction parameters may, for example, include a reference image index, a motion vector, and the like. Supported segmentation patterns may include, for example, a non-segmentation mode, i.e., a selection where the current block 40 is not further segmented; a horizontal segmentation mode, i.e., a selection where the current block 40 is subdivided along a horizontal extension line into an upper or top portion and a bottom or lower portion; and a vertical segmentation mode, i.e., a selection where the current block 40 is vertically subdivided along a vertical extension line into a left portion and a right portion. In addition, supported segmentation patterns may also include a selection where the current block 40 is further regularly subdivided into four further blocks, each assuming to be one-quarter of the current block 40. Further, the segmentation may be all blocks 40 of the image 20 or only a suitable subset thereof, e.g., those with an associated encoding pattern, such as an inter-frame prediction pattern. Similarly, it should be noted that, by itself, merging may only be applicable to certain blocks, e.g., those encoded in an inter-frame prediction pattern. According to an embodiment further described below, the states generally described above also signal whether each block is an inter-frame prediction mode rather than an intra-frame prediction mode. Therefore, one of the flag states for block 40 can signal that this block is not further segmented and that merging and skipping modes are applied to the inter-frame prediction-coded blocks for which it is activated. However, as an auxiliary decision in the case of another flag state, each segment or sample subset 50 and 60 may each be attached with a further flag within bitstream 30 to signal whether merging will be applied to each segment 50 and 60. Furthermore, different subsets of the supporting segmentation modes may be available for block 40, for example, in the case of multi-tree fine-segmented leaf blocks, combined or individually depending on the block size and fine-segmentation level of block 40.

[0027] That is, image 20 is finely divided into blocks so that, in particular, blocks 40 can be fixed or transmitted within the bit stream. Similarly, the segmentation pattern to be used to further segment the current block 40 can be used to segment information in the form of transmission within the bit stream 30. Therefore, the segmentation information can thus be considered as an extension of image 20 being finely divided into blocks 40. On the other hand, the additional correlation of the original interval size that makes image 20 finely segmented into blocks 40 can still be maintained. For example, encoder 10 can be configured to transmit the encoding patterns of individual portions or blocks 40 of image 20 that will be used within the bit stream 30 using the interval size defined by block 40, while encoder 10 can be configured to vary the encoding parameters of individual encoding patterns within individual blocks 40 by increasing (fineer) interval size defined for individual segmentation patterns selected for individual blocks 40. For example, the encoding mode that transmits signals based on the interval size of block 40 can distinguish between intra-frame prediction modes, inter-frame prediction modes, and similar modes, such as temporal inter-frame prediction modes, inter-view prediction modes, etc. The type of encoding parameters associated with one or more sub-blocks (segments) resulting from the separation of individual blocks 40 then depends on the encoding mode assigned to each individual block 40. For example, for an intra-frame encoded block 40, the encoding parameters may include a spatial orientation along which image content of previously decoded portions of image 20 is used to fill the individual block 40. In the case of an inter-frame encoded block 40, the encoding parameters may include, in particular, a motion vector for motion compensation prediction.

[0028] Figure 1 exemplifies the current block 40, which is subdivided into two sub-blocks 50 and 60. In particular, a vertical partitioning pattern is exemplified. The smaller blocks 50 and 60 may also be referred to as sub-blocks 50 and 60, partitions 50 and 60, or prediction units 50 and 60. Specifically, encoder 10 can be configured to, in cases where one of the supported partitioning patterns is specified to make the current block 40 a subdivision of two or more further blocks 50 and 60, remove, from a set of coding parameter candidates available for each sub-block, coding parameter candidates with coding parameters identical to those associated with any sub-block (which, when merged with other sub-blocks, will form one of the supported partitioning patterns) from all further blocks except for a first sub-block of sub-blocks 50 and 60 in the coding order. For greater precision, for each supported partitioning pattern, a coding order is defined within the generated one or more partitioning patterns 50 and 60. In the case of Figure 1, the encoding order is illustrated using arrow 70, which defines that the left partition 50 is encoded before the right partition 60. In the horizontal partition mode, it can be defined that the upper partition is encoded before the lower partition. In any case, encoder 10 is configured to remove, for each second partition 60 in encoding order 70, the candidate encoding parameters that have the same encoding parameters as the associated first partition 50 from the candidate encoding parameters of that group for each second partition 60, in order to avoid this merging. That is, in fact, both partitions 50 and 60 will have the same encoding parameters associated with them. In practice, this can be equivalently generated at a lower encoding rate by selecting the non-partition mode for the current block 40.

[0029] For greater accuracy, encoder 10 can be configured to use block merging with block partitions in an efficient manner. Regarding block merging, encoder 10 can determine separate sets of encoding parameter candidates for partitions 50 and 60. The encoder can be configured to determine the sets of encoding parameter candidates for partitions 50 and 60 based on the encoding parameters associated with previously decoded blocks. In particular, at least some of the encoding parameter candidates within the set can be equal to, i.e., can be adopted from the encoding parameters of previously decoded partitions. Additionally, at least some of the encoding parameter candidates can be derived from encoding parameter candidates associated with more than one previously encoded partition through an appropriate combination (e.g., median, mean, or similar). However, because encoder 10 is configured to determine the reduction set of coding parameter candidates, and if more than one coding parameter candidate remains after removal, for each of the non-first segment 60, the selection among the remaining non-removed coding parameter candidates, in order to set the coding parameters associated with each segment according to a non-removed or selected coding parameter candidate, encoder 10 is configured to remove such candidates that it will effectively result in the removal of coding parameter candidates for the reunification of segments 50 and 60. That is, syntax clustering is effectively avoided, as encoding according to a valid segment would be more complex than directly communicating the segment by simply using the segment information.

[0030] Furthermore, since the set of coding parameter candidates is smaller, the amount of side information required to encode merging information into bitstream 30 may be reduced due to the lower number of elements in these candidate sets. In particular, since the decoder can determine and sequentially reduce the set of coding parameter candidates in the same way as the encoder in Figure 1, the encoder 10 in Figure 1 can utilize the reduced set of coding parameter candidates (by, for example, using fewer bits) to insert a syntax element into bitstream 30, indicating which of the non-removed coding parameter candidates will be used for merging. Of course, in the case where the number of non-removed coding parameter candidates for each segment is only one, the introduction of syntax elements into bitstream 30 can be completely suppressed. In any case, due to merging, that is, setting the coding parameters associated with each segment according to one of the remaining non-removed coding parameter candidates, or one of the selected ones, the encoder 10 can completely suppress the new embedding of one of the coding parameters for each segment into bitstream 30, thus also reducing side information. According to some embodiments of this application, encoder 10 can be configured to transmit information within bit stream 30 to refine information about the remainder or selected one of the respective separated coding parameter candidates.

[0031] Based on the previously described method of reducing the possibility of merging candidate lists, encoder 10 can be configured to determine which merging candidates will be removed by comparing their encoding parameters with the encoding parameters of the partitions, the merging of which will produce another supporting partition pattern. This method of handling encoding parameter candidates will effectively remove at least one encoding parameter candidate in the case shown in Figure 1, for example, assuming that the encoding parameters of the left partition 50 form an element of the set of encoding parameter candidates for the right partition 60. However, in the case where they are equal to the encoding parameters of the left partition 50, further encoding parameter candidates may also be removed. However, according to another embodiment of the invention, encoder 10 can be configured to determine the set of candidate blocks for each of the second and subsequent partitions in the encoding order by removing one or more candidate blocks from a set of candidate blocks (which, when merged with each of the respective partitions, will produce one of the supporting partition patterns). In some sense, this means the following. Encoder 10 can be configured to determine merge candidates for each of the separate partitions 50 or 60 (i.e., the first and the next in the encoding order), such that each element of the candidate set has exactly one of the previously encoded blocks 40 or a partition of the current block 40 associated with the respective encoding parameters of the partition to which the candidate uses the relevant partition. For example, each element of the candidate set may be equal to, i.e., adopted from one of the encoding parameters of the previously encoded partitions, or may be derived from the encoding parameters of at least one of the previously encoded partitions, for example, by further scaling or refining using additionally transmitted refining information. However, encoder 10 may also be configured to have further elements or candidates attached to this candidate set, i.e., encoding parameter candidates, which have been derived from a combination of encoding parameters of more than one of the previously encoded partitions, or have been derived by modifying the encoding parameters of a previously encoded partition, for example, by taking only the encoding parameters of a moving parameter list. For the "combined" elements, there is no 1:1 correlation between the encoding parameters of individual candidate elements and an individual partition. According to the first option illustrated in Figure 1, encoder 10 can be configured to remove all candidates whose encoding parameters are equal to the encoding parameters of partition 50 from the overall candidate set. According to the latter option illustrated in Figure 1, encoder 10 can be configured to remove only elements associated with the candidate set of partition 50. Reconciling the two arguments, encoder 10 can be configured to remove candidates by exhibiting a 1:1 correlation to some (e.g., neighboring) portions of the candidate set of previously encoded partitions, without extending this removal (and searching for candidates with equal encoding parameters) to the remainder of the candidate set with encoding parameters obtained using combination. However, of course, if a combination would also result in redundant representations, this can be resolved by removing redundant encoding parameters from the list or by performing a redundancy check on the combined candidates.

[0032] Before describing the decoder embodiment suitable for the embodiment of Figure 1 just described, an implementation example of an encoding device, namely an encoder, described in more detail according to Figure 1, will be described below in more detail with reference to Figure 2. Figure 2 shows an encoder comprising a segmenter 72 (configured to segment image 20 into blocks 40), a merger 74 (configured to merge blocks 40 into one or more sample set groups as described above), an encoder or encoding level 76 (configured to encode image 20 by varying encoding parameters across image 20 in units of sample set groups), and a bitstream generator 78. Encoder 76 is configured to encode image 20 by predicting image 20 and to encode a prediction residual for predetermined blocks. That is, encoder 76 encodes, as described above, not all prediction residuals of blocks 40. Instead, some of them have actuated skip patterns. Bitstream generator 78 is configured to embed prediction residuals and encoding parameters together into bitstream 30, while indicating one or more syntax elements for at least a subset of blocks 40, whether each block 40 will be merged into one of the block groups with another block, and whether each block uses a skip mode. As described above, the fine segmentation information under the fine segmentation of fine segmenter 72 can also be encoded into bitstream 30 using bitstream generator 78. This is indicated by dashed lines in Figure 2. The merging decision performed by merger 74 and the skip mode decision performed by encoder 76, as described above, are generally encoded into bitstream 30 using bitstream generator 78, such that one of the possible states of one or more syntax elements of current block 40 is indicated, regarding whether each block will be merged into one of the block groups with another block of image 20, and no prediction residual is encoded and embedded into bitstream 30. Bitstream generator 78, for example, may use entropy coding techniques for embedding. Segmenter 72 may be responsible for segmenting image 20 into blocks 40, and further segmenting them individually into segments 50 and 60. Merger 74 is responsible for the merging decisions described above, while encoder 76, for example, may determine the skipping pattern of block 40. Of course, all these decisions collectively affect bitrate / distortion measurements, and therefore device 10 can be configured to experiment with many decision options to determine which is better.

[0033] Following the description of an encoder according to an embodiment of the present invention with respect to Figures 1 and 2, a device for decoding, namely a decoder 80 according to an embodiment, will be described with reference to Figure 3. The decoder 80 of Figure 3 is configured to decode bitstream 30, in which image 20 is encoded as described above. In particular, the decoder 80 is configured to, for a current sample set or block 40, generally respond to a first determination regarding whether the aforementioned flags within bitstream 30 will be used to determine whether the encoding parameters associated with the current block 40 will be set according to a merge candidate or obtained from bitstream 30, and a second determination regarding whether the current block 40 of image 20 will be reconstructed solely based on a prediction signal according to the encoding parameters associated with the current block 40 without any residual data, or will be reconstructed using residual data within bitstream 30 by refining the prediction signal according to the encoding parameters associated with the current block 40.

[0034] That is, the decoder's function is primarily consistent with that of the encoder described in Figures 1 and 2. For example, decoder 80 can be configured to perform fine segmentation of image 20 into block 40. This fine segmentation may be known to decoder 80, or decoder 80 may be configured to extract individual segmentation information from bitstream 30. Whenever block 40 is merged, decoder 80 can be configured to obtain encoding parameters associated with block 40 by setting encoding parameters based on a merge candidate. To determine the merge candidate, decoder 80 can perform the determination of the merge candidate set or list described above in exactly the same manner as the encoder. According to some embodiments of this application, this even includes a reduction of the initial set / list of merge candidates to avoid redundancy between block segmentation on one side and block merging on the other side as described above. Whenever merging is actuated, the selection among the determined merge candidate set or list can be performed by decoder 80 using an individual merge index extracted from bitstream 30. The merge index indicates the merge candidates to be used from the (reduced) merge candidate set or list determined above. Further, as mentioned above, decoder 80 can also be configured to allow block 40 to be segmented according to one of the supporting segmentation patterns. Of course, one of these segmentation patterns may include a non-segmentation pattern, according to which block 40 is not further segmented. In the case of a flag indicating a generally defined state for the merging and skipping pattern actuation of a block 40, decoder 80 can be configured to reconstruct the current block 40 based solely on the predicted signal and not its combination with any residual signal. In other words, in this case, decoder 80 suppresses residual data extraction from the current block 40 and reconstructs the image 20 within the current block 40 solely by using the predicted signal derived from the current block encoding parameters. Also as previously stated, decoder 80 can interpret the general state of the flag as a signal for the current block 40, which is an inter-frame prediction block and / or a block that is not further segmented. That is, the decoder 80 can be configured to obtain the encoding parameters associated with the current block 40 by setting these encoding parameters according to a merge candidate, and if the flag signaling associated encoding parameters of the current block 40 in the discussion of the current block 40 in the 30-bit stream are to be set using merging, then the current block 40 of the image 20 is reconstructed based solely on the prediction signal according to the encoding parameters of the current block 40 without any residual data.However, if the flag in question, indicating that the current block 40 is not subject to merging or skipping mode is not used, then decoder 80 can respond to another flag within the 30-bit stream. Decoder 80, based on this other flag, obtains the encoding parameters associated with the current block by setting encoding parameters according to a separate merge candidate, obtains the residual data for the current block from bitstream 30, and reconstructs the current block 40 of image 20 based on the prediction signal and the residual data; or extracts the encoding parameters associated with the current block 40 from bitstream 30, obtains the residual data for the current block 40 from bitstream 30, and reconstructs the current block 40 of image 20 based on the prediction signal and the residual data. As described above, decoder 80 can be configured to anticipate the presence of another flag within bitstream 30 only if the first flag does not assume a general merging state and merging and skipping mode actuation are simultaneously merging. Decoder 80 then extracts the other flag from the bitstream to determine whether merging will occur without skipping mode. Of course, in the case where the second flag signal merge is cancelled, and the third flag signal skipping mode is activated or cancelled, the decoder 80 can be additionally configured to wait for another third flag within the current bitstream 30 of block 40.

[0035] Similar to Figure 2, Figure 4 illustrates a possible implementation of the decoding apparatus used in Figure 3. Therefore, Figure 4 shows an apparatus for decoding, namely, a decoder 80, which includes a segmenter 82 (configured to segment the image 20, to be encoded into a bitstream 30, into blocks 40), a merger 84 (configured to merge the blocks 40 into groups of one or more blocks), a decoder 86 (configured to decode or reconstruct the image 20 by varying the encoding parameters across the image 20 using sample set groups), and an extractor 88. The decoder 86 is also configured to decode the image 20 for the predetermined blocks 40, i.e., those with disabled skip modes, by predicting the image 20, decoding the prediction residuals for the predetermined blocks 40, and combining the prediction residuals with a prediction generated from the self-predicted image 20. Extractor 88 is configured to extract prediction residuals and encoding parameters together with bitstream 30, and one or more syntax elements of each of at least a subset of blocks 40, indicating whether each block 40 will be merged with another block 40 into one of such groups, wherein merger 84 is configured to perform the merge in response to one or more syntax elements, wherein one of the possible states of the one or more syntax elements indicates that each block 40 will be merged with another block 40 into one of the block groups and does not have prediction residuals that are encoded and embedded in bitstream 30.

[0036] Therefore, comparing Figure 4 and Figure 2, the fine segmenter 82 functions similarly to the fine segmenter 72 in order to recover the fine segments generated by the fine segmenter 72. The fine segmenter 82 identifies the fine segments of the image 20 by extracting fine segmentation information from the bit stream 30 using a predefined value or via the extractor 88. Similarly, the merger 84 merges the blocks 40 and is actuated regarding the blocks 40 and the block portions via the aforementioned communication within the bit stream 30. The decoder 86 uses the encoding parameters within the bit stream 30 to generate a prediction signal for the image 20. In the merging case, the decoder 86 copies the encoding parameters of the current block 40 or the current block's partition from the adjacent blocks / partitions, or sets its encoding parameters differently depending on the merging candidate.

[0037] As described above, extractor 88 is configured to interpret one of the possible states of a flag or syntax element of a current block as a signal for simultaneous signaling merging and skipping modes. Simultaneously, extractor 88 can interpret this state as also signaling one of the predetermined split patterns for the current block 40 within the supported split patterns. For example, the predetermined split pattern could be a non-split pattern under which block 40 remains unsplit, and it itself thus forms a split. Therefore, only when the simultaneous signaling state is uncertain for individual flags or syntax elements, extractor 8 predicts that bitstream 30 includes splitting information for signaling the split of block 40. As will be described in more detail below, the splitting information can be transmitted within bitstream 30 via a syntax element that simultaneously controls the encoding mode of the current block 40, i.e., splitting block 40 into one of the frames being encoded and one of the in-frame encodings. In this case, the general signaling state of the first flag / syntax element can also be interpreted as one of the signaling modes in the inter-frame predictive coding scheme. For each segment generated by the self-signaling segmentation information, in the case where the general signaling state of the first flag / syntax element of block 40 is uncertain, and both signaling merging and skipping mode activation are simultaneously implemented, the extractor 88 can extract another merging flag from the bitstream. In this case, the skipping mode can necessarily be interpreted as off by the extractor 88, and although merging can be individually activated for each segment using bitstream 30, the residual signal for this current block 40 is extracted from bitstream 30.

[0038] Therefore, the decoder 80 of Figure 3 or 4 is configured to decode the bitstream 30. As described above, the bitstream 30 can signal one of the supporting segmentation patterns for a current block 40 of the image 20. If the signaled supporting segmentation pattern specifies that the current block 40 is subdivided into two or more segments 50 and 60, the decoder 80 can be configured to remove all segments except the first segment 50 of those segments from a set of individual segmentation parameter candidates that have the same or equal coding parameters associated with any of those segments, in coding order 70. That is, the segment 60 in the example shown in Figures 1 and 3, where such segments, when merged with individual segments, would result in one of the supporting segmentation patterns, i.e., one that is not signaled within the bitstream 30, but is one of those supporting segmentation patterns.

[0039] For example, if the number of non-removed coding parameter candidates is not zero, decoder 80 can be configured to set the coding parameters associated with each segment 60 according to one of the non-removed parameter candidates. For example, decoder 80 sets the coding parameters of segment 60 individually, with or without further refinement and / or with or without scaling, based on the time distance involved in the coding parameters, so as to make them equal to one of the non-removed coding parameter candidates. For example, the merging of coding parameter candidates and non-removed candidates may have another associated reference image index, in addition to a reference image index explicitly signaled for segment 60 within bitstream 30. In this case, the coding parameters of the coding parameter candidates may define a movement vector for each with respect to a separate reference image index, and decoder 80 can be configured to scale the movement vector of the last selected non-removed coding parameter candidate according to the ratio between the two reference image indices. Therefore, according to the selection just mentioned, the coding parameters accepted for merging will include movement parameters, and thus the reference image index will be separated from there. However, as indicated above, according to another embodiment, these reference image indices may also be part of the merged encoding parameters.

[0040] Similarly, this merging behavior applies to the encoders of Figures 1 and 2 and the decoders of Figures 3 and 4, and can be restricted to inter-frame prediction blocks 40. Therefore, decoder 80 and encoder 10 can be configured to support both intra-frame and inter-frame prediction modes for the current block 40, and merging only occurs when the current block 40 is encoded in inter-frame prediction mode. Thus, only the encoding / prediction parameters of the previously encoded segments of these inter-frame predictions can be used to determine / construct the candidate list.

[0041] As discussed above, the encoding parameters can be prediction parameters, and the decoder 80 can be configured to use the prediction parameters of segments 50 and 60 to derive a prediction signal for each segment. Of course, the encoder 10 also derives the prediction signal in the same manner. However, the encoder 10 additionally sets the prediction parameters along with all other syntax elements within the bitstream 30 to achieve some optimization in a suitable sense.

[0042] Furthermore, as previously stated, the encoder can be configured to embed an index into a (non-removed) encoded parameter candidate only if the number of (non-removed) encoded parameter candidates for a given segment is greater than 1. Therefore, the decoder 80 can be configured to, depending on the number of (non-removed) encoded parameter candidates (e.g., for segment 60), if the number of (non-removed) encoded parameter candidates is greater than 1, only include bitstream 30 indicating which (non-removed) encoded parameter candidate is used in the merged syntax element. However, cases where the number of candidate sets becomes less than 2 can generally be excluded, as described above, by extending the candidate list / set using combined encoded parameters, i.e., these parameters are derived from combinations of encoded parameters from more than one – or more than two previously encoded segments, limiting the performance of candidate set reduction by taking from or deriving from those candidates obtained from exactly one previously encoded segment. The opposite is also possible; that is, all candidates for encoded parameters that have the same value for those separators that result in another supporting separator pattern are usually removed.

[0043] Regarding the decision, decoder 80 functions similarly to encoder 10. That is, decoder 80 can be configured to determine the set of merging candidates for a block 40 based on encoding parameters associated with previously decoded segments. In other words, an encoding order is defined not only within the individual blocks 40 segments 50 and 60, but also within block 40 of image 20 itself. All segments previously encoded at segment 60 can therefore serve as the basis for determining the set of merging candidates for any sequential segment (e.g., segment 60 in Figure 3). Also as mentioned above, encoders and decoders can limit the determination of the merging candidate set segments to spatial and / or temporal adjacency. For example, decoder 80 can be configured to determine the merging candidate set based on encoding parameters associated with previously decoded segments adjacent to the current segment, where these segments can be placed both outside and inside the current block 40. Of course, the determination of merging candidates can also be performed for the first segment of the encoding order. Only removal is possible.

[0044] Similar to the description in Figure 1, in addition to one encoded in an in-frame prediction mode, decoder 80 can be configured to determine a set of encoding parameter candidates for each non-first segment 60 from an initial set of previously decoded segments.

[0045] Furthermore, in the case where the encoder introduces fine segmentation information into the bit stream so that the finely segmented image 20 becomes blocks 40, the decoder 80 can be configured to recover the fine segments of image 20 into these coded blocks 40 based on the fine segmentation information in the bit stream 30.

[0046] Regarding Figures 1 to 4, it should be noted that the residual signal of the current block 40 can be transmitted via bitstream 30 at intervals that may differ from the interval size defined by the separation of coding parameters. For example, for blocks where skipping modes are not activated, encoder 10 of Figure 1 can be configured to subdivide block 40 into one or more transition blocks in a manner parallel to or regardless of the separations, becoming separations 50 and 60. The encoder can transmit further subdivisions of each transition block of block 40 using further subdivision information. Decoder 80, then, can be configured to recover the further subdivisions of block 40 into one or more transition blocks based on the further subdivision information in the bitstream, and obtain one residual signal of the current block 40 from the bitstream in units of these transition blocks. The meaning of transition block separation can be a transition in the encoder, such as DCT, and a corresponding inverse transition in the decoder, such as IDCT, performed within each transition block of block 40. To reconstruct the image 20 within block 40, encoder 10 then combines, for example, separately, the predicted signals derived from the respective partitions 50 and 60 by applying coding parameters, and the residual signals. However, it should be noted that the residual coding may not involve any transformation and inverse transformation separately, and the predicted residual, for example, is encoded in the spatial domain.

[0047] Before further detailing the possible embodiments below, the possible internal structures of the encoder and decoder of Figures 1 to 4 will be described with respect to Figures 5 and 6, but the merger and splitter are not shown in these figures to focus on the hybrid coding properties. Figure 5 illustrates, exemplarily, how the encoder 10 can be internally configured. As shown, the encoder 10 may include a subtractor 108, a converter 100, and a bitstream generator 102, which, as indicated in Figure 5, can perform entropy coding. Elements 108, 100, and 102 are connected in series between an input 112 of the received image 20 and an output 114 of the bitstream 30. In particular, the subtractor 108 has a non-inverting input connected to the input 112, and the converter 100 is connected between the output of the subtractor 108 and a first input of the bitstream generator 102, which has an output then connected to the output 114. The encoder 10 in Figure 5 further includes an inverting converter 104 and an adder 110 connected in series to the output of the converter 100 in the aforementioned order. The encoder 10 further includes a predictor 106 connected between the output of the adder 110 and a further input of the adder 110 and the inverted input of the subtractor 108.

[0048] The components in Figure 5 interact as follows: Predictor 106 predicts portions of image 20, and the prediction result, i.e., the prediction signal, is applied to the inverting input of subtractor 108. The output of subtractor 108 then represents the difference between the prediction signal and each portion of image 20, i.e., the residual signal. The residual signal undergoes conversion encoding in converter 100. That is, converter 100 can perform a conversion, such as a DCT or the like, and a subsequent quantization on the converted residual signal (i.e., the conversion coefficients) to obtain the conversion coefficient level. Inverter 104 reconstructs the last residual signal output by converter 100 to obtain a reconstructed residual signal corresponding to the residual signal input to converter 100 (except for information loss due to quantization in converter 100). The reconstructed residual signal and the prediction signal output by predictor 106 are added together to result in the reconstruction of each portion of image 20 and are transmitted from the output of adder 110 to the input of predictor 106. Predictor 106 operates in different modes as described above, such as an in-frame prediction mode, an inter-frame prediction mode, and similar modes. The prediction mode of the predicted signal and the corresponding encoding or prediction parameters are applied by predictor 106 to obtain the prediction signal and are then transmitted to entropy encoder 102 for embedding in the bit stream.

[0049] A possible implementation of the internal structure of the decoder 80 in Figures 3 and 4, corresponding to a possible encoder shown in Figure 5, is illustrated in Figure 6. As shown therein, the decoder 80 may include a bitstream decimator 150, which, as shown in Figure 6, is implemented as an entropy decoder, an inverter 152, and an adder 154, connected in the aforementioned order between the decoder input 158 ​​and the output 160. Further, the decoder in Figure 6 includes a predictor 156 connected between the output of the adder 154 and its further inputs. The entropy decoder 150 is connected to the parameter input of the predictor 156.

[0050] Briefly describing the decoder function in Figure 6, the entropy decoder 150 is used to extract all information contained in the bitstream 30. The entropy coding mechanism used can be variable-length coding or arithmetic coding. In this way, the entropy decoder 150 recovers the conversion coefficient levels representing the residual signal from the bitstream and transmits them to the inverter 152. Further, the entropy decoder 150 acts as the decimator 88 described above and recovers all coding patterns and associated coding parameters from the bitstream and transmits them to the predictor 156. Additionally, separation information and merging information are extracted from the bitstream using the decimator 150. The signal is then converted in reverse, that is, the reconstructed residual signal and the predicted signal derived using the predictor 156 are combined, for example, by addition using the adder 154, and then the reconstructed signal is output at output 160 and transmitted to the predictor 156.

[0051] As can be clearly seen from the comparison of Figures 5 and 6, components 152, 154 and 156 functionally correspond to components 104, 110 and 106 in Figure 5.

[0052] In the description of Figures 1 to 6 above, many different possibilities regarding the possible fine division of image 20 and the corresponding interval sizes for some parameter variations contained in the encoded image 20 have been presented. This possibility is again illustrated with reference to Figures 7a and 7b. Figure 7a shows a portion of image 20. According to the embodiment in Figure 7a, the encoder and decoder are configured to first finely divide image 20 into root blocks 200. This root block is shown in Figure 7a. The image 20, finely divided into root blocks, is regularly divided in columns and rows, as shown by dashed lines. The size of the root blocks 200 can be selected by the encoder and transmitted to the decoder using bitstream 30. Alternatively, the size of these root blocks 200 can be fixed using predetermined values. These root blocks 200 are finely divided using a quadtree partition to produce blocks 40 consistent with those above, which may be referred to as encoded blocks or encoded units. These coded blocks or units are drawn with thin solid lines in Figure 7a. In this way, the encoder accompanies the segmentation information to each root block 200 and inserts this segmentation information into the bitstream. This segmentation information indicates how the root block 200 is segmented into blocks 40. The prediction mode varies within the image 20 with the spacing and units of these blocks 40. As indicated above, each block 40 – or blocks having a certain prediction mode, such as inter-frame prediction modes – is accompanied by segmentation information about which supporting segmentation pattern is used for each block 40. However, recalling the aforementioned flag / syntax elements, when assuming a generally transmitted state, one of the supporting segmentation patterns for each block 40 can also be transmitted simultaneously; therefore, explicit transmission of the other segmentation information for this block 40 can be suppressed and not expected at the encoder end, and thus, at the decoder end as well. In the case illustrated in Figure 7a, for many coded blocks 40, a non-segmentation mode has been selected, thus spatially aligning coded blocks 40 with their corresponding segmentations. In other words, coded block 40 is simultaneously a segmentation with one of its associated prediction parameters. The prediction parameters are then classified according to the mode associated with each coded block 40. Other coded blocks, however, are exemplary shown to be further segmented. The coded block 40 at the top right-hand corner of the root block 200, for example, is shown to be segmented into four segments, and thus the coded block at the bottom right-hand corner of the root block 200 is exemplary shown to be vertically subdivided into two segments. The subdivisions used to separate the majority of segments are shown using dashed lines. Figure 7a also shows the coding order among the thus defined majority of segments. As shown, a depth-first pass order is used. Crossing the edge of the root block, the encoding order can continue in a scanning order (according to which the column of root block 200 is scanned from the top to the bottom of image 20 in a column manner).This measure maximizes the chance that a segmentation has a previously encoded segment adjacent to its top and left edges. Each block 40 – or blocks with a prediction pattern, such as an inter-frame prediction pattern – can have a merge toggle indicator within the bitstream indicating whether merging is actuated for the corresponding segment. It should be noted that a segmentation block becoming a segment / prediction unit can be limited to a maximum of two segments, with the only exception being the minimum possible block size of block 40. In the case of using a quadtree to refine the segmentation to obtain block 40, this avoids redundancy between the refinement information used to refine image 20 into block 40 and the segmentation information used to refine block 40 into segments. Additionally, only one or two segments are allowed, with or without asymmetric segments.

[0053] Figure 7b illustrates a finely segmented tree. The fine segmentation of the root block 200 is shown by solid lines, while dashed lines mark the leaf blocks of the quartic tree's fine segmentation, which are coded blocks 40. That is, the segmentation of the coded blocks represents an extension of a quartic fine segmentation.

[0054] As mentioned above, each coding block 40 can be subdivided in parallel into transformation blocks, and thus the transformation blocks can represent different subdivisions of each coding block 40. For each of these transformation blocks, which are not shown in Figures 7a and 7b, the transformation of the residual signal of the transformation coding block can be performed separately.

[0055] Below, further embodiments of the invention will be described. While the embodiments above focus on the relationship between block merging on one side and block splitting on the other, the following description also includes arguments of this application regarding other coding principles known in current codecs, such as skip / direct mode. However, the following description is not to be regarded as merely describing separate embodiments, i.e., those separate embodiments described above. Of course, the following description also discloses possible implementation details of the embodiments described above. Therefore, the following description uses reference numerals as previously described in the figures, and thus one of the individual possible implementations described below will also define possible variations of the embodiments described above. Most of these variations can be transferred separately to the embodiments above.

[0056] In other words, embodiments of this application describe a method for reducing the side rate of information in image and video coding applications by combining merged signals and the absence of residual data in the sample set. In other words, by combining syntax elements indicating the use of the merging mechanism and syntax elements indicating the absence of residual data, the side rate of information in image and video coding applications is reduced.

[0057] Furthermore, before explaining these changes and further details, an overview of image and video codecs is presented.

[0058] In image and video coding applications, an array of samples associated with an image is typically divided into a set of specific samples (or sample sets), which may represent rectangular or square blocks or any other set of samples containing regions of arbitrary shapes, triangles, or any other shape. The fine division of the sample array can be fixed or finely divided (at least partially) within the bitstream by a syntax. To maintain a small information rate using the information transmitted by the fine division information, the syntax typically allows only a limited number of choices resulting in simple divisions (e.g., fine divisions that make a block into smaller blocks). A commonly used division mechanism is to divide a square block into four smaller square blocks, or into two rectangular blocks of the same size, or into two rectangular blocks of different sizes, wherein the division actually used is transmitted within the bitstream. The sample set is associated with specific coding parameters, which may indicate predictive information or residual coding patterns, etc. In video coding applications, a division is often performed for motion rendering purposes. All samples within a block (within a segmented pattern) are associated with the same set of motion parameters, which may include parameters indicating the prediction type (e.g., list 0, list 1, or bidirectional prediction; and / or shift or affine prediction or prediction with a different motion model), parameters indicating the reference image used, parameters indicating the motion relative to such reference images (e.g., displacement vector, affine motion parameter vector, or motion parameter vector for any other motion model), which are typically sent to the predictor as a difference, parameters indicating the accuracy of the motion parameters (e.g., half-sample or quarter-sample accuracy), parameters indicating the weighting of the reference sample signal (e.g., for brightness compensation purposes), or parameters indicating the interpolation filter used to derive the motion-compensated prediction signal for the current block. It is assumed that for each sample set, separate coding parameters (e.g., for indicating prediction and / or residual coding) are sent. To achieve improved coding efficiency, the present invention proposes a method and specific embodiments for merging two or more sample sets to form a so-called sample set family. All sample sets in this population share the same encoding parameters, which can be sent together with one of the sample sets in the population. In this way, the encoding parameters do not need to be sent separately to each sample set in the population; instead, the encoding parameters are sent only once for the entire population.

[0059] Therefore, the information rate used to transmit the encoding parameters is reduced and the overall encoding efficiency is improved. In another method, further refinement of one or more encoding parameters can be transmitted to one or more sample sets within a sample set family. This refinement can be arbitrarily applied to all sample sets of a family or only to the sample set to be transmitted.

[0060] Some embodiments of the present invention combine a merging process with the division of a block into sub-blocks 50, 60 (as described above). Typically, image or video coding systems support various division patterns for block 40. As an example, a square block may not be divided, or it may be divided into four square blocks of the same size, or into two rectangular blocks of the same size (wherein the square blocks may be divided vertically or horizontally), or into rectangular blocks of different sizes (horizontally or vertically). The above-described example division patterns are shown in Figure 8. In addition to the above description, the division may even include more than one division level. For example, square sub-blocks may also be selectively further divided using the same division pattern. When this division process is combined with a merging process (allowing a (square or rectangular) block to be merged with, for example, one of its adjacent blocks), the issue arises that the same resulting division can be achieved through different combinations of division patterns and merging signals. Therefore, the same information can be transmitted in a bitstream using different codewords, which is clearly approximately optimal for coding efficiency. As a simple example, consider a square block that is not further subdivided (as shown in the top left corner of Figure 8). This subdivision can be directly communicated by transmitting a syntax element that transmits the block 40 without further subdivision. However, the same pattern can also be communicated by transmitting a syntax element that specifies the block, for example, that it is subdivided into two vertically (or horizontally) aligned rectangular blocks 50 and 60. We can then send a merge message indicating that the second of these rectangular blocks is merged with the first rectangular block, resulting in the same subdivision as when we transmit the block without further subdivision. This can also be achieved by first specifying that the block is subdivided into four square sub-blocks and then sending a merge message that effectively merges all four blocks. This concept is clearly approximately optimal (because we have different codewords for communicating the same event).

[0061] Some embodiments of the present invention reduce side information rate and thus increase coding efficiency for combinations of merging concepts and concepts providing different partition patterns for a block. Referring to the partition pattern example in Figure 8, when we prohibit (i.e., exclude bitstream syntax from specifying) merging a rectangular block with a first rectangular block, the “simulation” of further partitioning any partition pattern having two rectangular blocks can be avoided. Referring further to this issue, it is also possible to “simulate” an undivided pattern by merging a second rectangle with any other adjacent (i.e., not the first rectangular block) associated with the same parameters (e.g., information indicating prediction) as the first rectangular block. Redundancy can be avoided by adjusting the transmission of merging information so that when these merging parameters result in a pattern that can also be achieved by supporting one of the partition patterns via signaling, the transmission of specific merging parameters does not include bitstream syntax. As an example, if the current partition pattern indicates a subdivision into two rectangular blocks, as shown in Figures 1 and 3, for instance, before transmitting the merge information for the second block, i.e., block 60 in the cases of Figures 1 and 3, it can be checked which possible merge candidates have the same parameters as the first rectangular block, i.e., block 50 in the cases of Figures 1 and 3 (e.g., parameters used to indicate the prediction signal). All candidates with the same movement parameters (including the first rectangular block itself) are removed from the merge candidate set. The coded words or flags sent to signal the merge information are applicable to the generated candidate set. If the candidate set becomes empty due to parameter checks, no merge information can be sent. If the candidate set contains just one item, it only signals whether the block is merged, but the candidate does not need to be signaled because it can be derived at the decoder. The same concept is also applied to the partition pattern of dividing a square block into four smaller square blocks, as in the example above. Here, the appropriate way to transmit the merge flag is neither to indicate that either a partition pattern without subdivisions or a partition pattern subdivided into two rectangular blocks of the same size can be achieved using a combination of merge flags. Although we have illustrated the concept of a particular partition pattern in the examples above, it should be understood that the same concept (the description of a particular partition pattern that avoids being combined by another partition pattern and the corresponding merge information) can be applied to any other set of partition patterns.

[0062] Another argument to consider is that the concept of merging is, in some sense, similar to the skip / direct mode found in video coding design. In skip / direct mode, fundamentally no motion parameters are sent for a current block; instead, they are inferred from a spatial and / or temporal neighbor. In one particularly valid concept of skip / direct mode, a list of motion parameter candidates (reference frame indices, displacement vectors, etc.) is generated from a spatial and / or temporal neighbor, and an index indicating which candidate parameter is selected is sent. For a bidirectional prediction block (or multiple hypothetical frames), a separate candidate can be signaled to each reference list. Possible candidates may include blocks up to the top of the current block, blocks to the left of the current block, blocks to the top left of the current block, blocks to the top right of the current block, intermediate predictors of various candidates, and blocks located in the same position within one or more previous reference frames (or any other previously encoded blocks, or combinations thereof).

[0063] The combination of the hop / direct concept means that a block can be encoded using either a hop / direct or a merge mode. While the hop / direct and merge concepts are similar, there are differences between them, which will be explained in more detail in Part 1. The main difference between hop and direct is that the hop mode further transmits the signal without any residual signal being sent. When the merge concept is used, a flag is typically sent indicating whether a block contains a non-zero transition coefficient level.

[0064] To achieve improved coding efficiency, as described in the embodiments above and below, the combination of whether the sample set uses coding parameters from another sample set and whether there is no residual signal for the block being transmitted is considered. A combination flag indicates that the sample set uses coding parameters from another sample set and that no residual data is transmitted. In this case, only one flag, instead of two, needs to be transmitted.

[0065] As described above, some embodiments of the present invention also provide an encoder with greater freedom in generating one bitstream because the merging method significantly increases the possibility of selecting image sample array partitions without introducing redundancy in the number of bitstreams. Because the encoder can choose among more options, for example, minimizing a specific bitrate / distortion metric, coding efficiency can be improved. As an example, some other patterns (e.g., the pattern in Figure 9) that can be represented by a combination of sub-segmentation and merging can be further tested (using corresponding block sizes for motion estimation and mode determination), and the optimal pattern provided using simple partitioning (Figure 8) and partitioning and merging (Figure 9) can be selected based on a specific bitrate / distortion metric. Furthermore, for each block, it can be tested whether merging with any previously encoded candidate set produces a specific reduction in bitrate / distortion metric, and then the corresponding merging flag is set during the encoding process. In summary, there are many possibilities for operating an encoder. In a simple method, the encoder can first determine the optimal partitioning of the sample array (as in current technology encoding mechanisms). Next, it checks whether merging with another sample set or another sample set population reduces a specific bit rate / distortion cost measure for each sample set. At this point, the prediction parameters associated with the merged sample set population can be re-estimated (e.g., by performing a new move search) or the prediction parameters for the previously determined merged current sample set and candidate sample sets (or sample set populations) can be estimated for the considered sample set population. In a more general approach, a specific bit rate / distortion cost measure can be estimated for another sample set candidate population. As a particular example, when testing various possible splitting patterns (e.g., referring to Figure 8), some or all of the patterns represented by the splitting and merging combination (e.g., referring to Figure 9) can be additionally tested. That is, for all patterns, a specific move estimation and pattern determination procedure is performed, and the pattern that produces the minimum bit rate / distortion measure is selected. This process can also be combined with a low-complexity process as described above, so that for the generated block, it is additionally tested whether the merging with previously encoded blocks (e.g., outside of the patterns in Figure 8 and Figure 9) results in a reduction in bit rate / distortion measurements.

[0066] Below, some possible detailed implementations of the above embodiments are described, for example, for the first, second, and fifth image encoders and the third, fourth, and sixth image decoders. As noted above, the same can be used for image and video coding. As described above, an image, or a specific set of sample arrays for an image, can be decomposed into blocks, which are associated with specific coding parameters. Such images typically contain multiple sample arrays. In addition, an image may also be associated with additional auxiliary sample arrays, for example, which specify transparency information or depth maps. The image sample array (including auxiliary sample arrays) can be clustered into one or more so-called planar clusters, where each planar cluster includes one or more sample arrays. A planar cluster of an image can be encoded independently or, if the image is associated with more than one planar cluster, predicted from other planar clusters of the same image. Each planar cluster is typically decomposed into blocks. Such blocks (or corresponding blocks of sample arrays) are predicted using either inter-frame-image prediction or intra-frame-image prediction. Such blocks may have different sizes and may be square or rectangular. Image segmentation into blocks can be fixed using syntax, or it can be signaled (at least partially) within the bitstream. Typically, syntax elements are sent, signaling the finer segmentation of blocks of a predetermined size. These syntax elements can indicate whether and how a block is segmented into smaller blocks and are associated with encoding parameters, such as those used for prediction purposes. A possible segmentation pattern example is shown in Figure 8. The decoding of the encoding parameters associated with all samples (or corresponding blocks in a sample array) of a block is specified in a certain way. In this example, all samples of a block are predicted using the same set of prediction parameters, such as a reference index (identifying a reference image in a set of previously encoded images), a movement parameter (indicating the measurement of block movement between the reference image and the current image), parameters indicating the interpolation filter, in-frame prediction mode, etc. The movement parameter can be represented using a displacement vector with horizontal and vertical components or using higher-order movement parameters, such as affine movement parameters with six components. It is also possible that more than one set of specific prediction parameters (e.g., reference index and movement parameter) are associated with a single block. Therefore, for each set of these specific prediction parameters, a single intermediate prediction signal is generated for one of the blocks (or corresponding blocks of the sample array), and the final prediction signal is constructed using a combination of these overlapping intermediate prediction signals. The corresponding weighting parameters and possibly a fixed offset (added to the weighted sum) may be fixed for an image, a reference image, or a set of reference images, or they may be contained within the set of prediction parameters for the corresponding block. The difference between the original block (or the corresponding sample array block) and their prediction signals, also known as the residual signal, is typically transformed and quantized. Typically, a two-dimensional transformation is applied to the residual signal (or the corresponding sample array for the residual block).For transformation coding, these blocks (or corresponding sample array blocks), in which a specific set of prediction parameters has been used, can be further segmented before the transformation is applied. These transformation blocks can be equal to or smaller than the blocks used for prediction. It is also possible that a transformation block contains more than one block used for prediction. Different transformation blocks can have different sizes and can represent square or rectangular blocks. In the example above for Figures 1-7, it should be noted that the leaf nodes that may be initially finely segmented, i.e., the coding block 40, can be further segmented in parallel to define the coding parameter interval size, and two-dimensional transformations are applied to the transformation blocks separately. After the transformation, the resulting transformation coefficients are quantized and the so-called transformation coefficient levels are obtained. These transformation coefficient levels and the prediction parameters, and, if presented, the segmentation information is entropy-coded. In particular, the coding parameters for these transformation blocks are called residual parameters. These residual parameters and the prediction parameters, and, if presented, the segmentation information can be entropy-coded. In the current H.264 video coding standard, one of the flags called the Coded Block Flag (CBF) signals that all transition coefficient levels are zero and therefore no residual parameters are encoded. According to the present invention, this signal is combined into a merged actuation signal.

[0067] In current image and video coding standards, the ability to subdivide an image (or planar group) into blocks provided by the syntax is very limited. Typically, it only indicates whether (and how) a block of a predetermined size can be subdivided into smaller blocks. For example, the largest block size in H.264 is 16x16. A 16x16 block is also called a megablock, and each image is divided into megablocks in the first stage. For each 16x16 megablock, it can be communicated whether it is encoded as one 16x16 block, or two 16x8 blocks, or two 8x16 blocks, or four 8x8 blocks. If a 16x16 block is subdivided into four 8x8 blocks, then each of these 8x8 blocks can be encoded as one 8x8 block, or two 8x4 blocks, or two 4x8 blocks, or four 4x4 blocks. The advantage of using a small set of possible partitions to define blocks in current video and image coding standards is that the side information rate (can) for transmitting fine-segment information can be kept small. However, its disadvantage is that the bit rate required to transmit prediction parameters for that block becomes significant, as explained below. The side information rate used to transmit prediction information typically represents the majority of the total bit rate of a block. When this side information is reduced, for example, by using a larger block size, coding efficiency can be increased. Compared to H.264, it is also possible to increase the set of partition patterns supported. For example, the partition patterns shown in Figure 8 can be provided to square blocks of all sizes (or selected sizes). A true image or picture in a video sequence includes objects of arbitrary shape with specific properties. As an example, these objects or object portions are characterized by a unique texture structure or unique movement. Typically, the same set of prediction parameters can be applied to these objects or object portions. However, object boundaries typically do not coincide with the possible block boundaries of large prediction blocks (e.g., 16x16 giant blocks in H.264). An encoder typically determines finer partitions (within a finite set of possibilities), which minimizes a specific bit rate / distortion cost metric. For arbitrarily shaped objects, this can result in a large number of small blocks. This means that realism is maintained as more partition patterns (as described above) are provided. It should be noted that the number of partition patterns should not be too large, as there will then be a lot of side information and / or encoder / decoder complexity to transmit and process these patterns. Therefore, arbitrarily shaped objects typically result in a large number of small blocks due to partitioning. And because each of these small blocks is associated with a set of prediction parameters that needs to be transmitted, the side information rate can become a major part of the total bit rate. However, because many small blocks still represent regions of the same object or part of the same object, the prediction parameters for some of the resulting blocks are the same or very similar. Intuitively, coding efficiency can be increased when the grammar extends in a way that not only allows for the fine division of a block, but also allows for the merging of two or more blocks resulting from the fine division. Thus, we obtain a group of blocks encoded with the same prediction parameters.The prediction parameters for this block group only need to be encoded once. In the example of Figures 1-7 above, for example, if merging occurs, the encoding parameters for the current block 40 are not sent. That is, the encoder does not send the encoding parameters associated with the current block, and the decoder does not expect bitstream 30 to contain the encoding parameters for the current block 40. Instead, according to a particular embodiment, only refinement information can be transmitted for the merged current block 40. The candidate set and its reduction, as well as the merging and other decisions, are also performed for the other coded blocks 40 of image 20. These coded blocks typically form coded block groups and coding chains, where the encoding parameters for these groups are sent only once in the bitstream.

[0068] If the bit rate saved by reducing the number of encoded prediction parameters is greater than the additional bit rate spent on encoding merged information, then the merging described above leads to increased coding efficiency. It should be further mentioned that the syntax extension (for merging) described above provides the encoder with additional degrees of freedom to choose to segment an image or planar group into blocks without introducing redundancy. The encoder is not limited to first segmenting and then checking whether some of the resulting blocks have the same set of prediction parameters. In a simpler alternative, the encoder could first decide on segmentation, as with current coding techniques. Then, for each block, it would check whether merging with one of its neighboring blocks (or related previously decided block groups) reduces the bit rate / distortion cost measure. In this case, the prediction parameters associated with the new block group can be re-estimated (e.g., by performing a new move search) or the previously determined prediction parameters for the current block and neighboring blocks or block groups can be estimated for the block group. An encoder can also directly examine the pattern (or a subset thereof) provided by the combination of cutting and merging; that is, movement estimation and pattern determination can be performed according to the shape generated above. This merging information can be transmitted on a block basis. Effectively, this merging can also be interpreted as inference about the prediction parameters of the current block, wherein these inferred prediction parameters are set equal to the prediction parameters of one of the adjacent blocks.

[0069] For modes other than hop, an additional flag, similar to a CBF, is required to ensure that no residual signal is transmitted. Currently, the H.264 video coding standard offers two different hop / direct modes, selected based on the image level: temporal direct mode and spatial direct mode. Both direct modes are only applicable to B-images. In temporal direct mode, the reference index for reference image list 0 is set to 0, and the reference index for reference image list 1, as well as the motion vectors for both reference lists, are derived based on the motion data of the same location block in the first reference image of reference image list 1. Temporal direct mode uses motion vectors from temporally same location blocks and adjusts the motion vector size based on the time distance between the current and same location blocks. In spatial direct mode, the reference indices and motion vectors for both reference image lists are derived essentially based on spatially adjacent motion data. The reference index is selected as the smallest of the corresponding reference indices among spatial neighbors, and each motion vector component is set to the median of the corresponding motion vector components among spatial neighbors. Skip mode can only be used to encode 16x16 giant blocks in H.264 (in P and B images), while direct mode can be used to encode either 16x16 giant blocks or 8x8 sub-giant blocks. Compared to direct mode, if merging is applied to the current block, all prediction parameters can be copied from the current block to the block it is merged with. Merging can also be applied to any block size that results in the aforementioned more flexible partitioning patterns, where all samples of a pattern are predicted using the same prediction parameters.

[0070] The basic concept described in this invention is to reduce the bit rate required to transmit the CBF flag by using combination merging and CBF flags. If a sample set uses merging and no residual data is transmitted, then a flag is transmitted to signal both.

[0071] To reduce the information rate in image and video coding applications, a specific set of samples (which may represent rectangular or square blocks, regions of arbitrary shape, or any other set of samples) is typically associated with a specific set of coding parameters. For each of these sample sets, the coding parameters are contained in the bitstream. These coding parameters may represent prediction parameters, indicating how the corresponding sample is predicted using previously encoded samples. The segmented image sample arrays becoming sample sets can be fixed using a syntax or can be signaled using corresponding segmentation information within the bitstream. Multiple segmentation patterns are allowed for a block. The coding parameters for a sample set are sent in a predetermined order, using the syntax provided. They can be signaled for merging with one or more other sample sets (e.g., for prediction purposes) to form a current sample set family. The possible set of values ​​corresponding to the merged information can be adapted to the segmentation pattern being used, such that a particular segmentation pattern cannot be represented using a combination of other segmentation patterns and corresponding merged data. The coding parameters for a sample set family only need to be sent once. In addition to the prediction parameters, residual parameters (e.g., transformation and quantization side information and transformation coefficient levels) can be sent. If the current sample set is merged, side information describing the merging procedure is sent. This side information will be further referred to as merge information. Embodiments of the present invention illustrate a concept that combines the transmission of merge information with the transmission of encoded block flags (indicating whether residual data is present for a block).

[0072] In one specific embodiment, the merge information includes a combined flag, called `mrg_cbf`, which is equal to 1 if the current sample set is merged and no residual data is sent. In this case, no further encoded parameters or residual parameters are sent. If the combined `mrg_cbf` flag is equal to 0, it indicates whether another flag applied to the merge is encoded. Further flags indicating no residual parameters are sent are encoded. In CABAC and Context-Adaptive VLC, the context for probabilistic derivation (and VLC list switching) of syntax elements related to the merge information can be selected as a function of previously sent syntax elements and / or decoded parameters (e.g., the combined `mrg_cbf` flag).

[0073] In a preferred embodiment, the merging information including the combined mrg_cbf flag is encoded before the encoding parameters (e.g., prediction information and segmentation information).

[0074] In a preferred embodiment, the merged information including the combined mrg_cbf flag is encoded after a subset of encoded parameters (e.g., prediction information and fine-segmentation information). For each sample set, the merged information can be encoded after the fine-segmentation information is generated.

[0075] In the embodiments illustrated below with reference to Figures 11 to 13, mrg_cbf is referred to as skip_flag. Generally, mrg_cbf may be referred to as merge_skip to demonstrate that it is another skip version regarding block merging.

[0076] The preferred embodiments described below are for sample sets representing rectangular and square blocks, but they can be directly extended to regions of any shape or other sample sets. The preferred embodiments describe combinations of merging mechanism syntax elements and syntax elements indicating no residual data. Residual data includes residual side information and transformation coefficient levels. For all preferred embodiments, no residual data is indicated using a Coded Block Flag (CBF), but it can also be represented using other methods or flags. A CBF equal to 0 indicates the case where no residual data is transmitted. 1. Combination of the merged flag and the CBF flag

[0077] Below, the auxiliary merge actuation flag is called mrg, and later, for figures 11-13, the same one is called merge_flag. Similarly, the merge index is currently called mrg_idx, and merge_idx will be used later.

[0078] The possible combinations of the merge flag and the CBF flag using a single syntax element are described in this section. The description of these possible combinations described below can be transferred to the illustrations in Figures 1 through 6 above, which illustrate any of them.

[0079] In a preferred embodiment, up to three syntax elements are sent to indicate merge information and CBF.

[0080] The first syntax element, hereinafter referred to as `mrg_cbf`, indicates whether the current set of samples is to be merged with another set of samples and whether all corresponding CBFs are equal to 0. The `mrg_cbf` syntax element can only be encoded if the derived set of a candidate sample set is not empty (after removing candidates that can be merged without utilizing different split patterns). However, its original value, which guarantees that the list of mergeable candidates will not disappear, ensures that at least one or even at least two mergeable candidates are available. In a preferred embodiment of the invention, the `mrg_cbf` syntax element is encoded if the derived set of a candidate sample set is not empty, as described below. – If the current block is merged and CBF is equal to 0 for all components (e.g., luminance and dichromatic components), then the mrg_cbf syntax element is set to 1 and encoded. Otherwise, the mrg_cbf syntax element is set to 0 and encoded. The numerical values ​​0 and 1 of the mrg_cbf syntax elements can also be toggled.

[0081] The second syntax element, further called `mrg`, indicates whether the current sample set is to be merged with another sample set. If the `mrg_cbf` syntax element is equal to 1, then the `mrg` syntax element is not encoded and is inferred to be equal to 1. If the `mrg_cbf` syntax element is not presented (because the derived candidate sample set is empty), then the `mrg` syntax element is also not presented, but is inferred to be equal to 0. However, its original value, which guarantees that the list of mergeable candidates will not disappear, ensures that at least one or even at least two mergeable candidates are available.

[0082] The third syntax element, further referred to as `mrg_idx`, is encoded only if the `mrg` syntax element is equal to 1 (or inferred to be equal to 1), indicating which set of candidate samples is used for merging. In a preferred embodiment, the `mrg_idx` syntax element is encoded only if the derived set of a candidate sample set contains more than one candidate sample set. In a further preferred embodiment, the `mrg_idx` syntax element is encoded only if at least two sets of samples in the derived set of a candidate sample set are associated with different encoding parameters.

[0083] It should be mentioned that the candidate list for merging can be fixed to improve analytical productivity and robustness against information loss by decoupling analysis and reconstruction. For greater accuracy, decoupling can be achieved using a fixed configuration of list items and codewords. This eliminates the need for a fixed list length. However, using additional candidates while fixing the list length allows compensation for the coding efficiency loss of a fixed (longer) codeword. Therefore, as mentioned above, the merge index syntax element can be sent only if the candidate list contains more than one candidate. However, this requires deriving the list before analyzing the merged index, preventing these two procedures from being performed in parallel. To allow for increased analytical productivity and make the analysis more robust against transmission errors, this dependency can be removed by using a fixed codeword and a fixed number of candidates for each index value. If this number cannot be achieved using candidate selection, auxiliary candidates may be derived to complete the list. These additional candidates may include so-called combined candidates, whose shift parameters from candidates that may differ from those already in the list, and zero shift vectors are established.

[0084] In a preferred embodiment, the merged information for a sample set is encoded after a subset of the prediction parameters (or, more generally, specific encoded parameters associated with the sample set) has been sent. This subset of prediction parameters may include one or more reference image indices, one or more motion parameter vector components, or a reference image index and one or more motion parameter vector components, etc.

[0085] In a preferred embodiment, the `mrg_cbf` syntax element of the merged information is encoded only for a reduced set of delimiter patterns. One possible set of delimiter patterns is presented in Figure 8. In a preferred embodiment, this reduced set of delimiter patterns is limited to one and corresponds to a first delimiter pattern (top-left of the list in Figure 8). As an example, `mrg_cbf` is encoded only if a block is not further delimited. As a further example, `mrg_cfb` may be encoded only for square blocks.

[0086] In another preferred embodiment, the `mrg_cbf` syntax element of the merge information is encoded only for one block of a partition, where this partition is one of the possible partition patterns shown in Figure 8, for example, a partition pattern with four left-bottom blocks. In a preferred embodiment, if more than one block is merged with one of these partition patterns, the merge information of the first merged block (in decoding order) includes `mrg_cbf` syntax elements for complete partitioning. For all other blocks subsequently decoded with the same partition pattern, the merge information only includes `mrg` syntax elements indicating whether the current sample set is merged with another sample set. Information on whether residual data is present is inferred from the `mrg_cbf` syntax elements encoded in the first block.

[0087] In a further preferred embodiment of the invention, merging information for a sample set is encoded before prediction parameters (or, more generally, specific encoding parameters associated with the sample set). This merging information, including the syntax elements `mrg_cbf`, `mrg`, and `mrg_idx`, is encoded as described in the first preferred embodiment above. These prediction or encoding parameters and these residual parameters are sent only if the merging information signal indicates that the current sample set is not being merged with another sample set and the CBF, for at least one component, is equal to 1. In a preferred embodiment, if the `mrg_cbf` syntax element indicates that the current block is being merged and the CBF for all components is equal to 0, then no further signaling is required for this current block after the merging information.

[0088] In another preferred embodiment of the invention, the syntax elements mrg_cbf, mrg, and mrg_idx are combined and encoded into one or two syntax elements. In one preferred embodiment, mrg_cbf and mrg are combined into a syntax element that indicates any of the following: (a) the block is merged and it does not contain residual data, (b) the block is merged and it contains residual data (or may contain residual data), (c) the block is not merged. In another preferred embodiment, the syntax elements mrg and mrg_idx are combined into a syntax element. If N is the number of merge candidates, the combined syntax element indicates one of the following: the block is not merged, the block is merged with candidate 1, the block is merged with candidate 2, ..., the block is merged with candidate N. In a further preferred embodiment of the invention, the syntax elements mrg_cfb, mrg, and mrg_idx are combined into a syntax element that indicates one of the following (N is the number of candidates): the block is not merged, the block is merged with candidate 1 and does not contain residual data, the block is merged with candidate 2 and does not contain residual data, ..., the block is merged with candidate N and does not contain residual data, the block is merged with candidate 1 and contains residual data, the block is merged with candidate 2 and contains residual data, ..., the block is merged with candidate N and contains residual data. The combined syntax element can be sent using variable-length code, arithmetic encoding, or binary arithmetic encoding using any specific binary mechanism. 2. Combinations of merged flags and CBF flags, and jump / direct patterns.

[0089] Skip / direct mode can be supported for all or only specific block sizes and / or block shapes. In the skip / direct mode extension described in the current H.264 video coding standard, a set of candidate blocks is used in skip / direct mode. The difference between skip and direct modes is whether residual parameters are transmitted. These skip and direct parameters (e.g., for prediction) can be inferred from any of the corresponding candidates. A candidate index is encoded, indicating which candidate is used to infer these encoded parameters. If multiple predictions are combined to form the final prediction signal for the current block (such as a bidirectional prediction block used in an H.264 B-frame), each prediction may involve different candidates. Therefore, a candidate index can be encoded for each prediction.

[0090] In a preferred embodiment of the invention, the skip / direct candidate list may include different candidate blocks from the merged candidate list. An example is shown in Figure 10. This candidate list includes the following blocks (currently represented by Xi): l Movement vector (0,0) l Middle (between the left, top, and corners) left block (Li) The upper block (Ai) l Corner blocks (in order: top right (Ci1), bottom left (Ci2), top left (Ci3)) l is different, but the same location block in the previously encoded image The following symbols are used to illustrate the following embodiments: `let_mvp_ori` is the candidate set used in one of the skip / direct modes. This set consists of {middle, left, top, corner, same position}, where `middle` is the middle value (the middle value of the ordered set of left, top, and corner), and `same position` is the closest reference frame given (or the first reference image in one of the reference image lists), and the corresponding movement vector is measured according to the time distance. Movement vectors with two components equal to 0 can be additionally inserted into the candidate list, for example, if there are no left, no top, and no corner blocks. l set_mvp_comb is a subset of set_mvp_ori.

[0091] In a preferred embodiment, both skip / direct mode and block merging mode are supported. Skip / direct mode uses the original candidate set, set_mvp_ori. Merging information for block merging mode may include the combined mrg_cbf syntax elements.

[0092] In another embodiment, both skip / direct mode and block merging mode are supported, but skip / direct mode uses a modified candidate set, `set_mvp_comb`. This modified candidate set can be a specific subset of the original set `set_mvp_ori`. In a preferred embodiment, the modified candidate set consists of corner blocks and blocks in the same position. In another preferred embodiment, the modified candidate set consists only of blocks in the same position. Further subsets are also possible.

[0093] In a preferred embodiment, the merge information containing the mrg_cbf syntax element is encoded before the skip mode related parameters.

[0094] In a preferred embodiment, the skip mode related parameters are encoded before the merge information containing the mrg_cbf syntax elements.

[0095] According to another embodiment, the direct mode may not be activated (or even presented) and block merging has an extended set of candidates for skipping modes replaced by mrg_cbf.

[0096] In a preferred embodiment, the candidate list for block merging may contain different candidate blocks. An example is shown in Figure 10. This candidate list contains the following blocks (currently represented by Xi): l Movement vector (0,0) left block (Li) The upper block (Ai) l is different, but the same location block in the previously encoded image l Corner blocks (in order: top right (Ci1), bottom left (Ci2), top left (Ci3)) l Combined bidirectional predictive candidates l Non-scale adjusted bidirectional prediction candidates

[0097] It should be noted that the block merging candidate positions can be the same as the inter-frame - predicted MVP list in order to save memory access.

[0098] Furthermore, the list can be "fixed" as described above to decouple parsing and reconstruction, thereby improving parsing and making it more robust to information loss. 3. CBF encoding

[0099] In a preferred embodiment, if the `mrg_cfb` syntax element is equal to 0 (its message is that the block is not merged or contains non-zero residual data), a flag is sent indicating whether all components of the residual data (e.g., luminance and chroma components) are zero. If `mrg_cfb` is equal to 1, this flag is not sent. In a particular configuration, if `mrg_cfb` is equal to 0, this flag is not sent and the `mrg` syntax element indicates that the block is merged.

[0100] In another preferred embodiment, if the mrg_cfb syntax element is equal to 0 (its message is that the block is not merged or contains non-zero residual data), a syntax element is sent for each component and the message is sent for whether the residual data of that component is zero.

[0101] Different network patterns can be used with mrg_cbf.

[0102] Therefore, the apparatus for encoding an image described in the above embodiment includes: A fine segmenter, configured to finely segment the image into a sample set of samples; A merger configured to merge such sample sets into a mutually exclusive set of one or more sample sets; An encoder configured to encode an image by varying encoding parameters across the image in units of mutually exclusive sets of samples, wherein the encoder is configured to encode the image for a predetermined set of samples by predicting the image and encoding a prediction residual; and A bitstream generator is configured to embed the prediction residual and the encoding parameters together with one or more syntax elements for at least one subset of the sample sets into a bitstream, the syntax elements indicating whether each sample set is merged with another sample set to form one of the mutually exclusive sets.

[0103] Furthermore, an apparatus for decoding a bitstream in which an image is encoded has been described, comprising: A fine segmenter, configured to finely segment the image into a set of multiple samples; A merger configured to merge such sample sets into mutually exclusive sets, each of which is one or more sample sets; A decoder is configured to decode an image by varying the encoding parameters across the image in units of mutually exclusive sets of samples, wherein the decoder is configured to decode the image by predicting the image for a predetermined set of samples, decoding a prediction residual for the predetermined set of samples, and combining the prediction residual with a prediction generated by predicting the image itself; An extractor configured to extract the prediction residual and the coding parameters from the bitstream together with one or more syntax elements for at least a subset of the sample sets, the syntax elements signaling whether each sample set will be merged into one of the mutually exclusive sets along with another sample set, wherein the merger is configured to perform the merging in response to the syntax element.

[0104] One of the possible states of one or more grammatical elements will be signaled regarding the individual sample set being merged into one of the mutually exclusive sets along with another sample set, and the absence of a predictive residual being encoded and embedded in the bitstream.

[0105] The extractor is also configured to extract fine segmentation information from the bit stream, and the fine segmenter is configured to segment the image into a sample set in response to the fine segmentation information.

[0106] The extractor and the merger are configured to sequentially step through the sample sets according to a sample set scanning order, and for a current sample set, Extract the first binary syntax element (mrg_cbf) from this bitstream; If the first binary syntax element is determined to be a first binary state, then by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with the mutex set, the current sample set is merged into one of the mutex sets, skipping the extraction of the prediction residual for the current sample set and stepping to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be a second binary state, then extract a second syntax element (mrg, mrg_idx) from the bitstream; and According to the second grammatical element, the current sample set is merged into one of the mutually exclusive sets by inferring that the encoding parameters of the current sample set are equal to the encoding parameters of the mutually exclusive sets, or the encoding parameters of the current sample set are extracted, the extraction relating to at least one further grammatical element of the prediction residual for the current sample set.

[0107] For at least one subset of such sample sets, one or more syntactic elements are also communicated, if each sample set is to be merged into any set of mutually exclusive sets along with another sample set, then which set of predetermined candidate sample sets adjacent to that sample set will be merged with.

[0108] The extractor is also configured so that if one or more syntax elements do not communicate with respect to their respective sample sets, they will be merged into either mutually exclusive set along with the other sample set. Then one or more further syntax elements (skip / direct mode) are extracted from the bitstream to signal whether, and which of the further sets of predetermined candidate sample sets from the neighboring individual sample sets, at least a portion of the encoding parameters for each sample set will be predicted.

[0109] In this case, the set of predetermined candidate samples and the further set of predetermined candidate samples may be a set of a few predetermined candidate sample sets and the further set of predetermined candidate sample sets that are mutually exclusive or intersecting.

[0110] The extractor is also configured to extract fine-segmentation information from the bitstream, and the fine-segmentation unit is configured to hierarchically segment the image into sample sets in response to the fine-segmentation information. The extractor is configured to sequentially step through a parent sample set and its child sample sets, wherein the parent sample set consists of the sample sets from which the image has been finely segmented, and for a given child sample set... Extract the first binary syntax element (mrg_cbf) from this bitstream; If the first binary syntax element is determined to be a first binary state, then by inferring that the encoding parameter for the current subsystem sample set is equal to the encoding parameter associated with the mutual exclusion set, the current subsystem sample set is merged into one of the mutual exclusion sets, skipping the extraction of the prediction residual for the current subsystem sample set, and stepping to the next subsystem sample set; If the first binary syntax element is determined to be a second binary state, then extract a second syntax element (mrg, mrg_idx) from the bitstream; and According to the second grammatical element, by inferring that the encoding parameters of the current sub-family sample set are equal to the encoding parameters of the associated population, the current sub-family sample set is merged into one of the populations, or the encoding parameters of the current sub-family sample set are extracted, which extracts at least one further grammatical element related to the prediction residual of the current sub-family sample set, and then proceeds to the next sub-family sample set. For the next subsystem sample set, if the first binary syntax element of the current subsystem sample set is determined to be the first binary state, then skip the extraction of the first binary syntax element and start extracting the second syntax element instead. If the first binary syntax element of the current subsystem sample set is determined to be the second binary state, then extract the first binary syntax element.

[0111] Suppose, for example, a kinship set (CU) is split into two sub-kinship sets (PU). Then, if for the first PU, the first binary syntax element (merge_cbf) has a first binary state: 1) the first PU uses merging and the first and second PUs (the entire CU) have no residual data in the bit stream, and 2) for the second PU, the second binary syntax element (merge_flag, merge_idx) is signaled. However, if the first binary syntax element has a second binary state for the first PU, then 1) for the first PU, the second binary syntax element (merge_flag, merge_idx) is signaled, and residual data is in the bit stream; however, 2) for the second PU, the first binary syntax element (merge_cbf) is signaled. Therefore, merge_cbf can also be signaled at a PU level, that is, for sequential sub-kinship sets, if for all previous sub-kinship sets, merge_cbf is in the second binary state. If, for a sequential set of subsystem samples, merge_cbf is in the first binary state, then all subsystem samples below this set have no residual data in the bitstream. For example, for a CU split into, say, 4 PUs, merge_cbf might be in the first binary state for the second PU, indicating that the third and fourth PUs in the encoding order have no residual data in the bitstream, but the first PU may have or could have it.

[0112] The first and second binary syntax elements can be encoded using context-adaptive variable-length encoding or context-adaptive (binary) arithmetic encoding, and the context used to encode these syntax elements is derived from the values ​​of these syntax elements in previously encoded blocks.

[0113] As illustrated in another preferred embodiment, the syntax element `merge_idx` can only be sent if the candidate list contains more than one candidate. This needs to be derived from the list before analyzing the merged indexes to prevent these two procedures from being performed in parallel. To allow for increased analysis productivity and make the analyzer more robust to transmission errors, this correlation may be removed by using a fixed codeword for each index value and a fixed number of candidates. If this number cannot be reached by selecting a candidate, auxiliary candidates may be derived to complete the list. These additional candidates may include so-called merged candidates, which can be established from the movement parameters of possibly different candidates already in the list, as well as zero movement vectors.

[0114] In another preferred embodiment, the syntax used to signal which block of the candidate set is simultaneously tuned to the encoder and decoder. If, for example, three options for merging blocks are given, those three options appear only in the syntax and are considered for entropy encoding. The probability of all other options is considered to be 0, and the entropy encoder and decoder are tuned simultaneously in the encoder and decoder.

[0115] Because the prediction parameters inferred from the merge procedure results can represent the complete set of prediction parameters associated with a block, or they can represent a subset of these prediction parameters (e.g., a hypothetical prediction parameter for a block in which multiple hypothetical predictions are used).

[0116] In a preferred embodiment, the syntax elements of the merged information are entropy-encoded using a contextual pattern.

[0117] The method of transferring the above embodiments to a specific syntax is illustrated below with reference to the figures below. In particular, figures 11-13 show different parts of a syntax that takes advantage of the above embodiments. In particular, according to the embodiment described below, image 20 is first split upwards into coding tree blocks, the image content of which is encoded using the syntax coding_tree shown in figure 11. As shown therein, for entropy_coding_mode_flag=1, which relates to, for example, context-adaptive binary arithmetic coding or another specific entropy coding mode, the current quartic subdivision of the coding tree block is signaled within the syntax part coding_tree via a flag called split_coding_unit_flag in flag 400. As shown in figure 11, according to the embodiment described hereafter, the root block is subdivided to the depth shown in figure 7a—first sequentially signaled using split_coding_unit_flag. Whenever a leaf node is reached, that same one represents a coding unit, which is directly encoded using the syntax function coding_unit. This can be seen from Figure 11, when looking at the if-clause at 402, it checks whether the current split_coding_unit_flag is set. If yes, the function coding_tree is called repeatedly, causing further split_coding_unit_flag to be sent / extracted in the encoder and decoder respectively. If not, that is, if split_coding_unit_flag=0, then the current child block of the root block 200 in Figure 7a is a leaf block and in order to encode this coding unit, the function coding_unit in Figure 10 is called at 404.

[0118] In the currently described embodiment, the above selection is used, according to which merging is only used for images where the inter-frame prediction mode is available. That is, intra-frame encoded slices / images are not merged. This is as can be seen from Figure 12, where the skip_flag is sent only at 406 if the slice type is not equal to the intra-frame image slice type, that is, if the current slice to which the current encoding unit belongs allows separation by inter-frame encoding. Merging is only related to the prediction parameters of inter-frame prediction according to this embodiment. According to this embodiment, the skip_flag is signaled for the entire encoding unit 40, and if the skip_flag is equal to 1, this flag value is also signaled to the decoder: 1) The current coding unit separation mode is a non-separation mode. According to this mode, identical units are not separated, and the unit itself serves as the unique separator for that coding unit. 2) Currently, the encoding unit / separator is inter-frame encoding, that is, it is assigned to the inter-frame encoding mode. 3) Currently, encoding units / delimiters accept merging, and 4) Currently, the encoding unit / separator accepts skip mode, that is, it has a passive skip mode.

[0119] Therefore, if `skip_flag` is set, the function `prediction_unit` is called at 408, indicating that the current encoding unit is the prediction unit. However, this is not the only possibility for switching to the merge selection. Instead, if `skip_flag` for the overall encoding unit is not set at 406, the prediction type for encoding units that are not within the frame—image slices—is signaled using the syntax element `pred_type` at 410, depending on which, in the case where the current encoding unit is not further segmented, for example, at 412, the function `prediction_unit` is called for any segmentation of the current encoding unit. In Figure 12, only four different segmentation selections are shown, but the other segmentation selections shown in Figure 8 are also available. Another possibility is that the segmentation selection `PART_NxN` is unavailable, but the others are available. The association between the names of the segmentation patterns used in Figure 12 and the segmentation selections shown in Figure 8 is indicated by the respective subscripts under each segmentation selection in Figure 8. Please note that the predictive type syntax element `pred_type` not only signals the predictive pattern, i.e., intra-frame or inter-frame encoding, but also signals the delimiters in the inter-frame encoding pattern. The inter-frame encoding pattern is discussed further. The function `prediction_unit` is called for each delimiter, for example, delimiters 50 and 60 in the encoding order described above. The function `prediction_unit` begins at 414 by checking `skip_flag`. If `skip_flag` is set, `merge_idx` follows at 416. The check in step 414 checks whether the `skip_flag` signaled at 406 regarding the overall encoding unit has been set. If not, a `merge_flag` is again signaled at 418, and if the latter is set, a `merge_idx` follows at 420, indicating the current merge candidate for the delimiter. Again, the merge_flag is signaled for the current segment only in 418 if the current prediction mode of the current coding unit is an inter-frame prediction mode (ref 422). That is, if the skip_flag is not set, the prediction mode is signaled via pred_type in 410, where, for each prediction unit, assuming that the inter-frame coding mode is activated by pred_type signaling (ref 422), a merge-specific flag, i.e., merge_flag, is sent for each subsequent segment if merging is activated for each segment via the merge index merge_idx.

[0120] As can be seen from Figure 13, when the prediction parameters used for the current prediction unit are sent at 424, according to this embodiment, merging is only performed in cases where the prediction parameters are not used for the current prediction unit. That is, because merging is not activated by skip_flag, nor by each separate merge_flag.

[0121] As indicated above, skip_flag=1 simultaneously indicates that no residual data is sent. This can be derived from the fact that residual data transmission in Figure 12, 426, occurs only when skip_flag equals 0, and also from the fact that residual data transmission is checked immediately after its transmission if the skip_flag state is within the else selection of if-clause 428.

[0122] So far, the embodiments in Figures 11 to 13 have been described only under the assumption that `entropy_coding_mode_flag` equals 1. However, the embodiments in Figures 11 to 13 also include embodiments of the above embodiments in the case where `entropy_coding_mode_flag` = 0, in which another entropy coding mode is used to entropy-encode the syntax elements, for example, variable-length coding, and, for more precision, context-adaptive variable-length coding, for example, especially, the possibility of simultaneous signaling merging actuation and signaling skipping mode exists in the above embodiments, according to which, generally, the signaling state is only one of more than two sets of states for each syntax element. This will now be described in more detail. However, it should be noted that the possibility of switching between the two entropy coding modes is optional, and therefore, different embodiments can be easily derived from Figures 11 to 13 by allowing only one of the two entropy coding modes.

[0123] For example, see Figure 11. If `entropy_coding_mode_flag` equals 0 and the `slice_type` syntax element indicates that the current root block belongs to an inter-frame coded slice, i.e., the inter-frame coding mode is available, then a syntax element `cu_split_pred_part_mode` is sent at 430, and this syntax element indicates, as indicated by its name, further subdivision information of the current coded unit, actuation or deactivation of skip modes, merging, and actuation or deactivation of prediction modes, as well as individual splitting information. See Listing 1: List 1 [cu_split_pred_] [part_mode] split_coding_ unit_flag skip_flag [merge_flag] Predictive patterns Separation mode 0 1 - - - - 1 0 1 - MODE_SKIP PART_2Nx2N 2 0 0 1 MODE_INTER PART_2Nx2N 3 0 0 0 MODE_INTER PART_2Nx2N 4 0 - - MODE_INTER PART_2NxN 5 0 - - MODE_INTER PART_Nx2N 6 0 - - MODE_INTRA PART_2Nx2N

[0124] Listing 1 indicates the importance of the possible states of the syntax element `cu_split_pred_part_mode` in cases where the current encoding unit is not the smallest size in the current root block's quartet subdivision. These possible states are listed in the outermost left row of Listing 1. Since Listing 1 indicates cases where the current encoding unit is not the smallest size, there is a state `cu_split_pred_part_mode`, i.e., state 0, which indicates that the current encoding unit is not an actual encoding unit but needs to be further subdivided into four units, which are then traversed in depth-first traversal order, again as described in the call function `coding_tree` in 432. That is, `cu_split_pred_part_mode=0` indicates that the current quartet subdivision unit of the current root block will be further subdivided into four smaller units, i.e., `split_coding_unit_flag=1`. However, if cu_split_pred_part_mode is assumed to be any other possible state, then split_coding_unit_flag = 0 and the current unit forms a leaf block of the current root block, i.e., one coding unit. In this case, one other possible state of cu_split_pred_part_mode represents the general signaling state described above, where the current coding unit is simultaneously signaled to accept merging and has a skip mode activated, indicated by skip_flag equal to 1 in the third row of Listing 1, while the current coding unit is simultaneously signaled not to undergo further splitting, i.e., PART_2Nx2N is selected as the splitting mode. cu_split_pred_part_mode also has a possible state where the signaling merge activation and skip mode are not activated. This is possible state 2, corresponding to skip_flag = 0 and merge_flag = 1, and no splitting mode is activated, i.e., PART_2Nx2N. That is, in this case, merge_flag is signaled beforehand instead of within the prediction_unit syntax. In the other possible states of cu_split_pred_part_mode, inter-frame prediction modes with other splitting modes are signaled, and these splitting modes now encode more than one splitting unit. List 2 cu_split_pred _part_mode split_coding_ unit_flag skip_flag [merge_flag] Predictive patterns Separation mode 0 0 1 - MODE_SKIP PART_2Nx2N 1 0 0 1 MODE_INTER PART_2Nx2N 2 0 0 0 MODE_INTER PART_2Nx2N 3 0 - - MODE_INTER PART_2NxN 4 0 - - MODE_INTER PART_Nx2N 5 (Remove symbol) 0 - - MODE_INTRA PART_2Nx2N MODE_INTRA PART_NxN MODE_INTER PART_NxN

[0125] Listing 2 shows the importance or syntax of the possible states of `cu_split_pred_part_mode` in the case of the current coding unit with the smallest possible size for the current root block's quartet subdivision. In this case, with `split_coding_unit_flag=0`, all possible states of `cu_split_pred_part_mode` correspond to no further subdivision. However, this possible state 0 signals `skip_flag=1`, that is, it signals that merging is triggered and the skip mode is active. Furthermore, it also signals that no splitting occurs, i.e., the split mode `PART_2Nx2N`. Possible state 1 corresponds to possible state 2 in Listing 1, and the same applies to possible state 2 in Listing 2, which corresponds to possible state 3 in Listing 1.

[0126] Although most of the functions and syntax have been described above in the embodiments shown in Figures 11-13, some further information is presented below.

[0127] `skip_flag[x0][y0]` equal to 1 indicates the current encoding unit (see Figure 40). When decoding P or B slices, no more syntax elements are parsed after `skip_flag[x0][y0]` except for the move vector predictor index (`merge_idx`). `skip_flag[x0][y0]` equal to 0 indicates that the encoding unit is not skipped. The array indices x0 and y0 indicate the position (x0, y0) of the top-left luminance sample of the considered encoding unit relative to the top-left luminance sample of the image (Figure 20).

[0128] When skip_flag[x0][y0] is not displayed, it will be inferred to be equal to 0.

[0129] As mentioned above, if skip_flag[x0][y0] equals 1, - The predicted pattern is inferred to be equal to MODE_SKIP - The partition pattern is inferred to be equal to PART_2Nx2N

[0130] `cu_split_pred_part_mode[x0][y0]` specifies `split_coding_unit_flag` and when the coding unit is not split. `skip_flag[x0][y0]` and `merge_flag[x0][y0]` are the prediction mode and splitting mode of the coding unit. The array indices `x0` and `y0` indicate the position (x0, y0) of the top-left luminance sample of the coding unit considered relative to the top-left luminance sample of the image.

[0131] The merge_flag[x0][y0] indicates whether the inter-frame prediction parameters for the current prediction unit (50 and 60 in the image, i.e., the separation within the coded unit 40) are inferred from the inter-frame prediction separation. The array index x0, y0 indicates the position (x0, y0) of the top-left luminance sample of the considered prediction block relative to the top-left luminance sample of the image.

[0132] merge_idx[x0][y0] indicates the merge candidate index in the merge candidate list, where x0, y0 indicates the position (x0, y0) of the top-left brightness sample of the considered prediction block relative to the top-left brightness sample of the image.

[0133] Although not explicitly indicated in the illustrations of Figures 11-13 above, the determination of merge candidates or the merge candidate list is exemplary in this embodiment, using not only the encoding or prediction parameters of spatially adjacent prediction units / segments, but also the formation of the candidate list using temporally adjacent temporally adjacent segmentation prediction parameters and previously encoded images. Furthermore, combinations of prediction parameters for spatially and / or temporally adjacent prediction units / segments are used and included in the merge candidate list. Of course, only a subset may be used. In particular, Figure 14 illustrates one possibility for determining spatial adjacency, i.e., spatially adjacent segmentations or prediction units. Figure 14 exemplifies a prediction unit or segment 60 and pixels B0 to B2 and A0 and A1 directly adjacent to the boundary 500 of segment 60. Specifically, B2 is the top-left pixel diagonally adjacent to segment 60, B1 is the top pixel vertically above segment 60 and adjacent to the top-right pixel, B0 is the top-right pixel diagonally to segment 60, A1 is the horizontal left pixel of segment 60 and adjacent to the bottom-left pixel, and A0 is the bottom-left pixel diagonally to segment 60. The segment containing at least one pixel of B0 to B2 and A0 and A1 forms a spatially adjacent segment, and its prediction parameters form a merge candidate.

[0134] To perform the above selection by removing those candidates that would make another separating pattern also available, the following function can be used: In particular, candidate N, i.e., the encoding / prediction parameters from the covered pixel N=(B0,B1,B2,A0,A1), i.e., the predicted unit / segment restoration at position (xN,yN), is removed from the candidate list if any of the following conditions are true (see PartMode in Figure 8 and the corresponding segment index (PartIdx) for retrieving individual segments within the encoding unit): - Currently, the prediction unit's segmentation mode is PART_2NxN with a segmentation index equal to 1, and the prediction units covering luminance positions (xP, yP–1) (PartIdx=0) and luminance positions (xN, yN) (Cand.N) have the same movement parameter: mvLX[xP,yP–1] == mvLX[xN,yN] refIdxLX[xP,yP–1] ==refIdxLX[xN,yN] predFlagLX[xP,yP–1] == predFlagLX[xN,yN] - Currently, the prediction unit segmentation mode is PART_Nx2N with PartIdx equal to 1, and the prediction units covering the luminance positions (xP–1, yP) (PartIdx=0) and luminance (xN, yN) (Cand.N) have the same shift parameter: mvLX[xP–1,yP] == mvLX[xN,yN] refIdxLX[xP–1,yP] == refIdxLX[xN,yN] predFlagLX[xP–1,yP] == predFlagLX[xN,yN] - Currently, the prediction unit's segmentation mode is PART_NxN, and PartIdx is equal to 3. Prediction units covering luminance positions (xP–1, yP) (PartIdx=2) and luminance positions (xP–1, yP–1) (PartIdx=0) have the same movement parameter. mvLX[xP–1,yP] == mvLX[xP–1,yP–1] refIdxLX[xP–1,yP] == refIdxLX[xP–1,yP–1] predFlagLX[xP–1,yP] == predFlagLX[xP–1,yP–1] Furthermore, the prediction units covering both the brightness position (xP, yP–1) (PartIdx=1) and the brightness position (xN, yN) (Cand.N) have the same shift parameter: mvLX[xP,yP–1] == mvLX[xN,yN] refIdxLX[xP,yP–1] == refIdxLX[xN,yN] predFlagLX[xP,yP–1] == predFlagLX[xN,yN] - Currently, the prediction unit's segmentation mode is PART_NxN, and PartIdx is equal to 3. Prediction units covering luminance positions (xP, yP–1) (PartIdx=1) and luminance positions (xP–1, yP–1) (PartIdx=0) have the same movement parameter. mvLX[xP,yP–1] == mvLX[xP–1,yP–1] refIdxLX[xP,yP–1] == refIdxLX[xP–1,yP–1] predFlagLX[xP,yP–1] == predFlagLX[xP–1,yP–1] Furthermore, the prediction units for both the brightness positions (xP–1, yP) (PartIdx=2) and (xN, yN) (Cand.N) are the same. mvLX[xP–1,yP] == mvLX[xN,yN] refIdxLX[xP–1,yP] == refIdxLX[xN,yN]

[0135] In this regard, note that the position (xP, yP) indicates the topmost pixel of the current separation / prediction unit. That is, according to the first item, all coding parameter candidates are checked, which are derived by directly taking the coding parameters of the adjacent prediction units, i.e., prediction units N. However, other coding parameter candidates can be checked in the same way regarding whether they are the same as the coding parameters of the individual prediction units, and merging them will result in a separation of another separation pattern also supported by the syntax. According to the embodiment just described above, the equality of coding parameters includes checking the equality of the movement vector, i.e., mvLX, the reference index, i.e., refIxLX, and the prediction flag predFlagLX, which indicates the parameters associated with the reference list X, i.e., the movement vector and the reference index, having X as 0 or 1, used for inter-frame prediction.

[0136] Please note that the previously mentioned possibility of removing candidate encoding parameters for adjacent prediction units / segments can also be applied to the case of supporting asymmetric segmentation patterns, as shown in the right half of Figure 8. In this case, pattern PART_2NxN can represent all horizontal segmentation patterns and PART_Nx2N can correspond to all vertical segmentation patterns. Furthermore, pattern PART_NxN can be excluded from supporting segmentation patterns or segmentation patterns, and in this case, only the initial secondary removal check will be performed.

[0137] Regarding the embodiments in Figures 11-14, it should also be noted that the predicted segments within the frame may be excluded from the candidate list, that is, their encoding parameters are, of course, not included in the candidate list.

[0138] Furthermore, it should be noted that the three sets of flags can be used for skip_flag, merge_flag, and merge_idx, respectively.

[0139] While some arguments are described in terms of apparatus, it should be understood that these arguments also represent descriptions of corresponding methods, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, arguments described in terms of method steps also represent descriptions of a corresponding block or item or a feature of a corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware device, such as a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, some or more of the most important method steps can be performed using these devices.

[0140] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. This implementation can be carried out using a digital storage medium having electronically readable control signals stored thereon, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which works (or is capable of working) with a programmable computer system, such that separate methods are performed. Therefore, the digital storage medium can be read by a computer.

[0141] Some embodiments of the present invention include a data carrier having electronically readable control signals, which can be used with a programmable computer system such that one of the methods described herein is performed.

[0142] Typically, embodiments of the present invention can be implemented as a computer program product having code that, when executed on a computer, is available for operation of one of the methods. This code, for example, can be stored on a machine-readable carrier.

[0143] Other embodiments include a computer program for performing one of the methods described herein, which is stored on a machine-readable carrier.

[0144] In other words, one embodiment of the method of the present invention is, therefore, when a computer program is executed on a computer, the computer program is a computer program of program code used to perform one of the methods described herein.

[0145] A further embodiment of one of the methods of the present invention is therefore a data carrier (or digital storage medium, or computer-readable medium) containing a computer program recorded thereon for performing one of the methods described herein. The data carrier, the digital storage medium, or the recording medium is a general physical entity and / or non-transitory.

[0146] A further embodiment of the method of the present invention is therefore a data stream or a signal sequence representing a computer program used to perform one of the methods described herein. This data stream or signal sequence, for example, can be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0147] A further embodiment includes a processing component, such as a computer or a programmable logic device, which is configured or adapted to perform one of the methods described herein.

[0148] A further embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0149] A further embodiment of the invention includes an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for one of the methods described herein to a receiver. The receiver is, for example, a computer, a mobile device, a memory device, or the like. The apparatus or system, for example, includes a file server for transmitting the computer program to the receiver.

[0150] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field-programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed using any hardware device.

[0151] The embodiments described above are merely illustrative of the principles of the invention. Those skilled in the art will understand that various modifications and variations can be made to the configuration of the invention and the details described herein. Therefore, it is intended to be limited only to the scope of the pending patent application and not to the specific details described and set forth in the embodiments herein.

[0152] From the above discussion, it will be understood that the present invention can be embodied in various embodiments, including but not limited to the following:

[0153] Example 1: An apparatus configured to decode a bitstream encoded from an image, wherein the sample array of the image is divided into sample sets associated with each of the respective encoding parameters, the apparatus being configured such that, for a current sample set, generally responding to a flag within the bitstream, a first determination is made regarding whether the encoding parameters associated with the current sample set will be set based on a merge candidate or obtained from the bitstream, and a second determination is made regarding whether the current sample set of the image will be reconstructed solely based on a prediction signal associated with the encoding parameters associated with the current sample set, without any residual data, or will be reconstructed using residual data within the bitstream by refining the prediction signal associated with the encoding parameters associated with the current sample set.

[0154] Example 2: The apparatus according to Example 1, wherein the apparatus is configured such that the decoder is: If the flag signaling within the bitstream is associated with the encoding parameters of the current sample set, these parameters will be set based on a merge candidate. The encoding parameters associated with the current sample set are obtained by setting encoding parameters based on a merged candidate, and the current sample set of the image is reconstructed based solely on a prediction signal according to these encoding parameters without any residual data. If the flags within the bitstream are not associated with the current sample set, then these encoding parameters will be set based on a merge candidate. Then it responds to another flag within that bitstream, such that the device responds according to that other flag. By setting encoding parameters based on a merge candidate to obtain encoding parameters associated with the current sample set, residual data for the current sample set is obtained from the bitstream, and the current sample set of the image is reconstructed based on the prediction signal and the residual data. The encoding parameters associated with the current sample set are extracted from the bit stream, residual data for the current sample set is obtained from the bit stream, and the current sample set of the image is reconstructed based on the prediction signal and the residual data.

[0155] Example 3: A decoder according to Example 1 or 2, wherein the decoder is configured such that: If the flag signaling within the bitstream is associated with the encoding parameters of the current sample set, these parameters will be set based on a merge candidate. Then, one of the support partition patterns provided for the current sample set is pre-defined and applied to the current sample set. If the flags within the bitstream are not signaled to the encoding parameters of the current sample set, they will be set based on a merge candidate. The response is to transmit segmentation information within the bitstream for use with one of the supporting segmentation patterns in the current sample set.

[0156] Example 4: A decoder according to Example 3, wherein one of the predetermined modes among the supported separation patterns is a non-separated mode and the decoder is configured such that the decoder: If the flag signaling within the bitstream is associated with the encoding parameters of the current sample set, these parameters will be set based on a merge candidate. The encoding parameters associated with the current sample set are obtained by setting encoding parameters based on a merged candidate, and the current sample set of the image is reconstructed based solely on a prediction signal according to these encoding parameters without any residual data. If the flags within the bitstream are not associated with the current sample set, these encoding parameters will be set according to a merge candidate. The response is to transmit segmentation information within the bitstream for use with one of the supporting segmentation patterns in the current sample set. Then, residual data for the current sample set is obtained from that bit stream. For each sample subset of the current sample set that supports the segmentation pattern, there is another flag in response within the bitstream, such that the decoder responds according to that other flag. By setting encoding parameters based on a merge candidate, encoding parameters associated with each individual sample subset can be obtained, or Extract the encoding parameters associated with each individual sample subset from the bitstream; and The current sample set of the image is reconstructed based on the coding parameters of each sample subset and a prediction signal of the residual data.

[0157] Example 5: A decoder according to Example 4, wherein the decoder is configured to: If the communication support partition pattern specifies that the current sample set is further partitioned into one of two or more sample subsets, Then move In an encoded order, all sample subsets except for the first sample subset of such sample subsets, From the combination and candidates of one of the individual sample subsets, A merge candidate having the same encoding parameters as any of the associated sample subsets will form one of the supporting partition patterns when merged with the respective sample subsets.

[0158] Example 6: An apparatus for decoding a bitstream in which an image is encoded, the apparatus comprising: A fine segmenter, configured to finely segment the image into a set of multiple samples; A merger configured to merge such sample sets into groups, each consisting of one or more sample sets; A decoder configured to decode an image using encoding parameters that vary across the image in units of a set of samples, wherein the decoder is configured to decode the image by predicting the image for a predetermined set of samples, decoding a prediction residual for the predetermined set of samples, and combining the prediction residual with a prediction generated by predicting the image itself; An extractor configured to extract the prediction residual and the coding parameters from the bitstream, together with one or more syntax elements for at least a subset of the sample sets, the syntax elements indicating whether each sample set will be merged into one of the populations along with another sample set, wherein the merger is configured to perform the merging in response to one or more syntax elements. One of the possible states of one or more grammatical elements will be communicated regarding the individual sample set being merged into one of the groups along with another sample set, and the unpredictable residual being encoded and embedded in the bitstream.

[0159] Example 7: According to the apparatus of Example 6, the extractor is also configured to extract fine segmentation information from the bit stream, and the fine segmenter is configured to segment the image into a sample set in response to the fine segmentation information.

[0160] Example 8: The apparatus according to Example 6 or 7, wherein the extractor and the merger are configured to sequentially step through the sample sets according to a sample set scanning order, and, for a current sample set, Extract the first binary syntax element (mrg_cbf) from this bitstream; If the first binary syntax element is determined to be a first binary state, then by inferring that the encoding parameter for the current sample set is equal to the encoding parameter associated with the group, the current sample set is merged into one of the groups, skipping the extraction of the prediction residual for the current sample set and stepping to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be a second binary state, then extract a second syntax element (mrg, mrg_idx) from the bitstream; and According to the second grammatical element, the current sample set is merged into one of the groups by inferring that the encoding parameters of the current sample set are equal to the encoding parameters of the associated group, or the encoding parameters of the current sample set are extracted, the extraction relating to at least one further grammatical element of the prediction residual for the current sample set.

[0161] Example 9: According to any of Examples 6 to 8, the apparatus also transmits one or more grammatical elements for at least one subset of the sample sets, if each sample set is to be merged into any set of mutually exclusive sets along with another sample set, then the individual sample set is to be merged with which of the predetermined candidate sample sets adjacent to the individual sample set.

[0162] Example 10: An apparatus according to any one of Examples 6 to 9, wherein the extractor is also configured to extract fine segmentation information from the bit stream, and the fine segmenter is configured to progressively segment the image into a sample set in response to the fine segmentation information, and the extractor is configured to sequentially step through a parent sample set of a child sample set, wherein the parent sample set consists of the sample set of the image being finely segmented, and for a current child sample set, Extract the first binary syntax element (mrg_cbf) from this bitstream; If the first binary syntax element is determined to be a first binary state, then by inferring that the encoding parameter for the current subsystem sample set is equal to the encoding parameter associated with the mutual exclusion set, the current subsystem sample set is merged into one of the mutual exclusion sets, skipping the extraction of the prediction residual for the current subsystem sample set, and stepping to the next subsystem sample set; If the first binary syntax element is determined to be a second binary state, then extract a second syntax element (mrg, mrg_idx) from the bitstream; and According to the second grammatical element, by inferring that the encoding parameters of the current sub-family sample set are equal to the encoding parameters of the associated population, the current sub-family sample set is merged into one of the populations, or the encoding parameters of the current sub-family sample set are extracted, which extracts at least one further grammatical element related to the prediction residual of the current sub-family sample set, and then proceeds to the next sub-family sample set. For the next subsystem sample set, if the first binary syntax element of the current subsystem sample set is determined to be the first binary state, then skip the extraction of the first binary syntax element and start extracting the second syntax element instead. If the first binary syntax element of the current subsystem sample set is determined to be the second binary state, then extract the first binary syntax element.

[0163] Example 11: According to the apparatus of Example 10, the first and second binary syntax elements are encoded using context-adaptive variable-length coding or context-adaptive (binary) arithmetic coding, and the context used to encode the syntax elements is derived based on the values ​​of these syntax elements in previously encoded blocks.

[0164] Example 12: An apparatus for encoding an image into a bitstream, wherein a sample array of the image is divided into sample sets associated with individual encoding parameters, the apparatus being configured to encode a flag into the bitstream for a current sample set, and to generally communicate whether the encoding parameters associated with the current sample set will be set according to a merge candidate or will be obtained from the bitstream, and whether the current sample set of the image will be reconstructed solely based on a prediction signal according to one of the encoding parameters associated with the current sample set, without any residual data, or will be reconstructed using residual data in the bitstream by refining the prediction signal according to the encoding parameters associated with the current sample set.

[0165] Example 13: An apparatus for encoding an image, the apparatus comprising: A fine segmenter, configured to finely segment the image into a set of multiple samples; A merger configured to merge such sample sets into groups, each consisting of one or more sample sets; An encoder is configured to encode an image using encoding parameters that vary across the image in units of a subset of sample sets, wherein the encoder is configured to encode the image for a predetermined set of samples by predicting the image and encoding a prediction residual; A one-bit stream generator is configured to embed the prediction residual and the encoded parameters, along with one or more syntax elements for at least one subset of the sample sets, into a one-bit stream, the syntax elements indicating whether each sample set will be merged into one of the populations along with another sample set. One of the possible states of one or more grammatical elements will be communicated regarding the individual sample set being merged into one of the groups along with another sample set, and the unpredictable residual being encoded and embedded in the bitstream.

[0166] Example 14: A method for decoding a bitstream encoded from an image, wherein the sample array of the image is divided into sample sets associated with individual coding parameters, the method comprising, for a current sample set, generally responding to a flag in the bitstream, a first determination regarding whether the coding parameters associated with the current sample set will be set based on a merge candidate or obtained from the bitstream, and a second determination regarding whether the current sample set of the image will be reconstructed solely based on a prediction signal according to one of the coding parameters associated with the current sample set, without any residual data, or will be reconstructed using residual data in the bitstream by refining the prediction signal according to the coding parameters associated with the current sample set.

[0167] Example 15: A method for decoding a bitstream in which an image is encoded, the method comprising the following steps: The image is further segmented into a set of multiple samples; These sample sets are merged into groups, each consisting of one or more sample sets; The image is decoded by varying the encoding parameters across the image in units of such sample sets, wherein the decoder is configured to decode the image for a predetermined sample set by predicting the image, decoding a prediction residual, and combining the prediction residual with a prediction generated by self-predicting the image; Along with one or more syntax elements for at least one subset of these sample sets, the prediction residual and the coding parameters are extracted from the bitstream, the syntax elements indicating whether each sample set will be merged into one of the populations along with another sample set, wherein the merger is configured to perform the merging in response to one or more syntax elements. One of the possible states of one or more grammatical elements will be communicated regarding the individual sample set being merged into one of the groups along with another sample set, and the unpredictable residual being encoded and embedded in the bitstream.

[0168] Example 16: A method for encoding an image as a bitstream, the sample array of the image being divided into sample sets associated with individual coding parameters, the method comprising, for a current sample set, encoding a flag into the bitstream, and generally communicating whether the coding parameters associated with the current sample set will be set according to a merge candidate or obtained from the bitstream, and whether the current sample set of the image will be reconstructed solely based on a prediction signal according to one of the coding parameters associated with the current sample set, without any residual data, or will be reconstructed using residual data in the bitstream by refining the prediction signal according to the coding parameters associated with the current sample set.

[0169] Example 17: A method for encoding an image, the method comprising the following steps: The image is further segmented into a set of multiple samples; These sample sets are merged into groups, each consisting of one or more sample sets; The image is decoded by varying the encoding parameters across the image in units of such sample sets, wherein the encoder is configured to encode the image for a predetermined sample set by predicting the image and encoding a prediction residual; The predicted residual and the encoded parameters are embedded together with one or more syntax elements for at least one subset of the sample sets into a one-bit stream, the syntax elements signaling whether each sample set will be merged into one of the populations along with another sample set. One of the possible states of one or more grammatical elements will be communicated regarding the individual sample set being merged into one of the groups along with another sample set, and the unpredictable residual being encoded and embedded in the bitstream.

[0170] Example 18: A computer program having a code, which, when executed on a computer, performs the method according to either Example 14 or 17.

[0171] 10,76: Encoder 20: Image 30: Bit Stream 40: Block 50, 60: Sub-blocks 70: Encoding order 72, 82: Fine divider 74,84: Merger 78: Bit Stream Generator 80, 86: Decoder 88: Extractor 100: Converter 102: Entropy Encoder 104,152: Inverter 106,156: Predictor 108: Subtractor 110,154: Adder 112,158: Input 114,160: Output 150: Entropy Decoder 200: Root Block 500: Boundary

Claims

1. An apparatus for decoding a bitstream encoded with an image (20), comprising: a segmenter (82) configured to segment the image (20) into a plurality of sample sets (40); a merger (84) configured to merge the plurality of sample sets (40) into a plurality of groups, each having one or more sample sets; and a decoder (86) configured to decode the image (20) using a plurality of encoding parameters transmitted in the bitstream in units of the groups of the sample sets, wherein the decoder (86) is configured to decode the image for a predetermined sample set by predicting the image (20), decoding a prediction residual for the predetermined sample set, and combining the prediction residual with a prediction generated by predicting the image (20). An extractor (88) is configured to extract from the bitstream (30) one or more syntax elements of each of the prediction residual and the coding parameters for at least a subset of the sample set (40), indicating whether an individual sample set (40) will be merged with another sample set (40) into one of the plurality of groups, wherein the merger (84) is configured to perform the merge in response to the one or more syntax elements, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set (40) will be merged with another sample set (40) into one of the plurality of groups and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by the conversion coefficient level.

2. The apparatus according to claim 1, wherein the extractor is further configured to extract fine segmentation information from the bitstream, and the fine segmenter is configured to segment the image into a plurality of sample sets in response to the fine segmentation information.

3. The apparatus according to claim 1, wherein the extractor and the merger are configured to sequentially step through the plurality of sample sets according to a sample set scanning order, and, for a current sample set, the extractor and the merger are configured to: extract a first binary syntax element from the bitstream; if the first binary syntax element is determined to be a first binary state, merge the current sample set into the group by inferring that the encoding parameters for the current sample set are equal to the encoding parameters associated with one of the plurality of groups, skip the extraction of the prediction residual for the current sample set, and step to the next sample set in the sample set scanning order; If the first binary syntax element is determined to be the second binary state, then the second syntax element is extracted from the bitstream; and depending on the second syntax element, the current sample set is merged into one of the groups by inferring that the encoding parameters for the current sample set are equal to the encoding parameters associated with one of the multiple groups, or the extraction of the encoding parameters for the current sample set is performed, while at least one additional syntax element associated with the prediction residual for the current sample set is extracted.

4. The apparatus according to claim 1, wherein one or more syntax elements for each of at least a subset of the sample set further communicate, if the individual sample set is to be merged with any of the plurality of groups along with another sample set, then the individual sample set is to be merged with which of the predetermined candidate sample sets adjacent to the individual sample set.

5. The apparatus of claim 1, wherein the extractor is further configured to extract fine segmentation information from the bitstream, and the fine segmenter is configured to hierarchically segment the image into a plurality of sample sets in response to the fine segmentation information, and the extractor is configured to sequentially step through a parent sample set of a child sample set, the parent sample set being composed of the plurality of sample sets from which the image is finely segmented, and, for a current child sample set, the extractor is configured to: extract a first binary syntax element from the bitstream; if the first binary syntax element is determined to be a first binary state, merge the current child sample set into one of the groups by inferring that the encoding parameters for the current child sample set are equal to the encoding parameters associated with the group, skip the extraction of the prediction residual for the current child sample set, and step to the next child sample set; If the first binary syntax element is determined to be the second binary state, then the second syntax element is extracted from the bitstream; and the current subsystem sample set is merged into one group by inferring that the encoding parameters for the current subsystem sample set are equal to the encoding parameters associated with one of the plurality of groups, or the extraction of the encoding parameters for the current subsystem sample set is performed, while at least one additional syntax element associated with the prediction residual for the current subsystem sample set is extracted, and then the process proceeds to the next subsystem sample set. For the next subsystem sample set, if the first binary syntax element of the current subsystem sample set is determined to be the first binary state, then the extraction of the first binary syntax element is skipped, and the extraction of the second syntax element is started instead; and if the first binary syntax element of the current subsystem sample set is determined to be the second binary state, then the first binary syntax element is extracted.

6. The apparatus according to claim 5, wherein the first binary syntax element and the second binary syntax element are encoded using context-adaptive variable-length coding or context-adaptive binary arithmetic coding, and the context used to encode the syntax elements is derived based on the values ​​of these syntax elements in previously encoded blocks.

7. The apparatus of claim 1, wherein the bitstream further includes a depth map associated with the image.

8. The apparatus according to claim 1, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

9. An apparatus for encoding an image, the apparatus comprising: a segmenter (72) configured to segment the image into a plurality of sample sets; a merger (74) configured to merge the plurality of sample sets into a plurality of groups, each having one or more sample sets; an encoder (76) configured to encode the image using a plurality of encoding parameters transmitted in a one-bit stream on a group-by-group basis, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined sample set; and a stream generator (78) configured to insert the prediction residual and the encoding parameters together with one or more syntax elements for each of at least a subset of the plurality of sample sets into a one-bit stream, the one or more syntax elements indicating whether an individual sample set is merged into one of the plurality of groups along with another sample set. One of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the multiple groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by the level of the transformation coefficient.

10. The apparatus of claim 9, wherein the bitstream further includes a depth map associated with the image.

11. The apparatus according to claim 9, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

12. A method for decoding a bitstream encoded with an image (20), comprising: dividing the image (20) into a plurality of sample sets (40); merging the plurality of sample sets (40) into a plurality of groups, each having one or more sample sets; and decoding the image (20) using a plurality of encoding parameters transmitted in the bitstream in units of the groups of the sample sets, wherein a decoder (86) is configured to decode the image for a predetermined sample set by predicting the image (20), decoding a prediction residual for the predetermined sample set, and combining the prediction residual with a prediction generated by predicting the image (20). Extract one or more syntax elements from the bitstream (30) for each of the prediction residual and the coding parameters and at least one subset of the sample set (40), indicating whether an individual sample set (40) will be merged into one of the plurality of groups along with another sample set (40), wherein a merger (84) is configured to perform the merge in response to the one or more syntax elements, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set (40) will be merged into one of the plurality of groups along with another sample set (40) and the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by the conversion coefficient level.

13. The method according to claim 12, wherein the bitstream further includes a depth map associated with the image.

14. The method according to claim 12, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

15. A method for encoding an image, the method comprising: segmenting the image into a plurality of sample sets; merging the plurality of sample sets into a plurality of groups, each having one or more sample sets; encoding the image using a plurality of encoding parameters transmitted in a one-bit stream on a group-by-group basis, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined sample set; inserting the prediction residual and the encoding parameters together with one or more syntax elements for each of at least a subset of the plurality of sample sets into a one-bit stream, the one or more syntax elements indicating whether an individual sample set is merged into one of the plurality of groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the plurality of groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a transformation coefficient level.

16. The method according to claim 15, wherein the bitstream further includes a depth map associated with the image.

17. The method according to claim 15, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

18. A digital storage medium for storing a bitstream, wherein an image is encoded into the bitstream by: dividing the image into a plurality of sample sets; merging the plurality of sample sets into a plurality of groups, each having one or more sample sets; encoding the image using a plurality of encoding parameters transmitted in a bitstream in units of the groups of the sample sets, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined sample set; inserting the prediction residual and the encoding parameters together with one or more syntax elements for each of at least a subset of the plurality of sample sets into the bitstream, the one or more syntax elements indicating whether an individual sample set is merged into one of the plurality of groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the plurality of groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a conversion coefficient level.

19. A digital storage medium for storing a bitstream, wherein an image is encoded into the bitstream, wherein the bitstream comprises: a prediction residual and a plurality of encoding parameters, wherein the encoding parameters are transmitted in the bitstream in units of groups of sample sets of the image, the groups being generated from a plurality of groups that subdivide the image into a plurality of sample sets and merge the plurality of sample sets into a sample set; one or more syntax elements for each of at least a subset of the plurality of sample sets, the one or more syntax elements indicating whether an individual sample set will be merged into one of the plurality of groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the plurality of groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a transformation coefficient level.

20. The digital storage medium according to claim 19 further includes a depth map associated with the image.

21. The digital storage medium according to claim 19 or 20, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

22. A method for decoding a bitstream, wherein the method comprises: receiving and decoding the bitstream, the bitstream being encoded by: segmenting the image into a plurality of sample sets; merging the plurality of sample sets into a plurality of groups, each having one or more sample sets; encoding the image using a plurality of encoding parameters transmitted in the bitstream in units of the groups of the sample sets, wherein the encoder (76) is configured to encode the image by predicting the image and encoding a prediction residual for a predetermined sample set; inserting the prediction residual and the encoding parameters together with one or more syntax elements for each of at least a subset of the plurality of sample sets into the bitstream, the one or more syntax elements indicating whether an individual sample set is merged into one of the plurality of groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the plurality of groups along with another sample set and the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a conversion coefficient level.

23. A method for decoding a bitstream, wherein the method comprises: receiving and decoding the bitstream, the bitstream comprising: A prediction residual and multiple coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, the groups being generated from multiple groups of segments of the image into multiple sample sets and merging the multiple sample sets into a sample set; one or more syntax elements for each of at least one subset of the sample sets, the one or more syntax elements indicating whether an individual sample set will be merged into one of the multiple groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the multiple groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a transformation coefficient level.

24. A method for storing an image, comprising: storing a bitstream on a digital storage medium, the bitstream comprising: A prediction residual and multiple coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, the groups being generated from multiple groups of segments of the image into multiple sample sets and merging the multiple sample sets into a sample set; one or more syntax elements for each of at least one subset of the sample sets, the one or more syntax elements indicating whether an individual sample set will be merged into one of the multiple groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the multiple groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a transformation coefficient level.

25. A method for transmitting an image, comprising: transmitting a bitstream on a transmission medium, the bitstream comprising: A prediction residual and multiple coding parameters, wherein the coding parameters are transmitted in the bitstream in units of groups of sample sets of an image, the groups being generated from multiple groups of segments of the image into multiple sample sets and merging the multiple sample sets into a sample set; one or more syntax elements for each of at least one subset of the sample sets, the one or more syntax elements indicating whether an individual sample set will be merged into one of the multiple groups along with another sample set, wherein one of the possible states of the one or more syntax elements indicates that an individual sample set will be merged into one of the multiple groups along with another sample set and that the individual sample set (40) has a zero prediction residual, wherein the prediction residual is represented by a transformation coefficient level.

26. The method of any one of claims 22 to 25, wherein the bitstream further includes a depth map associated with the image.

27. The method according to any one of claims 22 to 25, wherein the information sample array is one of sample arrays relating to different planes of the image, the different planes being encoded independently of each other.

Images