Effective partition coding with high degree of division freedom

Abstract

To provide a decoder and an encoder that allow contour block division considering the high degree of freedom.SOLUTION: A decoder 600 includes: a divider 602 that binarizes an image 215 from a data stream 304 to divide a reference block 216 arranged at the same position as a prescribed block 210 of a depth and parallax map 213, into a first partition 202a and a second partition 202b, thereby obtaining bi-segmentation; and a spatial transfer unit 604 that transfers the bi-segmentation of the reference block to the prescribed block of the depth and parallax map, in order to obtain a first partition and a second partition of the prescribed block.SELECTED DRAWING: Figure 22

Description

[Technical field]

[0001] The present invention relates to sample array coding using contour block partitioning, or block partitioning, which allows for a high degree of freedom. [Background technology]

[0002] Many coding schemes compress sample array data using subdivision of sample arrays into blocks. The sample arrays may define the spatial sampling of textures (i.e., images), however, other sample arrays may be compressed using similar coding techniques, such as depth maps. Due to the different nature of the information spatially sampled by each sample array, different coding concepts are optimal for different kinds of sample arrays. However, regardless of such sample arrays, many of these coding concepts use a block subdivision process to assign individual coding options to the blocks of the sample array, thereby finding a good tradeoff between the side information rate for coding the coding parameters assigned to each block on the one hand and the residual coding rate for coding the prediction residual for the prediction misprediction of each block on the other hand, or finding a good configuration in the rate / distortion sense with or without residual coding.

[0003] Typically, the blocks are rectangular or square. Clearly, it is advantageous to be able to adapt the shape of the coding units (blocks) to the content of the sample array to be coded. Unfortunately, however, adapting the shape of the blocks or coding units to the content of the sample array requires the expenditure of additional side information to signal the block division. A Wedgelet-type division of the blocks has been found to be a suitable compromise between the possible blocks and the associated side information overhead. The Wedgelet-type division leads to a division of the blocks into, for example, Wedgelet partitions for which specific coding parameters can be used.

[0004] However, even the restriction on Wedgelet partitioning introduces a significant amount of additional overhead for signaling the partitioning of blocks, and it would therefore be preferable to have a more efficient coding concept at hand that allows greater freedom in partitioning blocks in encoding the sample array in a more efficient manner. Summary of the Invention [Problem to be solved by the invention]

[0005] This object is achieved by the subject matter of the pending independent claims. [Means for solving the problem]

[0006] The main idea underlying the present invention is that, even though Wedgelet-based partitioning seems to represent a good trade-off between side information rate on the one hand and achievable change in partitioning possibilities on the other hand, compared to contour partitioning, the ability to relax the partitioning constraint to the extent that the partition must be a Wedgelet partition in order to derive a good predictor for the two segmentations in the depth / disparity map allows a relatively uncomplicated statistical analysis to be applied to the texture information of spatially overlapping samples. Thus, it is an increase in the freedom to reduce the indirect signaling that provides that it is precisely the texture information that is co-located in the form of an image according to the first aspect.

[0007] Another idea on which a further aspect of the invention is based is that the idea just outlined, according to the derivation of the two segmentations based on the same location as the reference block within the image with the next transfer of the two segmentations in the current block of the depth / disparity map, is only reasonable if the possibility of achieving a good approximation of the contents of the current block of the depth / disparity map is high enough to justify the specification of respective default values ​​of the corresponding coding option identifiers to cause the transfer mode of the two segmentations. In other words, the coding option identifiers in the case of the transfer of the respective two segmentations, in any case, when entropy coding is selected, the side information rate can be preserved by avoiding the need to take into account the respective default values ​​of the coding option identifiers for the current block of the depth / disparity map.

[0008] Further sub-aspects are the subject of dependent claims.

[0009] Preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 shows a block diagram of a multi-view encoder in accordance with an embodiment of the present invention. [Diagram 2] FIG. 2 shows a schematic diagram of a portion of a multi-view signal for illustrating information reuse across viewpoint and image depth / disparity boundaries. [Diagram 3] FIG. 3 shows a block diagram of a decoder conforming to FIG. [Figure 4] FIG. 4 shows Wedgelet partitions of quadratic blocks in contiguous (left) and separate signal spaces (right). [Diagram 5] FIG. 5 shows schematic diagrams of six different geometric arrangements of Wedgelet block partitions. [Figure 6]FIG. 6 shows examples of Wedgelet partition patterns for block sizes 4×4 (left), 8×8 (middle) and 16×16 (right). [Figure 7] FIG. 7 shows the approximation of the depth signal with a Wedgelet model by combining the partition information and CPV (average value of the depth signal in the partition region). [Figure 8] FIG. 8 illustrates the generation of a Wedgelet partition pattern. [Figure 9] FIG. 9 shows contour partitions of secondary blocks in contiguous (left) and separate signal spaces (right). [Figure 10] FIG. 10 shows an example of a contour partition pattern for block size 8×8. [Figure 11] FIG. 11 shows the approximation of the depth signal with a contour model by combining the partition information and CPV (average value of the depth signal in the partition region). [Figure 12] FIG. 12 shows intra prediction of a Wedgelet partition (blue) for a scenario in which the reference block above is a type Wedgelet partition (left) or regular intra direction (right). [Figure 13] FIG. 13 shows the prediction of Wedgelet (blue) and Contour (green) partition information from reference to texture luminance information (luma). [Figure 14] Figure 14 shows the CPV of a block partition. We show the CPV prediction from adjacent samples of adjacent blocks (left) and a cross-section of the block (right), showing the relationship between different CPV types. [Figure 15] FIG. 15 shows the pre-selection of modes based on differences in the luminance information (luma) of the textures. [Figure 16] FIG. 16 shows a block diagram of a decoder according to an embodiment. [Figure 17] FIG. 17 shows a block diagram of an encoder conforming to FIG. [Figure 18] FIG. 18 shows a block diagram of a decoder according to an embodiment. [Figure 19] FIG. 19 shows a block diagram of an encoder conforming to FIG. [Figure 20] FIG. 20 shows a block diagram of a decoder according to an embodiment. [Figure 21] FIG. 21 shows a block diagram of an encoder conforming to FIG. [Figure 22] FIG. 22 shows a block diagram of a decoder according to an embodiment. [Figure 23] FIG. 23 shows a block diagram of an encoder conforming to FIG. [Figure 24] FIG. 24 shows a block diagram of a decoder according to an embodiment. [Diagram 25] FIG. 25 shows a block diagram of an encoder conforming to FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following description of preferred embodiments of the present invention starts with possible environments in which the embodiments of the present invention can be advantageously used. In particular, a multi-view codec according to the embodiment is described with reference to Figs. 1-3. However, it should be emphasized that the embodiments described hereafter are not limited to multi-view coding. Nevertheless, some aspects described further below can be better understood and have special synergistic effects when combined with multi-view coding or, in particular, with coding of depth maps. Thus, after Figs. 1-3, the description proceeds from an introduction to irregular block partitioning and the problems involved therewith. This description refers to Figs. 4-11 and forms the basis for the description of the embodiments of the present invention described thereafter.

[0012] As just mentioned, further embodiments outlined below are applicable to the coding of depth maps, in particular, for example, to represent the geometry of a scene, using non-rectangular or irregular block partitioning and modeling functions in image and video coding applications. However, these embodiments are also applicable to conventional image and video coding. Further, embodiments outlined below provide concepts for the use of non-rectangular block partitioning and modeling functions in image and video coding applications. The embodiments are applicable to the coding of depth maps, in particular, for example, to represent the geometry of a scene, but are also applicable to conventional image and video coding.

[0013] In multi-view video coding, two or more views of a video scene (captured simultaneously by multiple cameras) are coded in a single bitstream. The primary goal of multi-view video coding is to provide an end user with an enhanced multimedia experience by presenting a 3D viewing impression. When two views are coded, the two reconstructed video sequences can be displayed in a conventional stereoscopic display (wearing glasses). However, the necessary use of glasses for conventional stereoscopic displays is often cumbersome for the user. Enabling a high-quality stereo viewing impression without glasses is currently an important topic of research and development. A promising technology for such autostereoscopic displays is based on lenticular lens systems. In principle, an array of cylindrical lenses is mounted on a conventional display in such a way that multiple views of a video scene are displayed simultaneously. Each view is displayed in a small cone. As a result, each eye of the user sees a different image; this effect creates a stereo impression without special glasses. However, this kind of autostereoscopic display generally requires 10-30 viewpoints of the same video scene (more viewpoints will be required if the technology is further improved). More than two viewpoints can also be used to provide the possibility to interactively select a viewpoint for a video scene. However, the coding of multiple viewpoints of a video scene significantly increases the required bit rate compared to multiple viewpoints of a conventional monocular (2D) video. Usually, the required bit rate increases almost linearly with the number of viewpoints coded. A concept to reduce the amount of transmitted data for an autostereoscopic display consists in transmitting only a small number of viewpoints (perhaps 2-5 viewpoints), but in addition transmitting a so-called depth map representing the depth (distance of real-world objects relative to the cameras) of the image samples for one or more viewpoints. Providing a small number of viewpoints coded together with a corresponding depth map (for high-quality intermediate viewpoints (virtual viewpoints located between the coded viewpoints) and additional viewpoints extending a little to one or both ends of the camera array) can be created at the receiver side by appropriate rendering techniques.

[0014] In state-of-the-art image and video coding, an image or a particular set of sample arrays for an image is usually decomposed into blocks that are associated with certain coding parameters. An image usually comprises several sample arrays (luminance and chrominance signals). In addition, an image may be associated with additional auxiliary sample arrays, which may specify, for example, transparency information or a depth map. Each image or sample array is usually decomposed into blocks. The blocks (or the corresponding blocks of the sample arrays) are predicted by inter-picture or intra-picture prediction. The blocks may have different sizes and may be square or rectangular. The division of an image into blocks may be regulated by syntax or it may be signaled (at least in part) inside the bitstream. Often, syntax elements are transmitted that signal the subdivision for blocks of a defined size. Such syntax elements may clarify whether and how a block is subdivided into smaller blocks and, for example, for prediction purposes, the associated coding parameters. Decoding the associated coding parameters for all samples of a block (or a corresponding block of a sample array) may be specified in a specific manner. In an example, all samples in a block are predicted using the same set of prediction parameters, such as reference indexes (identifying reference images in a set of already coded images), motion parameters (specifying magnitudes for motion in the block between the reference images and the current image), parameters for specifying an interpolation filter, intra-prediction modes, etc. The motion parameters may be represented by a displacement vector having horizontal and vertical components, or by higher-order motion parameters, such as affine motion parameters consisting of six components. It is also possible that more than one set of a particular prediction parameter (such as reference indexes and motion parameters) is associated with a single block.Then, for each of these specific sets of prediction parameters, a single intermediate prediction signal for the block (or the corresponding block of the sample array) is generated, and the final prediction signal is formed by a combination that includes superposing the intermediate prediction signals. The corresponding weight parameters and potentially fixed offsets (added to the weighted sum) may also be adjusted for the image, or the reference image, or the set of reference images, or they are included in the set of prediction parameters for the corresponding block. The difference between the original blocks (or the corresponding blocks of the sample array) and their prediction signals, also called the residual signal, is usually transformed and quantized. Often, a two-dimensional transform is applied to the residual signal (or the corresponding sample array for the corresponding residual block). For transform coding, the block (or the corresponding block of the sample array), for which a specific set of prediction parameters was used, may be further divided before applying the transform. The transform block is equal to or smaller than the block used for prediction. It is also possible that the transform block includes two or more blocks used for prediction. Different transform blocks may have different sizes and may represent square or rectangular blocks. After transformation, the resulting transform coefficients are quantized and so-called transform coefficient levels are obtained. The transform coefficient levels as well as the prediction parameters and, if present, the subdivision information are entropy coded.

[0015] State-of-the-art coding techniques such as ITU-T Rec. H.264 | ISO / IEC JTC 1 14496-10, or the current working model for HEVC, can also be applied to depth maps, where the coding tools are specifically designed for coding natural video. Depth maps have different characteristics as images of natural video sequences. For example, depth maps contain less spatial details. They are mainly characterized by sharp edges (representing object boundaries) and large regions of nearly constant or slowly varying sample values ​​(representing object regions). If depth maps are coded more efficiently by applying coding tools specifically designed to exploit the properties of depth maps, the overall coding efficiency of multi-view video coding with depth maps can be improved.

[0016] To serve as a basis for a possible encoding environment in which the subsequently described embodiments of the present invention may be advantageously used, a possible multi-view encoding concept is further described with respect to Figs. 1-3.

[0017] Fig. 1 shows an encoder for encoding a multi-view signal according to an embodiment. The multi-view signal of Fig. 1 is illustratively shown at 10 to include two views 121 and 122, although multiple views are possible. Moreover, according to the embodiment of Fig. 1, each view 121 and 122 includes video 14, depth / disparity map data 16. However, many of the advantageous principles of the embodiment described below may also be advantageous when used in conjunction with a multi-view signal having some views that do not include depth / disparity map data.

[0018] The images 14 of each viewpoint 121 and 122 represent spatio-temporal sampling of a projection of a common scene along different projection / viewing directions. Preferably, the temporal sampling rates of the images 14 of the viewpoints 121 and 122 are equal to each other, although this constraint does not necessarily have to be met. As shown in Fig. 1, each image 14 preferably comprises a sequence of frames with each frame associated with a respective timestamp t, t-1, t-2, .... In Fig. 1, the image frames are denoted by v view number,time stamp number For each frame v i,t represents a spatial sampling of the scene i along each viewing direction at each timestamp t and thus includes one or more sample arrays, such as, for example, one sample array for luminance samples and two sample arrays with chrominance samples, or simply a sample array for luminance samples or other color components, such as color components of an RGB color space. The spatial resolution of the one or more sample arrays may differ within both one image 14 and the images 14 of the different viewing points 121 and 122.

[0019] Similarly, the depth / disparity map data 16 represents a spatio-temporal sampling of the depths of scene objects of a common scene measured along the viewing directions of each of the viewpoints 121 and 122. The temporal sampling rate of the depth / disparity map data 16 may be equal to or different from the temporal sampling rate of the associated video of the same viewpoint, as illustrated in Fig. 1. In the case of Fig. 1, each video frame v has associated therewith a respective depth / disparity map d of the depth / disparity map data 16 of each of the viewpoints 121 and 122. In other words, in the embodiment of Fig. 1, each video frame v of viewpoint i and timestamp t has a temporal sampling rate of d, i.e., i,t is the associated depth / disparity map d i,tWith regard to the spatial resolution of the depth / disparity map d, the same applies as shown above for the video frames, i.e. the spatial resolution may vary with the difference between the depth / disparity maps of different viewpoints.

[0020] To efficiently compress the multi-view signal 10, the encoder of FIG. 1 encodes the views 121 and 122 in parallel into the data stream 18. However, the encoding parameters used to encode the first view 121 are reused to adapt the same or to predict the second encoding parameters used in encoding the second view 122. By this measure, the encoder of FIG. 1 exploits the fact that the parallel encoding of the views 121 and 12 results in the encoder determining the encoding parameters for these views in the same way. As a result, the redundancy between these encoding parameters can be exploited to effectively increase the compression ratio or rate / distortion ratio (measured distortion, e.g., average distortion of both views and rate measured as the encoding rate of the entire data stream 18).

[0021] In particular, the encoder of Figure 1 is generally indicated by the reference numeral 20 and includes an input for receiving a multi-view signal 10, and an output for outputting a data stream 18. As can be seen in Figure 2, the encoder 20 of Figure 1 includes two encoding branches for each view point 121 and 122, i.e., one for video data and the other for depth / disparity map data. Thus, the encoder 20 includes an encoding branch 22 for the video data of view point 1. v,1 , Encoding branch 22 for depth / disparity map data of viewpoint 1 d,1 , an encoding branch 22 for the second viewpoint video data; v,2 , and an encoding branch 22 for the depth / disparity map data of the second viewpoint. d,2 Each of these encoding branches 22 is similarly constructed. To describe the structure and functionality of the encoder 20, the following description is limited to the encoding branches 22. v,1We start with the structure and functionality of the branch 22, which is common to all branches 22. Then, the individual features of the branches 22 are described.

[0022] Coding Branch 22 v,1 is for encoding the image 141 of the first view 121 of the multi-view signal 12, and therefore branch 22 v,1 has an input for receiving video 141. In addition, branch 22 v,1 comprises, connected in series with one another in the mentioned order, a subtractor 24, a quantization / transform module 26, a requantization / inverse transform module 28, an adder 30, a further processing module 32, a decoded image buffer 34, which in turn are connected in parallel to two prediction modules 36 and 38, and a combiner or selector 40 connected on the one hand between the outputs of the prediction modules 36 and 38 and on the other hand to an inverting input of the subtractor 24. The output of the combiner 40 is further connected to an input of the adder 30. The non-inverting input of the subtractor 24 receives the image 141.

[0023] Coding Branch 22 v,1 The elements 24 to 40 cooperate to encode the video 141. The encoding is performed by encoding the video 141 in units of specific parts. For example, in encoding the video 141, a frame v 1,k is segmented, for example, into blocks, or other segments of sample groups. The segmentation may be constant in time or may vary in time. Furthermore, the segmentation may be known to the encoder and decoder by default or may be signaled within the data stream 18. The segmentation may be a standard segmentation of the frame into blocks, such as a non-overlapping arrangement of blocks in rows and columns, or may be a quadtree-based segmentation into blocks of various sizes. The currently coded segment of the image 141 being input to the non-inverting input of the subtractor 24 is referred to as the current block of the image 141 in the following description of Figures 1-3.

[0024] The prediction modules 36 and 38 are for predicting a current block and, for this purpose, the prediction modules 36 and 38 have their inputs connected to the decoded image buffer 34. In practice, both prediction modules 36 and 38 use a previously reconstructed part of the image 141 present in the decoded image buffer 34 to predict the current block coming from the non-inverting input of the subtractor 24. In this regard, the prediction module 36 operates as an intra predictor that spatially predicts the current part of the image 141 from the spatial neighbourhood of an already reconstructed part of the same frame of the image 141, whereas the prediction module 38 operates as an inter predictor that temporally predicts the current part from a previously reconstructed frame of the image 141. Both modules 36 and 38 perform their prediction according to or as described certain prediction parameters. More specifically, the latter parameters are determined by the encoder 20 in some optimization framework to optimize some optimization objective, such as optimizing the rate / distortion ratio under some constraints or without constraints, such as the maximum bit rate.

[0025] For example, the intra prediction module 36 may determine spatial prediction parameters for the current portion, such as an intra prediction direction along adjacent content, in which an already reconstructed portion of that frame of video 141 is extended / copied onto the current portion to predict the latter.

[0026] Inter prediction module 38 may use motion compensation to predict the current portion from a previously reconstructed frame, and the inter prediction parameters included therewith include a motion vector, a reference frame index, motion prediction subdivision information for the current portion, a hypothesis number, or some combination thereof.

[0027] A combiner 40 may combine one or more predictions provided by modules 36 and 38, or simply select one of them. The combiner or selector 40 forwards the current partial resulting prediction to the adder input of the subtractor 24, and further to each of the adders 30.

[0028] At the output of the subtractor 24, a residual of the prediction of the current part is output, and the quantization / transform module 26 is configured to transform this residual signal with quantization of the transform coefficients. The transformation can be, for example, some spectrally decomposing transformation, such as DTC. Due to the quantization, the resulting process of the quantization / transform module 26 is lossy, i.e. coding losses occur. The output of the module 26 is a residual signal 421 to be transmitted within the data stream. Not all blocks rely on residual coding. Rather, some coding modes can suppress residual coding.

[0029] The residual signal 421 is dequantized and inverse transformed in module 28 in order to reconstruct the residual signal as far as possible, i.e. to correspond to the residual signal as output by subtractor 24, despite the quantization noise. Adder 30 combines this reconstructed residual signal with a prediction of the current part by addition. Other combinations are also possible. For example, subtractor 24 can operate as a divider to measure the residual in proportion and, according to a variant, the adder can be implemented as a multiplier to reconstruct the current part. In this way, the output of adder 30 represents a preliminary reconstruction of the current part. However, further processing in module 32 can optionally be used to expedite the reconstruction. Further processing of this kind can include, for example, deblocking, adaptive filtering, etc. All available reconstructions are buffered so far in decoded picture buffer 34. In this way, decoded picture buffer 34 buffers previously reconstructed frames of video 141 and previously reconstructed parts of the current frame to which the current part belongs.

[0030] To enable the decoder to reconstruct the multi-view signal from the data stream 18, the quantization / transform module 26 sends the residual signal 421 to a multiplexer 44 of the encoder 20. At the same time, the prediction module 36 sends intra-prediction parameters 461 to the multiplexer 44, the inter-prediction module 38 sends inter-prediction parameters 481 to the multiplexer, and the processing module 32 sends further processing parameters 501 to the multiplexer 44, and in turn all this information is multiplexed or inserted into the data stream 18.

[0031] As has become apparent from the above description of the embodiment of FIG. v,1 The coding of the video 141 according to is self-contained in that the coding is independent of the depth / disparity map data 161 and any data of other viewpoints 122. v,1 is assumed to be coded video 141 into data stream 18 by determining coding parameters to obtain correction data, i.e. the above-mentioned residual signal 421, and predicting a current portion of video 141 from a previously coded portion of video 141 according to the first coding parameters, which is then coded by encoder 20 into data stream 18 from the coded portion of the current portion, and determining a prediction error of the prediction of the current portion. The coding parameters and the correction data are inserted into data stream 18.

[0032] Coding Branch 22 v,1 The above-mentioned coding parameters inserted into the data stream 18 by may include any one or any combination of the following:

[0033] First, the coding parameters for the video 141 may define / transmit a segmentation of the frames of the video 141, as will be discussed shortly.

[0034] Furthermore, the coding parameters may include coding mode information indicating for each segmentation or current portion, which coding mode may be used to predict the respective segment, such as intra-prediction, inter-prediction, or a combination thereof.

[0035] The coding parameters may also include the mentioned prediction parameters, such as intra prediction parameters for parts / segments predicted by intra prediction, and inter prediction parameters for inter predicted parts / segments.

[0036] However, the coding parameters may additionally include further processing parameters 501 signaling to the decoder side how to process the current or already reconstructed portion of the video 141 before using it to predict a subsequent portion of the video 141. These further processing parameters 501 may include indices that index the respective filters, filter coefficients, etc.

[0037] - The prediction parameters 461, 481 and further processing parameters 501 additionally include sub-segmentation data in order to define the accuracy of the mode selection or to define further sub-segmentations in relation to the aforementioned segmentations defining completely independent segmentations, such as the application of different adaptive filters for different parts of a frame within the scope of further processing.

[0038] The coding parameters also influence the determination of the residual signal and thus may also be part of the residual signal 421. For example, the spectral transform coefficient levels output by the quantization / transform module 26 are considered as correction data, while the quantization step size is likewise signaled within the data stream 18 and the quantization step size parameter may be considered as a coding parameter.

[0039] The coding parameters may further define prediction parameters that define a second stage prediction of the prediction residual in the first prediction stage described above. Intra / Inter prediction may be used in this respect.

[0040] To increase the coding efficiency, the encoder 20 includes a coding information exchange module 52 which receives all the coding parameters and further information that affects or is affected by the processing within the modules 36, 38 and 32, as illustratively shown by the vertically extending arrows shown in the coding information exchange module 52 below the respective modules. The coding information exchange module 52 is responsible for distributing the coding parameters and any further coding information between the coding branches 22 so that the branches predict or adapt the coding parameters from each other. In the embodiment of Fig. 1, instructions are defined in the data entries, i.e., the images and depth / disparity map data of the views 121 and 122 of the multi-view signal 10, to achieve this purpose. In particular, the image 141 of the first view 121 precedes the image 142, followed by the depth / disparity map data 161 of the first view, and then the depth / disparity map data 162 of the second view 122, and so on. It should be noted that this strictness in the data elements of the multi-view signal 10 does not have to be strictly applied for the encoding of all multi-view signals 10. However, for easier consideration, in the following, this strictness is assumed to be constant. The strictness in the data elements naturally also defines the strictness in the associated branches 22 therewith.

[0041] As already indicated above, the coding branch 22 d,1 ,twenty two v,2 and 22 d,2 Further encoding branches 22 such as v,1However, due to the mentioned instructions in the depth / disparity maps 161 of the images and viewpoints 121 and 122, respectively, and the corresponding instructions defined in the encoding branch 22, e.g. d,1 has additional freedom in predictive coding parameters to be used to code the current portion of the depth / disparity map of the first viewpoint 121. This is because of the above ordering of the different viewpoints of video and depth / disparity map data. For example, each of these elements may be coded with its own reconstructed portion as well as the elements prior to it in the above ordering of these data elements. Thus, in coding the depth / disparity map data 161, the coding branch 22 d,1 is allowed to use information known from a previously reconstructed portion of the corresponding image 141. d,1 However, it utilizes the reconstructed portion of the video 141 to predict some characteristics of the depth / disparity map data 161, which allows for better compression ratios of the depth / disparity map data 161, which are theoretically unlimited. d,1 may predict / adapt coding parameters related to encoding the video 141 as described above to obtain coding parameters for encoding the depth / disparity map data 161. In case of adaptation, the signaling of some coding parameters related to the depth / disparity data map 161 within the data stream 18 may be suppressed. In case of prediction, only the prediction residuals / correction data related to these coding parameters have to be signaled within the data stream 18. Such prediction / adaptation of coding parameters is also described further below.

[0042] Surprisingly, in addition to the modes mentioned above for modules 36 and 38, the coding branch 22 d,1may have additional coding modes available for coding blocks of the depth / disparity map 161. Such additional coding modes are further described below with respect to irregular block partitioning modes. In an alternative perspective, irregular partitioning may be viewed as a continuation of the subdivision of the depth / disparity map into blocks / partitions, as described below.

[0043] In any case, additional prediction functions are present for the following data elements: image 142 of second view 122 and depth / disparity map data 162. For these coding branches, the inter prediction module may perform inter-view prediction as well as temporal prediction. The corresponding inter prediction parameters include similar information as compared to the temporal prediction, i.e., inter-view prediction segment, disparity vector, view index, reference frame index, and / or an indication of the number of hypotheses, i.e., the number of inter predictions participating in forming the inter-view prediction by summation. Such inter-view prediction is performed in the same way as branch 22 for image 142. v,2 Branch 22 on depth / disparity map data 162 d,2 1. The inter prediction parameters are available for the inter prediction module 38. Of course, these inter prediction parameters also represent coding parameters that can serve as the basis for adaptation / prediction for the next view data of the three views that may not be shown in FIG.

[0044] Due to the above-mentioned factors, the amount of data inserted into the data stream 18 by the multiplexer 44 is further reduced. d,1 ,twenty two v,2 and 22 d,2 The amount of coding parameters of the coding branch 22 can be significantly reduced by adapting the coding parameters of the preceding coding branch or by simply inserting the prediction residuals associated with it. Due to the ability to choose between temporal and inter prediction, the coding branch 22 v,2 and 22 d,2There is also a lower amount of residual data 423 and 424. The reduction in the amount of residual data overcompensates for the additional coding effect in distinguishing between the temporal and inter-view prediction modes.

[0045] To explain the principles of coding parameter adaptation / prediction in more detail, reference is made to Fig. 2. Fig. 2 shows a typical portion of a multi-view signal 10. Fig. 2 shows a video frame v when segmented into segments or portions 60a, 60b and 60c. 1,t For simplicity reasons, we use frame v 1,t Only three parts of a video frame v are shown, although segmentation may separate the frame into segments / parts seamlessly and without gaps. 1,t The segmentation of v may be modified or varied in time, and the segmentation may or may not be signaled within the data stream. FIG. 2 shows a frame v of which portions 60a and 60b are representative in this case. 1,t-1 1. It is shown that the motion vectors 62a and 62b are also temporally predicted from reconstructed versions of several reference frames of the image 141, which are the current frame v in display time order 64. As is well known, the coding orders in the frames of the image 141 may not coincide with the representation orders in these frames. And, therefore, the reference frames are the current frame v in display time order 64. 1,t For example, portion 60c may be an intra predicted portion for which intra prediction parameters are inserted into data stream 18.

[0046] Depth / Disparity Map 1,t In the encoding of d,1 may utilize one or more of the above mentioned possibilities, which are exemplified below with respect to FIG.

[0047] -For example, depth / disparity maps 1,t In the encoding of d,1 is coding branch 22 v,1 As used by1,t Therefore, for a video frame v 1,t If there are segmentation parameters within the range of the coding parameters for the depth / disparity map data d 1,t Alternatively, the retransmission can be avoided due to the coding branch 22 d,1 sends a video frame v via data stream 18. 1,t A depth / disparity map d for signaling deviations of the segmentation in relation to 1,t As a basis / prediction for segmentation, a video frame v 1,t FIG. 2 shows the coding branch 22. d,1 is the depth / disparity map d 1,t As a pre-segmentation of video frame v 1,t In this example, the coding branch 22 is used as the segmentation. d,1 is the video v 1,t The presegmentation is adapted from or predicted from the segmentation of .

[0048] -Furthermore, coding branch 22 d,1 is the video frame v 1,t The depth / disparity map d 1,t The coding mode of the portions 66a, 66b and 66c of the video frame v may be adaptive or predictable. 1,t and depth / disparity map d 1,t In the case of different segmentations between video frames v 1,t The coding mode adaptation / prediction from video frame v 1,t of a video frame v, as obtained from a co-located portion of the segmentation of 1,t The selection / prediction of the coding mode from the depth / disparity map d can be controlled. A suitable definition of co-location is as follows: 1,t The video frame v for the current part in 1,tThe co-located parts in, for example, the depth / disparity map d 1,t In the case of the prediction coding mode, the coding branch 22 includes the part located at the same place in the upper left corner of the current frame in d,1 represents the number of video frames v explicitly signaled within the data stream 18. 1,t The depth / disparity map d associated with the coding mode in the range 1,t The deviation of the coding modes of the parts 66a to 66c can be signaled.

[0049] - As far as prediction parameters are concerned, coding branch 22 d,1 is the same depth / disparity map d 1,t or to spatially adapt or predict prediction parameters used to encode adjacent portions within a video frame v 1,t For example, FIG. 2 shows a depth / disparity map d 1,t The portion 66a of the video frame v is an inter-predicted portion, and the corresponding motion vector 68a is 1,t 4 illustrates that the motion vectors 62a of the co-located portion 60a of the data stream 18 may be adapted or predicted from the motion vector 62a of the co-located portion 60a of the data stream 18. In the case of prediction, only the motion vector difference is inserted into the data stream 18 as part of the inter prediction parameters 482.

[0050] - In terms of coding efficiency, coding branch 22 d,1 A depth / disparity map d uses irregular block division. 1,t It would be advantageous to have the ability to subdivide the segments of the presegmentation of v. 1,t In order to derive partition information such as the Wadgellet separation line 70 from the depth / disparity map d, some irregular block partition modes of the embodiment described further below are referred to. 1,tThe pre-segmentation blocks of are then subdivided. For example, the depth / disparity map d 1,t The block 66c is further divided into two wedgelet-shaped partitions 72a and 72b. d,1 may be configured to code these sub-segments 72a and 72b separately. In the case of FIG. 2, both sub-segments 72a and 72b are exemplarily shown to be inter-predicted using respective motion vectors 68c and 68d. According to Sections 3 and 4, coding branch 22 d,1 may have the freedom to select between several coding options for irregular block partitioning and signal the selection to the decoder as side information within the data stream 18.

[0051] In the encoding of the image 142, the encoding branch 22 v,1 In addition to the available coding modes, the coding branch 22 v,2 has the option of inter-view prediction.

[0052] For example, Figure 2 shows a video frame v 2,t The segmentation portion 64b of the first viewpoint image 141 uses the disparity vector 76 to represent the temporally corresponding image frame v 1,t 13 shows an example of cross-view prediction from

[0053] Despite this difference, coding branch 22 v,2 In addition, the video frame v, such as the coding parameters used in these coding operations, 1,t Encoding of and depth / disparity map d 1,t Therefore, the coding branch 22 can effectively use all available information forming the v,2 is the aligned video frame v 1,t and depth / disparity map d 1,t 1 from some or a combination of the motion vectors 62a and 68 of the respective co-located portions 60a and 66a of the video frame v 2,t10 may adapt or predict motion parameters, including motion vector 78, for temporally inter predicted portion 74a of the frame 10. If any, prediction residuals are signaled with respect to the inter prediction parameters for portion 74a. In this regard, motion vector 68a, having already been subjected to prediction / adaptation from motion vector 62a itself, must be undone.

[0054] Video frame v 1,t and the corresponding depth / disparity map d 1,t Since both coding parameters are available, the available common data delivered by module 52 is increased and, moreover, the depth / disparity map d 1,t Regarding the encoding of video frame v, as described above, 2,t Other possibilities for adaptive / predictive coding parameters for coding are given in the coding branch 22. v,2 By video frame v 2,t This can be applied to the encoding of

[0055] Then, coding branch 22 d,2 is coding branch 22 d,1 Depth / disparity map by 1,t Similarly to the coding of the depth / disparity map d 2,t This is done by encoding, for example, a video frame v 2,t Therefore, this is true for all of the coding parameter adaptation / prediction cases. However, in addition, d,2 is the depth / disparity map d of the previous viewpoint 121 1,t In addition, the encoding branch 22 also has the opportunity to adapt / predict the coding parameters from the coding parameters that were used to code the d,2 is coding branch 22 v,2 For example, inter-view prediction may be used as described with respect to the above.

[0056] After describing the encoder 20 of FIG. 1, it is noted that it may be implemented, for example, in software, hardware or firmware, i.e., programmable hardware. Although the block diagram of FIG. 1 suggests that the encoder 20 structurally includes parallel encoding branches, i.e., one encoding branch per image and depth / disparity map of the multi-view signal 10, this need not be the case. For example, software routines, circuitry or programmable logic configured to perform the tasks of elements 24-40 may each be used sequentially to accomplish the tasks for each of the encoding branches. In parallel processing, the processing of the parallel encoding branches may be performed in parallel processor cores or circuitry running in parallel.

[0057] Figure 3 shows a decoder capable of decoding data stream 18, for example, to reconstruct one or several view pictures corresponding to a scene represented by a multi-view signal from data stream 18. For the most part, the structure and functionality of the decoder of Figure 3 is similar to the encoder of Figure 20, so that the reference signs of Figure 1 are reused as far as possible to indicate that the description of the functionality provided above with respect to Figure 1 also applies to Figure 3.

[0058] The decoder of Fig. 3 is generally indicated by the reference sign 100 and comprises an input for a data stream 18 and an output for outputting a reconstruction of one or several views 102 as described above. The decoder 100 comprises a demultiplexer 104 and a pair of decoding branches 106 for each of the data elements of the multi-view signal 10 (Fig. 1) represented by the data stream 18, as well as a view extractor 108 and a coding parameter exchanger 110. As was the case with the encoder of Fig. 1, the decoding branches 106 comprise the same decoding elements in the same interconnections, and it is therefore the decoding branch 106 which is typically responsible for the decoding of the image 141 of the first view 121. v,1 In particular, each encoding branch 106 is described with respect to the multi-view signal 10, i.e., the decoding branch 106v,1 1, the decoding branch 106 comprises an input connected to a respective output of the multiplexer 104 and an output connected to a respective input of the viewpoint extractor 108 in order to output to the viewpoint extractor 108 a respective data element of the image 141 in the case of the first coding branch 106. In between, each encoding branch 106 comprises an inverse quantization / inverse transform module 28, an adder 30, a further processing module 32 and a decoded image buffer 34 connected successively between the multiplexer 104 and the viewpoint extractor 108. The adder 30, the further processing module 32 and the decoded image buffer 34 form a loop with a combiner / selector 40 following the parallel connection of the prediction modules 36 and 38 between the decoded image buffer 34 and the input of the adder 30. As indicated with the same reference numbers as in FIG. 1, the structure and functionality of the elements 28-40 of the decoding branch 106 are similar to the corresponding elements of the encoding branch in FIG. 1 in that the elements of the decoding branch 106 emulate the process of the encoding process by use of information conveyed within the data stream 18. Of course, the decoding branch 106 simply reverses the encoding procedure with respect to the encoding parameters finally selected by the encoder 20, but the encoder 20 of FIG. 1 must find an optimal set of encoding parameters, optionally in some optimization sense, such as encoding parameters optimizing a rate / distortion cost function subject to certain constraints (e.g., maximum bitrate, etc.).

[0059] The demultiplexer 104 is for distributing the data stream 18 to the various decoding branches 106. For example, the demultiplexer 104 provides to an inverse quantization / inverse transform module 28 with residual data 421, to a further processing module 32 with further processing parameters 501, to an intra prediction module 36 with intra prediction parameters 461, and to an inter prediction module 38 with an inter prediction module 481. A coding parameter exchanger 110 operates like the corresponding module 52 of FIG. 1 for distributing common coding parameters and other common data between the various decoding branches 106.

[0060] The viewpoint extractor 108 receives the multi-view signals as reconstructed by the parallel decoding branches 106 and extracts therefrom one or several viewpoints 102 corresponding to viewing angles or observation directions defined by externally provided intermediate extraction control data 112.

[0061] Due to the similar structure of the decoder 100 in relation to the corresponding portion of the encoder 20, its functionality up to the interface to the viewpoint extractor 108 is similarly and easily explained in the above description.

[0062] In fact, decryption branch 106 v,1 and 106 d,1 The scaling parameters (421), parameters 461, 481, 501, and corresponding non-adaptive parameters, and prediction residuals, the second branch 16 d,1 4, the decoding branch 106 operates together to reconstruct the first view 121 of the multi-view signal 10 from the data stream 18 by predicting the current portion of the first view 121 from a previous reconstructed portion of the multi-view signal 10 according to first coding parameters contained in the data stream 18 (such as coding parameters 422, parameters 462, 482, 502), and correcting a prediction error of the prediction of the current portion of the first view 121 using first correction data within 421 and 422 reconstructed from the data stream 18 prior to the reconstruction of the current portion of the first view 121 and contained in the data stream 18. v,1 is responsible for decoding the video 141, while the encoding branch 106 d,1 is responsible for reconstructing the depth / disparity map 161. See, for example, FIG. 2. v,1reconstructs the image 141 of the first view 121 from the data stream 18 by predicting a current portion of the image 141, such as 60a, 60b or 60c, from a previously reconstructed portion of the multi-view signal 10 according to the corresponding coding parameters read from the data stream 18, i.e., the scaling parameters with 421, parameters 461, 481, 501, and correcting a prediction error of this prediction using corresponding correction data obtained from the data stream 18, i.e., from the transform coefficient levels within 421. For example, the decoding branch 106 v,1 The encoder 40 processes the video 141 in units of segments / portions using the coding order between video frames and for coding of segments within frames, such as the corresponding coding branch of the encoder for coding between segments of these frames. Thus, all previously reconstructed portions of the video 141 are available for prediction for the current portion. The coding parameters of the current portion may include one or more intra prediction parameters 501, inter prediction parameters 481, filter parameters for the further processing module 32, etc. Correction data for correcting prediction errors may be represented by spectral transform coefficient levels within the residual data 421. Not all of these coding parameters need to be transmitted in their entirety. Some of them may be spatially predicted from the coding parameters of adjacent segments of the video 141. For example, a motion vector for the video 141 may be transmitted within the bitstream as a difference motion vector between the motion vectors of adjacent portions / segments of the video 141.

[0063] Second Decryption Branch 106 d,1 As far as the residual data 422 and the corresponding prediction and filter parameters and demultiplexer 104 are concerned, each decoding branch 106 is decoded by the coding parameters that are not predicted by crossing the boundary between the views, i.e., by the residual data 422 and the corresponding prediction and filter parameters and demultiplexer 104 that transmit signals within the data stream 18. d,1104 to the decoding branch 106 via the coding parameters and demultiplexer 104. v,1 or any information as distributed via the coded information conversion module 110. Thus, the decoding branch 106 d,1 10, a decoding branch 106 for a first viewpoint 121 from the portion of the coding parameters sent via the demultiplexer 104 v,1 and 106 d,1 , and determines its coding parameters for reconstructing the depth / disparity map 161, which in part is handled by the decoding branch 106. v,1 For example, on the other hand, frame v 1,t 4, as a motion vector difference for another adjacent portion of the block 102, and on the other hand as a motion vector difference explicitly transmitted within the range 482, for example, from the motion vector 62a explicitly transmitted within the range 481. Additionally or alternatively, the decoding branch 106 d,1 may use reconstructed portions of the image 141 as described above for prediction of Wedgelet separation lines to derive irregular block partitions, as described above for decoding the depth / disparity map data 161 temporarily, and as outlined in more detail below.

[0064] More specifically, the decryption branch 106 d,1 is at least partially decrypted by branch 106 v,1 (or adapted therefrom) from the coding parameters used by the decoding branch 106 v,1 The decoding branch 106 reconstructs the depth / disparity map 141 of the first viewpoint 121 from the data stream by using the coding parameters predicted from the reconstructed portion of the image 141 in the decoded image buffer 34 of the first viewpoint 121. The prediction residuals of the coding parameters may be obtained from the data stream 18 via the demultiplexer 104. d,1Other coding parameters for the depth / disparity map data 161 may be transmitted entirely within the data stream 108 or for other references, i.e., referring to some previously reconstructed part of the depth / disparity map data 161 itself, to the coding parameters used for the coding. Based on these coding parameters, the decoding branch 106 d,1 predicts the current portion of the depth / disparity map data 141 from a previously reconstructed portion of the depth / disparity map data 161 and performs a decoding branch 106 prior to the reconstruction of the current portion of the depth / disparity map data 161. d,1 , and the respective correction data 422 is used to correct a prediction error in the prediction of the current portion of the depth / disparity map data 161 .

[0065] As already mentioned above with respect to encoding, the decoding branch 106 for the second viewpoint 122 v,2 and 106 d,2 The functionality of the pair is similar to that of the first view 121. Both branches cooperate to reconstruct the second view 122 of the multi-view signal 10 from the data stream 18 by using specific coding parameters. Only a part of these coding parameters is required for these two decoding branches 106. v,2 and 106 d,2 1 and 142, and optionally the residuals of the inter-view predicted parts. The current part of the second view 122 is predicted from a previously reconstructed part of the multi-view signal 10 and reconstructed from the data stream 18 by one of the decoding branches 106 before the reconstruction of the current part of the second view 122, and thus the current part of the second view 122 is predicted from a previously reconstructed part of the multi-view signal 10 and reconstructed from the data stream 18 by one of the decoding branches 106. v,2 and 106 d,2 The correction data sent by the demultiplexer 104 to this pair, namely 423 and 424, is used to correct the prediction error.

[0066] Decryption branch 106d,2 Decryption branch 106 v,1 ,106 d,1 and 106 v,2 1, the data stream 18 may determine its coding parameters at least in part by adaptation / prediction from the reconstructed image 142 and / or the reconstructed depth / disparity map data 162 of the first viewpoint 121 from coding parameters used by either the image 141, the depth / disparity map data 161 and the image 142 or a true subset thereof. For example, the data stream 18 may signal for the current portion 80b of the depth / disparity map data 162 whether the current portion 80b is adapted or predicted from co-located portions of either the image 141, the depth / disparity map data 161 and the image 142 or a true subset thereof. For example, the coding parameters of interest may include motion vectors, such as 84, or disparity vectors, such as the disparity vector 82. Furthermore, for irregularly partitioned blocks, other coding parameters may be determined by the decoding branch 106. d,2 It can be derived by:

[0067] In any case, the reconstructed portion of the multi-view data 10, the views of which are contained therein, reaches the viewpoint extractor 108, which is based on the viewpoint extraction of the new viewpoints, i.e., the images related to these new viewpoints. This viewpoint extraction may include or involve a reprojection of the images 141 and 142, by using the depth / parallax map data associated therewith. Frankly, in the reprojection of the images to other intermediate viewpoints, the portions of the images corresponding to the scene parts located closer to the viewer are shifted along a parallax direction, i.e. a vector parallax direction of a different orientation, than the portions of the images corresponding to the scene parts located further from the viewer's position.

[0068] It should be mentioned that the decoder does not necessarily include the viewpoint extractor 108. Rather, the viewpoint extractor 108 may be absent. In this case, the decoder 100 is simply to reconstruct any of the viewpoints 121 and 122, such as one, some or all of them. In the case where no depth / disparity data exists for the individual viewpoints 121 and 122, the viewpoint extractor 108 may nevertheless perform intermediate viewpoint extraction by effectively using disparity vectors associating corresponding parts of the mutually adjacent viewpoints with each other. Using these disparity vectors to support the disparity vectors of the disparity vector field associated with the views of the adjacent viewpoints, the viewpoint extractor 108 may construct an intermediate viewpoint view from such views of the adjacent viewpoints 121 and 122 by applying this disparity vector field. For example, for a video frame v 2,t Assume that v has 50% of the cross-view predictions of that part / segment. That is, for 50% of the parts / segments, a disparity vector exists. For the remaining parts, the disparity vectors can be determined by the viewpoint extractor 108 by interpolation / extrapolation in the spatial impression. Temporal interpolation using disparity vectors for parts / segments of previously reconstructed frames of the video 142 can also be used. 2,t and / or a reference video frame v 1,t can then be warped according to these disparity vectors to obtain intermediate viewpoints. To this end, the disparity vectors are scaled according to intermediate viewpoint positions of intermediate viewpoints between the viewpoint positions of the first viewpoint 121 and the second viewpoint 122. Details regarding this procedure are outlined in more detail below.

[0069] However, the below outlined embodiment may be advantageously used in the framework of Figs. 1-3 if it is merely the coding of one view including the video and corresponding depth / disparity map data such as the first view 121 in the above outlined embodiment. In that case, the transmitted signal information, i.e., the single view 121, may be referred to as a signal corresponding to view synthesis, i.e., a signal enabling view synthesis. Adding the video 141 with the depth / disparity map data 161 enables the view extractor 108 to perform some sort of view synthesis by the view 121 reprojecting to an adjacent new view by effectively using the depth / disparity map data 161. Gains in coding efficiency are obtained by using irregular block partitioning. Also, further, the below described irregular block partitioning may be used within the scope of the single view coding concept independent from the exchange of inter-view coding information in the manner described above. In particular, the above embodiment of Figs. 1-3 may be used with the branch 22, 10 ... v / d,2 and to the extent that the relevant viewpoint 122 is missing.

[0070] Thus, Figures 1-3 show an example for a multi-view coding concept in which the irregular block partitioning described thereafter can be advantageously used. However, it is emphasized again that the coding modes described below can be used in conjunction with other kinds of sample array coding, irrespective of the sample array, be it a depth / disparity map or not. The coding modes described below do not even require the coexistence of a depth / disparity map in addition to a corresponding texture map.

[0071] In particular, the embodiments outlined below include some coding modes, whereby the signal of a block is represented by a model that splits the samples of the signal into two sets of samples and represents each set of samples by a fixed sample value. Some of the coding modes described below may be used to directly represent the signal of the block or to generate a prediction signal for the block that is further refined by coding additional residual information (e.g., transform coefficient levels). If one of the coding modes described thereafter is applied to the depth signal, advantages, in addition to other positive aspects, may result from the fact that the depth signal is mainly characterized by gradually changing regions and sharp edges between the gradually changing regions. While the gradually changing regions may be efficiently represented by a transform coding approach (i.e., based on the DCT), the representation of a sharp edge in two approximately fixed regions requires a large number of transform coefficients to be coded. As described with respect to some embodiments outlined below, this kind of block containing edges may be better represented by using a model that splits the block into two regions, each with fixed sample values.

[0072] In the following, different embodiments of the invention are described in more detail. In sections 1 and 2, the basic concept for partitioning a block into two regions of fixed sample values ​​is described. Section 3 describes different embodiments for specifying how a block is partitioned into different regions and what parameters need to be transmitted to represent the partition as well as the sample values ​​for the regions. The embodiments include concepts for signaling partition information independent of any other block, for signaling partition information based on transmitted data for spatially adjacent blocks, and for signaling partition information based on already transmitted texture images (conventional video images) for the depth map to be encoded. Then, section 4 describes embodiments of the invention with respect to encoding of mode information, partition information and fixed sample values ​​in conjunction with some embodiments for handling irregularly placed blocks.

[0073] Although the following description is primarily directed to coding of depth maps (particularly in the context of multi-view video coding), and the following description is based on a given depth block, some embodiments of the present invention may also be applied to conventional video coding. Hence, the description may be applied to other signal types if the term "depth block" is replaced with the general term "signal block." Furthermore, the following description sometimes focuses on square blocks, but the present invention may also be applied to rectangular blocks or other connected or simply connected sets of samples.

[0074] 1. Wedgelets In block-based hybrid video coding, a frame is subdivided in rectangular blocks, for example as shown in Figures 1-3. Often these blocks are square, and the processing for each block follows the same functional structure. Note that although most of the examples in this section use square blocks, Wedgelet block partitions and all related methods are not limited to square blocks, but rather are possible for any rectangular block size.

[0075] 1.1 Wedgelet Block Partitions The basic principle of Wedgelet block partitioning is to divide the area of ​​a block 200 into two regions 202a, 202b separated by a line segment 201, as illustrated in Fig. 4, where the two regions are labeled P1 and P2. The separating line segment is determined by a start point S and an end point E, both of which are located on the boundary of the block. Sometimes, in the following, region P1 will be referred to as Wedgelet partition 202a, while region P2 will be referred to as Wedgelet partition 202b.

[0076] For a continuous signal space (see the left side of Figure 4), the starting point is located at S(x S ,yS ), and the end point position is E(x E ,y E ), with the block size restrictions being 0≦x≦x E and 0≦y≦y E (where one of the coordinates is either the minimum (0) or the maximum (x E or y E ). With these definitions, the equation of the separating line is: JPEG2025041906000002.jpg17152

[0077] This equation is x S ≠x E Note that this is only valid for . Then, two regions P1 and P2 are defined as the area to the left and the area to the right of the line segment, respectively.

[0078] Typically, in digital image processing, a discrete signal space (see the right side of FIG. 4) is often used, where a block consists of an integer number of samples 203 exemplified by a grid square. Here, the start and end points S and E are given by the position S(u S ,v S ) and E(u E ,v E ) and the block size is limited to 0≦x≦u. E , and 0≦y≦v E In the individual separation case, the equation of the separation line can be formulated according to equation (1). However, as illustrated in the right of FIG. 4, the definition of the regions P1 and P2 is now different, so that only complete samples are assigned as part of the two regions. This assignment problem can be solved algorithmically, as described in section 1.4.1.

[0079] The Wedgelet block partitions 202a, 202b require their start and end points 204 to be located on different edges of the block 200. Thus, six different geometric arrangements of the Wedgelet block partitions 202a, 202b are distinguished by the rectangular or square block 200, as illustrated in FIG.

[0080] 1.2 Wedgelet partition pattern To use the Wedgelet block partitions in the encoding process, the partition information may be stored in the form of a partition pattern. Such a pattern may be stored as an array of size u B ×v B and each element contains binary information whether the corresponding sample belongs to region P1 or P2. Figure 6 shows, for example, Wedgelet partition patterns for different block sizes. Here, the binary region information, i.e., the two segmentations, is represented by black or white for the sample 203.

[0081] 1.3 Wedgelet Modeling and Approximation For modeling a block of depth signal with Wedgelets, the required information conceptually consists of two elements. One is the partition information (see section 1.1), for example in the form of a partition pattern, which assigns each sample 203 to one of two regions (see section 1.2). The other information element required is the value to be assigned to the samples of the region. The value of each of the two Wedgelet regions may be defined as a constant. This is an example of some implementation outlined below. This value is thus referred to as the fixed partition value (CPV). The second information element then constitutes the two representative samples for the specified region.

[0082] In order to approximate the signal of the depth block by Wedgelets, as shown in FIG. 7, the CPV of a given partition can be calculated as the average value of the original depth signal of the corresponding region. On the left side of FIG. 7, a grayscale portion of a representative non-depth map is shown. A block 200, which is currently the subject of the Wedgelet-based division, is shown as an example. In particular, its illustrated location within the original depth signal 205 is shown, as well as an enlarged grayscale version. First, the partition information, i.e., the two possible segmentation regions P1 and P2, are covered by the block 200. Then, the CPV of a region is calculated as the average value of all samples covered by the respective region. If the partition information of the example in FIG. 5 matches the depth signal 205 well, the resulting prediction of the block 200 based on the Wedgelet model, i.e., the outlined Wedgelet division mode, with a low CPV for region P1 (dark gray) and a high CPV for region P2 (light gray) represents a good approximation of the depth block.

[0083] 1.4 Wedgelet Processing 1.4.1 Wedgelet Pattern List For the purpose of efficient processing and signaling of Wedgelet block partitions, the partition patterns may be organized in a lookup list. The Wedgelet pattern list may contain patterns for all possible combinations of start and end point locations for the region separation segments, or may contain a suitable subset of all possible combinations. Thus, one lookup list may be generated for each prediction block size. To enable signaling between the encoder and decoder (see Section 3 for details) that relies on the location or index of a particular pattern within the list for a particular block size, the same list may be made available at the encoder and decoder. This may be implemented either by including a predefined set of patterns or by running the same generation algorithm as part of the encoder and decoder initialization.

[0084] As illustrated in Figure 8, the central function for creating a Wedgelet partition pattern lookup list is the generation of one list element. This can be achieved as described below (or by a similar algorithm). An empty pattern (u of binary elements) is B ×v BGiven the coordinates of the starting point S and the ending point E (in Fig. 8, left) as well as the coordinates of the starting point S and the ending point E, the first step is to draw the separating line segment. For this purpose, the Bresenham line segment algorithm can be applied. In general, the algorithm plots samples 203 to form a close approximation of a straight line between the two given points. In the case of a Wedgelet partition pattern, all elements 203 close to the line segment between the starting point S and the ending point E are marked (black boxes in Fig. 8, middle left). The last step is to fill one of the two resulting separated regions with the marked samples. Here, the assignment problem mentioned above needs to be stated. Since the elements of the pattern are binary, the separating line segment marked by the Bresenham algorithm will be part of one region. Intuitively, this seems unstable if the line segment sample is theoretically part of both regions. However, it is possible to assign the separating line segment sample to one region without significant loss. This is ensured by the fact that the marking of both line segments as well as the algorithm filling the area are geometrically aware, i.e., related to the underlying corner according to the geometrical arrangement. Based on the fact that the corner area is completely delimited by the separating line segments, filling this area is relatively simple. The filling algorithm starts from the underlying corner element 206 and marks successively all pattern elements, like columns and line segments, until it reaches the already marked element and thus the part of the line segment 207 (see FIG. 8 (middle right)). As a result, the Wedgelet partition pattern for a given start point position and end point position is represented by a binary value (FIG. 8 right).

[0085] The generation method for a Wedgelet partition pattern lookup list for a particular block size successively creates list elements for line segment start and end positions. This is achieved by iterating through the six geometric configurations shown in FIG. 5. For each geometric configuration, the start position is located on one of the edges of the block and the end position is located on the other edge of the block, and the list generation process executes the Wedgelet pattern generation method introduced above for each possible combination of start and end positions. For efficient processing and signal transmission, the Wedgelet pattern list should only contain certain unique patterns. Therefore, before a new pattern is added to the list, a check is made whether it is or is not identical to any of the patterns already in the list. In such a case, the pattern is redundant and therefore discarded. In addition, surface patterns, i.e. all samples are assigned to one area and are also removed from the list if they do not represent a valid Wedgelet block partition.

[0086] As an extension to the described Wedgelet pattern list, the resolution of the line segment start and end positions used for the generation of the patterns can be optimally increased and decreased, for example depending on the block size. The purpose of this extension is to find a better trade-off between coding efficiency and complexity. Increasing the resolution leads to a list with more patterns, while decreasing the resolution results in a shorter list resolution compared to the normal resolution. Thus, the resolution increases for small block sizes and decreases for large block sizes. It is important to note the independence of the resolution for the start and end positions, and the Wedgelet partition patterns stored in the list usually need to have the standard resolution, i.e. the original block size. Reducing the resolution can be simply achieved by generating patterns only for use with a subset of start and end positions, as described above. For example, half the resolution means limiting the generation of the patterns to every second start and end position. On the other hand, increasing the resolution is more difficult. To cover all start and end positions, a temporary pattern with increased resolution is first generated using the algorithm described above. In a second step, the resulting patterns are down-resolved to a constant resolution. Note that due to binary data, the down-resolution process does not support interpolation, which results in a large number of identical patterns in the case of increased resolution.

[0087] As a final result of the above-mentioned Wedgelet pattern generation, an ordered list of Wedgelet patterns is derived at both the encoder side and the decoder side. In actual implementation, these patterns may be predefined by the encoding algorithm / encoding standard used. Moreover, it is not necessary to generate the patterns by the actual algorithm described above, and improvements of this algorithm may also be used. It is only important that both the encoder and the decoder generate (and later use) the same list of Wedgelet patterns for the encoding and decoding processes.

[0088] 1.4.2 Wedgelet Search for Minimum Distortion Based on the lookup list mentioned above, the best approximation of the block signal by Wedgelet partitions may be found by a search algorithm. For a Wedgelet-based coding algorithm, the best approximation may be understood as the Wedgelet model that results in the minimum distortion. In other words, the search tries to find the best match of a Wedgelet partition pattern for a given block. The search makes use of derived pattern lists, which include all possible Wedgelet partition patterns for a given block size (see section 1.4.1 for details). These lists help to limit the processing time of the search, in case the patterns do not need to be generated again each time the minimum distortion Wedgelet search is performed. Each search step may consist of the following steps:

[0089] Calculation of CPV value from a given partition pattern and original block signal. Distortion between the original block signal and the Wedgelet model D W,cur Calculation of. D W,cur >D W,min If true, update the Wedgelet information with the minimum distortion and W,min =D W,curStore the list index of the current partition pattern by setting

[0090] Instead of the distortion, the magnitude of the Lagrangian cost can be used to find the Wedgelet pattern to use. The magnitude of the Lagrangian constant is a weighted sum D+λ·R, where the weighting of the distortion D is given by a particular Wedgelet pattern with the rate R required to transmit the associated parameters given by the Lagrangian multiplier λ.

[0091] Different methods are possible for the search algorithm, ranging from exhaustive search to fast search methods. An exhaustive search means that all elements of the Wedgelet pattern list are successively checked for the minimum distortion. This method ensures that the global minimum is found, without the expense of being slow (which is important especially for the encoder). A fast search method advances the method, reducing the number of required search steps. A fast search method could be, for example, successive refinements. In a first phase, the minimum distortion Wedgelet for a subset of partition patterns resulting from a limited number of start and end positions, e.g., every four boundary samples only, is searched for. In a second phase, the start and end positions are refined, for example, by allowing every second boundary sample, but limiting the range of start and end positions checked as a result of the first phase. Finally, the minimum distortion Wedgelet is found by refining the step size every cycle. In contrast to a full search, such a fast search method only makes it possible to find a local minimum, but the number of Wedgelet patterns to be examined is significantly lower and, therefore, the search is faster. Note that the step size of the first phase, for example as a function of the block size, does not have to be a constant value but can be set optimally.

[0092] The index just discussed which determines the course of a wedgelet segment or wedgelet pattern is called wedge_full_tab_idx.

[0093] 2. Contour Note that although most of the implementations in this section use square blocks, the contour block partitions and all related implementations are not limited to square blocks, but rather any rectangular block size is possible.

[0094] 2.1 Contour block partition The basic principle of contour block partitioning is to divide the area of ​​a block 200 into two regions 202a, 202b. Unlike Wedgelet block partitioning, the separation line 201 between the regions cannot be described by a geometric formulation. As illustrated by the two regions labeled P1 and P2 in Figure 9, the contour regions can be of any shape and they do not even need to be connected.

[0095] Figure 9 also illustrates the difference between continuous and disjoint signal spaces for contour block partitions. Also, only complete samples can be assigned as part of either of the two regions for the disjoint signal space (right of Figure 9). Since the contour partition information is derived from the disjoint signal space (see Section 3.2.2 for details), no geometric formulation is made and the assignment problem for Wedgelet block partitions need not be considered here.

[0096] 2.2 Contour Partition Pattern In Wedgelet partition pattern matching (see Section 1.2), the contour block partition information can be stored in the form of a partition pattern. Such a pattern can be a vector of size u B ×v Bwhere each element contains the binary information of whether the matching sample belongs to region P1 or P2. Figure 10 shows an example of a contour partition pattern that represents the binary region information by the sample color of black or white.

[0097] 2.3 Contour Modeling and Approximation The principle of approximating the depth signal of a block with a contour is the same as the Wedgelet concept described in section 1.3. Also, the required information consists of two elements partition information and partition filling indication, which is calculated as the average value of the original depth signal of the corresponding region, and may in turn include one fixed partition value (CPV) for each of the two regions.

[0098] Contour approximation is illustrated in Fig. 11, where the original depth signal of prediction block 200 is shown highlighted and enlarged to show its environment. Also, the partition information for regions P1 and P2 is first overlaid by the block, and then the CPV is calculated as the average value of all samples covered by the region. In the example in Fig. 11, if the partition information matches the depth signal well, the resulting contour model with low CPV for region P1 (dark grey) and high CPV for region P2 (light grey) represents a good approximation of the depth block.

[0099] 3. Block Partition Coding For example, within a coding framework for multi-view video and depth (MVD), such as the coding environment of Figures 1-3, new coding routines or modes should be defined to use the methods and algorithms described in the previous sections, and the necessary tools should be implemented in the encoder and decoder.

[0100] For example, the encoder or encoding branch 22 of FIG. v / d,1For hybrid video encoders, such as those described above, these tools can be classified as part of evaluation, prediction, or signal transmission. Evaluation summarizes tools that are only part of the encoding process since they depend on the original input information (e.g., uncompressed images). In contrast, prediction summarizes tools that are part of the encoding and decoding process since they only depend on transmitted and / or reconstructed information (e.g., decoded images). Signal transmission summarizes tools for coding information transmitted in the bitstream from the encoder to the decoder. Thus, they require the use of the same syntax and the same entropy coding states.

[0101] It should be noted that for the evaluation tool, the distortion can be derived by measuring the difference between the distortion and the original depth signal of the block, e.g. as the mean squared error (MSE) or mean absolute difference (MAD), as known from classical video coding approaches, or by measuring the difference in the synthesized viewpoint caused by displaced samples for the distorted depth signal of the block, as the synthesized viewpoint distortion.

[0102] The concept for irregular block partition coding according to the embodiment outlined below can be split into one for processing partition information (see Sections 3.1 and 3.2) and one for processing CPV (see Section 3.3).

[0103] 3.1. Wedgelet-based intra coding This section presents two intra-coding modes based on Wedgelet block partitions (see Section 1). Both modes can be combined with the delta CPV method (see Section 3.3.2).

[0104] 3.1.1. Intra-modeling of Wedgelet Block Partitions The basic principle of this mode is to find the best matching Wedgelet partition at the encoder and transmit the partition information explicitly in the bitstream. At the decoder, the signal of the block is reconstructed using the partition information transmitted explicitly. Therefore, the main tools for this mode are part of the estimation and signal transmission.

[0105] The Wedgelet partition information for this mode is not predicted, but is retrieved in the evaluation process at the encoder. For this purpose, as described in section 1.4.2, a minimum distortion Wedgelet search is performed using the original depth signal of the current block as a reference. The search yields the best matching Wedgelet partition with respect to the transformation method used.

[0106] To reconstruct the block at the decoder, the Wedgelet partition information must be signaled in the bitstream. This is achieved by explicitly transmitting the position or index of the matching pattern in the list (see Section 1.4.1). The list index is signaled by a constant number of bins. Given a Wedgelet pattern list with N elements, the index of the pattern used is coded using a fixed-length code or a variable-length code or an arithmetic coding (including context-adaptive binary arithmetic coding), or any other entropy coding method. Advanced methods of signaling the Wedgelet partition information can include sorting the list based on the probability of each partition pattern, or using an alternative representation of the partition information, e.g., the start and end positions of a line segment, or the start position and slope of a line segment.

[0107] 3.1.2. Intra Prediction of Wedgelet Block Partitions The basic principle of this mode is to predict a Wedgelet partition from information available for previously coded blocks in the same image, i.e. intra prediction. For a better approximation, the predicted partition is refined, for example, at the encoder, for example by varying the end positions of the line segments. A single transmission of an offset to the end positions of the line segments in the bitstream may be sufficient, and at the decoder the signal of the block may be reconstructed using the partition information resulting from combining the predicted partition and the transmitted refinement information such as the offset. Thus, the main tools for this mode are prediction, estimation and part of the signal transmission.

[0108] The prediction of Wedgelet partition information for this mode works internally with a Wedgelet representation consisting of the start position and gradient of the separating line segment. For further processing, i.e. adapting the end position offset of the line segment and reconstructing the signal of the block, the prediction result is converted into a representation consisting of the start position and end position of the line segment. The prediction process of this mode derives the start position and gradient of the line segment from the information of previously coded blocks, e.g. the neighboring blocks to the left and above the current block. In FIG. 12, only the current block 210 and the above neighboring blocks 212 are shown. It should be noted that for some blocks, one or both of the neighboring blocks are unavailable. In such cases, the process for this mode continues with skipping or setting the missing information to a meaningful default value.

[0109] As shown in Fig. 12, two main concepts are distinguished for predicting Wedgelet partition information according to the presently suggested embodiment. The first concept covers the case where the block 212 is exemplary dependent on Wedgelet partitioning when one of the two adjacent reference blocks is of Wedgelet type, which is shown in the left-hand embodiment of Fig. 12. The second concept covers the case where the block 212 is exemplary dependent on intra-coding when the two adjacent reference blocks are not of Wedgelet type but of intra-direction type, which may be the default intra-coding type, which is shown in the right-hand embodiment of Fig. 12.

[0110] If the reference block 212 is of Wedgelet type, the prediction process can work as follows: According to FIG. 12, left side, the gradient m ref is the starting position S of the first step ref and end position E ref The principle of this concept is to extend the reference wedgelet of the current block 210, i.e. the wedgelet separating line segment 201', within the current block 210, which is only possible if the extension of the separating line segment 201' of the reference wedgelet 212 actually crosses the current block 210. Therefore, the next step is to check whether it is possible to continue extending the reference wedgelet. The example on the left side of FIG. 12 shows a scenario in which it is possible, but if the start and end positions of the reference wedgelet are at the left and right edges of the block, the extension of the line segment will not cross the block below. If the check is positive, the start position S P and end position E P is predicted in the final step. The gradient m p By definition, m ref Since the position is simply calculated as the intersection point of the extension line with the block boundary sample.

[0111] If the reference block 212 is of intra-direction type, the prediction process can work as follows: According to FIG. 12, right side, the gradient m ref is derived from the first step intra prediction direction 214. If the intra direction 214 is only provided in the form of an abstract index, the mapping or transformation function can be expressed as the gradient m ref Unlike the concept of predicting from a Wedgelet type reference block 212, no separation line information is provided by the intra-direction type reference block 212. Therefore, the starting position S p is derived from information also available to the decoder, i.e., the neighboring samples of the left and above neighboring blocks. They are shown hatched in Fig. 12, right side. The density of the hatching represents the value of the neighboring samples. From these neighboring samples, the pair of neighboring samples with the maximum slope, as shown in Fig. 12, right side, is adjacent to the starting position S p where the slope is understood as the absolute difference between the values ​​of two consecutive samples. For a Wedgelet partition, the line segment starting point S p separates two regions 202a, 202b with different values ​​of one edge 216 of the block 210. Thus, the maximum slope point among adjacent samples of adjacent blocks is S p By redefining the gradient, m p m ref and S p With the end position E p can be calculated as a final step.

[0112] The two presented concepts are complementary. Prediction from a Wedgelet type reference block has better matching partition information, but this is not always possible, prediction from an intra-direction type reference block is always possible, but the partition information is worse matched. Therefore, it is beneficial to combine the two concepts into one prediction mode. To achieve this without additional signaling, the following processing hierarchy can be defined: if the above reference block is Wedgelet type, try to predict the partition; otherwise, if the left reference block is Wedgelet type, try to predict the partition; otherwise, predict the partition from the above and left reference information. For the latter, different decision criteria for deciding between the above and left directions are possible, ranging from simply prioritizing the above to advanced methods that jointly evaluate the direction and the gradient of the neighboring samples. If the upper and left reference blocks are Wedgelet type, such advanced criteria can also be applied.

[0113] The line segment end offset for refining the Wedgelet partition cannot be predicted, but can be searched for by the encoder in the evaluation process. For the search, the candidate partition is calculated based on the predicted Wedgelet partition and the line segment end offset E off By iterating over a range of offset values ​​and comparing the distortions of the different resulting Wedgelet partitions, the offset value of the best matching Wedgelet partition is determined for the deformation method used.

[0114] To reconstruct the block at the decoder, the line end offset value is to be signaled in the bitstream. The same can be signaled by the use of three syntax elements. The first signaling is to indicate any offset E offis present, i.e., the same is zero. The second one is the sign of the offset, i.e., indicates clockwise or counterclockwise deviation if the offset is non-zero. And the third indicates the absolute offset value -1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. In pseudocode, these syntax elements can be included as follows: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. JPEG2025041906000003.jpg28154dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag are DmmDeltaEnd, i.e., E off can be used to derive, and is as follows: DmmDeltaEnd[ x0 ][ y0 ] = ( 1 ? 2 *dmm_delta_end_sign_flag[ x0 ][ y0 ] ) * (dmm_delta_end_abs_minus1[ x0 ][ y0 ] + 1)

[0115] The most likely case is that the offset value is zero. For efficient signal transmission, the first bin is sent, which has the function of a flag indicating whether the offset is zero or not. Then, if the offset is not zero, k+1 additional bins are sent, which are ±2 k where the first bin represents the sign and the remaining k bins represent the absolute value of the offset. For efficient coding, k is typically a small number and may be set adaptively, for example depending on the block size. The line segment end position offsets may be transmitted by fixed-length codes, variable-length codes, or any other entropy coding technique, including arithmetic coding (including context-adaptive binary arithmetic coding).

[0116] 3.2. Inter-component prediction for block partition coding This section presents two coding modes based on predicting partition information from texture. Both modes can be combined with the delta CPV method (see Section 3.3.2). The texture information (i.e., the conventional video image) is assumed to be transmitted before the associated depth map.

[0117] The basic principle of these modes, as Wedgelets or as contour block partitions, can be described as predicting partition information from texture reference blocks. This kind of prediction can be called inter-component prediction. Unlike temporal or inter-view prediction, no motion or disparity compensation is needed here, as the texture reference pictures show the scene simultaneously and from the same perspective. Since no partition information is transmitted for these modes, inter-component prediction uses the reconstructed texture image as a reference. Depending on the color space used for texture coding, one or more components of the texture signal are considered for inter-component prediction. For video coding, the YUV color space is generally used. Here, the luma component contains the most important information to predict the signal of depth blocks, i.e. edges between objects. Thus, simple inter-component prediction methods only utilize the information of the luma component, while advanced methods further utilize the chroma component for joint prediction or to refine the luma prediction result.

[0118] 3.2.1 Texture-based prediction of Wedgelet block partitions The basic principle of this mode is to predict the Wedgelet partition of the depth block 210 of the depth map 213 from the texture reference block 216. This is achieved by searching for the best matching Wedgelet partition for the reconstructed texture image, as shown in Fig. 13. For this purpose, a minimum distortion Wedgelet search is performed using as reference the reconstructed texture signal 215, more specifically the luma block 216 having the same position and size as the depth block 210, as described in section 1.4.2. The resulting Wedgelet partition pattern 218 is used for the prediction 220 of the depth block. In Fig. 13, this is highlighted by the upper box, and for the example shown, the predicted Wedgelet partition (in the middle) approaches the depth block 210 very well. Since the described Wedgelet prediction can be performed simultaneously in the encoder and the decoder, no signaling of partition information is required for this mode.

[0119] Texture-based prediction of contour block partitions The basic principle of this mode is to predict the contour partition of the depth block from the texture reference block. As shown in Fig. 10, this is realized by deriving a contour partition 218' for the reconstructed texture image 215. For this purpose, a contour approximation is performed using as reference the reconstructed texture signal 215, more specifically a luminance block 216 having the same position and size as the depth block 210. Since this kind of contour prediction can be performed simultaneously in the encoder and the decoder, signaling of partition information is not required for this mode.

[0120] The contour partition pattern can be generated by calculating the average value of the reference block 216 and setting it as a threshold. Depending on whether the value of the samples of the reference block 216 is below or above the threshold, the matching positions are marked as part of the region P1 or P2 in the partition pattern 218'. The resulting contour partition pattern 218' is used for the prediction 220 of the depth block 210. In FIG. 13, this is highlighted by the lower box, and for the example shown, the predicted contour partition (middle) 218' reliably approximates the depth block 210. However, a threshold approach that does not approximate the depth signal well would potentially result in an unstable pattern with many isolated small parts. To improve the consistency of the contour pattern, the deviation process can be extended, for example by filtering or segmentation approaches.

[0121] The binary partition pattern dmmWedgeletPattern[x,y] defining the contour partition pattern denotes the sample positions within the block to be divided, (x,y), where x,y=0..nT-1, obtained from the luma samples of the located texture image block videoLumaSample[x,y], where x,y=0..nT-1, as follows: The threshold tH is derived as: tH=sumDC / (nT*nT), where sumDC+=videoLumaSamples[x,y] for x,y=0..nT-1 The pattern value is set as follows: If videoLumaSamples[x,y] is greater than tH, the following applies: dmmWedgeletPattern[x,y]=1 Otherwise the following applies: dmmWedgeletPattern[x,y]=0

[0122] 3.3 CPV encoding The concepts for CPV coding are provided in this section. Since both partition types, Wedgelets and contours, have two partition regions with constant values ​​by definition, they can be equally applied to all four modes for predicting or estimating block partition information (see Sections 3.1 and 3.2). Therefore, the CPV process does not need to distinguish between partition types or coding modes, but rather assumes that a partition pattern is given to the current depth block.

[0123] 3.3.1 Predicted CPV For a better understanding of CPV prediction, three types of CPV are distinguished: original CPV, predicted CPV and delta CPV. Their relationship is illustrated in a schematic diagram in FIG. 14, right side for a cross section of a block (FIG. 14, left dotted line 230). Here, line segment 232 represents the original signal of block 200 along line segment 230. According to the explanations in sections 1.3 and 2.3, the original CPV (FIG. 14, right side line segments 234 and 236) is calculated as the average value of the signals covered by the corresponding regions P1 and P2, respectively.

[0124] Original CPV W orig,P1 and W orig,P2leads to the best approximation of the original signal (left side or line segment 232 in Fig. 14) for a given block partition, but since the original signal is not available at the decoder, it is necessary to transmit the values ​​in the bitstream. This is quite expensive in terms of bit rate and can be avoided by applying the principle of CPV prediction. In contrast to the original CPV, the predicted CPV is derived from information also available at the decoder, i.e., the neighboring samples of the left and upper neighboring blocks, as illustrated in Fig. 14, left hatched sample 203. Here, the neighboring samples are marked in grey, and the predicted CPV for each region of a given partition results from calculating the average value of those samples neighboring the corresponding region (Fig. 14, left side line segments 238 and 240). It should be noted that the left or upper neighboring blocks are not always available. In such cases, the respective neighboring samples can be set to a default value.

[0125] On the right side of Fig. 14, the predicted CPV W pred,P1 , and W pred,P2 is represented by line segments 238 and 240, and the illustration emphasizes that the original and predicted CPVs can differ significantly. P1 and ΔW P2 The difference between the original and predicted values ​​of depends on the similarity between the original signal 232 of the current block 200 and the boundary signal of the reconstructed neighboring block shown on the hatched sample 203. This difference is defined as the delta CPV of the corresponding region. This is called the delta CPV ΔW P1 and ΔW P2 This means that if CPV is estimated at the encoder and transmitted in the bitstream, it is possible to reconstruct the original CPV at the decoder by adding the delta CPV to the predicted CPV. Simply transmitting the delta instead of the original value leads to a significant reduction in the required bitrate.

[0126] The predicted fixed partition values ​​CPV can be called dmmPredPartitionDC1 and dmmPredPartitionDC2 and can be derived from the neighboring samples p[x,y] as follows. In the following, dmmWedgeletPattern means to partition the surrounding current block such that samples (x,y) for x,y=0..nT-1 are exemplified. That is, the sample positions adjacent to the top edge are located at (x-1), for x=0..nT-1, and the sample positions adjacent to the left edge are located at (-1,y), for y=0..nT-1. The neighboring sample values ​​already reconstructed are denoted p[x,y]. sumPredDC2, sumPredDC1, numSamplesPredDC2 and numSamplesPredDC1 are initially set to zero: For x = 0..nT-1 the above neighboring samples are summed up as: ? If dmmWedgeletPattern[ x, 0 ] is equal to 1 (partition P1, for instance), the following applies: sumPredDC2 += p[ x, -1 ] and numSamplesPredDC2 += 1 ? Otherwise (partition P2, for instance), the following applies: sumPredDC1 += p[ x, -1 ] and numSamplesPredDC1 += 1 For y = 0..nT-1 the left neighboring samples are summed up as: ? If dmmWedgeletPattern[ 0, y ] is equal to 1, the following applies: sumPredDC2 += p[ -1, y ] and numSamplesPredDC2 += 1 ? Otherwise, the following applies: sumPredDC1 += p[ -1, y ] and numSamplesPredDC1 += 1 The predicted constant partition values are derived as follows. ? If numSamplesPredDC1 is equal to 0, the following applies: dmmPredPartitionDC1 = 1 << ( BitDepth Y ? 1 ) ? Otherwise, the following applies: dmmPredPartitionDC1 = sumPredDC1 / numSamplesPredDC1 ? If numSamplesPredDC2 is equal to 0, the following applies: dmmPredPartitionDC2 = 1 << ( BitDepth Y ? 1 ) ? Otherwise, the following applies: dmmPredPartitionDC2 = sumPredDC2 / numSamplesPredDC2

[0127] 3.3.2 Quantization and Adaptive Delta CPV Based on the principles of CPV prediction, a concept for efficient processing of delta CPV is introduced in this section. Transmitting delta CPV in the bitstream serves the purpose of reducing the distortion of the reconstructed signal for block partition coding. However, since the difference between the original and predicted signal is also covered by the transform coding of the disparity, the bit rate required to signal the delta CPV values ​​determines the benefit of this method. Therefore, quantization of delta CPV can be introduced as follows: the values ​​are linearly quantized after being evaluated at the encoder and dequantized before reconstruction at the decoder. Transmitting the quantized delta CPV has the effect that the bit rate is reduced, while the reconstructed signal from the dequantized values ​​differs only slightly from the best approximation. In conclusion, this results in a lower rate-distortion cost compared to the case without quantization. With regard to the linear quantization step size, the performance can be further improved by applying the principles known from transform coding, i.e., defining the quantization step size as a function of QP and not as a fixed value. Let the quantization step size be qΔ CPV =2 QP / 10 where 1≦qΔ CPV Setting delta CPV as ≦max(ΔCPV) / 2 has been found to be efficient and robust.

[0128] A possible signaling of delta CPV in the bitstream for the two regions of a split block can be configured as follows: JPEG2025041906000004.jpg27154

[0129] The transmission on the bitstream for a particular block may be made dependent on the syntax element DmmDeltaFlag, which may be either explicitly sent or derived from some coding mode syntax element.

[0130] The dmm_dc_1_abs, dmm_dc_1_sign_flag, dmm_dc_2_abs, and dmm_dc_2_sign_flag can be used to derive the values ​​of DmmQuantOffsetDC1 and DmmQuantOffsetDC2 as follows: DmmQuantOffsetDC1[ x0 ][ y0 ] = ( 1 ? 2 *dmm_dc_1_sign_flag[ x0 ][ y0 ] ) * dmm_dc_1_abs[ x0 ][ y0 ] DmmQuantOffsetDC2[ x0 ][ y0 ] = ( 1 ? 2 *dmm_dc_2_sign_flag[ x0 ][ y0 ] ) * dmm_dc_2_abs[ x0 ][ y0 ]

[0131] The dequantized offsets dmmOffsetDC1 and dmmOffsetDC2 can be derived from DmmQuantOffsetDC1 and DmmQuantOffsetDC2 as follows: dmmOffsetDC1 = DmmQuantOffsetDC1 * Clip3( 1, ( 1 << BitDepth Y ) ? 1, 2 (QP' Y / 10)- 2 ) dmmOffsetDC2 = DmmQuantOffsetDC2 * Clip3( 1, ( 1 << BitDepth Y ) ? 1, 2 (QP' Y / 10)- 2 )

[0132] BitDepth Y may be the bit width that DmmQuantOffsetDC1 and DmmQuantOffsetDC2 are internally, within the encoder and decoder, and QP' may be, for example, the just mentioned quantization parameter QP included in the coded transform coefficient levels of the prediction residual of the current slice.

[0133] The fixed partition value CPV can then be obtained by adding the dequantized offset to the predicted CPV: For the first partition: dmmPredPartitionDC1 + dmmOffsetDC1 For the second partition: dmmPredPartitionDC2 + dmmOffsetDC2

[0134] As already mentioned at the beginning of section 3, the distortion for the evaluation tool can be measured in two different ways. For delta CPV, these transformation methods strongly affect the evaluation process. In case the distortion is measured as the difference between the distortion of the block and the original depth signal, the evaluation process searches for the closest approximation of the original CPV by simply calculating and quantizing the delta CPV as described above. In case the distortion is measured for a synthesized viewpoint, the evaluation process can be extended to make the delta CPV better adapted to the quality of the synthesized viewpoint. This is based on the fact that those delta CPVs that necessarily lead to the best approximation of the original CPV do not necessarily lead to the best synthesized viewpoint quality. To find the delta CPV that leads to the best synthesized viewpoint quality, the evaluation process is extended by a minimum distortion search (compare section 1.4.2), which is then applied iteratively over all possible delta CPV combinations for the two partitions. For efficient processing and signal transmission, the range of tested values ​​can be limited. The search results in the combination of delta CPVs that produces the minimum distortion in the synthesized view, and these values ​​are finally quantized for transmission.

[0135] Note that the delta CPV method potentially allows skipping the conversion / quantization and transmission of the (remaining) residual. Due to the close approximation to the original or optimal depth signal, respectively, the impact of omitting disparity is limited, especially when evaluated with respect to the quality of the depicted viewpoint.

[0136] 4. Mode Encoding 4.1 Mode signal transmission In the encoding process, one mode is selected for every block by rate-distortion optimization, and the mode information is signaled in the bitstream, e.g., before the partition and CPV information. According to Section 3, the following four block partition modes can be defined (e.g., in addition to the non-irregular partition mode):

[0137] Wedgelet_ModelIntra: Intra modeling of Wedgelet block partitions (see Section 3.1.1) Wedgelet_PredlIntra: Intra prediction for Wedgelet block partitions (see Section 3.1.2) Wedgelet_PredTexture: Texture-based prediction of Wedgelet block partitions (see Section 3.2.1) Contor_PredTexture: Texture-based prediction of contour block partitions (see Section 3.2.2)

[0138] Each of the four modes can be applied with or without a method for delta CPV processing (see Section 3.3.2), resulting in eight different mode_IDs for signaling to the decoder which type of processing should be applied for prediction and reconstruction of the block.

[0139] If the block partition mode derived above is implemented as an additional set of block coding modes to an existing coding framework such as one of Figures 1-3, an additional flag preceding the mode information may be transmitted in the bitstream, signaling whether the block partition mode is used or not. If this flag is not set, normal block coding mode signaling follows. Otherwise, mode_ID is signaled, which indicates the actual block partition mode and also whether delta CPV is transmitted or not. In the bitstream, mode_ID is represented by three bins.

[0140] 4.2 Mode Preselection The idea behind mode preselection is to reduce the processing and signaling efforts for block partition coding (see Section 3) by implementing the concept of filtering out modes that are unlikely to be selected for the current block.

[0141] The first mode preselection concept disables modes that have a very low probability due to small block sizes. This means that in most cases the distortion is high compared to the rate required to signal the mode information. Among the four modes defined in Section 4.1, this applies to Wedgelet_PredIntra and Contour_PredTexture. Based on statistical analysis, these two modes are disabled for block sizes 4x4 and below.

[0142] The second mode preselection concept applies to two modes based on inter-component prediction, namely Wedgelet_PredTexture and Contour_PredTexture. The idea behind this concept is to adaptively exclude these modes when it is very unlikely that a meaningful block partition pattern can be derived from the texture reference block. This kind of blocks are characterized by being flat without relatively significant edges and contours. To identify these blocks, the differences of the texture reference block are analyzed. The criterion for disabling the two mentioned modes is that the differences are below a certain threshold. This mode preselection method is performed as follows: the differences are measured as the mean absolute error (MAE) between the average values ​​of the luma samples and the reference block (see 216 in Fig. 13). Instead of a fixed value, the threshold is set as a function of the quantization parameter (QP). If the QP is higher and vice versa, the threshold is set based on the results of a statistical analysis of the MAE values. MAE Set as (QP)=QP / 2, these two modes have the effect of excluding more blocks.

[0143] FIG. 15 shows a visualization of this mode preselection method with details on two texture intensity blocks 250 and 2502 and the average value of absolute difference versus right 2521 and 2522, respectively. Block 2501 has a very flat spatial sample value appearance with almost no structure, which is reflected by a very low variance. Modes Wedgelet_PredTexture and Contour_PredTexture are not considered since no meaningful partition information can be predicted from this block 2501. In contrast to block 2502, which has high variance, significant edges and contours result. Thus, the two modes are considered since the partition information derived from block 2502 is probably a good predictor for the partition of the corresponding depth block. JPEG2025041906000005.jpg69160 Table 1: Modes as determined by preselection.

[0144] Table 1 summarizes the effect of the two mode preselection concepts on the available modes. By excluding certain modes, the number of mode_IDs that must be signaled in the bitstream is reduced. Since each method reduces the number of bins required to signal a mode_ID by one, and the combination of both modes reduces the number of bins by two, the table shows that the two methods can be combined into an efficient method.

[0145] 5. Generalization After describing some possible irregular partitioning modes, their conceptual subdivision into two segmentation decisions (see 3.1 and 3.2) on the one hand and two partitions (see 3.3) on the other hand, as well as the coding of coding parameters, their possible operation in the coding framework and a description of the coding environment that such modes may additionally provide, the resulting embodiments for the respective decoders and encoders shall be described partly in more general terms. In particular, the following sections highlight certain advantageous details outlined above and provide a more comprehensive explanation than described above of how these details can be used in the decoders and encoders in a more general sense than described above. In particular, as outlined below, some of the advantageous aspects used in the above modes can be utilized individually.

[0146] 5.1. Extension of the Wedgelet Separation Segment across the Block Boundary As has become clear from the above discussion, the use of Wedgelet partitioning forms a possible compromise between the signaling overhead for signaling the partitioning on the one hand and the achievable diversity with irregular partitioning on the other hand. Nevertheless, a significant amount of side information data is required to unambiguously transmit the partition information, i.e., the location of the Wedgelet separation line segment, e.g., by using an index of the location of the Wedgelet separation line segment, e.g., according to the concepts outlined above with respect to Section 3.1.1.

[0147] Thus, the extension of the Wedgelet separation segments across the block boundaries forms one possible way of solving the problem just outlined. The above description in section 3.1.2 described a specific example for utilizing a solution to this problem. However, more generally, when utilizing the idea of ​​the extension of the Wedgelet separation segments over the block boundaries, a decoder may be constructed according to an embodiment of the invention, as outlined below with respect to FIG. 16. Nevertheless, all details described in sections 3.1.2 and other sections in 3-4 are to be understood as possible implementation details that may be combined with the description individually presented below.

[0148] The decoder of Fig. 16 is generally indicated by reference numeral 300 and is configured to reconstruct a sample array 302 from a data stream 304. The decoder is configured to perform the reconstruction by block-based decoding. For example, the sample array 302 may be part of a series of sample arrays and the decoder 300 may be implemented as a block-based hybrid decoder supporting different coding modes per block 304. The sample array may be any spatially sampled information, such as a texture or a depth map. For example, the decoder 300 of Fig. 16 may be implemented to reconstruct one viewpoint including the texture / image and depth / disparity map representing the sample array 302. To that extent, the decoder 300 may be implemented in the decoding branch 106.d,1 106 more v,1 or individually as a pair of decryption branches 106 d,1 That is, with or without coding of the prediction residual, the decoder 300 may be configured to reconstruct the sample array 302 using a coding mode such as intra prediction, temporal (motion compensated) prediction and / or inter-view (disparity compensated) prediction. The coding mode may, for example, comprise an explicit Wedgelet coding mode for each block, and the location of the Wedgelet separation line segment is transmitted explicitly in the data stream 304, for example the modes outlined in section 3.1.1.

[0149] In any event, the decoder 300 is configured to perform for a current block 210, e.g., a block for which a default encoding mode option is signaled in the data stream 304, the steps currently outlined. The functionality involved in these steps may be incorporated within the intra prediction module 36 or the intra prediction module and swap module 52.

[0150] The steps performed by the decoder 300 for one block of each mode are Wedgelet separation line position prediction 306, followed by position refinement 308 and decoding 310. In particular, the decoder of FIG. 16 is configured to predict in step 306a. The position 312 of the Wedgelet separation line within the block 210 of the sample array 302 depends on the Wedgelet separation line 201' of the neighboring block 212 of the block 210, such that the Wedgelet separation line at the predicted position 312 creates an extension or continuation of the Wedgelet separation line 201' of the neighboring block 212 into the current block 210. The decoder 300 could also derive the position of the Wedgelet separation line 201' of the neighboring block 212 to which the sample array 302 belongs by a respective explicit signaling for the block 212 from the data stream 304 or by some other coding option, e.g. by edge detection in the texture sample array. Other possibilities are described above and further below.

[0151] As mentioned above, the Wedgelet separation segment of block 210, whose position 312 is predicted in 306, may be a straight line, as was the case with the above description of section 3.1.2. However, instead, the line segment may be defined more generally, for example using a series of sample position hops, i.e., a series of symbols belonging to the separation segment, each defining the next pixel in the line segment. The line segment may have a predefined analytically determined curvature, which may also be predicted from line segment 201', or may be derived from some other previously processed portion of data stream 304.

[0152] In particular, the prediction 306 may then be configured such that the wedgelet separation line of the block 210 is pre-determined with respect to the general extension direction as well as the position transverse to the general extension direction of the wedgelet separation line. In the case of a curve, curve approximation using, for example, a polynomial function may be used to extrapolate the separation line of the block 212 and respectively determine the position of the separation line of the block 210. In the case of a straight line, the slope and the position in the transverse direction to the wedgelet separation line are determined.

[0153] It should also be stated that with respect to the prediction 306, neighborhood and extension do not necessarily have to be defined in spatial terms. Rather, blocks 210 and 212 may also be adjacent in time. For example, block 212 could be a co-located block of a sample array of a sample array sequence that is adjacent in time to sample array 302. In that case, the extension of the Wedgelet separation segment 201 to block 210 is a "temporal continuation."

[0154] The obvious possibilities of how prediction 306 may be performed are outlined above in section 3.1.2 and an explanation is given here. Refine Position 308 is for refining predicted position 312. That is, the decoder 300 is configured in Refine Position 308 to refine predicted position 312 of Wedgelet separation segment 301 of block 210 using refinement information signaled in data stream 304. Thus, refined Wedgelet separation segment 201 divides block 210 into first and second Wedgelet partitions 202a and 202b.

[0155] As mentioned above, the decoder 300 can be configured to occupy a collinear position in space such that the Wedgelet separation segment 201 at the predicted position 312 forms a collinear extension of the Wedgelet separation segment 201' of the adjacent block 212, and, to form a collinear extension, the refinement can be limited such that the starting position 314 of the Wedgelet separation segment of a given block 210 adjacent to the adjacent block 212 is maintained relative to the predicted position 312, regardless of the refinement information. That is, in the case of a straight Wedgelet separation segment, only its slope can be refined, while the starting point of the Wedgelet separation segment 201 at the edge 316 of the block 210 separating the blocks 210 and 212 remains unchanged. For example, an offset of the opposite end of the wedgelet separation segment 201 along the periphery of the block 210 from the end position 320 according to the predicted wedgelet separation segment position 312, i.e., the end portion 318 of the wedgelet separation segment 201, may be signaled in the data stream 304 as described above with respect to section 3.1.2.

[0156] In section 3.1.2, the offset is E off As described in this section, the decoder 300 may be configured to extract refinement information from the data stream using entropy decoding, where various possible offsets from a direct extension sample position 320 along the periphery of the block 210, measured in units of sample position pitch along the periphery, have associated probability estimates that increase monotonically from larger offsets to smaller offsets, such that smaller offsets have a higher probability than larger offsets associated therewith. For example, the VLC codeword length may decrease monotonically.

[0157] As mentioned above, the three syntax elements are offThe first signal transmission may be used to transmit any offset E off is present, i.e., whether the same is zero. The second is the sign of the offset, i.e., if the offset is non-zero, it signifies clockwise or counterclockwise deviation. And the third indicates the absolute offset value -1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. In the pseudo code, these syntax elements can be included as: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. JPEG2025041906000006.jpg28157dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag are DmmDeltaEnd, i.e. E off can be used to derive, and is as follows: DmmDeltaEnd[ x0 ][ y0 ] = ( 1 ? 2 *dmm_delta_end_sign_flag[ x0 ][ y0 ] ) * ( dmm_delta_end_abs_minus1[ x0 ][ y0 ] + 1)

[0158] The decoder 300 is then configured to decode the given block 210 by the first and second Wedgelet partitions 202a and 202b in a decoding 310. In the description given above in Sections 3 and 4, particularly Section 4, the decoding 310 decodes the first fixed partition value W in the first Wedgelet partition 202a. pred,P1 to the samples at the sample array locations and a second fixed partition value W pred,P2to the samples at the sample array positions. The advantage of this realization of the decoding procedure 310 is that the amount of side information can be kept low. In particular, this possible realization is particularly advantageous in the case of such information carried by sample arrays having the above outlined properties consisting of relatively flat plateaus with sharp edges between them, such as depth maps. However, it is even possible that the decoder has assigned other coding parameters to the Wedgelet partitions 202a and 202b individually. For example, motion and / or disparity compensated predictions can be applied to the partitions 202a and 202b individually in the decoding 310, thereby obtaining respective motion and / or disparity parameters for the partitions 202a and 202b, such as individual vectors, individually. Alternatively, the partitions 202a and 202b can be intra-coded individually in the decoding 306, for example by applying an intra-coding instruction to each of the same individually.

[0159] According to FIG. 16, the following information may be present in the data stream for block 210: 1) a coding option identifier with respective predefined states that trigger steps 306-310; 2) refinement information such as an end position offset; 3) optionally, coding parameters such as CPV or CPV residual for one or both partitions 202a, 202b, optionally since the same can be predicted spatially or temporally from adjacent samples / blocks; 4) optionally, coding parameter residual such as delta CPV.

[0160] Furthermore, the decoder 300 of FIG. 16 may be configured such that the coding mode realized by steps 306-310 is simply a default coding mode option among two coding mode options triggered by a respective common predefined value of the respective coding option identifier in the data stream 304. If the coding option identifier has a predefined value, the decoder 300 of FIG. 16 may be configured, for example, to retrieve the coding option identifier from the data stream 304 by checking whether any of a set of candidate blocks adjacent to the predefined block has a Wedgelet separation line segment reaching the block 210. For example, the candidate blocks may include spatially adjacent blocks 304 of the sample array 302 preceding the current block 210 in the coding order applied by the decoder 300 of the decoding block 304 of the sample array 302 - or in the decoding order. For example, the coding order may scan the block 304 row-wise from left to right and from top to bottom, and in that case the candidate blocks may include blocks immediately adjacent to the left of the current block 210 and blocks immediately adjacent above the current block 210, such as block 212. If the check reveals that there is such a Wedgelet block in the set of candidate blocks, the decoder 300 may perform prediction 306, refinement 308 and decoding 310 in an unaltered manner. However, if not, the decoder 300 may perform prediction 306 instead. As described above in section 3.1.2 and as outlined in more detail with respect to the next section, the decoder 300 may then be configured to predict the position of the Wedgelet separation line 201 in the current block 210 by setting the extension direction of the Wedgelet separation line 201 in the current block 210 depending on the reconstructed neighboring samples adjacent to the current block 210 or depending on the intra-prediction direction of one or more intra-predicted blocks of the candidate block.As far as possible implementations for the prediction of fixed partition values ​​of the decoder 310 are concerned, reference is made to the above and below descriptions.

[0161] Furthermore, it should be noted that certain effects result when the extension of the Wedgelet separation line segments across the block boundary is combined with a coding mode that allows more freedom of segmentation of the current block, e.g., the contour mode as outlined above and described further below. In particular, the decoder 300 can be configured to support the modes realized by blocks 306-310 as well as the contour division mode, thereby allowing the coding overhead to be appropriately adapted to the needs of the block.

[0162] In any case, the block as decoded / reconstructed by steps 306-310 can serve as a reference for the prediction loop of the decoder 300. That is, the prediction result when using bi-level prediction can serve as a reference for, for example, motion and / or disparity compensated prediction. Furthermore, the reconstructed value obtained by decoding 310 can serve as a spatial neighboring sample in the intra prediction of any block 304 of the sample array 302 following in the decoding order.

[0163] FIG. 17 shows a possible encoder compatible with the decoder of FIG. 16. In particular, FIG. 17 shows an encoder 330 configured to predict the position of a Wedgelet separation line in a given block of a sample array depending on the Wedgelet separation line of an adjacent block of the given block, so that the Wedgelet separation line of the predicted position forms an extension of the Wedgelet separation line to a given block of the adjacent block, in order to encode the sample array into a data stream. This function is indicated by 332. Furthermore, the decoder 330 has a function 334 to refine the predicted position of the Wedgelet separation line using refinement information, and the Wedgelet separation line of the given block divides the given block into a first and a second Wedgelet partition. The encoder 330 also has an insertion function 336 in which the refinement information is inserted into the data stream, and an encoding function in which the encoder 330 encodes the given block in units of the first and second Wedgelet partitions.

[0164] 5.2 Wedgelet separation line segment extension direction prediction from the intra prediction direction of adjacent blocks As described above, even Wedgelet-based block partitioning requires a significant amount of side information to inform the decoder about the location of the Wedgelet separation segments.

[0165] The idea on which the embodiments outlined below are based is that the intra-prediction direction of adjacent, intra-predicted blocks can be used to predict the extension direction of the Wedgelet separation line segment of the current block, thereby reducing the rate of side information required to transmit the partitioning information.

[0166] In the above description, Section 3.1.2 has been presented as a possible implementation of the embodiment outlined below, which is also described in more general terms so as not to limit the set of irregular partitioning modes outlined above in Sections 3 and 4. Rather, the ideas just mentioned may be advantageously used independently of the other details described in Section 3.1.2, as described in more detail below. Nevertheless, all details described in Section 3.1.2 and other sections should be understood as possible implementation details, which may be individually combined with the description presented below.

[0167] Thus, Fig. 18 shows an embodiment for a decoder 400, which, as far as possible additional functionality is concerned, can be implemented as described above with respect to section 3.1.2 and / or Figs. 1-3, making use of the ideas just outlined. That is, the decoder 400 of Fig. 18 is configured to reconstruct the sample array 302 from the data stream 304. The decoder 400 of Fig. 18 can be implemented as described above in section 5 with respect to the decoder 300 of Fig. 16, except for the coding modes defined by the functions 306-310, which are generally optional for the decoder of Fig. 18. That is, the decoder 400 of Fig. 18 can operate to reconstruct the sample array 302 of Fig. 18 by block-based decoding, for example block-based hybrid decoding. Among the coding modes available to the blocks 303 of the sample array 302, there is an intra-prediction mode, which is further outlined with respect to the functional module 402 of the decoder 400. As is the case with the decoder 300 of FIG. 18, the subdivision of the sample array 302 into blocks 303 can be fixed by default or the respective subdivision information can be signaled in the data stream 304. In particular, the decoder 400 of FIG. 18 operates in a similar manner as with respect to FIGS. 1-3, i.e., the encoding branch 106. v / d,1 or simply 106 d,1The decoder may be configured as outlined above, such as any of the decoders of FIG. 3 or a viewpoint decoder, such as a depth decoder etc.

[0168] In particular, the decoder 400 of FIG. 18 has a function 402 for predicting a first block 212 of a sample array 302 using intra prediction. For example, an intra prediction mode is signaled in a data stream 304 for the block 212. The decoder 400 may be configured to perform intra prediction 402 by filling the first block 212 by copying to the first block 212 reconstructed values ​​of samples 404 of the sample array 302 adjacent to the first block 212 along an intra prediction direction 214. The intra prediction direction 214 may also be signaled in the data stream 304 for the block 212, for example by indexing one of several possible directions. Alternatively, the intra prediction direction 214 of the block 212 itself may be subjected to prediction. For example, the decoding branch 106 d,1 1-3, in which the intra predictor 36 of the decoder 400 may be configured to perform step 402. In particular, the neighboring samples 404 may belong to blocks 303 of the sample array 302 whose reconstructions are already available and which have already been passed in decoding order by the decoder 400, such that the block 212 includes reconstructed values ​​of the neighboring samples 404 that are adjacent to the block 212. As mentioned above, various coding modes may be used by the decoder 400 to reconstruct these preceding blocks that are preceding in decoding order.

[0169] Furthermore, the decoder 400 of FIG. 18 can be configured to predict the position 312 of the Wedgelet separation line segment 201 in the second block 210 adjacent to the first block 212 by setting the extension direction of the Wedgelet separation line segment 201 in the second block 210 depending on the intra-prediction direction 214 of the adjacent block 212, the Wedgelet separation line segment 201 dividing the second block 210 into the first and second Wedgelet partitions 202a and 202b. For example, the decoder 400 of FIG. 18 can be configured to set the extension direction of the Wedgelet separation line segment 201 to be as equal to the intra-prediction direction 214 as possible with respect to the quantization of the representation of the extension direction of the Wedgelet separation line segment 201. In case of a straight separation line segment, the extension direction simply corresponds to the slope of the line segment. In case of a curved separation line, the intra-prediction direction can for example be adapted as the local slope of the separation line segment of the current block at the boundary with the adjacent block. For example, the decoder 400 of FIG. 18 selects an extension direction from among a set of possible extension directions that forms the best approximation of the intra-prediction direction 214 .

[0170] Thus, in prediction 404, the decoder 400 predicts the position 312 of the Wedgelet separation segment 201 of the current block 210, at least as far as its extension direction is concerned. The derivation of the position of the Wedgelet separation segment 201 of the second block 210 can be determined by leaving the extension direction 408 unchanged. For example, although it was described in section 3.1.2 that the prediction of the starting point 314 of the Wedgelet separation segment 201 can be performed by the decoder 400 when deriving the Wedgelet separation segment position in step 406, the decoder 400 can instead be configured to derive this starting point 314 by explicit signaling in the data stream 304. Furthermore, the decoder 400 of FIG. 18 could be configured to spatially position the Wedgelet separation segment 201 in a derivation 406 parallel to the extension direction 306, for example during maintenance of the extension direction 316, by temporally predicting the lateral distance from a Wedgelet block located at the same position in a previously decoded sample array, or by spatially predicting the lateral position from other sample arrays belonging to a different viewpoint compared to the sample array 302.

[0171] However, when the decoder 400 derives the position of the Wedgelet separation line segment 201 within the second block 210 of the sample array 302, it is preferred to place the starting point 314 of the Wedgelet separation line segment 201 at the position of the maximum change between successive ones of the series of reconstructed values ​​of the samples of the sample series extending adjacent to the second block 210 along a part of the periphery of the second block 210. The sample line segment is indicated by reference number 410 in FIG. 18 with the samples represented by small crosses. The sample line segment 410 may be limited to the samples of the spatially adjacent blocks available. In any case, it should be emphasized that the sample line segment 410 for which the maximum change is determined may extend around one of the corners of the rectangular block 210 as shown in FIG. 18. Thus, according to this procedure, the decoder 400 may be configured to place the Wedgelet separation line segment in the derivation 406 to start between the adjacent samples and the sample line segment 410, where there is a maximum difference in the reconstructed values ​​parallel to the extension direction 408.

[0172] Thus, the rate of side information is conserved since a good prediction is found to derive the location of the Wedgelet separation segment 201 by other means than explicit signal transmission in the data stream 304 .

[0173] Then, decoding 412 by the decoder 400 occurs just as it was described with respect to FIG. 16, in which the decoder 400 decodes the second block in units of the first and second Wedgelet partitions 202a and 202b.

[0174] Of course, the decoder 400 of Figure 18 may also be modified to include the refinement function 308 of Figure 16. Thus, as derived in step 406, the offset of the end position 318 of the Wedgelet separation segment 201 of the current block 210 relative to the end position 320 of the position of the Wedgelet separation segment - which may or may not be constrained to be straight, as shown above - may be signaled in the data stream 304.

[0175] As mentioned above, the three syntax elements have an end offset of this kind, E off , the first signal transmission can be used to transmit off is present, i.e., whether the same is zero or not. The second one is the sign of the offset, i.e., signifies clockwise or counterclockwise deviation if the offset is non-zero. And the third indicates the absolute offset value -1: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. In the pseudo code, these syntax elements can be included as: dmm_delta_end_flag, dmm_delta_end_sign_flag, dmm_delta_end_abs_minus1. JPEG2025041906000007.jpg26156dmm_delta_end_abs_minus1 and dmm_delta_end_sign_flag are DmmDeltaEnd, i.e., E off can be used to derive, and is as follows: DmmDeltaEnd[ x0 ][ y0 ] = ( 1 ? 2 *dmm_delta_end_sign_flag[ x0 ][ y0 ] ) * ( dmm_delta_end_abs_minus1[ x0 ][ y0 ] + 1)

[0176] However, alternative procedures are possible as well. For example, instead of signaling an end position offset, a direction or angle offset relative to the extension direction set according to the intra-prediction direction 214 could be signaled in the data stream 304 for the block 202.

[0177] According to FIG. 18, the following information may be present in the data stream for block 210: 1) a coding option identifier, each with a predefined state that triggers steps 406-412; 2) optionally, refinement information such as an end position offset; 3) optionally, coding parameters, such as CPV or CPV residual, for one or both partitions 202a, 202b, optionally because they can be predicted spatially or temporally from adjacent samples / blocks; 4) optionally, coding parameter residual, such as delta CPV.

[0178] Regarding possible modifications of the decoding step 412 in relation to the description in section 3.3, reference is made to the above description of step 310 in FIG.

[0179] 18 may be configured to consider steps 406 and 412 as a coding mode option enabled by a coding option identifier in the data stream 304, where the Wedgelet separation segment position derivation 406 forms a dependent measure for deriving the Wedgelet separation segment position, the extension of which continues up to the current block 210, in case none of the set of candidate blocks in the neighborhood of the current block 210 already has the Wedgelet separation segment.

[0180] Figure 19 shows an embodiment for an encoder compatible with the decoder of Figure 18. The encoder of Figure 19 is generally indicated by reference numeral 430 and is configured to encode a sample array into a data stream 304. Internally, the encoder 430 is configured to predict a first block of the sample array using intra prediction in block 432 and a line segment derivation based on the description of block 406 of Figure 18 in block 434. The encoder 430 then encodes the second block that was the subject of the line segment derivation in 434 by the first and second partitions in coding block 436.

[0181] Of course, the encoder 430 may be configured to operate to mirror the functionality of the decoder of Figure 18 beyond the functionality shown in Figure 19. That is, the encoder 430 may operate using block-based, e.g., block-based hybrid coding. Although not explicitly stated, the same is also true with respect to the encoder of Figure 17 when compared to the decoder of Figure 16.

[0182] 5.3 Deriving the Wedgelet Separation Segment by Positioning Its Starting Point According to the Reconstructed Values ​​of the Adjacent Samples Further methods of reducing the side information required to transmit information about the position of the Wedgelet separation line of a Wedgelet block form the basis of the embodiments outlined below. In particular, the idea is to place the start of the Wedgelet separation line at the position of the maximum change between successive ones of the reconstructed values ​​of the samples of the sample series extending adjacent to the current block along the perimeter, so that the previously reconstructed samples, i.e. the reconstructed values ​​of the blocks preceding the current block according to the encoding / decoding order, at least allow the prediction of the correct positioning of the Wedgelet separation line. In this way, similar to the possibilities outlined above with respect to sections 5.1 and 5.2, the required side information rate can be reduced in order to enable the decoder to correctly position the Wedgelet separation line. The idea underlying the embodiments outlined below was therefore also utilized in the above description of section 3.1.2, in which possible realizations of the embodiments outlined below are described.

[0183] Thus, Fig. 20 shows an embodiment for a decoder 500, which may be implemented as described above with respect to section 3.1.2 and / or Figs. 1-3, making use of the ideas just outlined, as far as possible further functionalities are concerned. That is, the decoder 500 of Fig. 20 is configured to reconstruct the sample array 302 from the data stream 304. Typically, the decoder 500 of Fig. 20 may be implemented as described above in sections 5.1 or 5.2, for example with respect to the decoder 300 of Fig. 16, except for the coding modes defined by functions 306-310, which are optional for the decoder of Fig. 18, and except for the coding modes defined by functions 402, 406 and 412, which are optional for the decoder of Fig. 20, for example with respect to the decoder 400 of Fig. 18. That is, the decoder 500 of Fig. 20 may operate to reconstruct the sample array 302 of Fig. 20 by block-based decoding, for example block-based hybrid decoding. When that is exactly the case with the decoder 300 of Figure 18, the subdivision of the sample array 302 into blocks 303 can be fixed by default or with the respective subdivision information signalled in the data stream 304. In particular, the decoder 500 of Figure 20 operates as outlined above with respect to Figures 1-3, i.e. like the decoder of Figure 3, or like the encoding branch 106. v / d,1 A viewpoint decoder such as d,1 The depth decoder can be configured as follows:

[0184] To put it bluntly, the decoder of FIG. 20, designated by reference numeral 500, corresponds primarily to the decoder of FIG. 18. However, in blocks 402 and 406, the functions outlined with respect to FIG. 18 merely represent optional steps with respect to FIG. 20. Rather, the decoder 500 of FIG. 20 is configured to derive, in step 406′, a position of a Wedgelet separation line segment 201 within a predefined block 210 of the sample array 302 by locating a starting point 314 of the Wedgelet separation line segment 201 at a maximum change position between successive ones of a series of reconstructed values ​​of a line segment 410 of samples extending adjacent to the predefined block 210 along a portion of the periphery of the predefined block 210, the Wedgelet separation line segment 201 dividing the predefined block 210 into first and second Wedgelet partitions 202a and 202b. In step 412', the decoder 500 performs decoding of the resulting partitions 202a and 202b in the manner outlined above with respect to FIG.

[0185] Specifically, in derivation 406', the decoder 500 orders the reconstructed values ​​of the samples of the already decoded neighboring blocks of the block 210 according to their order of occurrence when traversing these samples in a counterclockwise or clockwise direction. The resulting sequence of reconstructed values ​​is shown in FIG. 20 at 502. As can be seen, the largest difference between successive reconstructed values ​​occurs between the n-th and (n+1)-th neighboring samples, and therefore the decoder of FIG. 20 will place the Wedgelet separation line at edge 316, whereby this pair of neighboring samples is adjacent between the samples of the block 210 that are directly adjacent to this pair of neighboring samples in turn. As outlined above, the decoder 500 can use a row-wise block scan direction, and therefore the neighboring samples of the sample line 410 can extend along the left edge and the top edge of the block 210. The same can be achieved using a mix of row-wise scanning of tree root blocks scanned row-wise according to the decoding / encoding order, where for each currently visited tree root block, a quadtree subdivision is performed and its leaf root blocks are scanned in a depth-first traversal order. When using this kind of order, the probability of having the maximum number of adjacent samples already reconstructed increases compared to using a breadth-first traversal order.

[0186] In the derivation 406', the decoder 500 of FIG. 20 can use the derivation of the Wedgelet separation segment extension direction 408 as described with respect to FIG. 18 and in section 3.1.2, as well as in any other manner. Alternatively, the Wedgelet separation segment extension direction in which the decoder 500 places the Wedgelet separation segment 201 of the current block 210 can be different, for example, it can be predicted in time from a Wedgelet block placed at the same position of a previously decoded sample array of the sample array sequence including the sample array 302. Alternatively, an explicit signaling of the end point 318 of the Wedgelet separation segment can be used. The explicit signaling can indicate that the offset of the end point 318 from the sample position is located in the opposite position relative to the start position 314 across the middle of the block 210. Other solutions are of course possible.

[0187] In this regard, it should be noted that the starting point 314 may be defined by the decoder 500 in step 406' to correspond to the nth sample position, the (n+1)th sample position, or a sub-pixel position therebetween.

[0188] Many of the combination possibilities mentioned above in sections 5.1 and 5.2 are also transferable to the embodiments of the current section. For example, the coding mode of the decoder 500 implemented by blocks 406' and 412' can represent an auxiliary alternative system function triggered by a common default value of a common coding option identifier with the Wedgelet separation line extension concept of section 5.1 representing a default coding mode executed instead whenever one of the set of candidate neighboring blocks has a Wedgelet separation line continuing to the current block 210. Other generalizations and modifications are also possible. For example, the decoder 500 could support a contour division mode, etc.

[0189] According to FIG. 20, the following information may be present in the data stream for block 210: 1) a coding option identifier with respective predefined states that trigger steps 406'-412'; 2) optionally, refinement information such as an end position offset; 3) optionally, coding parameters - such as CPV or CPV residual - for one or both partitions 202a, 202b - optional, because the same can be predicted - spatially or temporally from adjacent samples / blocks; 4) optionally, coding parameter residual such as delta CPV.

[0190] Figure 21 shows an encoder that is compatible with the decoder of Figure 20. It is configured to be able to perform line segment derivation 434' following step 406 and encoding 436', indicated by reference numeral 530, as outlined in relation to Figure 19 with respect to block 436.

[0191] 5.4 Two-tile (pixel)-based segmentation of depth / disparity maps by thresholding co-located parts of the image As becomes clear from the above description, Wedgelet-based partitioning represents a kind of trade-off between the side information rate on the one hand and the achievable diversity of partitioning possibilities on the other hand. In comparison, partitioning into contours seems more complicated in terms of side information rate.

[0192] The idea underlying the embodiments described further below is that the ability to relax the partitioning constraint to the extent that the partitions must be Wedgelet partitions allows for applying relatively uncomplicated statistical analysis to overlaid spatially sampled texture information to derive good predictors for the two segmentations of the depth / disparity map. Thus, according to this idea, the freedom to precisely reduce signaling overhead is increased if there is co-located texture information in the shape of the image and where meaningful texture changes can be observed. Possible implementations of this idea, which utilize this idea, were described above in section 3.2.2, but are described in more detail below under more general terms. Also, all details described in section 3.2.2 and other sections should be understood as possible implementation details, which may be combined with the descriptions individually presented below.

[0193] In particular, Figure 22 shows a decoder 600 according to such an embodiment of the invention. The decoder 600 is for reconstructing a given block of a depth / disparity map 213 associated with an image 215 from a data stream 304. The decoder includes a segmenter 602, a spatial transformer 604 and a decoder 606. The decoder 600 is implemented by the decoding branch 106. d,1 / 2 2. The decoder 600 may be configured as described above with respect to any of the above. That is, the decoder 600 may operate on a block-by-block basis. Moreover, the same may be implemented as a hybrid video decoder. The subdivision of the depth / disparity map 213 into blocks may derive entirely from the subdivision of the image 215 into blocks, or may deviate therefrom, where the subdivision of the depth / disparity map may be signaled in the data stream 304 or may be otherwise known to the decoder 600.

[0194] The segmenter 602 is configured to segment a reference block 216 of the image 215, which is located at the same position as a predefined block 210 of the depth / disparity map 213, by binarizing the image 215 within the reference block 216 to obtain two segmentations of the reference block into a first and a second partition.

[0195] The spatial transferor 604 then transfers the two segmentations of the reference block 216 of the image to the predefined block 210 of the depth / disparity map 213 to obtain the first and second partitions 202a and 202b of the predefined block 210.

[0196] The decoder 606 is configured to decode the predefined block 210 per first and second partitions 202a and 202b. The functionality of the decoder 606 corresponds to the functionality described above with respect to boxes 310, 412 and 412'.

[0197] 1-3, the segmenter and forwarder functions may be included within the switching modules 52 and 110, respectively. Meanwhile, for example, the functionality of the decoder 606 may be realized in, for example, an intra prediction module.

[0198] As described above with respect to section 3.2.2, which may present possible implementation details for the elements of FIG. 22, the segmenter 602 may be configured to check the values ​​of the image 215 individually in the tiles 608 of the two-dimensional subdivision of the reference block 216 in the binarization process, such that each of the first and second partitions 202a and 202b of the reference block 216 of the image 215 may be a set of tiles that completely cover the reference block 216 of the image 215 and are complementary to each other, as to whether the respective values ​​are greater or smaller than a respective predefined value. That is, the binarization process may be performed at a sample resolution, where the tiles 608 correspond to individual samples 610 of the image 215. It should be mentioned that the decoder 600 may be responsible for the reconstruction of the image 215, in that the values ​​that are the subject of the individual checks in the binarization process are reconstructed values ​​of the reconstructed image 215. In particular, as described with respect to FIGS. 1-3, the decoder 600 may be configured to reconstruct the image 215 in front of the depth / disparity map 213 associated therewith.

[0199] As already mentioned above, the segmenter 602 may be configured to apply morphological hole filling and / or low-pass filtering on the result of the binarization process when segmenting, in order to obtain two segmentations into a first and a second partition of the reference block 216. This avoids the occurrence of too many separated parts of the partitions of the two segmentations obtained from the reference block 216 spatially transferred by the spatial transformer 604, however, the probability of sudden depth changes of this kind occurring visible to the eye is quite low. Naturally, the encoder performs the same.

[0200] Furthermore, the decoder 600 and the segmenter 602 may be configured to perform the binarization process by determining a magnitude of central tendency of the reconstructed sample values ​​of the reference block 216 of the image 215 and comparing each reconstructed sample value of the reference block 216 of the image 215 with a respective threshold value that depends on the determined magnitude. As mentioned above, the threshold value may be defined globally among the samples 610 in the reference block 216. As central tendency some average value may be used, such as the arithmetic mean or the median.

[0201] As described above in section 4.2, the decoder 600 could be configured to validate the coding mode represented by the blocks 602-606 only if the a priori determined variance of the values ​​of the samples in the reference block 216 of the image 215 is above a predefined threshold. If this is not the case, the two segmentations found by the binarization process would likely not form a good predictor for the appearance of the block 210 in the depth / disparity map, and therefore this coding mode cannot take this block into account. By suppressing the mode possibilities, a disadvantageous and unnecessary increase in the number of symbol values ​​of each coding option identifier for which the entropy probability evaluation must be taken into account is avoided.

[0202] According to FIG. 22, the following information may be present in the data stream for block 210: 1) a coding option identifier with a respective predefined state that triggers steps 602-604; 2) optionally, information steering the two segmentations, such as subsequent filtering / hole filtering directivity, e.g. meeting a threshold, directivity, etc; 3) optionally, coding parameters - such as CPV or CPV residual - for one or both partitions 202a, 202b - optional, since the same can be predicted - spatially or temporally from neighboring samples / blocks; 4) optionally, coding parameter residual, such as delta CPV.

[0203] All further variations mentioned above with respect to the embodiments of Figures 16, 18 and 20 are also applicable to the embodiment of Figure 22. This is especially true for the reference of the prediction loop of the decoder 600 and the use of the prediction result as part of the coding mode combination possibilities of Figure 22, for any of the other coding modes described in any of Sections 5.1-5.3 or for the variations mentioned above in Sections 3-4.

[0204] Figure 23 shows a possible implementation of an encoder compatible with the decoder of Figure 22. The encoder of Figure 23 is generally indicated by reference numeral 630 and is configured to encode a predefined block of a depth / disparity map associated with an image into the data stream 304. The encoder includes a segmenter 632 and a spatial transformer 634 that operate like components 602 and 604 of Figure 22 in that they operate on an internally reconstructed version of a previously encoded portion of the data stream 304. An encoder 636 encodes the predefined block in terms of the resulting partition.

[0205] 5.5 Dependence of the Effectiveness of the Transfer of the Two Segmentations to Depth / Disparity Maps on the Sample Value Distribution within the Reference Blocks from Image to Image The idea underlying the embodiment outlined below has already been mentioned in section 4.2, namely that the idea of ​​following the derivation of the two segmentations based on a co-located reference block within the image with the next displacement of the two segmentations onto the current block of the depth / disparity map is only reasonable if the possibility of achieving a good approximation of the contents of the current block of the depth / disparity map is high enough to justify the condition of the respective default values ​​of the corresponding coding option identifiers to trigger the two segmentation displacement modes. In other words, when entropy coding this coding option identifier in case each of the two segmentation displacements is very unlikely to be selected anyway, the side information rate can be preserved by avoiding the need to take into account the respective default values ​​of the coding option identifiers for the current block of the depth / disparity map.

[0206] Thus, according to the modified embodiment of the decoder 600 of FIG. 22, the reference is generally configured similarly to the decoder of FIG. 20, as the above description has been given, as far as possible implementations of the decoder 600 are concerned. However, the segmenter 602 is not limited to be configured to segment the reference block 216 into a contour partition and also into a Wedgelet partition. Rather, the segmenter 602 is merely configured to segment the reference block 216 of the image 215 according to the texture features of the image 215 within the reference block 216, in order to obtain the first and second partitions from the two segmentations of the reference block. For example, the segmenter 602 can use edge detection to detect possible extension directions of the Wedgelet separation line segment in order to transfer the line segment thus located spatially from the block 216 onto the depth / disparity block 210 by the spatial transferer 604, according to this kind of modified embodiment. Other possible two segmentations by the segmenter 602 are also feasible.

[0207] However, beyond that the decoder 600 would be configured according to this embodiment such that the segmentation by the segmenter 602, the spatial transfer by the spatial transferer 604 and the decoding form one of the first set of coding options of the decoder 600, which is not part of the second set of coding options of the decoder 600, where the decoder would be further configured to determine the variance of the sample values ​​in the reference block 216 of the image 215, to retrieve the coding option identifier from the data stream 304, and to use the coding option identifier as an index to the first set of coding options in case of a variance exceeding a predefined threshold, with decoding of the boxes 602-606 onto the predefined block 210 if the segmentation, spatial transfer and coding options indicate one coding option, and inheriting the predefined threshold as an index to the second set of coding options in case of a variance. In this way, signaling overhead for signaling the coding option identifier can be saved. As the variance, the mean absolute difference, the standard deviation or the variance can be used.

[0208] Regarding further modifications of the embodiment of the just mentioned modification of FIG. 22, reference is made to section 5.4 and the explanations regarding section 4.2.

[0209] A corresponding encoder can be derived from the encoder of FIG.

[0210] 5.6 Efficient Prediction with Two-Fold Using Predictions for One or Both Fixed Partition Values ​​from Adjacent Samples As outlined above with respect to the various embodiments already described so far, the method of predicting a current block by assigning fixed partition values ​​to the two partitions of the block is quite effective, especially when encoding sample arrays, often like depth / disparity maps, where the content of these sample arrays consists mostly of plateaus or simply connected regions of similar values ​​separated from each other by sharp edges. Nevertheless, even the transmission of such fixed partition values ​​requires a considerable amount of side information that must be avoided.

[0211] The idea underlying the embodiments described further below is that the rate of this side information can be reduced if the average value of the values ​​of adjacent samples associated with or adjacent to each partition is used as a predictor for the fixed partition value. The inventors have discovered that this kind of coding mode for blocks of sample arrays can even leave signaling of refinement of each fixed partition value.

[0212] FIG. 24 shows a decoder 700 for reconstructing the sample array 302 from the data stream 304. The decoder 700 can be configured to reconstruct the sample array 302 using block-based decoding or can be configured to use hybrid decoding. Generally, all possible implementations described above in sections 5.1-5.5 also apply to the decoder 700 of FIG. 24. Naturally, all possible implementations for splitting the current block 210 into two partitions merely represent any alternatives for the decoder 700 of FIG. 24 that may also be performed differently in this respect.

[0213] In particular, the decoder 700 is configured to perform different tasks or functions to derive a prediction of the current block 210. In particular, the decoder 700 is configured to perform a derivation 702 from two partitions of the given block 210 of the sample array 302 into a first partition, indicated by hatched samples, and a second partition, indicated by non-hatched samples. Furthermore, the decoder 700 is configured to perform an association 704 of each of the adjacent samples of the sample array 302, which are adjacent to the given block 210, with a respective one of the first and second partitions, such that each adjacent sample is adjacent to the partition with which it is associated. In FIG. 24, the adjacent samplings, which are the subject of the association 704, are shown with two different kinds of hatching, namely dot hatching and cross hatching. The dot hatching indicates samples adjacent to samples of the block 210, which belong to one partition of the block 210, while the cross hatching indicates samples adjacent to samples of the block 210 that belong to the other partition. As described above with respect to Sections 5.1-5.4, the decoder 700 may use an appropriate encoding / decoding order within the blocks 303 of the sample array 302 to achieve a high probability for the usefulness of adjacent samples of the blocks 303 of the sample array 302 that have already been reconstructed by the decoder 700.

[0214] Of course, it may occur that the available adjacent samples, i.e., adjacent samples of block 210 that are located in already reconstructed block 303 of sample array 302, simply join one of the partitions of block 210. In that case, data stream 304 may explicitly transmit a fixed partition value for each other partition that does not neighbor any of the adjacent samples. Instead, some other alternative procedure may then be performed by decoder 700. For example, decoder 700 may then set this missing fixed partition value to a default value or a long-term average among previously reconstructed values ​​of sample array 302 and / or a value determined from some other previously reconstructed sample array.

[0215] Finally, in prediction 706, the decoder 700 predicts the given block 210 by assigning the average value of the values ​​of adjacent samples associated with the first partition to the sample of the sample array located in the first partition and / or the average value of the values ​​of adjacent samples associated with the second partition to the sample of the sample array located in the second partition.

[0216] The decoder 700 may be configured to use the refinement information in the data stream to refine a prediction of a given block 210, i.e. by applying a first refinement value within the refinement information on an average value of values ​​of neighboring samples associated with a first partition and / or by applying a second refinement value within the refinement information on an average value of values ​​of neighboring samples associated with a second partition. In this regard, the decoder 700 may be further configured, when applying the first and / or second refinement value, to, for example, linearly combine and add the first and / or second refinement value and the average value of values ​​of neighboring samples associated with the first partition and / or the average value of values ​​of neighboring samples associated with the second partition, respectively. The decoder 700 may be configured to, when applying the first and / or second refinement values, retrieve the first and / or second refinement values ​​from the data stream and scale the retrieved first and / or second refinement values ​​using a quantization step size that depends on a reference quantization step size for which a predefined spatially sampled component associated with the sample array - texture and / or depth map - is transmitted in the data stream. The sample array may be, for example, a depth map, and the reference quantization step size may be used by the decoder 700 to reconstruct a texture sample array from a bitstream with which the depth map is associated. Further reference is made in detail to the respective parts of section 3.3.2.

[0217] The decoder is configured to predict a position of a Wedgelet separation line within a given block of the sample array that is dependent on a Wedgelet separation line of an adjacent block of the given block, such that the Wedgelet separation line at the predicted position forms an extension of the Wedgelet separation line of the adjacent block to the given block when deriving the first and second partitions from the two partitions of the given block of the sample array. The decoder is further configured to refine the predicted position of the Wedgelet separation line using refinement information in the data stream, such that the Wedgelet separation line of the given block separates the given block into the first and second partitions.

[0218] As mentioned above, the decoder 700 can perform the bipartition using any of the ideas described in sections 5.1-5.5. The decoder 700 can be configured to predict a reference block of the sample array 302 adjacent to the predefined block 210 along an intra-prediction direction to the reference block by using intra-prediction by filling the reference block by copying reconstructed values ​​of samples of the sample array adjacent to the first block. In deriving the first and second partitions from the two partitions of the predefined block of the sample array, the decoder 700 can predict the location of a Wedgelet separation line segment in the predefined block 210 by setting an extension direction of the Wedgelet separation line segment in the predefined block according to the intra-prediction direction, and the Wedgelet separation line segment divides the predefined block into the first and second partitions.

[0219] Alternatively, if the sample array 302 is a depth / disparity map associated with the image, the decoder 700 may be configured to segment a reference block of the image that is co-located with the predefined block 210 by binarizing the image within the reference block to obtain first and predefined partitions from the two segmentations of the reference block, and to spatially transfer the two segmentations of the reference block of the image onto the predefined block of the depth / disparity map to obtain the first and second partitions.

[0220] The decoder may further be configured to use the default block as a reference for a prediction loop of the decoder.

[0221] Figure 25 shows a possible embodiment for an encoder 730 adapted for the decoder 700 of Figure 24. The encoder 730 for encoding a sample array in a data stream is configured to derive 732 a first and a second partition from two partitions of a given block of the sample array, and to associate 734 each of adjacent samples of the sample array adjacent to the given block with a respective one of the first and second partitions such that each adjacent sample is adjacent to the partition with which it is associated. The encoder is further configured to predict 736 the given block by assigning the average value of the values ​​of samples adjacent to a sample of the sample array located in the first partition associated with the first partition and the average value of the values ​​of samples adjacent to a sample of the sample array located in the second partition associated with the second partition.

[0222] Although some aspects are described in relation to an apparatus, it will be apparent that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in relation to a method step represent a description of the corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or more of the most important steps may be performed by such an apparatus.

[0223] Depending on specific implementation requirements, embodiments of the present invention may be implemented in hardware or in software. The implementation may be implemented using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, having electronically readable control signals stored thereon, which cooperate (or can cooperate) with a programmable computer system so that the respective methods are executed. The digital storage medium may thus be computer readable.

[0224] Some embodiments according to the invention include a data carrier having an electronically readable signal capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0225] Typically, embodiments of the invention are implemented as a computer program product having program code, which, when the computer program product runs on a computer, is operated to perform one of the methods. The program code may, for example, be stored on a machine readable carrier.

[0226] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0227] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0228] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) comprising recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium or recorded medium is generally tangible and / or transitory.

[0229] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, for example a data stream or a sequence of signals adapted to be transmitted via a data communication connection, for example the Internet.

[0230] A further embodiment comprises a processing means, for example a computer, or a programmable logic circuit, configured to or adapted to perform one of the methods described herein.

[0231] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0232] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory element, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0233] In some embodiments, a programmable logic circuit (e.g., a Field Programmable Gate Array) may be used to perform some or all of the functions described herein. In some embodiments, the Field Programmable Gate Array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

[0234] The above described embodiments merely represent examples of the principles of the present invention. It is understood that modifications and variations of the devices described herein will be apparent to others skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the detailed description of the specification as set forth in the description and illustration of the invention.

Claims

1. 1. A decoder for reconstructing a predefined block (210) of a depth / disparity map (213) associated with an image (215) from a data stream (304), the decoder comprising: segmenting the reference block (216) of the image (215) by a binarization process of the image (215) within the reference block (216) co-located with the predetermined block (210) to obtain two segmentations of the reference block into a first and a second partition; spatially transferring the two segmentations of the reference block (216) of the image to the predefined block (210) of the depth / disparity map (213) to obtain first and second partitions (202a, 202b) of the predefined block (210); and A decoder configured to decode the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b).

2. 2. The decoder of claim 1, wherein each of the first and second partitions (218') of the reference block (216) of the image (215) is a set of tiles that together completely cover the reference block (216) of the image (215) and are configured for individually checking the values ​​of the image (215) within the reference block (216) in the tiles of a two-dimensional subdivision of the reference block (216) as to whether the respective values ​​are greater or lower than a respective predefined value in a binarization process, so as to be complementary to each other.

3. 3. A decoder as claimed in claim 2, arranged for, in a binarisation process, checking individually the image values ​​within the reference block at sample resolution, such that each tile corresponds to a sample position of the reference block.

4. 4. A decoder according to claim 1, configured for applying, during segmentation, morphological hole-filling and / or a low-pass filter to a result of the binarization process in order to obtain the two segmentations of the reference block (216) into the first and second partitions.

5. 5. A decoder according to claim 1, arranged to perform a binarization process by determining a measure of central tendency of reconstructed sample values ​​of the reference block (216) of the image (215) and by comparing each reconstructed sample value of the reference block (216) of the image (215) with a respective threshold value which is dependent on the determined measure.

6. The decoder is configured such that the segmentation, spatial transfer and prediction form one of a first set of coding options and are not part of a second set of coding options, where the decoder is further configured to: determining a variance of sample values ​​within the reference block (216) of the image (215); receiving an encoding option identifier from the data stream; 6. A decoder as claimed in claim 1, configured to perform segmentation, spatial transformation and prediction on the given block if an index indicates the first set of coding options, and to use the coding option identifier as the index to the first set of coding options if the variance exceeds a predetermined threshold, and to use the coding option identifier as the index to the second set of coding options if the variance follows the predetermined threshold.

7. 1. A decoder for reconstructing a predefined block of a depth / disparity map associated with an image from a data stream, the decoder comprising: segmenting the reference block (216) of the image (215) depending on texture features of the image (215) within the reference block (216) co-located with the predetermined block to obtain two segmentations of the reference block into a first and a second partition; spatially transferring the two segmentations of the reference block of the image to the predefined block (210) of the depth / disparity map to obtain first and second partitions of the predefined block; and configured to decode the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); wherein the decoder is configured such that the segmentation, spatial transfer and prediction form one of a first set of coding options for the decoder and are not part of a second set of coding options for the decoder, wherein the decoder is further configured to: determining a variance of sample values ​​within the reference block (216) of the image (215); receiving an encoding option identifier from the data stream (304); and if the index indicates the first set of coding options, performing segmentation, spatial transformation and prediction on the given block (210), and using the coding option identifier as the index to the first set of coding options if the variance exceeds a predetermined threshold, and using the coding option identifier as the index to the second set of coding options if the variance follows the predetermined threshold.

8. The decoder of claim 7 , wherein the decoder is configured to retrieve the encoding option identifier by entropy decoding.

9. A decoder as claimed in claim 1 to 8, further configured to use the predefined block as a reference in a prediction loop of the decoder.

10. The decoder, in decoding the predetermined block (210), 9. The decoder of claim 1, configured to predict the predetermined block (210) by assigning first fixed partition values ​​to samples (203) of the sample array located within the first Wedgelet partition (202a) and assigning second fixed partition values ​​to samples (203) of the sample array located within the second Wedgelet partition (202b).

11. 11. The decoder of claim 10, further comprising: in a process of predicting the predetermined block, the decoder is configured to associate each adjacent sample of the sample array that is adjacent to the predetermined block with a respective one of the first and second Wedgelet partitions, such that each adjacent sample is adjacent to the Wedgelet partition with which it is associated; and configured to predict the predetermined block by assigning an average value of values ​​of the adjacent samples associated with the first partition to samples of the sample array located within the first partition, and by assigning an average value of values ​​of the adjacent samples associated with the second partition to samples of the sample array located within the second partition.

12. 12. A decoder as claimed in claim 10 or claim 11, configured to refine the prediction of the default block by applying a first refinement value within the refinement information to the average value of values ​​of the adjacent samples associated with the first partition and / or by applying a second refinement value within the refinement information to the average value of values ​​of the adjacent samples associated with the second partition.

13. 13. The decoder of claim 12, configured for, in the process of applying the first and / or second refinement value, linearly combining the first and / or second refinement value with the average value of values ​​of the neighboring samples associated with the first partition and / or the average value of values ​​of the neighboring samples associated with the second partition, respectively.

14. A decoder as claimed in claim 12 or claim 13, configured in the process of applying the first and / or second refinement value to retrieve the first and / or second refinement value from the data stream and to scale the first and / or second refinement value such that a predefined sample component associated with the sample array is spatially retrieved using a quantization step size that depends on a reference quantization step size transmitted within the data stream.

15. 15. The decoder of claim 14, wherein the sample array is a depth map, and the decoder is configured to use the reference quantization step size to reconstruct a texture sample array from the bitstream associated with the depth map.

16. 1. An encoder for encoding a predefined block of a depth / disparity map associated with an image into a data stream, the encoder comprising: segmenting the reference block of the image by binarizing the image within the reference block co-located with the predetermined block to obtain two segmentations of the reference block into a first and a second partition; - spatially transferring the two segmentations of the reference block of the image to the predefined block of the depth / disparity map to obtain first and second partitions of the predefined block; and An encoder configured for encoding the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b).

17. 1. An encoder for encoding a predefined block of a depth / disparity map associated with an image into a data stream, the encoder comprising: segmenting a reference block (216) of the image (215) depending on texture features of the image (215) within the reference block (216) co-located with the predetermined block to obtain two segmentations of the reference block into a first and a second partition; spatially transferring the two segmentations of the reference block of the image to the predetermined block of the depth / disparity map to obtain first and second partitions of the predetermined block; and configured to encode the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); wherein the encoder is configured such that the segmentation, spatial transfer and encoding form a first set of encoding options for the encoder and are not part of a second set of encoding options for the encoder, wherein the encoder is further configured to: determining a variance of sample values ​​within the reference block (216) of the image (215); encoding an encoding option identifier into said data stream (304); an encoder configured to, when an index indicates the first set of encoding options, use the encoding option identifier as the index to the first set of encoding options if the variance exceeds a predefined threshold, and to use the encoding option identifier as the index to the second set of encoding options if the variance follows the predefined threshold.

18. 1. A method for reconstructing a predefined block (210) of a depth / disparity map (213) associated with an image (215) from a data stream (304), the method comprising: segmenting the reference block of the image (215) by a binarization process of the image (215) within the reference block (216) co-located with the predetermined block (210) to obtain two segmentations of the reference block into a first and a second partition; - spatially transferring the two segmentations of the reference block (216) of the image to the predefined block (210) of the depth / disparity map (213) to obtain first and second partitions (202a, 202b) of the predefined block (210); decoding the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); A method comprising:

19. 1. A method for reconstructing a predefined block of a depth / disparity map associated with an image from a data stream, the method comprising: segmenting the reference block (216) of the image (215) depending on texture features of the image (215) within the reference block (216) co-located with the given block in order to obtain two segmentations of the reference block into a first and a second partition; - spatially transferring the two segmentations of the reference block of the image to the predefined block of the depth / disparity map to obtain first and second partitions of the predefined block; and decoding the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); Including, wherein the method is performed such that the segmentation, spatial transfer and decoding form a first set of encoding options and are not part of a second set of encoding options of the method, wherein the method further comprises: determining the variance of values ​​of samples within said reference block (216) of said image (215); retrieving an encoding option identifier from the data stream; if the index indicates the first set of coding options, performing the segmentation, spatial transformation and prediction on the predetermined block (210) and using the coding option identifier as the index to the first set of coding options if the variance exceeds a predetermined threshold, and using the coding option identifier as the index to the second set of coding options if the variance follows the predetermined threshold; A method comprising:

20. 1. A method for encoding a predefined block of a depth / disparity map associated with an image into a data stream, the method comprising: - segmenting the reference block of the image by binarizing the image within the reference block co-located with the predetermined block to obtain two segmentations of the reference block into a first and a second partition; - spatially transferring the two segmentations of the reference block of the image to the predefined block of the depth / disparity map to obtain first and second partitions of the predefined block; and encoding the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); A method comprising:

21. 1. A method for encoding a predefined block of a depth / disparity map associated with an image into a data stream, the method comprising: segmenting the reference block (216) of the image (215) depending on texture features of the image (215) within the reference block (216) co-located with the given block in order to obtain two segmentations of the reference block into a first and a second partition; - spatially transferring the two segmentations of the reference block of the image to the predefined block of the depth / disparity map to obtain first and second partitions of the predefined block; encoding the predetermined block (210) in units of the first and second Wedgelet partitions (202a, 202b); Including, wherein the method is performed such that the segmentation, spatial transfer and encoding form one of a first set of encoding options of the method and are not part of a second set of encoding options of the method, the method further comprising: determining the variance of values ​​of samples within said reference block (216) of said image (215); encoding an encoding option identifier into said data stream (304); if the index indicates the first set of coding options, performing the segmentation, spatial transformation and prediction on the predetermined block (210) and using the coding option identifier as the index to the first set of coding options if the variance exceeds a predetermined threshold, and using the coding option identifier as the index to the second set of coding options if the variance follows the predetermined threshold; A method comprising:

22. Computer program having a program code for performing the method according to claim 18 or claim 21, when the program is run on a computer.

Images