Efficient Subpicture Extraction

Abstract

To provide an encoding method and an encoder, and a decoding method and a decoder that provide the concept of efficient sub-picture extraction.SOLUTION: In a video coding method, a sub-picture extraction process 14 is made less complex by providing each slice with start position information 42 which indicates a starting position 40 from which onward a picture 18 is, along a coding path 22, encoded into respective slice 26, relative to a picture onset position 24 at which the coding path starts traversing a segment 30 where the starting position is located. Alternatively or additionally, a sub-picture extraction process is processed similar to choosing one layer out of a multi-layer data stream.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present application relates to the concept of efficient sub-picture extraction. [Background technology]

[0002] Sub-picture extraction is a process in which a data stream in which a video picture is coded is adjusted to a sub-picture area without the need for re-encoding. HEVC DAM3 MCTS extraction allows, for example, to extract a sub-picture specific data stream from the original full picture data stream without the need for any re-encoding, as far as, for example, motion compensated prediction and residual coding are concerned, for example, by continuously dividing the picture into tiles that are coded independently of each other and grouping them into tile sets that are restricted such that even motion compensated prediction does not cross tile set boundaries. This MCTS extraction, however, requires the adjustment of each NAL unit carrying slice data to adjust the slice segment addresses of the slice headers it carries.

[0003] Thus, while sub-picture extraction processes are known, it would be advantageous to have a concept at hand that reduces the complexity of the tasks involved. Summary of the Invention [Problem to be solved by the invention]

[0004] It is an object of the present invention to provide a data stream, a decoder and / or an encoder according to a concept that reduces the complexity of sub-picture extraction. [Means for solving the problem]

[0005] This object is achieved by the subject matter of the independent claims of the present application.

[0006] According to a first aspect of the present application, the complexity of the sub-picture extraction is reduced by providing each slice with starting point information indicating the starting position from which the picture is coded into the respective slice along the coding path, relative to the onset position where the coding path starts to traverse the segment in which the starting position is located. The picture is subdivided into segments that are traversed by the coding path successively in segments, each slice being coded with a fragment of the segment or one or more segments completely, and the picture is coded into segments without coding interdependence. The picture is coded into the data stream along the coding path in slice units in which the picture is partitioned. In addition, each slice includes segment information indicating the segment in which the starting position is located. Thus, the starting position information and the segment information together make it possible to determine in which segment and where in this segment the starting position of each slice is located. By coding the segments without coding interdependence, it is possible to remove one or more slices for one segment without affecting the decodability of another segment. Also, even if segments of a picture are used to compile a new sub-picture by shuffling or rearranging the segments and / or discarding some of the segments of the picture, it is possible to form each sub-picture specific data stream based on the original data stream by discarding slices that do not code any segments forming the sub-picture and taking the non-discarded slices that still refer to the correct segments, and by relative position indications indicate the correct position of the start position of each slice in the respective segment. Thus, according to the first aspect of the present application, the data stream allows easy sub-picture extraction without the need to correct the start position information and segment information, despite rearranging the segments of the picture in the sub-picture data stream and / or excluding some segments.In other words, it makes the sub-picture extraction process easier by having the opportunity to employ the remaining slices without having to correct the starting position information and simply remove or discard slices that are not related to any segment contained within the sub-picture to be extracted.

[0007] According to one embodiment, the start position information comprises a variable length coded start address syntax element. Variable length coding can be used without any penalty during sub-picture extraction, since the start position information does not need to be inverted in any way during sub-picture extraction.

[0008] According to yet further embodiments, the data stream can be switched by a start position coding mode flag between start position information indicating the start position of each slice relative to the onset position of the segment or absolutely relative to the picture onset position where the coding path starts traversing the picture. With this option, existing codecs that previously use absolute start position indications can be extended to take advantage of the relative position indications discussed herein. A decoder may be able to understand both types of start position indications provided by the start position information or just one of them.

[0009] According to yet a further embodiment, the data stream further comprises a base address data field defining, for each segment, a base address of the onset position of the respective segment that addresses an onset position along the coding path relative to the picture onset position. The communication of these base addresses reuses the computational overhead for calculating base addresses based solely on the subdivision of pictures into segments.

[0010] According to one embodiment, the segment information includes a segment syntax element in which a segment index is encoded that indexes the segment in which the start position is located. The syntax element may be encoded using a fixed length code. The data stream may include an index data field that defines the relevance of the segment index value to each segment. In other words, the index data field may explicitly associate a tag, i.e., a segment index value, to the segment, and the segment information of a slice is set to the tag, or segment index value, associated with the segment in which the start position of the respective slice is located. This tagging may be easily modified during sub-picture extraction.

[0011] Another aspect of the present application, which may be combined with the first aspect, is to make the sub-picture extraction process easier by treating it similarly to picking one layer out of a multi-layer data stream. That is, according to the second aspect, the extractable data stream includes at least one sub-picture decoding option and a parameter set that also indicates a reduced picture size for the at least one sub-picture decoding option and a second decoder capability level required to decode the sub-picture from the extracted version of the data stream, as well as an indication of the size of the pictures of the data stream and the first decoder capability level required to decode the pictures from the data stream. This sub-picture decoding option is treated similarly to a sub-layer for a higher layer corresponding to a complete picture, and the extracted version of the data stream results from the data stream by removing or discarding slices from the data stream in which segments that are spatially offset with respect to the segment clusters of which the sub-pictures are composed are encoded. These slices may or may not include starting position information and segment information as discussed above with respect to the first aspect. However, it may be sufficient if each slice contains segment information indicating the segments that the respective slice covers. Thus, according to the second aspect, the process of sub-picture extraction is similar to the transition between several layers of a multi-layer data stream, and only requires the exclusion of slices that do not relate to the intended layer, i.e. slices that do not cover any of the segments contained in the sub-picture are now extracted. No "conversion" or "modification" of the slices of the data stream to be adopted in the extracted data stream is necessary.

[0012] Advantageous aspects are the subject of the dependent claims. Preferred embodiments of the present application are described below with reference to the drawings. [Brief description of the drawings]

[0013] [Figure 1] A schematic diagram showing the concept of a data stream that can be extracted using relative start position indication for slices to alleviate the sub-picture extraction process, showing a data stream that has not yet been extracted, as well as the extraction process and the extracted data stream, with participating entities including an encoder, a decoder, and optionally a network device. [Diagram 2] FIG. 2 is a schematic diagram showing pictures and fragments of a data stream to illustrate an embodiment in which the data stream is switchable between relative and absolute slice start position indications; [Diagram 3] FIG. 2 is a schematic diagram showing an implementation of a data stream as presented with respect to FIG. 1 as a variant of an HEVC data stream. [Figure 4] FIG. 4 shows pseudocode illustrating content conveyed within a parameter set, here illustratively a picture parameter set, that may be used in the implementation of FIG. 3. [Diagram 5] FIG. 4 is a schematic diagram of a picture and its segmentation into segments, along with the parameters used in the pseudocode presented with respect to the example implementation provided with respect to FIG. 3. [Figure 6] 4 illustrates pseudocode illustrating possible content conveyed in a slice header according to the example implementation of FIG. 3. [Figure 7] FIG. 4 shows pseudocode illustrating a tiling syntax that may be used to implement the example codec of FIG. [Figure 8] FIG. 8 shows pseudocode illustrating possible contents of a parameter set, here illustratively a sequence parameter set, using the tiling syntax of FIG. 7. [Figure 9] FIG. 8 shows pseudocode illustrating possible contents of a parameter set, here exemplarily a picture parameter set, using the tiling syntax of FIG. 7. [Figure 10]FIG. 7 shows pseudocode illustrating possible contents of a slice header for implementing the example of FIG. 3, which differs from FIG. 6 in that a segment may be a composition of one or more tiles. [Figure 11] FIG. 11 shows pseudocode illustrating possible contents of a picture parameter set, here exemplarily a sequence parameter set, to illustrate the possibility that the parameter set conveys information not just about complete picture decoding options, but additionally about at least one sub-picture decoding option. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] A first embodiment of the present application will be described with reference to Fig. 1. Fig. 1 shows a data stream 10 and a sub-picture specific data stream 12 derived therefrom by a sub-picture extraction 14. The data stream 10 may have been generated by an encoder 16. In particular, the encoder 16 has encoded a picture 18 in the data stream 10. It should be noted that, although Fig. 1 shows the picture 18 as one picture of a video 20, the embodiments of the present application are not limited to video data streams. Rather, the embodiments described herein can be easily transferred to a picture codec. The following description, however, shows the embodiment in terms of video coding.

[0015] When encoding picture 18 into data stream 10, encoder 16 follows or uses a coding path 22 that traverses picture 18 from a picture onset position 24, which may be located in the upper left corner of picture 18, as shown in FIG. 1, toward an opposite picture corner, such as the lower right corner, for example, to continue traversing to another picture of video 20. Following or using coding path 22 may, for example, determine the availability of spatially neighboring portions of picture 18, such as samples or parameters derived therefrom by encoder 18, such as syntax elements, for coding the current portion of picture 18. For example, encoder 16 may use predictive coding. To this end, encoder 16 may predict the current portion, such as its sample content or syntax elements describing the sample content, based on spatially neighboring portions of picture 18, provided that these neighboring portions precede the current portion along coding path 22. Additionally or alternatively, the encoder 16 may use other spatial coding dependencies for the current portion, such as deriving an entropy context for coding syntax elements describing the current portion of the picture 18 based on adjacent portions of the picture 18. In addition, the encoder 16 partitions or packets the data in which the picture 18 is coded, using a coding path 22. To this end, the encoder 16 subdivides the coded data into slices along the coding path 22. Thus, each slice 26 in the data stream 10 containing data for the picture 18 covers or encodes a corresponding portion, i.e., a slice 28 of the picture 18, which is traversed by the coding path 22 consecutively, without interleaving the slices 28. In other words, the coding path 22 traverses each of the slices 28 into which the picture 18 is subdivided only once.

[0016] As also shown in FIG. 1, the encoder 16 also follows the subdivision of the picture 18 into segments 30 when encoding the picture 18 into the data stream 10. FIG. 1 exemplarily shows the subdivision of the picture 18 into four segments 30 arranged in a 2x2 array. The number of segments is not important, however, and may differ. The subdivision of the picture 18 into segments 30 may, however, be such that the segments 30 are arranged in rows and columns. The subdivision of the picture 18 into segments 30 may be such that the segments within a segment row are of the same height and the segments within a segment column are of the same width, completely covering the picture 18 without gaps. However, the rectangular segments 30 may thereby differ in size, in that the segment columns and / or segment rows may differ in terms of width and height, respectively. Following the subdivision of the picture 18 into segments 30 may relate to coding interdependencies. The encoder 16 encodes the picture 18 into the data stream 10 such that the coding interdependencies do not cross segment boundaries. For this reason, no part of picture 18 within a segment 30 is coded with any dependency on any part of picture 18 outside this segment. In other words, encoder 16 encodes segments 30 independent of each other. As long as the relationship of slices 28 on the one hand and segments 30 on the other is considered, a slice 28 may completely cover one or more segments 30, or may only be present in one segment 30. For illustrative purposes only, FIG. 1 shows each segment 30 to be composed of two slices 28. This means that coding path 22 traverses segments 30 consecutively, i.e., each segment 30 is completely traversed by coding path 22 before coding path 22 traverses the next segment 30 in coding order.

[0017] In this manner, encoder 16 encodes picture 18 into data stream 10 in slices 28. Each slice 26 in data stream 10 thus has a corresponding encoded picture slice 28. For example, each slice 26 has payload data 32 that encodes the content of a corresponding slice portion 28 of picture 18 in the form of one or more of prediction parameters, e.g., a prediction mode distinguishing between intra- or inter-coding modes, motion parameters for intra-predicted blocks, intra-prediction sub-modes for intra-predicted blocks, subdivision information for subdividing picture 18 into blocks, and residual data, e.g., transform coefficients representing prediction residuals.

[0018] Before proceeding to a description of the slice header 34 of the slice 26 and its components or syntax elements, respectively, a brief description of how the encoder 16 codes the picture 18 in the data stream 10 follows. As mentioned before, the segments 30 are segments coded independently of each other. However, the description so far focuses only on the encoding of the picture 18. However, if the picture 18 is a picture of the video 20, it may be that the encoder 16 adopts a subdivision of the picture into segments 30 for the sequence of pictures of the video 20, i.e., subdivision of the pictures of the picture sequence in the same way, i.e., so that the other pictures are segmented into the same number of segments 30 and the segments 30 are of equal size with the segment boundaries spatially coinciding between the pictures. This is, for example, the situation shown in FIG. 1, where the pictures preceding and following the picture 18 are shown as being subdivision in the same way as the picture 18, i.e., into four segments 30. In addition to the mutual independence of the coding segments 30 within one picture, the encoder 16 may code each segment of the picture 18 such that the encoding of the respective segment 30 of the picture 18 does not depend on a portion of another picture, called a reference picture, that is outside or spatially offset from the segment colocated with the respective segment of the picture 18. In other words, the colocated segments 30 of pictures in a sequence of pictures of the video 20 may form a spatio-temporal region in the video 20 in which the encoder 16 performs encoding independently of the spatio-temporal region formed by the other segments of these pictures. Thus, the aforementioned inter-prediction, i.e., motion-compensated prediction, will be restricted such that a block in one segment 30 of the picture 18 coded using inter-prediction will not reference, by a motion vector, a portion of a reference picture that is outside the segment of the reference picture colocated with the segment in which the current block is located. Rather, the encoder 16 will instead choose, for example, intra-prediction for that block.

[0019] Still further, it should be noted that in addition to the slice headers 34 of the slices 26, which will be described in more detail below, the data stream 10 includes further information including higher level coding parameters, hereinafter referred to as parameter sets. This parameter set 36 may be distributed over the data stream 10 and may be defined hierarchically in terms of scope. In other words, different parts of the parameter set 36 may have different scopes. To this end, some coding parameters may be valid for only one picture, such as picture 18, other coding parameters may represent coding parameters valid for a sequence of pictures, including, for example, picture 18, and still other coding parameters may be valid for the entire video 20. The coding parameters defined within the parameter set 36 will also be described below, so a brief discussion will suffice for the moment. The coding parameters included within parameter set 36 may indicate, for example, the picture size of the pictures of video 20, the subdivision of the pictures into segments 30, an indication of the decoder capability level required to decode pictures 18 and video 20, respectively, from data stream 10, and / or the decoding timestamps, coded picture buffer retrieval times, and / or buffer sizes to be used to decode data stream 10.

[0020] In addition to the details described above, it is noted that the encoder 16 may form the data stream 10 such that slices 26 included in the data stream 10 for one picture, such as picture 18, are not interleaved with other slices 26 in the data stream 10 for another picture of the video 20. Rather, they form a contiguous portion of the data stream 10, called an access unit 36. The access unit 36 ​​thus groups together the slices 26 for one picture, such as picture 18. In addition to the slices 26, other data may be included in an access unit 36 ​​as well, such as a portion of a parameter set 36. The data in an access unit 36 ​​may adhere to some predefined order or ordering convention such that a receiver of the data stream 10, such as a decoder 38, can recognize the start of the next access unit in the data stream 10 based on monitoring the order of the data in the data stream 10. Even the slices in an access unit 36 ​​may be ordered relative to one another according to the order in which the coding path 22 traverses the corresponding slices 28 in the picture 18.

[0021] After having described the general framework for coding the data stream 10, the description proceeds to a description of the slice headers 34 and their contents. In particular, the encoder 16 provides each slice header 34 with start position information indicating a start position 40 from which the picture 18 is to be coded along the coding path 22 into the respective slice 26 that contains the respective slice header 34. Knowledge of the start position 40 is intended to enable the decoder 38 to correctly locate the picture content coded in the respective slice 26 for the preceding slice portion 28 along the coding path 22, i.e. the picture content for the corresponding slice portion 28 of the picture 18, even if any preceding slice for the picture 18 is missing in the data stream 10.

[0022] Preferably, however, the start position information 42 indicates a start position 40 of the slice 26 within the slice header 34 in which it is contained, relative to an onset position 44 at which the coding path 22 begins traversing the segment 30 in which the start position 40 is located, as illustratively indicated by arrow 46 in Figure 1. The relative indication of the start position 40 simplifies sub-picture extraction 14, as will be explained further below.

[0023] In order to compensate for the relative indication of the start position 40 by the start position information 42 and its associated ambiguity, the slice header 34 additionally includes segment information 48 indicating the segment 30 in which the start position 40 of the slice 26 is located, to which the segment information 48 belongs. In the example of Fig. 1, for example, the slice 28 whose start position 40 is indicated by the arrow 46 is located in the segment 30 indexed by the capital letter B in Fig. 1, and accordingly, the segment information 48 of that slice will indicate that this slice 26 belongs to segment B. Thus, the start position information 42 and the segment information 48 together are sufficient to determine the absolute position of the start position 40 in the picture 18, i.e., the relative position or start position 40 with respect to the origin, i.e., the picture onset position 24.

[0024] A decoder 38 receiving the data stream 10 can decode from the data stream 10 the start position information 42 of each slice 26 and use the start position information 42 to locate the start position 40 of each slice relative to an onset position 44 of the segment in which the start position 40 is located. Furthermore, the decoder 38 decodes from the data stream 10 the corresponding segment information 48 of each slice 26 so that the decoder 38 knows as to which segment 30 the onset position 44 belongs for which the start position information 42 indicates the start position 40. The decoder 38 can therefore locate the onset position 44 based on a parameter set 36 indicating the subdivision of the picture 18 into segments 30 and accordingly combine the relative position indication of the start position 40 indicated with respect to the onset position 44 with the onset position 44 in the picture 18, this combination resulting in an absolute position of the start position 40 in the picture 18 relative to the picture onset position 24. To reduce computational overhead for the decoder 38, a more detailed embodiment described further below illustrates the possibility that the parameter set 36, or some other part within the data stream 10, such as supplemental extension information, may expressively indicate the onset position 44 for each segment 30, so that the decoder 38 can use this indication rather than calculating the onset position of the segment 30 itself.

[0025] It has already been outlined above that a relative indication of the start position 40 with respect to the onset position of the corresponding segment 30 is a preferred situation. This will be discussed in more detail below. In particular, the preferred situation becomes more apparent when examining the sub-picture extraction process 14. The latter process may be performed by any entity that processes the data stream 10. For example, the extraction process may be performed by a network device 50 connected to the transmission path between the encoder 16 and a decoder 52 that receives the extracted data stream 12. The decoder 52 may be identical to the decoder 38, i.e. a decoder that would also have been able to decode the complete or original data stream 10, or a decoder that is able to decode the extracted data stream 12 but is not able to decode the complete data stream 10, for example due to a decoder capability level lower than that of the decoder 38. The extraction process 14 may, however, be performed within the decoder 52 / 38 itself.

[0026] The extraction process 14 aims at the formation of a data stream 12 in which a picture 54 is encoded which, compared to picture 18, is composed solely of a suitable subset of the segments 30 of picture 18 and / or differs in the mutual arrangement of the segments 30 within the picture areas of pictures 18 and 54, respectively. Although Fig. 1 shows the case in which picture 54 of data stream 12 is composed solely of one of the four segments 30 of picture 18 of the original data stream 10, namely segment B, this is merely an example and picture 54 may alternatively be composed of two or more of the segments 30 of picture 18, as long as a rectangular picture area of ​​picture 54 is provided.

[0027] Notably, for slices 26 included in the data stream 10, the extraction process 14 only requires discarding or excluding slices 26 for segments 30 of pictures 18 that are not included in the picture 54, while slices 26 for any segments 30 that contribute to the composition of the picture 54 are adopted or left in the extracted data stream 12 without any modification, in particular without any modification of the start position information 42 and the segment information 48. In particular, since the start position information 42 indicates that the start position 40 of a slice 26 that falls in a segment 30 whose start position contributes to the picture 54 is indicated relative to the onset position of that segment, this relative indication is still accurate in the reduced data stream 12. Moreover, the segment information 48 indicates, for all slices 26 adopted in the extracted data stream 12, which segment the corresponding slice 26 belongs to, i.e., in which segment its start position is located. This segment information 48 is still used in the extraction process 14 to identify slices 26 that are to be excluded or discarded.

[0028] The only information present in the extracted data stream 12 that has not yet been discussed with respect to the original data stream 10 is the coding parameters associated with a modified configuration of the pictures 54 of the extracted data stream 12 based on the selected set of segments 30. Regarding this topic, several possibilities exist. According to a first alternative, the extraction process 14 may require modification of said parameter set 36' in the extracted data stream 12 to reflect a modified picture size of the pictures 54 of the extracted video 56, a subdivision of the pictures 54 into segments, a decoder capability level required to decode the extracted data stream 12 that may be lower than the decoder capability level required to decode the complete video 20 or pictures 18 from the data stream 10, and / or modified decoding timestamps, coded picture buffer retrieval times and / or buffer sizes for decoding the extracted data stream 12. However, according to an alternative embodiment, the aspect of using the relative start position indication in the start position information 42 is combined with an aspect of the present application, in which even the parameter set 36' of the extracted data stream 12 may be left unchanged, in other words by providing both items of information in the parameter set 36 of the original data stream 10. According to this alternative, the parameter set 36 would indicate all the coding parameters for decoding the whole picture 18 from the data stream 10, and at the same time, would indicate all the coding parameters for decoding the picture 54 from the data streams 10 or 12. The coding parameters indicated by the parameter set 36 for the extracted data stream 12 and its picture 54 may be indicated in the parameter set 36 at least partially relative to or in a different manner than the coding parameters indicated by the parameter set 36 for decoding the picture 18 from the data stream 10.For example, as will be outlined in more detail below, parameter set 36 may indicate a subdivision of picture 18 into segments 30 such that the size of segments 30 is clear from this portion of parameter set 36. The coding parameters of parameter set 36 for picture 54 may rely on this knowledge and may simply indicate as to which segments contribute to picture 54 and as to which mutual arrangement of these contributing segments within picture 54 applies. Even the aforementioned base address indication for onset position 44 within picture 18 may also be repeated in data stream 10 for picture 54 of extracted data stream 12. And as will be further explained below, parameter set 36 may thus not only indicate one extraction or sub-picture decoding option for data stream 10, but several extraction or sub-picture decoding options that differ in terms of the configuration of picture 54 based on segments 30 of picture 18. Thus, in the case of the alternative where the parameter set 36' remains unchanged with respect to the parameters 36 in the original data stream 10, the decoder 52 will receive a decoding option indication 58 indicating, for example, whether picture decoding, i.e., decoding of the picture 18, should be performed on the data stream 12, or sub-picture decoding, i.e., decoding of the picture 54, should be performed on the data stream 12. If the unused slice 26 has already been removed as part of a previously performed extraction process 14, this decoding option indication should not necessarily indicate picture decoding as the slice is missing. However, recall that the extraction process 14 may have been performed by the decoder 52 itself, which may be equivalent to the decoder 38. To this end, the decoder 52 may receive the complete original data stream 10 and then decide, based on the decoding option indication 58, to perform the extraction process 14 in that it removes the unused slice 26 itself, i.e., discards it. The decoding option indication may be, for example, an external signaling provided by an application or a user.Alternatively, the decoding option indication may be a specific indication of the decoder capability level of the decoder 52, which excludes in any case some of the decoding options of the data stream 10 from being decodable by the decoder 52. If picture decoding were to be performed for the complete picture 18, the decoding would be performed as discussed with respect to the decoder 38. If the sub-picture decoding option is applied, i.e., the decoder 52 decodes the picture 54 from the data stream 12, the decoding process is substantially the same with the following differences: the decoder 52 will use coding parameters of a particular parameter set 36' for this sub-picture decoding, i.e., for the picture 54. Furthermore, if the calculation is to be performed by the decoder 52 itself, the decoder 52 will calculate a different base address for the onset position 44 of the segment 30 in the picture 54. The different calculation result is a result of the different arrangement of the segment 30 in the picture 54. The start position information 42 indicating the relative position of the start position 40 of the slice is the same as that indicated by the start position information 42 for the onset position 44 of the segment. However, if the segment onset position 44 is indicated in parameter set 36' for the segment's placement within picture 54, decoder 52 may use these base addresses specific to sub-picture decoding to obtain picture 54 instead of its own calculations.

[0029] Further details regarding how the decoder 38 / 52 may operate in decoding the inbound data stream, whether 10 or 12, are provided at the end of the description regarding FIG.

[0030] In the following description, a specific example for implementing the embodiment outlined above is described. In so doing, it is assumed that the data stream includes a start position coding mode flag 60 in the parameter set 36 that switches between a relative start position indication 46 of the start position 40 of a slice relative to the segment onset position 44 as described with respect to FIG. 1 and an absolute start position indication 62 of the start position 40 of a slice relative to the picture onset position 24. In other words, the flag 60 signals whether the start position indication 42 in the slice 26 of the data stream 10 indicates the start position 40 as outlined with respect to FIG. 1, i.e., relative to the segment onset position 44 or absolutely to the picture onset position 24. The flag 16 can then be used to switch on the presence of a segment indication 48, as will be further discussed below. In the case of a merely relative start position indication 46, the segment indication 48 can be present in the slice 26.

[0031] A further detail that emerges from the next described embodiment is the fact that the slice position indication 42 may in fact be signaled by a combination of a flag and a conditionally signaled slice address. In the case of the relative indication option 46, the flag included by the start position information 42 will indicate whether the corresponding slice is the first slice in a segment, i.e. whether its start position 40 coincides with the onset position 44 of the corresponding segment. In that case, the start position information 42 will not include any start address syntax element. If not, however, the address syntax element points from the onset position 44 of the corresponding segment to the start position 40 of the slice. In the case of the absolute indication option 62, the flag of the start position information 42 will indicate whether the corresponding slice starts at the picture onset position 24, i.e. whether it is the first slice of a picture, and if so, the slice position information will not include any address syntax element. If not, however, the address syntax element indicates the start position 40 of the respective slice relative to the picture onset position 24. The flags contained by the starting position information 42 just mentioned can be used by the decoder 38 / 52 in the case of the absolute pointing mode to access unit boundary detection, i.e., detection of a new access unit 36. To compensate for the lack of ability to use this detectability in the case of the relative pointing mode, the encoder 16 can use alternative access unit boundary mechanisms, such as the placement of access unit delimiters, i.e., a number of packets, at the boundaries between successive access units. The decoder can then use these delimiters in the case of the relative pointing mode.

[0032] As a further remark before commencing with the subsequent more detailed description of the embodiment, it is noted that there are different possibilities for indicating the start position 40 by the start position information 42. For example, the picture 18 may be regularly subdivided into an array of blocks, as shown in the upper left corner of the picture 18 in FIG. 1. The coding path 22 may traverse these coding blocks 64 consecutively, without interleaving, i.e. one coding block 64 is completely traversed before proceeding to the next coding block 64 of the picture 18. The start position information 42 may indicate the position 42 measured in units of traversed coding blocks 64 along the coding path 22 to the start position 40. If indicated relatively 46, the start position information 42 will indicate the number of coding blocks 64 along the coding path 22 that separate the onset position 44 and the slice start position 40. In the case of the absolute indication option 62, the start position information 42 indicates the start position 40 in units of traversed coding blocks 64 along the coding path 22 when starting from the picture onset position 24. Thus, in this case, the start position indication 42 would use a linear, one-dimensional addressing domain, and the base address of the above-mentioned segment onset position 44 may be measured in the number of coding blocks 64 traversed when starting from the picture onset position 24 to the respective segment onset position 44. The sum of the base address 44 and the slice start position 40 would result in an absolute address measured from the start position 24 onwards. However, alternative ways of signaling the start position 40 by the start position information 42 would be feasible. For example, the relative position of the start position 40 with respect to the corresponding segment onset position 44 could be indicated as the x and y coordinates of a vector pointing from the onset position 44 to the slice start position 40, with the components x and y, respectively, measured in samples or units of the coding blocks 64.Furthermore, addressing may be performed according to the strictly row-wise linear addressing of coding blocks 64 discussed above, i.e., such that the addresses of the coding blocks in one complete row of picture 18 follow each other before the addresses increase for the first coding block of the next row in the picture, regardless of the segmentation into segments 30. Internally, decoders 38 and 52 may switch between different address systems or domains as needed.

[0033] Another possibility that becomes evident from the following description is the following: The segments 30 have been described so far as possible spatial cross-sections of spatio-temporal regions, where coding interdependencies do not cross the boundaries of these spatio-temporal regions. These segments 30 can for example be defined as a set of one or more tiles into which a picture 18 can be subdivided and which are coded independently of each other as far as the coding of one picture 18 is concerned. Thus, the segments 30 can be tiles if one segment 30 consists of one tile, and this is the case in variant A described below, whereas variant B assumes that the segments 30 can consist of one or more tiles. Moreover, according to the following description, the aforementioned coding blocks 64 are coding tree units (CTUs), meaning that these blocks 64 are tree root blocks that are further subdivided hierarchically by multi-tree subdivisioning into coding blocks, prediction blocks, and / or transform blocks, in units where the encoder 16 selects the coding mode to be used, i.e., inter-coding or intra-coding, of the corresponding coding block, in units where the prediction parameters of the corresponding selected coding mode are set, i.e., for each prediction block, which may be a leaf node of the coding block, and in units where the transformation of the prediction residual occurs, i.e., transform block units, which may also be leaf blocks of the coding block. The subdivision of the CTUs 64 into coding units, prediction blocks, and transform blocks may be signaled as part of the corresponding slice payload 32.

[0034] Therefore, in the following, an implementation of the embodiment described with respect to Fig. 1 is shown as a possible modification of the HEVC framework. As explained in the introductory part of the specification of this application, in HEVC DAM3 MCTS extraction, adjustments to the slice segment addresses of each slice header are necessary, which may even affect the byte alignment of each slice header due to the variable length coding of the slice addresses. In particular, when using HEVC DAM3 MCTS, the slice addresses of slices adopted in the MCTS extracted data stream will have to be corrected due to their absolute indication option, i.e., these slice addresses will have to be modified to point to new picture onset positions of picture 54, which may result in slice addresses of different lengths and therefore different byte alignment due to the variable length coding. The later described modification of the HEVC framework will overcome this. In particular, the relatively simple MCTS substream extraction results in an implicit or explicit association of slices with tiles / segments by signaling a part of the slice segment addresses.

[0035] In the following, a first variant of the modification of the HEVC framework is described, variant A. As outlined above with respect to Figure 1, according to this variant, instead of signaling a slice segment address for the first CTU of the picture, the slice segment address is signaled for the first CTU of the respective tile or tile configuration, i.e. the current segment 30. "First" means the leftmost and topmost CTU when using the HEVC coding order.

[0036] The difference between the two references, i.e., the first CTU of the current segment 30, or the first CTU of the picture, is derived by the decoder, i.e., decoder 38 or 52, by combining picture and tile command information from parameter sets 36 and 36', respectively, with the association of slice 28 / 26 to this segment 30, which may be a tile or set of tiles, and this association may be explicitly transmitted in the data stream as outlined below.

[0037] An array containing information about the segment size in the CTB and the slice address offset per segment 30 may be determined on the decoder side. The access unit 36 ​​starts the detection to be performed by the decoder 38 or 52, which may be performed based on a flag included in the start position information 42 in case of the absolute indication option 62, since this flag indicates the first slice of the access unit 36 ​​and the picture 18, respectively, and in case of the relative indication option 46, by the use of an access unit delimiter NAL unit in the data stream 10, i.e. a special packet separating consecutive access units in the data stream 10, used to signal the crossing between consecutive access units. As explained above, the first_slice_in_pic_flag may be included by the start position information 42 and may change to a flag indicating the start of the segment 30, which may be a tile or a configuration of tiles.

[0038] Fig. 3 gives an overview of an embodiment resulting from the HEVC technology framework. As shown, the parameter set 36 is distributed in a picture parameter set 70, a sequence parameter set 72, and a video parameter set 74, which are different in scope. Fig. 3 shows only a fragment from the data stream 10, i.e., two VCL NAL units 76 and 78, each including a NALU header 80 indicating that after the slice information, the corresponding slice 26 is included, which consists of a slice header 34 and a slice payload 32. The slice header 34 points to the corresponding picture parameter set 70, as indicated by the arrow 80, the picture parameter set 70 points to the corresponding sequence parameter set, as indicated by the arrow 82, and the sequence parameter set 72 points to the corresponding valid video parameter set, as indicated by the arrow 84. The parameter set 36 suggests, for example, that the picture parameter set 70 or the sequence parameter set 72 includes a tiling syntax 86, i.e., a syntax that defines, according to one alternative, the subdivision of the picture 14 into tiles 88 forming the segments 30 discussed above in Fig. 1. Based on the tiling syntax 86, the decoder is thus able to calculate, for example, a base slice segment address, as indicated by arrow 90. A sum 92 of the base slice segment address 94 and the offset slice segment address 96, conveyed as part of the starting position information 42 and the slice header 34, results in a corresponding slice segment address 98 of the corresponding slice 26, i.e., a slice segment address measured absolutely relative to the picture onset position 24. Figure 3 actually shows two slices 26 depicted as belonging to one tile of tiles 88, namely tile number 2, which is derivable to the decoder based on this segment information 48, which is also included by the slice header 34, as will become clear from the following description.

[0039] 4, the tiling structure or syntax 86 may be included in the picture parameter set 70. Alternative embodiments would be possible as well, for example in which the tiling syntax 86 would reside in the sequence parameter set 72 rather than the picture parameter set.

[0040] Specifically, as shown in Figure 4, part 100 of parameter set 70 indicates the subdivision of picture 18 into segments 30, here tiles. The number of segment columns is indicated at 102, the number of segment rows is indicated at 104, and flag 106 optionally provides the opportunity to signal within the data stream that the widths of the segment columns and the heights of the segment rows are set non-uniformly. If not present or if flag 106 is not set, the widths of the segment columns are indicated individually at 108 and the heights of the segment rows are indicated individually at 110.

[0041] Furthermore, if the flag slice_segment_base_addr_per_tile_enable_flag 112 is set, i.e., at 114, explicit tagging of segments is achieved by explicitly signaling a segment index value tile_id_in_pps[i] for each segment 30 of the picture 18. The segment index is conveyed in the data stream, i.e., in the parameter set 70, by using a default order among the regular array of segments 30, such as row-wise as indicated by the dashed arrow 116 in FIG.

[0042] The semantics of the syntax elements in Figure 4 are outlined in detail below.

[0043] slice_segment_base_addr_per_tile_enable_flag equal to 0 specifies that the slice segment variable CtbAddrInRs is derived solely from slice_segment_address. slice_segment_base_addr_per_tile_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on slice_segment_address and the tile-dependent base address. When slice_segment_base_addr_per_tile_enable_flag is not present, it is inferred to be equal to 0.

[0044] tile_id_in_pps[i] specifies the identification number of the tile in bitstream order. The value of tile_id_in_pps[i] shall range from 0 to 255, and if i is not equal to j, tile_id_in_pps[i] shall not have the same value as tile_id_in_pps[j]. When tile_id_in_pps is not present, it is inferred to be equal to 0.

[0045] When slice_segment_base_addr_per_tile_enable_flag is equal to 1, it may be a bitstream consistency constraint that an access unit delimiter NAL unit is present in the bitstream and tiles_fixed_structure_flag is equal to 1.

[0046] The following pseudocode illustrates how a decoder, such as decoder 38 or 52, calculates certain dimensions of segment 30 and the base address of segment 30 based on information available, for example, in the data stream.

[0047] The following describes the coding tree block raster and tile scan conversion process.

[0048] The list colWidth[i], for i in the range 0 to num_tile_columns_minus1, inclusive, specifying the width of the i-th tile column in coding tree block (CTB) units, is derived as follows (pseudocode 1): if(uniform_spacing_flag) for(i=0;i<=num_tile_columns_minus1;i++ ) colWidth[i]=((i+1)*PicWidthInCtbsY) / (num_tile_columns_minus1+1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1+1) else { colWidth[num_tile_columns_minus1]=PicWidthInCtbsY for(i=0;i <num_tile_columns_minus1;i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0049] colWidth[i] is the width of the i-th segment column in columns 0...num_tile_columns_minus1, measured within the coding tree block CTB64 num_tile_columns_minus1, where num_tile_columns_minus1 is the number of segment columns minus 1.

[0050] The list rowHeight[j], for j ranging from 0 to num_tile_rows_minus1, inclusive, specifying the height of the j-th tile row in CTB units, is derived as follows (pseudocode 2): if(uniform_spacing_flag) for(j=0;j<=num_tile_rows_minus1;j++) rowHeight[j]=((j+1)*PicHeightInCtbsY) / (num_tile_rows_minus1+1)- (j*PicHeightInCtbsY) / (num_tile_rows_minus1+1) else { rowHeight[num_tile_rows_minus1]=PicHeightInCtbsY for(j=0;j <num_tile_rows_minus1;j++){ rowHeight[j]=row_height_minus1[j]+1 rowHeight[num_tile_rows_minus1]-=rowHeight[j] } }

[0051] rowHeight[i] is the height of the i-th segment row in rows 0...num_tile_rows_minus1, measured within the coding tree block CTB64 num_tile_rows_minus1, where num_tile_rows_minus1 is the number of segment rows minus 1.

[0052] The list TileId[ctbAddrTs] for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, inclusive, that specifies the conversion from CTB addresses to tile IDs in a tile scan, is derived as follows (pseudocode 3): for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]]=tileIdx for(y = rowBd[j]; y < rowBd[j + 1]; y++) for(x = colBd[i]; x < colBd[i + 1]; x++) TileId[CtbAddrRsToTs[y * PicWidthInCtbsY + x]] = tileIdx

[0053] tileIdxTileIdMap[] is interpreted as mapping the N segment index values contained within the vector tile_id_in_pps[] to the tileIdx of that segment. tile_id_in_pps[] is a vector containing components 0…N - 1, where N is the number of segments 30, and the i-th component indicates the segment index value associated with the i-th segment in raster scan segment order 116. tileIdx indexes segment 30 in raster scan segment order 116. TileId[i] is interpreted as mapping the CTB address i, i.e., the address of block 64, measured in coding order or along coding path 22, to the tileIdx of the segment 39 in which this CTB is located. CtbAddrRsToTs[i] is a function that maps the rank i of block 64, which is the i-th block 64 of picture 18 in picture raster scan order leading horizontally through block 64 of picture 18, to its address measured in coding order 22. colBd[i] is a vector containing components 0…N, where N is one less than the number of segment columns, i < N, the i-th component indicates the left position of the i-th segment column from left to right measured within block 64, and the N-th component indicates the picture width in number of blocks.

[0054] It should be noted that although all parameters used in the pseudocode so far have been described as being used to determine base addresses in the case of full picture decoding, i.e., for picture 18, these parameters may equally be used to determine base addresses for the onset positions of segments within subpicture 54, where the parameters N, colWidth[i], num_tile_columns_minus1, rowHeight[i], num_tile_rows_minus1, tileIdxTileIdMap[], tile_id_in_pps[], CtbAddrRsToTs[i], colBd[i] are specific to this subpicture 54 and, respectively, its organization by segments 30 and its subdivision into corresponding blocks 64. Similarly, all references to coding path 22 and raster scan segment order 116 apply to the respective subpicture. tileIdxTileIdMap[] ensures the maintenance of the association between, on the one hand, the segment index value tile_id_in_sh, which assumes one of tile_id_in_pps[], and, on the other hand, the segments of the subpicture 54 and their indexing in the raster scan segment order 116, i.e. tileIdx. As a minor note, the syntax element subscript "_pps" is chosen here to reflect the example of signaling a segment index value to a segment association in the PPS, but as already outlined above, it should be noted that this is only an example and the signaling can also be realized elsewhere, such as in the SPS or VPS. In line with the latter description, alternatively the subscript "_ps" is used, with the corresponding substitutions being of course also applicable to the present embodiment. Similarly, the following example makes it clear that one may give up the 1:1 correspondence between tiles and segments. This is also outlined with reference to FIG. 1. Accordingly, corresponding subsequent embodiments will use the name components "Area" or "area_" or "_area" instead, as will become apparent from the following description.1, the explicit assignment of tags or segment index values ​​to segments may be omitted in alternative embodiments by implicitly deriving a mapping between segment index values ​​in segment information 48 and segments in sub-pictures 54, such as by using an index range tileIdx instead, in which case transmission of such an association in parameter set 36 for picture 18 may be omitted. However, the explicit assignment of segment index values ​​to segments 30 of picture 18 mitigates intermixing of tiles of two separate data streams.

[0055] Specifies the size of the kth tile in the picture in CTB units, range from 0 to ((num_tile_rows_minus1+1)) inclusive. * The list TileSizeInCtbsY[k] for k in the range (num_tile_columns_minus1+1)-1) is derived as follows (pseudocode 4). for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++) TileSizeInCtbsY[k]=colWidth[i] * rowHeight[j]

[0056] TileSizeInCtbsY[i] is a vector containing components 0...N-1, where N is the number of segments 30, and the i-th component indicates the number of blocks 64 contained within the i-th segment of the raster scan segment order 116.

[0057] Specifies the slice segment address offset, in CTB units, in the tile traversal of the first slice segment of the kth tile in the picture, in bitstream order, from 0 to ((num_tile_rows_minus1+1) inclusive. *The list TileSliceSegBaseAddr[k] for k in the range (num_tile_columns_minus1+1)-1) is derived as follows (pseudocode 5). TileSliceSegBaseAddr[0]=0 for(k=1;k<=((num_tile_rows_minus1+1) * (num_tile_columns_minus1+1)-1);k++) TileSliceSegBaseAddr[k]=TileSliceSegBaseAddr[k-1]+TileSizeInCtbsY[k-1]

[0058] TileSliceSegBaseAddr[i] is a vector containing components 0...N-1, where N is the number of segments 30, and the i-th component indicates the address of the onset position 44 of the i-th segment in the raster scan segment order 116, measured as the number of blocks 64 along the coding path from the picture onset position 24 to the segment onset position 44.

[0059] Here, "tile scan" denotes the scan 22 as shown in Fig. 1. TileSliceSegBaseAddr denotes the aforementioned base address of the segment onset position 44, which according to this example is equal to the tile. The calculations shown above can be used by the decoders 38 and 52, respectively, to calculate the base addresses. If sub-picture decoding is applied, the number of segments and their arrangement shall be adaptive.

[0060] Figure 6 shows what a slice segment header, i.e. the aforementioned slice header 34, may look like. In particular, the slice header 34 now includes a flag 120, i.e. first_slice_segment_in_area_flag, which together with the syntax element slice_segment_address 122 forms the start position information 42. If the flag 120 is not set, the slice_segment_address is present and indicates the offset of the slice start position 40 from the onset position 44 of the corresponding segment / tile 30. The latter is indicated by just one syntax element 124, i.e. the segment information 48 including tile_id_in_sh, which assumes a value outside the range of values ​​defined in the parameter set 70 of Figure 4, i.e. by syntax element 114.

[0061] In particular, the semantics can be expressed as follows: first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of a picture in decoding order when slice_segment_base_addr_per_tile_enable_flag is equal to 0. Otherwise, first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of a tile of a picture in decoding order when slice_segment_base_addr_per_tile_enable_flag is equal to 1. first_slice_segment_in_pic_flag equal to 0 specifies that the slice segment is not the first slice segment of a picture in decoding order or a tile in decoding order, respectively. Thus, slice_segment_base_addr_per_tile_enable_flag serves as an example for flag 60 discussed with respect to FIG. 2.

[0062] NOTE 1. This syntax element may be used for detection of boundaries between coded pictures that are consecutive in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of an IDR picture may cause problems in detecting boundaries between coded pictures. This may occur, for example, in the transmission of all intra-coded video in an error-prone environment. This problem may be mitigated by alternating two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) for two consecutive IDR pictures. The use of a temporal sub-layer zero index SEI message may also be useful, since that SEI message contains the syntax element irap_pic_id, whose values ​​are different for consecutive IRAP pictures in decoding order. Some system environments have other provisions that may be similarly useful for picture boundary detection, such as presentation timestamps in Rec. ITU-T H.222.0|ISO / IEC 13818-1 systems, access unit framing in the ISO / IEC 14496-12 ISO Base Media File Format, or the use of marker bits in IETF RFC 3550 Real-time Transport Protocol headers.

[0063] A dependent_slice_segment_flag equal to 1 specifies that the value of each absent slice segment header syntax element is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When absent, the value of dependent_slice_segment_flag is inferred to be equal to 0.

[0064] The variable SliceAddrRs is derived as follows: - If dependent_slice_segment_flag is equal to 0, SliceAddrRs is set equal to CtbAddrInRs. - Otherwise, SliceAddrRs is set equal to the SliceAddrRs of the preceding slice segment containing the coding tree block for which the coding tree block address is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs]-1], where the variable CtbAddrInRs is specified in the semantic of slice_segment_address.

[0065] tile_id_in_sh specifies the index of the tile to which the slice segment belongs. The value of area_id_in_sh shall range from 0 to 255. When area_id_in_sh is not present, it is inferred to be equal to 0. Two or more tiles in a picture shall not have the same value of tile_id_in_sh.

[0066] slice_segment_address specifies the address of the first coding tree block in the slice segment in a coding tree block raster scan of a picture (when slice_segment_base_addr_per_tile_enable_flag is equal to 1), or in a tile scan of a picture, as follows:

[0067] The variable maxNumCtbY is defined as (slice_segment_base_addr_per_tile_enable_flag ? TileSizeInCtbsY[tileIdxTileIdMap[tile_id_in_sh]]:PicSizeInCtbsY), where tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the currently decoded / encoded slice contained by its segment information 48 onto the correct tileIdx, i.e., refers to the correct segment in picture 18 or 54, and TileSizeInCtbsY gives, for this segment, the maximum value of the respective slice addresses that will be presented / representable by the slice_segment_address of its start position information 42, i.e., maxNumCtbsY-1. Decoders and encoders can calculate this information using pseudocode 1 to 4.

[0068] The length of the slice_segment_address syntax element is then Ceil(Log2(maxNumCtbsY)) bits. The value of slice_segment_address shall range from 0 to maxNumCtbsY-1, inclusive. When slice_segment_base_addr_per_tile_enable_flag is equal to 0, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same coded picture. When slice_segment_base_addr_per_tile_enable_flag is equal to 1, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit belonging to the same tile in the same coded picture. When slice_segment_address is not present, it is inferred to be equal to 0.

[0069] The variable CtbAddrInRs specifying the coding tree block address in a coding tree block raster scan of a picture is set equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_base_addr_per_tile_enable_flag? TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]]:0)]. The variable CtbAddrInTs specifying the coding tree block address in a tile scan is set equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal specifying the difference between the luma quantization parameter for a coding unit including cu_qp_delta_abs and its prediction is set equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr, which specify the values ​​to be used when determining the respective values ​​of the Qp'Cb and Qp'Cr quantization parameters for the coding unit containing cu_chroma_qp_offset_flag, are both set equal to 0. Thus, for relative indication mode 46, tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the currently decoded / encoded slice contained by its segment information 48 onto the correct tileIdx, i.e., references the correct segment within picture 18 or 54, and TileSliceSegBaseAddr gives a base address for this segment, which is then added to the relative slice address slice_segment_address of its start position information 42 to give the address of this slice. CtbAddrTsToRs simply converts the addresses measured along the coding path 22 to addresses measured along the raster scan block order between blocks 64. The decoder and encoder can use pseudocode 5 to calculate the base address of the segment onset location 44, or in the case of explicit information as exemplified below, the encoder can simply do this.

[0070] Regarding the order of VCL NAL units and their association to coded pictures, it can be said that:

[0071] Specifically, the following specifies the order of VCL NAL units and their association to coded pictures.

[0072] Each VCL NAL unit is a portion of a coded picture.

[0073] The order of VCL NAL units within a coded picture is constrained as follows: - The first VCL NAL unit of a coded picture shall have first_slice_segment_in_pic_flag equal to 1. - Let sliceSegAddrA and sliceSegAddrB be the values ​​of CtbAddrTsToRs[slice_segment_address(slice_segment_base_addr_per_tile_enable_flag?TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]:0)]] of any two coded slice segment NAL units A and B in the same coded picture. A coded slice segment NAL unit A shall precede a coded slice segment NAL unit B when any of the following conditions is true: - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]] and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0074] Decoders 38 and 52 can monitor these conditions to determine when a new access unit 36 ​​starts within the inbound data stream 10 or 12 or to determine that any NAL units are missing.

[0075] Variation A as described so far may be modified in different respects, for example to allow easier implementation and processing.

[0076] For example, in variant A, as already explained above with respect to Fig. 1, the segment 30 may not be strictly a tile. For this reason, the slice segment base address calculation may not occur exclusively for tile granularity, and multiple tiles may be joined together to form a segment 30 according to the explanation presented above, for which the slice segment base address may then be defined together, i.e., the slice address in the slice header is given relative to the first CTU belonging to a given area of ​​the multiple tiles forming the segment 30 in bitstream order.

[0077] Further, additionally or alternatively, according to variant B described below, it becomes clearer that instead of placing the burden of calculating the slice segment base address for each segment 30 on the client side or decoder, explicit signaling of the slice segment base address may be provided in the data stream.

[0078] Additionally or alternatively, for example, when slice_segment_base_addess_per_tile_enable_flag is equal to 1, while ensuring that tile_fixed_structure is equal to 1, the burden of repeating tile structure signaling within each PPS of the video bitstream may be reduced by allowing tiling syntax to be conveyed either within an SPS or within a PPS, and a PPS instance of the tiling syntax may override the tiling syntax settings conveyed within the SPS.

[0079] The following figures indicate, by highlighting, modifications to Variation A described above. For example, Figure 7 shows tiling syntax portions that may be used in either a PPS or an SPS, and when the tiling syntax of Figure 7 is conveyed in a PPS, it overrides the tiling syntax provided by the corresponding SPS, i.e., it has higher precedence. Figure 7 illustrates the possibility of parameter set 36 of Figure 1 conveying a base address for segment onset position 44 via base address data field 130.

[0080] The semantics can be as follows:

[0081] slice_segment_base_addr_per_area_enable_flag equal to 0 specifies that the slice segment variable CtbAddrInRs is derived solely from slice_segment_address. slice_segment_base_addr_per_area_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on slice_segment_address and a tile-dependent offset. When slice_segment_base_addr_per_area_enable_flag is not present, it is inferred to be equal to 0.

[0082] area_id_in_ps[i] specifies the id of the tile in bitstream order. The value of area_id_in_ps[i] shall range from 0 to 255, and area_id_in_ps[i] shall not have the same value as area_id_in_ps[j] if i is not equal to j. When area_id_in_sh is not present, it is inferred to be equal to 0.

[0083] slice_segment_base_address[i] specifies the base slice segment address in tile scan order for all slice segments that belong to an area with the same value of area_id_in_ps[i]. When not present, the value of slice_segment_base_address[i] is inferred to be equal to 0.

[0084] When area_id_enable_flag is equal to 1, the presence of an access unit delimiter NAL unit in the bitstream is a requirement for bitstream alignment.

[0085] It is a bitstream alignment requirement that for any two values ​​of i, when area_id_in_ps[i] do not have equal values, the values ​​of slice_segment_base_address[i-1] are not equal.

[0086] 8 and 9 provide an example of SPS and PPS syntax with respective enable flags for tiling syntax signaling, where the following semantics are used:

[0087] tiles_enabled_in_sps_flag equal to 1 specifies that SPS will propagate the default tiling syntax that describes the default structure of CVS.

[0088] tiles_enabled_in_pps_flag equal to 1 specifies that the PPS conveys tiling syntax describing the actual tiling structure of pictures in the CVS that reference the PPS and that overrides the tiling syntax in the referenced SPS.

[0089] The coding tree block raster and tiling scan conversion process in this case is as follows:

[0090] The list colWidth[i], for i ranging from 0 to num_tile_columns_minus1, inclusive, specifying the width of the i-th tile column in coding tree block (CTB) units, is derived as follows (pseudocode 1'): if(uniform_spacing_flag ) for(i=0;i<=num_tile_columns_minus1;i++) colWidth[i]=((i+1)*PicWidthInCtbsY) / (num_tile_columns_minus1+1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1+1) else { colWidth[ num_tile_columns_minus1]=PicWidthInCtbsY(6 3) for(i=0;i <num_tile_columns_minus1;i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0091] The list rowHeight[j], for j ranging from 0 to num_tile_rows_minus1, inclusive, specifying the height of the j-th tile row in CTB units, is derived as follows (pseudocode 2'): if(uniform_spacing_flag) for(j=0;j<=num_tile_rows_minus1;j++) rowHeight[j]=((j+1)*PicHeightInCtbsY) / (num_tile_rows_minus1+1)- (j*PicHeightInCtbsY) / (num_tile_rows_minus1+1) else { rowHeight[num_tile_rows_minus1]=PicHeightInCtbsY (6 4) for(j=0;j <num_tile_rows_minus1;j++){ rowHeight[j]=row_height_minus1[j]+1 rowHeight[num_tile_rows_minus1]-=rowHeight[j] } }

[0092] A list TileId[ctbAddrTs] for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, inclusive, which specifies the conversion from CTB addresses to tile IDs in tile scanning, is derived as follows (pseudo code 3'). for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) for(y=rowBd[j];y <rowBd[j+1];y++) for(x=colBd[i];x <colBd[i+1];x++) TileId[CtbAddrRsToTs[y*PicWidthInCtbsY+x]]=tileIdx

[0093] The meanings of the variables remain as taught above, but N indicates the number of tiles, which may be greater than the number M of segments 30 , due to the collection of tiles for the segments 30 .

[0094] The list TileSizeInCtbsY[k], for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), inclusive, specifying the size of the kth tile in the picture in CTB units, is derived as follows (pseudocode 4'): for(areaId=0;areaId<=255;areaId++) AreaSizeInCtbsY[areaId]=0 for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++) if(area_id_in_ps[j*(num_tile_columns_minus1+1)+i]==areaId) AreaSizeInCtbsY[areaId]+=(colWidth[i]*rowHeight[j])

[0095] area_id_in_ps[] is a vector containing elements 0...N-1, where N is the number of tiles and the ith element indicates the segment index value associated with the segment in which the ith tile is located in the tile raster scan segment order. The value entered in the element assumes a value between 0...256, or in other words, a value within the domain of possible values ​​for the segment index in segment information 48. Recall the dependency on the picture decoding option / target, i.e., whether the base address of the segment in picture 18 or the base address of the segment in subpicture 54 is to be calculated, where N is the number of tiles in picture 18 or subpicture 54, respectively. AreaSizeInCtbsY[ ] is interpreted as a vector containing an entry for each member of the range of possible values ​​for the segment index in segment information 48 indicating, for each segment index value occurring in picture 18 or subpicture 54, the number of blocks 64 covered by whatever the current decoding target / option is.

[0096] The mapping from AreaId to AreaIdx is derived as follows: NumTilesInArea[0...255]=0; NumAreas=1 for(k=0;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 If(NumTilesInArea[area_id_in_ps[k]]==1) AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas++;

[0097] NumTilesInArea[ ] is interpreted as being a vector containing an entry for each member of the universe of possible values ​​for the segment index in segment information 48 indicating, for each segment index value occurring in picture 18 or subpicture 54, the number of tiles covered by the respective segment having this segment index value, whatever the current decoding target / option. AreaIdToAreaIdx[ ] is interpreted to be a vector containing a component for each member of the range of possible values ​​for the segment index in segment information 48 indicating, for each segment index value occurring in picture 18 or subpicture 54, its rank or index as assigned or measured along the raster scan segment order leading from top left to bottom right in the segment row direction, whatever the current decoding target / option.

[0098] The list AreaSliceSegBaseAddr[k], for k ranging from 0 to (NumAreas-1), inclusive, specifying the slice segment address offset in CTB units of the first slice segment of the kth area in the picture, in bitstream order, is derived as follows (pseudocode 5'): AreaSliceSegBaseAddr[0]=0 for(k=1;k<(NumAreas);k++) AreaSliceSegBaseAddr[k] = slice_segment_base_address [k-1]

[0099] AreaSliceSegBaseAddr[i] is interpreted as a vector containing components 0...M-1, where M is the number of segments 30, equal to NumAreas, and in the i-th component indicates the base address, i.e., the segment onset position 44 for the i-th segment in the aforementioned raster scan segment order, using AreaIdx, which may be related to the area_id_in_sh of the slice's segment information 48 by the decoder and encoder using AreaIdToAreaIdx.

[0100] Therefore, if explicit transmission is also applied to the sub-picture decoding option, the explicit transmission of the base address for the first segment 30 of the picture 18 or sub-picture 54 in the raster scan segment order is excluded since it will be zero anyway, but it should be noted that the base address data field 130 defined by the syntax element slice_segment_base_address defines, for each segment 30, the base address of the onset position 44 of the respective segment, which addresses the onset position 44 relative to the picture onset position 24. The definition by the base address data field that this base address is zero is a consequence of the convention that the explicit base address transmission starts with the second segment in the raster scan segment order. Naturally, alternative embodiments can be assessed where explicit base address transmission for all segments is used instead.

[0101] If no explicit base address signaling is used, but an implicit derivation of the base address is used instead, the same can be performed by a decoder or encoder as follows: The following calculation is used by the encoder for the calculation of the base address even in the case of explicit base address signaling.

[0102] The list TileSliceSegBaseAddr[k], for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), inclusive, specifying the slice segment address offset, in CTB units, of the first slice segment of the kth tile in the picture, in bitstream order, and the list AreaSliceSegBaseAddr[k], for k ranging from 0 to (NumAreas-1), inclusive, specifying the slice segment address offset, in CTB units, of the first slice segment of the kth area in the picture, in bitstream order, are derived as follows (pseudocode 5"): TileSliceSegBaseAddr[0]=0 AreaSliceSegBaseAddr[area_id_in_ps[0]]=0 NumTilesInArea[0...255]=0; NumTilesInArea[area_id_in_ps[0]]=1 AreaIdToAreaIdx[area_id_in_ps[0]]=0 NumAreas=1 for(k=1;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 TileSliceSegBaseAddr[k]=TileSliceSegBaseAddr[k - 1]+TileSizeInCtbsY[k-1] If(NumTilesInArea[area_id_in_ps[k]]==1) AreaSliceSegBaseAddr[NumAreas]=TileSliceSegBaseAddr[k] AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas; NumAreas++;

[0103] Additionally, FIG. 10 shows what a slice header according to variant B might look like, and the following shows the possible semantics:

[0104] first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the picture in decoding order when tiles_enabled_flag is equal to 0. Otherwise, first_slice_segment_in_pic_flag equal to 1 specifies that the slice segment is the first slice segment of a tile of the picture in decoding order when tiles_enabled_flag is equal to 1. first_slice_segment_in_pic_flag equal to 0 specifies that it is not the first slice segment of the picture in decoding order.

[0105] NOTE 1. This syntax element may be used for detection of boundaries between coded pictures that are consecutive in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of an IDR picture may cause problems in detecting boundaries between coded pictures. This may occur, for example, in the transmission of all intra-coded video in an error-prone environment. This problem may be mitigated by alternating two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) for any two consecutive IDR pictures. The use of a temporal sub-layer zero index SEI message may also be useful, since its SEI message contains the syntax element irap_pic_id, whose value is different for consecutive IRAP pictures in decoding order. Some system environments have other provisions that may be similarly useful for picture boundary detection, such as presentation timestamps in Rec. ITU-T H.222.0|ISO / IEC 13818-1 systems, access unit framing in the ISO / IEC 14496-12 ISO Base Media File Format, or the use of marker bits in IETF RFC 3550 Real-time Transport Protocol headers.

[0106] A dependent_slice_segment_flag equal to 1 specifies that the value of each absent slice segment header syntax element is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When absent, the value of dependent_slice_segment_flag is inferred to be equal to 0.

[0107] The variable SliceAddrRs is derived as follows: - If dependent_slice_segment_flag is equal to 0, SliceAddrRs is set equal to CtbAddrInRs. - Otherwise, SliceAddrRs is set equal to the SliceAddrRs of the preceding slice segment containing the coding tree block for which the coding tree block address is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs]-1], where the variable CtbAddrInRs is defined in the semantic of slice_segment_address.

[0108] area_id_in_sh specifies the index of the tile to which the slice segment belongs. The value of area_id_in_sh shall range from 0 to 255. When area_id_in_sh is not present, it is inferred to be equal to 0.

[0109] slice_segment_address specifies the first coding tree block in the slice segment in the coding tree block raster scan of the picture (when slice_segment_base_addr_per_tile_enable_flag is equal to 1), and possibly in the tile scan of the picture, as follows: The variable maxNumCtbY is defined as (slice_segment_addr_offset_per_tile_enable_flag? AreaSizeInCtbsY[area_id_in_sh]:PicSizeInCtbsY).

[0110] The length of the slice_segment_address syntax element is Ceil(Log2(maxNumCtbsY)) bits. The value of slice_segment_address shall range from 0 to maxNumCtbsY-1, inclusive. When slice_segment_addr_offset_per_tile_enable_flag is equal to 0, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same coded picture. When slice_segment_addr_offset_per_tile_enable_flag is equal to 1, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same area in the same coded picture. When slice_segment_address is not present, it is inferred to be equal to 0.

[0111] The variable CtbAddrInRs, which specifies the coding tree block address in a coding tree block raster traversal of a picture, is set equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_addr_offset_per_tile_enable_flag?TileSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]]:0)]. The variable CtbAddrInTs, which specifies the coding tree block address in a tile traversal, is set equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal, which specifies the difference between the luma quantization parameter for the coding unit containing cu_qp_delta_abs and its prediction, is set equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr, which specify the values ​​to be used when determining the respective values ​​of the Qp'Cb and Qp'Cr quantization parameters for a coding unit that includes cu_chroma_qp_offset_flag, are both set equal to zero.

[0112] Regarding the order of the VCL NAL units and their association to coded pictures and access units 36, it can be said that: Each VCL NAL unit is a portion of a coded picture.

[0113] The order of VCL NAL units within a coded picture is constrained as follows: - The first VCL NAL unit of a coded picture shall have first_slice_segment_in_pic_flag equal to 1. - Let sliceSegAddrA and sliceSegAddrB be the values ​​of CtbAddrTsToRs[slice_segment_address+CtbAddrRsToTs[(slice_segment_base_addr_per_tile_enable_flag? AreaSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]]:0)]] of any two coded slice segment NAL units A and B in the same coded picture. A coded slice segment NAL unit A shall precede a coded slice segment NAL unit B when any of the following conditions is true: - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]] and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0114] The description of a possible HEVC codec extension for efficient sub-picture extraction presented above is, however, preliminarily extended to the aspect already shown with respect to FIG. 1, whereby the sub-picture extraction process is eased not only by an advantageous relative indication of the start position of a slice with respect to the onset position of the current segment, but also by providing information on the decoding of the sub-picture decoding option, or alternatively, of some sub-picture decoding options, so that the extraction process can in fact be limited only to activities related to the exclusion or discarding of slices that are not required according to said sub-picture decoding option. In other words, with this extension of the HEVC modification described above, the interest in merely decoding a sub-picture, i.e. the selection of a sub-picture decoding option, is handled in such a way that a layer of a multi-layer data stream is missed, by choosing all corresponding layers that correspond to the decoding of a complete picture area of ​​the original data stream. It will be noted later that the latter aspect may be used independently of the relative slice starting point indication discussed above.

[0115] Thus, the extensions described next allow an easier extraction process that does not involve rewriting parameter sets. As in the extraction process of layer bitstreams, removal of NAL units carrying slice data and parameter sets is sufficient to create a matching bitstream, and a decoder receiving the bitstream can easily determine the actual operation point represented in the extracted bitstream. In this context, an operation point resides in the space taken up by at least a subset of the segment grid or tiling grid, if tiles are used as segments, and optionally by scalability dimensions, such as spatial resolution, temporal resolution, SNR scalability, etc.

[0116] According to this embodiment, a parameter set such as a VPS or SPS defines several operating points in the context of a tiling, called an Output Tile Set (OTS), for which several properties are described below, where it is exemplarily assumed that the parameters for the OTS are contained within the SPS, and several embodiments are shown in FIG.

[0117] As shown in Figure 11, a parameter set, here exemplarily an SPS, indicates the decoding of a complete picture 18 (compared to Figure 1) using a tiling syntax 140, an example of which is presented in Figure 7, and indicates the subdivision of picture 18 into segments, optionally the association of segment index values ​​for these segments, respectively, and optionally the base addresses of the onset positions of these segments, as well as the size of picture 18, i.e., the horizontal size at 142 and the vertical size at 144, and, if i=0, the decoder capability level for decoding picture 18 at 146. Since the syntax element num_output_tile_format transmitted at 148 is greater than 1, the parameter set of Figure 11 additionally signals information 142 at 146 on further sub-picture decoding options indicating other picture sizes and associated decoder capability levels for picture 54. According to the example of Fig. 11, the various configurations of segments 30, here exemplarily tiles, which result in corresponding picture sizes as indicated by 142 and 144 for the output format, are also signaled separately. The num_output_tile_sets transmitted at 150 is in the syntax element and indicates the number of search configurations. Each configuration is associated with one of the output tile formats by an index 152. The number of tiles constituting the respective picture 18 or picture 54 is transmitted at 154 and a segment index is transmitted at 156 for the respective tile configuration. Finally, for each tile configuration of the number indicated by 150, the parameter set indicates at 158 ​​additional information regarding the decoding, i.e. parameters regarding the coded picture buffer with decoding timestamps and / or transfer times and / or the buffer size required to decode the respective segment configuration (or tile configuration) from the data stream.

[0118] When using the embodiment of FIG. 11, the decoding process taking place within the decoder 38 or 52 utilizes an additional input parameter, sometimes called the target output tile set, that indicates which of the defined OTSs is selected for decoding, which input parameter represents the signal 58 discussed above with respect to FIG. 1.

[0119] The parameter TargetOutputTileSet may be determined by various means. By external means such as indication by a system layer through signaling in a file format, transport stream, SDP, or otherwise. The presence of an SEI message in the bitstream indicating which tiles or OTS are present in the bitstream, i.e. a description of the previous extraction process that the bitstream has undergone. Thus, extraction information may be included in the data stream indicating one or more decodable OTS. · Analyze the values ​​of the syntax element tile_id_sh in the slice headers of the NAL units in the access unit to determine which of the defined OTSs may be decodable using the content of the bitstream.

[0120] For example, priorities may be assigned to the defined OTSs to aid in determining which OTS should be decoded by: Explicit signaling in the loop via num_output_tile_sets or SEI messages to assign priority to OTS Unit-of-Order within the loop via num_output_tile_sets A Tiles Present / Tiles Removed SEI message with an indication of all OTS contained in the bitstream.

[0121] Thus, parameter set 36 may include, for each decoding option signaled by 148 and, for that decoding option, for each segment collection as defined by 150 and 158, a priority indication indicating the priority with which this segment collection defining a particular sub-picture configuration is to be chosen for decoding target by decoders 38 and 52, respectively. The priority indication may indicate the priority overall among all of the segment collection options whose number is signaled by 150, or simply individually among groups belonging to one common sub-picture option, i.e., the same sub-picture size and corresponding decoder capability level.

[0122] The TileSliceSegBaseAddr derivation needs to be adjusted for variant A as follows (highlighted by turquoise): The list TileId[ctbAddrTs] for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, inclusive, that specifies the conversion from CTB addresses to tile IDs in tile traversal is derived as follows: for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]]=tileIdx for(y=rowBd[j];y <rowBd[j+1];y++) (6 9) for(x=colBd[i];x <colBd[i+1];x++) TileId[CtbAddrRsToTs[y*PicWidthInCtbsY+x]]=tileIdx

[0123] A list TileSizeInCtbsY[k], for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), inclusive, specifying the size of the kth tile in the picture, in CTB units, is derived as follows: for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++ ) TileSizeInCtbsY[k]=colWidth[i]*rowHeight[j]

[0124] For the i-th target OTS, the list TileSliceSegBaseAddr[k], for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), inclusive, specifying the slice segment address offset in the tile traversal of the first slice segment of the k-th tile in the picture in bitstream order in CTB units, is derived as follows: TileSliceSegBaseAddr[0]=0 cnt=1 for(k=1;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) if(tileIdxInTargetOTS) { TileSliceSegBaseAddr[cnt]=TileSliceSegBaseAddr[cnt-1]+TileSizeInCtbsY[k-1] cnt++ / / with cnt derivable from tile_id_in_sh with a mapping table to tile_id_in_sps }

[0125] tileIdxInTargetOTS is true for all tiles that belong to the TargetOutputTileSet, false otherwise. It will be noted above that the tile arrangement in the decoded pictures of the TargetOutputTileSet remains similar to the tile arrangement in the original bitstream. Raster to tile scan conversion and vice versa is done considering only the tiles in the targetOutputTileSet.

[0126] As already indicated above, the extension just described with respect to FIG. 11, which takes into account the parameter set 36 and its signaling of the picture size and its signaling of the corresponding decoder capability level not only for the original picture 18 but also for the sub-picture 54, may for example be used in a framework in which the aforementioned slice start position information 42 does not exist. For example, by convention, slices may be constrained not to start within the segment 30 but simply at the segment onset position. In that case, for example, the segment information 48 may be sufficient. Other examples may also be possible. In any case, the decoder 38 or 52 receiving the stream 10 or 12 (with or without the start position information 42) uses the information in the parameter set 36 equal to 36' as follows: In particular, the decoder decodes the parameter set 36 from the inbound data stream, which may be 10 or 12, and derives a decoding option indication 58 of whether picture or sub-picture decoding should be performed on the inbound data stream. Picture decoding indicates the target of decoding picture 18 from data stream 10, i.e., the construction of picture 10 from all segments 30. Therefore, to achieve this goal, all slices 26 need to be in data stream 10. Sub-picture decoding goal is the target of decoding picture 54 from data stream 12. "Picture 54" means that two or more sub-picture decoding options can be signaled in parameter set 36, with the same difference in picture size and shape and decoder capability level. For example, in the example of FIG. 1, different sub-picture decoding options can result from constructing forming sub-picture 54 from two adjacent segments 30, constructing sub-picture 54 from two segments 30 one by one, or constructing sub-picture 54 from only one segment 30, and these examples can be extended by constructing sub-picture from three segments 30, for example.

[0127] If picture decoding is to be performed on the data stream, the decoder derives from the parameter set 36 the size of the picture 18, such as via 142 and 144 for i=0, and from the data stream an indication of the decoder capability level required to decode the picture 18, such as via 146 for i=0, and the decoder derives from the parameter set 36 information regarding the subdivision of the picture 18 into segments 30, i.e. a first set of all segments, that are coded in the data stream without coding interdependence, such as from the tiling syntax 140. Furthermore, the decoder decodes the picture 18 from the data stream in slices 26, into which the picture 18 is partitioned along the coding path 22. If the individual slices 26 include starting position information, see the above description regarding the placement of the decoded picture content of the individual slices 26. The decoder may check the decoder capability level if it is met by the decoder 38.

[0128] However, in case of a decoding option indication 58 suggesting that sub-picture decoding is performed on the inbound data stream, the decoder derives from the parameter set 36 an indication of the further picture size of the sub-picture 54 and the decoder capability level required to decode the sub-picture from the data stream, such as via 142, 144, and 146 if i≠0, derives from the parameter set 36 information regarding the subdivision of the sub-picture 54 into a second set of segments whose segments are coded into the data stream without coding interdependence and which are a subset of the set of segments of the picture 18, and decodes the sub-picture 54 from the data stream by those slices 26 in which the sub-picture 54 is coded. Even here, the decoder uses or follows the coding path 22', which then traverses the other target objective, namely the sub-picture 54. Like the coding path 22 traversing the picture 18, the coding path 22' traverses the subpicture 54 successively, segment by segment, so that each slice 26 of the slices for any fraction of the subpicture 54 is coded from among a second set of segments for which one fraction of the segment, or one or more segments, are fully coded. It should be noted that the coding path or coding order 22' may traverse the subsets of the segments 30 participating in the construction of the subpicture 54 in a different order than the order in which the subsets are traversed by the coding path 22 in the original picture 18. However, this does not affect the decoding result, since the segment coding independence prevents coding dependencies across segment boundaries anyway. However, within each segment 30, the paths 22 and 22' coincide, which is important for maintaining the decoding synchronization and even the arrangement of the slice contents, when the use of the start position information 42 is not affected. Again, the decoder may check the decoder capability level for the sub-picture decoding option to see if that decoder capability level is met by the decoder 38 / 52.

[0129] As discussed above, the decoder may derive the decoding option indication 58 by various means. For example, the decoder 52 may analyze the segments covered by the slices present in the inbound data stream, i.e., 10 or 12, to determine whether the slices and corresponding segments have been removed, e.g., by exclusion, by the network device 50, and if so, which slices and corresponding segments have been removed. Furthermore, additionally or alternatively, external signaling, such as that shown by 58 in FIG. 1, may be used by the decoder 38 or 52 to determine the decoding option or target to be used. The external signaling may already uniquely identify the coding option to be used, or may simply exclude some coding options that should not be used or are not available, e.g., by the intermediate extraction process 14. This external signaling may then originate from the network device 50, or may positively identify, based on further information, a set of coding options from which the decoder should select one coding option. Further, additionally or alternatively, extrinsic information in the inbound data stream, such as explicit information about options that are still decodable, may be used by the decoder 38 or 52 to determine which decoding option or target to use. Similar to extrinsic information, the latter information may be included by the extraction entity, i.e., device 50. Furthermore, the decoder may check the decoder capability levels associated with the decoding options as present in the parameter set 54 to eliminate some from the list of possible decoding option candidates when they conflict with the decoder's capabilities. A combination of some or all of these hints may also form a decoding option indication. If any ambiguity remains, the decoder may use the above-mentioned priorities to determine which of the remaining possible decoding option candidates has the highest priority.

[0130] Although some aspects have been described in the context of an apparatus, it is clear that, where blocks or devices correspond to method steps or features of method steps, these aspects also represent a description of the corresponding method. Similarly, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

[0131] The data stream of the present invention may be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless or wired transmission medium, such as the Internet.

[0132] Depending on some implementation requirements, the implementation of the present invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, that stores electronically readable control signals that cooperate (or can cooperate) with a programmable computer system to perform the respective method. The digital storage medium can therefore be computer readable.

[0133] Some embodiments according to the invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system such that one of the methods described herein is performed.

[0134] Generally, embodiments of the present invention may be implemented as a computer program product comprising program code operable to perform one of these methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0135] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0136] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0137] A further embodiment of the invention is therefore a data carrier (or a digital storage medium, or a computer readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is generally tangible and / or non-transitory.

[0138] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be adapted to be transferred via a data communication connection, for example via the Internet.

[0139] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0140] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0141] Further embodiments according to the invention include an apparatus or a system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system comprises, for example, a file server for transferring the computer program to the receiver.

[0142] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0143] The apparatus described herein may be implemented using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0144] The apparatus described herein, or any components of the apparatus described herein, may be implemented at least partially in hardware and / or in software.

[0145] The methods described herein may be performed using a hardware apparatus, or using a computer, or using a combination of a hardware apparatus and a computer.

[0146] The methods described herein, or any of the components of the apparatus described herein, may be performed at least in part by hardware and / or by software.

[0147] The above-described embodiments merely illustrate the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to others skilled in the art. It is therefore intended to be limited only by the scope of the following claims, and not by the specific details presented by the description and illustration of the embodiments herein. [Explanation of symbols]

[0148] 10 Data Stream 12 Data Stream 14 Subpicture Extraction 16 Encoders 18 Pictures 20 Videos 22 Coding Pathways 22' coding pathway 24 Picture onset position 26 slices 28 slices 30 Segments 32 slice payload 34 Slice Header 36 Parameter Sets, Access Units 36' Parameter Set 38 Decoder 40 starting position 42 Starting position information 44 Onset Position 46 Relative start position 48 Segment Information 50 Network Devices 52 Decoder 54 Picture, Subpicture 54" Picture 56 Extracted Videos 58 Decryption option instructions 60 Start position coding mode flag 62 Absolute start position specification 64 coding blocks 70 Picture Parameter Set 72 Sequence Parameter Sets 74 Video Parameter Sets 76 VCL NAL units 78 VCL NAL units 80 NALU Header 86 Tiling Syntax 88 tiles 94 Base Slice Segment Address 96 Offset Slice Segment Address 98 slice segment address 106 Flags 112 Start position coding mode flag 114 Syntax Elements, 116 Raster Scan Segment Order 120 Flag 122 First Syntax Element 124 Secondary Syntax Element 130 Base Address Data Field 140 Tiling Index 142 Information about Subpicture Decoding Options 152 Index

Claims

1. Partitioning a picture (18) into slices (28) and segments (30); providing a slice header (34) including start position information (42) corresponding to each slice (26) of the slices (28), the start position information (42) indicating a start position (40) of the slice (26) relative to a segment onset position (44) of one of the segments (30); determining an absolute address measured from a start of the picture (18), said determining including performing a sum corresponding to the start location (40) and the segment onset location (44); encoding the segment onset positions (44), the slice headers (34), and each of the slices (26) into a data stream (10).

2. 2. The method of claim 1, wherein the pictures (18) are encoded into the segments (30) without any coding interdependence.

3. The method of claim 1 , wherein the segment (30) comprises one or more tiles.

4. The method of claim 1 , wherein the segment onset location (44) is indicated by a parameter set (36).

5. An encoder (16) comprising a processor executing instructions for carrying out the method according to any one of claims 1 to 4.

6. Receiving, via a data stream (10), a picture (18) partitioned into slices (28) and segments (30); decoding from the data stream (10) a segment onset position (44) of one of the segments (30) and a slice header (34) including start position information (42) corresponding to each slice (26) of the slices (28); locating a start position (40) of the slice (26) relative to the segment onset position (44) using the start position information (42); determining an absolute address measured from a start of the picture (18), said determining including performing a sum corresponding to the start location (40) and the segment onset location (44); A method of decoding comprising:

7. 7. The method of claim 6, wherein the pictures (18) are coded into the segments (30) without coding interdependence.

8. The method of claim 6 , wherein the segment (30) comprises one or more tiles.

9. The method of claim 6 , wherein the segment onset location (44) is indicated by a parameter set (36).

10. A decoder (38) comprising a processor for executing instructions for implementing the method according to any one of claims 6 to 9.

Images