Efficient sub-picture extraction

Abstract

To provide a data stream, a decoder, and / or an encoder that reduce complexity in sub-picture extraction.SOLUTION: A picture onset position 24 where a coding path 22 of an encoder 16 starts crossing of a segment 30 where its starting position 40 is located, is provided with starting position information 42 indicating a starting position 40 where a picture 18 thereafter is encoded into respective slices 26, 28 along the coding path 22, and thereby complexity in sub-picture extraction 14 is reduced. Alternatively or additionally, a sub-picture extraction process is processed similarly to selecting one layer from a multi-layer data stream.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to the concept of efficient sub-picture extraction.

Background Art

[0002] Sub-picture extraction is a process in which a data stream in which a video picture is encoded is adjusted to a sub-picture area without the need for re-encoding. HEVC DAM3 MCTS extraction, for example, continuously divides a picture into tiles that are independently coded with respect to each other, and groups those tiles into a tile set that is restricted so that motion compensation prediction does not cross tile set boundaries, so that, for example, as far as motion compensation prediction and residual coding are concerned, without any need for re-encoding, it is possible to extract a sub-picture specific data stream from the original full-picture data stream. This MCTS extraction, however, requires adjustment of each NAL unit carrying slice data in order to adjust the slice segment address of the carried slice header.

[0003] Therefore, although sub-picture extraction processes are known, it would be advantageous to have a concept that reduces the complexity of the required tasks at hand.

Summary of the Invention

Problems to be Solved by the Invention

[0004] It is an object of the present invention to provide a data stream, a decoder and / or an encoder in accordance with a concept for reducing the complexity of sub-picture extraction.

Means for Solving the Problems

[0005] This object is achieved by the subject matter of the independent claims of this application.

[0006] According to a first aspect of the present application, for an onset position at which a coding path starts crossing a segment where its starting position is located, start point information indicating a starting position from which the picture is encoded into respective slices along the coding path is provided to each slice, thereby reducing the complexity of sub-picture extraction. The picture is subdivided into segments that are continuously crossed in segment units by the coding path, and each slice has one fragment of a segment or one or more segments fully encoded, and the picture is encoded into segments without coding interdependence. The picture is encoded into a data stream along the coding path in slice units in which the picture is segmented. In addition, each slice includes segment information indicating the segment where the starting position is located. Therefore, both the start position information and the segment information make it possible to determine in which segment and where in this segment the start position of each slice is located. By encoding segments without coding interdependence, it is possible to remove one or more slices related to one segment without affecting the decodability of another segment. Also, even when segments of a picture are used to compile a new sub-picture by shuffling or rearranging the segments and / or discarding some of the segments of the picture, by discarding slices that do not encode any segments forming the sub-picture and adopting the non-discarded slices that still refer to the correct segments, a data stream specific to each sub-picture can be formed based on the original data stream, and it is possible to indicate the correct position of the starting position of each slice within each segment by relative position indication. Therefore, according to the first aspect of the present application, the data stream enables easy sub-picture extraction without the need to correct the start position information and the segment information regardless of the rearrangement of the segments of the picture in the sub-picture data stream and / or the exclusion of some segments.In other words, without the need to correct the start position information, there is an opportunity to simply remove or discard slices that are not related to any segment contained within the sub-picture and adopt the remaining slices, thereby facilitating the sub-picture extraction process.

[0007] According to one embodiment, the start position information includes a variable-length coded start address syntax element. Since the start position information does not need to be inverted in any way during sub-picture extraction, variable-length coding can be used without any penalty during sub-picture extraction.

[0008] According to yet another embodiment, the data stream can be switched between start position information indicating the start position of each slice with respect to the onset position of the segment or absolutely with respect to the picture onset position where the coding path starts to traverse the picture, by means of a start position coding mode flag. According to this option, existing codecs that have hitherto used absolute start position indications can be extended to utilize the relative position indications discussed herein. The decoder may be able to understand both types of start position indications provided by the start position information or simply one of them.

[0009] According to yet another embodiment, the data stream further includes a base address data field that defines a base address for the onset position of each segment, addressing the onset position along the coding path with respect to the picture onset position for each segment. The transmission of these base addresses re-uses the computational overhead for calculating the base address based on the individual subdivision of the picture into segments.

[0010] According to one embodiment, the segment information includes a segment syntax element in which a segment index that indexes the segment where the start position is located is encoded. The syntax element may be encoded using a fixed-length code. The data stream may include an index data field that defines the relevance of the segment index value for each segment. In other words, the index data field can explicitly associate a tag, i.e., a segment index value, with a segment, and the segment information of a slice is set to that tag or the segment index value associated with the segment where the start position of each slice is located. This tagging can be easily modified during sub-picture extraction.

[0011] Another aspect of the present application, which may be combined with the first aspect, is to make the sub-picture extraction process easier by processing it in the same way as selecting one layer from a multi-layer data stream. That is, according to the second aspect, the extractable data stream includes not only the size of the pictures in the data stream and an indication of the first decoder capability level required to decode pictures from the data stream, but also at least one sub-picture decoding option, and a parameter set that also indicates, for this at least one sub-picture decoding option, a reduced picture size and a second decoder capability level required to decode sub-pictures from the extracted version of the data stream. This sub-picture decoding option is processed in the same way as a sub-layer relative to a higher layer corresponding to a complete picture, and the extracted version of the data stream results from removing or discarding slices from a data stream in which segments that are spatially offset with respect to the segment clusters that make up the sub-picture are encoded. These slices may or may not include the start position information and segment information as discussed above with respect to the first aspect. However, it may be sufficient if each slice includes segment information indicating the segments covered by each slice. Thus, according to the second aspect, the process of sub-picture extraction is similar to the transition between some layers of a multi-layer data stream and only requires the exclusion of slices not related to the intended layer, that is, slices that do not cover any of the segments included in the sub-picture here are extracted. There is no need for "conversion" or "modification" of the slices of the data stream that will be employed in the extracted data stream.

[0012] Advantageous aspects are the subject matter of the dependent claims. Preferred embodiments of the present application are described below with reference to the drawings.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

[0014] A first embodiment of the present application will be described with reference to FIG. 1. FIG. 1 shows a data stream 10 and a sub-picture specific data stream 12 derived therefrom by sub-picture extraction 14. The data stream 10 may be generated by an encoder 16. Specifically, the encoder 16 encoded a picture 18 in the data stream 10. FIG. 1 shows the picture 18 as one picture of a video 20, but it should be noted that the embodiments of the present application are not limited to video data streams. Rather, the embodiments described herein can be easily migrated to picture codecs. The following description, however, shows embodiments related to video coding.

[0015] When encoding picture 18 within data stream 10, encoder 16 follows or uses coding path 22 that traverses picture 18 towards an opposite picture corner, such as the lower right corner, for example, to continue across to another picture of video 20, starting from picture onset position 24 that may be located at the upper left corner of picture 18 as shown in FIG. 1. Following or using coding path 22 determines the availability of spatially adjacent portions of picture 18, such as samples or parameters derived therefrom by encoder 18, such as syntax elements, for coding the current portion of picture 18. For example, encoder 16 may use predictive coding. For this purpose, encoder 16 may predict the current portion, such as its sample content or syntax elements describing its sample content, based on spatially adjacent portions of picture 18, provided that these adjacent portions precede the current portion along coding path 22. Additionally or alternatively, encoder 16 may use other spatial coding dependencies for the current portion, such as the derivation of an entropy context for coding syntax elements describing the current portion of picture 18 based on adjacent portions of picture 18. In addition, encoder 16 uses coding path 22 to segment or packetize the data for which picture 18 is being coded. For this purpose, encoder 16 subdivides the coded data into slices along coding path 22. Thus, each slice 26 within data stream 10 that contains data of picture 18 covers or codes a corresponding portion, i.e., a slice 28 of picture 18, and slice 28 of picture 18 is continuously traversed by coding path 22 without interleaving slice 28. In other words, coding path 22 traverses each of the subdivided slices 28 of picture 18 only once.

[0016] As also shown in FIG. 1, when encoding picture 18 into data stream 10, encoder 16 also follows the subdivision of picture 18 into segments 30. FIG. 1 exemplarily shows the subdivision of a picture into four segments 30 arranged in a 2x2 array. However, the number of segments is not important and may vary. However, the subdivision of picture 18 into segments 30 may be such that the segments 30 are arranged in rows and columns. The subdivision of picture 18 into segments 30 may be such that the picture 18 is completely covered without gaps, with the segments within one segment row having the same height and the segments within one segment column having the same width. However, thereby, the rectangular segments 30 may be of different sizes in that the segment columns and / or segment rows may differ from each other in terms of width and height. Following the subdivision of picture 18 into segments 30 may be related to coding dependencies. Encoder 16 encodes picture 18 into data stream 10 such that the coding dependencies do not cross segment boundaries. For this purpose, no part within segment 30 of picture 18 is coded depending on parts of picture 18 outside of this segment. In other words, encoder 16 codes segments 30 independently of each other. As long as the relationship between slice 28 on the one hand and segment 30 on the other hand is considered, a slice 28 completely covers one or more segments 30 or is present only within a single segment 30. For the sake of mere exemplification, FIG. 1 shows that each segment 30 is composed of two slices 28. This means that coding path 22 traverses segment 30 continuously, i.e., each segment 30 is completely traversed by coding path 22 before coding path 22 traverses the next segment 30 in coding order.

[0017] In this way, the encoder 16 encodes the picture 18 into the data stream 10 in units of slices 28. Each slice 26 in the data stream 10 thus has the corresponding encoded picture slice 28. For example, each slice 26 encodes the content of the corresponding slice portion 28 of the picture 18 in one or more forms of payload data 32, such as prediction parameters that distinguish, for example, an intra-coding mode or an inter-coding mode, motion parameters for intra-prediction blocks, an intra-prediction sub-mode for intra-prediction blocks, subdivision information for subdividing the picture 18 into blocks, and residual data such as transform coefficients representing prediction residuals.

[0018] Before proceeding to the description of each of the slice header 34 of slice 26 and its constituent or syntactic elements, a brief continuation of the description of how encoder 16 codes picture 18 into data stream 10 is provided. As mentioned above, segment 30 is a segment that is coded independently of each other. However, the description so far has focused only on the coding of picture 18. However, if picture 18 is a picture of video 20, encoder 16 may adopt a subdivision of the picture into segments 30 for the sequence of pictures of video 20, that is, subdivide the pictures of the picture sequence in the same way, that is, other pictures are segmented into the same number of segments 30, and the segments 30 are of equal size at spatially coincident segment boundaries between pictures. This is the situation shown in FIG. 1, for example, where the pictures preceding and following picture 18 are shown as being subdivided in the same way as picture 18, that is, into 4 segments 30. In addition to the mutual independence of the coding segments 30 within one picture, encoder 16 may code each segment of picture 18 such that the coding of each segment 30 of picture 18 does not depend on another part of a picture called a reference picture that is outside or spatially offset from the segment collocated with each segment of picture 18. In other words, the collocated segments 30 of the pictures within the picture sequence of video 20 may form a spatio-temporal region within video 20 where encoder 16 performs coding independently of the spatio-temporal region formed by other segments of these pictures. Therefore, the aforementioned inter prediction, that is, motion compensation prediction, is limited such that a block within one segment 30 of picture 18 coded using inter prediction does not refer to a part of a reference picture that is outside the segment of the reference picture collocated with the segment in which the current block is located by a motion vector. Rather, encoder 16 will, for example, instead select intra prediction for that block.

[0019] Furthermore, in addition to the slice header 34 of slice 26, which will be described in more detail below, it should be noted that the data stream 10 includes further information containing higher-level coding parameters, hereinafter referred to as a parameter set. This parameter set 36 may be distributed over the data stream 10 and may be hierarchically defined in terms of scope. In other words, different parts of the parameter set 36 may have different scopes. For this reason, some coding parameters may be valid for only one picture, such as picture 18, while other coding parameters may represent coding parameters valid for a sequence of pictures, including for example picture 18, and still other coding parameters may be valid for the entire video 20. The coding parameters defined within the parameter set 36 will suffice for the moment with a short discussion for the purpose of explanation below. The coding parameters included within the parameter set 36 may indicate, for example, the picture size of the pictures of the video 20, the subdivision of the pictures into segments 30, an indication of the decoder capability level required to decode picture 18 and video 20 respectively from the data stream 10, and / or decoding timestamps, coded picture buffer extraction times, and / or buffer sizes that will be used to decode the data stream 10.

[0020] In addition to the details described so far, it should be noted that the encoder 16 can form the data stream 10 such that the slices 26 contained within the data stream 10 for one picture, such as picture 18, are not interleaved with other slices 26 within the data stream 10 for another picture of the video 20. Rather, these form a continuous portion of the data stream 10, called an access unit 36. The access unit 36 thus groups together the slices 26 for one picture, such as picture 18. In addition to the slices 26, other data, such as a portion of the parameter set 36, may also be included within one access unit 36. The data within one access unit 36 can faithfully follow some predefined order or ordering rule such that the receiving side of the data stream 10, such as the decoder 38, can recognize the start of the next access unit within the data stream 10 based on monitoring the order of the data within the data stream 10. Even the slices within one access unit 36 can be ordered relative to each other according to the order in which the coding path 22 traverses the corresponding slice 28 within the picture 18.

[0021] After finishing the description of the general framework for coding the data stream 10, the description proceeds to the description of the slice header 34 and its contents. Specifically, the encoder 16 provides start position information indicating the start position 40 from which the picture 18 is to be encoded into respective slices 26 each including a respective slice header 34, along the coding path 22, to each slice header 34. Knowledge of the start position 40 enables the decoder 38 to accurately place the picture content encoded within each slice 26 for the respective slice portion 28 preceding along the coding path 22, i.e., the picture content for the corresponding slice portion 28 of the picture 18, even if any preceding slice for the picture 18 is missing within the data stream 10.

[0022] However, preferably, the start position information 42 indicates the start position 40 of the slice 26 within the slice header 34 in which the start position information 42 is included, relative to the onset position 44 at which the coding path 22 starts traversing the segment 30 in which the start position 40 is located, as illustratively shown by the arrow 46 in FIG. 1. The relative indication of the start position 40 reduces sub-picture extraction 14, as will be further described below.

[0023] To compensate for the relative indication of the start position 40 by the start position information 42 and its associated ambiguity, the slice header 34 additionally includes segment information 48 indicating the segment 30 in which the start position 40 of the slice 26 is located, to which the segment information 48 belongs. In the example of FIG. 1, for instance, the slice 28 whose start position 40 is indicated by the arrow 46 is located within the segment 30 indexed by the capital letter B in FIG. 1, and accordingly, the segment information 48 for that slice will indicate that this slice 26 belongs to segment B. Thus, the start position information 42 and the segment information 48 together are sufficient to determine the absolute position of the start position 40 within the picture 18, i.e., the relative position or start position 40 with respect to the origin, i.e., the picture onset position 24.

[0024] Decoder 38 that receives data stream 10 decodes the start position information 42 of each slice 26 from data stream 10, and can use the start position information 42 to locate the start position 40 of each slice relative to the onset position 44 of the segment where start position 40 is located. Further, so that decoder 38 knows about segment 30 to which onset position 44, for which start position information 42 indicates start position 40, belongs, decoder 38 decodes the corresponding segment information 48 of each slice 26 from data stream 10. Decoder 38 can thus locate onset position 44 based on parameter set 36 indicating the subdivision of picture 18 into segments 30, and in response, can combine the relative position indication of start position 40 indicated for onset position 44 with onset position 44 within picture 18, and this combination results in the absolute position of start position 40 within picture 18 relative to picture onset position 24. To reduce the computational overhead for decoder 38, a more detailed embodiment, further described below, indicates that some other part within data stream 10, such as parameter set 36 or supplementary extension information, can expressively show the onset position 44 for each segment 30 so that decoder 38 can use this indication rather than calculate the onset position of segment 30 itself.

[0025] It has been outlined above that a relative indication of the start position 40 with respect to the onset position of the corresponding segment 30 is a preferred situation. This will be discussed in more detail below. Specifically, the preferred situation becomes clearer when examining the sub-picture extraction process 14. The latter process may be executed by some entity that processes the data stream 10. For example, the extraction process may be executed by a network device 50 connected in the transmission path between the encoder 16 and the decoder 52 that receives the extracted data stream 12. The decoder 52 may be the same as the decoder 38, i.e., it may be a decoder that could also decode the complete or original data stream 10, or it may be a decoder that can decode the extracted data stream 12 but cannot decode the complete data stream 10 due to, for example, a decoder capability level lower than that of the decoder 38. However, the extraction process 14 may also be executed within the decoder 52 / 38 itself.

[0026] The extraction process 14 is simply composed of an appropriate subset of the segments 30 of the picture 18 compared to the picture 18 and / or, respectively, aims at the formation of the data stream 12 in which the picture 54 is encoded, which is different in the mutual arrangement of the segments 30 within the picture areas of the pictures 18 and 54. FIG. 1 shows an example where the picture 54 of the data stream 12 is simply composed of one of the four segments 30 of the picture 18 of the original data stream 10, i.e., segment B, but this is just an example, and the picture 54 may alternatively be composed of two or more of the segments 30 of the picture 18 as long as the rectangular picture area of the picture 54 is brought about.

[0027] Note that with respect to slice 26 contained within data stream 10, extraction process 14 simply requires the discard or exclusion of slice 26 with respect to segment 30 of picture 18 that is not contained within picture 54, whereas slice 26 with respect to any segment 30 that contributes to the composition of picture 54 is adopted, or left within the extracted data stream 12, without any modification, specifically, without any modification to start position information 42 and segment information 48. Specifically, start position information 42 indicates that the start position of slice 26 whose start position corresponds to segment 30 contributing to picture 54 is shown relative to the onset position of that segment, and this relative indication remains accurate within the reduced data stream 12. Moreover, segment information 48 indicates, for all slices 26 adopted within the extracted data stream 12, to which segment the corresponding slice 26 belongs, i.e., in which segment its start position is located. This segment information 48 is also used in extraction process 14 to identify slices 26 that are to be excluded or discarded.

[0028] The only information present in the extracted data stream 12 that has not yet been discussed with respect to the original data stream 10 is the coding parameters associated with the modified configuration of picture 54 of the extracted data stream 12 based on a selected set of segments 30. There are several possibilities with respect to this topic. According to a first alternative, the extraction process 14 may require a modification of the parameter set 36 such that the corresponding parameter set 36' within the extracted data stream 12 is lower than the decoder capability level required to decode the modified picture size of picture 54 of the extracted video 56, the subdivision of picture 54 into segments, the complete video 20 or picture 18 from the data stream 10, and / or the modified decoding timestamp, the coded picture buffer extraction time and / or the buffer size for decoding the extracted data stream 12. However, according to an alternative embodiment, the manner of using the relative start position indication within the start position information 42, i.e., by providing both information items to the parameter set 36 of the original data stream 10, can be combined with an aspect of the present application such that even the parameter set 36' of the extracted data stream 12 remains unchanged. According to this alternative, the parameter set 36 will indicate all coding parameters for decoding the entire picture 18 from the data stream 10 and at the same time all coding parameters for decoding picture 54 from the data stream 10 or 12. The coding parameters indicated by the parameter set 36 with respect to the extracted data stream 12 and its picture 54 may be at least partially indicated within the parameter set 36 in a manner that is different from or relative to the coding parameters indicated by the parameter set 36 for decoding picture 18 from the data stream 10.For example, as will be outlined in more detail below, parameter set 36 may indicate the subdivision of picture 18 into segments 30 such that the size of segment 30 is clear from this part of parameter set 36. The coding parameters of parameter set 36 for picture 54 may depend on this knowledge and may simply indicate which segments contribute to picture 54 and which mutual arrangement of these contributing segments within picture 54 applies. Even the aforementioned base address indication regarding the onset position 44 within picture 18 may also be repeated within data stream 10 for picture 54 of the extracted data stream 12. Also, as will be further explained below, parameter set 36 thus not only indicates one extraction option or sub-picture decoding option for data stream 10, but may be several extraction options or sub-picture decoding options that differ in terms of the composition of picture 54 based on segment 30 of picture 18. Thus, in the case of an alternative where parameter set 36' remains in an unchanged state with respect to parameter 36 within the original data stream 10, decoder 52 will receive a decoding option indication 58 that indicates, for example, whether picture decoding, i.e., the decoding of picture 18, should be performed on data stream 12 or whether sub-picture decoding, i.e., the decoding of picture 54, should be performed on data stream 12. If unused slice 26 has already been removed as part of the previously executed extraction process 14, this decoding option indication must not indicate picture decoding on the assumption that the slice is missing. However, it should be recalled that extraction process 14 may be performed by decoder 52 itself and decoder 52 may be equal to decoder 38. For this purpose, decoder 52 may receive the complete original data stream 10 and then decide to perform extraction process 14 in terms of removing, i.e., discarding, unused slice 26 itself based on decoding option indication 58. The decoding option indication may be, for example, external signaling provided by an application or a user.Alternatively, the decoding option indication may be a specific indication of the decoder capability level of decoder 52, which is excluded in any case because some of the decoding options of data stream 10 are decodable by decoder 52. Assuming that picture decoding is performed for a complete picture 18, the decoding is performed as discussed for decoder 38. When sub-picture decoding options are applied, i.e., when decoder 52 decodes picture 54 from data stream 12, the decoding process is substantially the same as the following differences. That is, decoder 52 will use the coding parameters of a specific parameter set 36' for this sub-picture decoding for picture 54. Further, if the calculation is to be performed by decoder 52 itself, decoder 52 will calculate a different base address for the onset position 44 of segment 30 within picture 54. The different calculation results are the result of different arrangements of segment 30 within picture 54. The start position information 42 indicating the relative position of the start position 40 of the slice is the same as that indicated by the start position information 42 for the onset position 44 of the segment. However, if the onset position 44 of the segment is indicated within parameter set 36' for the arrangement of the segments within picture 54, decoder 52 may use these base addresses specific to sub-picture decoding to obtain picture 54 instead of its own calculation.

[0029] Further details regarding how decoder 38 / 52 may operate when decoding an inbound data stream, whether it may be 10 or whether it may be 12, are described at the end of the description regarding FIG. 11.

[0030] In the following description, a specific example for implementing the embodiment outlined above will be described. In so doing, assume that the data stream includes within parameter set 36 a start position coding mode flag 60 that switches between a relative start position indication 46 of the start position 40 of a slice relative to the segment onset position 44 as described with respect to FIG. 1, and an absolute start position indication 62 of the start position 40 of the slice relative to the picture onset position 24. In other words, flag 60 signals whether the start position indication 42 within slice 26 of data stream 10 indicates start position 40 relative to the segment onset position 44, or alternatively, absolutely relative to the picture onset position 24, as outlined with respect to FIG. 1. Flag 16 may then be used to switch on the presence of segment indication 48, as further discussed below. In the case of simply the relative start position indication 46, segment indication 48 may be present within slice 26.

[0031] Further details that will become apparent from the embodiments described below are the fact that the slice position indication 42 can actually be signaled by a combination of a flag and a conditionally signaled slice address. In the case of the relative indication option 46, the flag included by the start position information 42 will indicate whether the corresponding slice is the first slice within the segment, i.e., whether its start position 40 coincides with the onset position 44 of the corresponding segment. In that case, the start position information 42 does not include any start address syntax elements. Otherwise, however, the address syntax elements point to the start position 40 of the slice from the onset position 44 of the corresponding segment. In the case of the absolute indication option 62, the flag of the start position information 42 represents whether the corresponding slice starts at the picture onset position 24, i.e., whether it is the first slice of the picture, and in that case, the slice position information does not include any address syntax elements. However, otherwise, the address syntax elements indicate the start position 40 of each slice relative to the picture onset position 24. As described above, the flag included by the start position information 42 can be used by the decoder 38 / 52 to access unit boundary detection, i.e., the detection of a new access unit 36, in the case of the absolute indication mode. To compensate for the lack of the ability to use this detectability in the case of the relative indication mode, the encoder 16 can use alternative access unit boundary mechanisms such as access unit delimiters at the boundaries between consecutive access units, i.e., the placement of some packets. The decoder can then use these delimiters in the case of the relative indication mode.

[0032] As a further note before starting a more detailed description of the embodiments, it is noted that there are different possibilities for indicating the start position 40 by the start position information 42. For example, picture 18 can be regularly subdivided into an array of blocks, as shown at the upper left corner of picture 18 in FIG. 1. Coding path 22 can traverse these coding blocks 64 continuously without interleaving, i.e., one coding block 64 is completely traversed before proceeding to the next coding block 64 of picture 18. The start position information 42 can indicate the position 42 measured in units of coding blocks 64 traversed along the coding path 22 up to the start position 40. When shown relatively 46, the start position information 42 will indicate the number of coding blocks 64 separating the onset position 44 and the slice start position 40 along the coding path 22. In the case of the absolute indication option 62, the start position information 42 indicates the start position 40 in units of coding blocks 64 traversed along the coding path 22 when starting from the picture onset position 24. Thus, in this case, the start position indication 42 will use a linear one-dimensional addressing area, and the base address of the onset position 44 of the above-mentioned segment may also be measured by the number of coding blocks 64 traversed when starting from the picture onset position 24 to each segment onset position 44. The sum of the base address 44 and the slice start position 40 will result in an absolute address measured earlier from the start position 24. However, alternative ways of signaling the start position 40 by the start position information 42 may also be feasible. For example, the relative position of the start position 40 with respect to the corresponding segment onset position 44 can be shown as the x and y coordinates of a vector pointing from the onset position 44 to the slice start position 40, and the components x and y are measured in samples or units of coding blocks 64, respectively.Furthermore, the address designation can be carried out in accordance with the strictly horizontal linear address designation of the aforementioned coding block 64, i.e., within one complete row of the picture 18, the addresses of the coding blocks follow each other before the address increases in the first coding block of the next row within the picture, regardless of the segmentation into segments 30. Internally, the decoders 38 and 52 can switch between different address systems or domains as required.

[0033] Another possibility that will become apparent from the following description is as follows. Segment 30 has been described so far as a possible spatial cross-section of a spatio-temporal region where the coding interdependencies do not cross the boundaries of these spatio-temporal regions. These segments 30 can be defined as a set of one or more tiles, for example, where picture 18 can be subdivided and, as long as it concerns the coding of one picture 18, can be coded independently of each other. Thus, segment 30 can be a tile if one segment 30 is composed of one tile, which is the case in deformation mode A described later, but deformation mode B assumes that segment 30 can be composed of one or more tiles. Moreover, according to the subsequent description, the aforementioned coding block 64 is a CTU (Coding Tree Unit), which means that these blocks 64 are units for selecting the coding mode in which encoder 16 will be used, i.e., inter-coding or intra-coding of the corresponding coding block, units in which the prediction parameters of the corresponding selected coding mode are set, i.e., for each prediction block that can be a leaf node of the coding block, and units in which the conversion of the prediction residue occurs, i.e., units of the conversion block that can also be a leaf block of the coding block, and is a tree root block that is hierarchically further subdivided by multi-tree subdivisioning into coding blocks, prediction blocks, and / or conversion blocks. The subdivision of CTU 64 into coding units, prediction blocks, and conversion blocks can be signaled as part of the corresponding slice payload 32.

[0034] Therefore, the following shows an implementation form of the embodiment described with respect to FIG. 1 as a possible modification of the HEVC framework. As described in the introduction part of the specification of the present application, in HEVC DAM3 MCTS extraction, adjustment to the slice segment address of each slice header is necessary, and this can even affect the byte alignment of each slice header due to the variable-length coding of the slice address. In particular, when using HEVC DAM3 MCTS, the slice addresses of the slices employed within the MCTS-extracted data stream have to be corrected by their absolute indication option, that is, these slice addresses have to be corrected to point to the new picture onset position of picture 54, which can result in slice addresses of different lengths, and thus different byte alignments, due to variable-length coding. The modifications described later for the HEVC framework will overcome this. Specifically, relatively simple MCTS substream extraction results in an implicit or explicit association of slices with tiles / segments by signaling a part of the slice segment address.

[0035] The following describes the first variant form, variant form A, of the modification of the HEVC framework. As outlined above with respect to FIG. 1, according to this variant form, the slice segment address is signaled not for the slice segment address for the first CTU of the picture, but for the first CTU of each tile or tile composition, that is, for the current segment 30. "First" means the leftmost and uppermost CTUs when using the HEVC coding order.

[0036] The differences between two references, namely the first CTU of the current segment 30 or the first CTU of the picture, are derived by the decoder, i.e., decoder 38 or 52, by combining the picture and tile command information from parameter sets 36 and 36' with the association of slice 28 / 26 to this segment 30 which can be a tile or a set of tiles, and this association can be explicitly transmitted within the data stream as outlined below.

[0037] An array containing information regarding the segment size within the CTB and the slice address offset per segment 30 can be determined on the decoder side. Access unit 36 initiates the detection to be performed by decoder 38 or 52 respectively. This detection can be performed based on the flag contained within start position information 42 for the case of absolute indication option 62, as this flag indicates access unit 36 and the first slice of picture 18 respectively, and for the case of relative indication option 46, it can be done by using the access unit delimiter NAL unit within data stream 10, i.e., a special packet used to separate consecutive access units within data stream 10 for signaling the crossing between consecutive access units. As described above, first_slice_in_pic_flag can be included by start position information 42 and can change to a flag indicating the start of segment 30 which can be a tile or a tile configuration.

[0038] FIG. 3 is a diagram giving an overview of an embodiment resulting from using the HEVC technology system. As shown, parameter set 36 is distributed among picture parameter set 70, sequence parameter set 72, and video parameter set 74 with different ranges. FIG. 3 simply shows two VCL NAL units 76 and 78 including NALU header 80 which includes a fragment from data stream 10, that is, each includes a corresponding slice 26 consisting of slice header 34 and slice payload 32 after slice information. Slice header 34 points to corresponding picture parameter set 70 as indicated by arrow 80, picture parameter set 70 points to the corresponding sequence parameter set as indicated by arrow 82, and sequence parameter set 72 points to the corresponding valid video parameter set as indicated by arrow 84. Parameter set 36 suggests that, for example, picture parameter set 70 or sequence parameter set 72 includes tiling syntax 86, that is, according to an alternative form, the syntax defining the subdivision of picture 14 into tiles 88 forming segment 30 discussed above in FIG. 1. Thus, based on tiling syntax 86, the decoder can calculate, for example, the base slice segment address as indicated by arrow 90. The sum 92 of base slice segment address 94 and offset slice segment address 96, which is transmitted as part of start position information 42 and slice header 34, results in the corresponding slice segment address 98 of the corresponding slice 26, that is, the slice segment address measured absolutely with respect to picture onset position 24. FIG. 3 actually shows two slices 26 depicted as belonging to one of the tiles 88, that is, tile number 2, which can be derived for the decoder based on this segment information 48 also included by slice header 34 as will become apparent from the following description.

[0039] Specifically, according to the embodiment shown in FIG. 4, the tiling structure or tiling syntax 86 may be included within the picture parameter set 70. For example, an alternative embodiment in which the tiling syntax 86 is present within the sequence parameter set 72 instead of the picture parameter set would be equally feasible.

[0040] Specifically, as shown in FIG. 4, the portion 100 of the parameter set 70 shows the segmentation 30, here the subdivision of the picture 18 into tiles. The number of segment columns is shown at 102, the number of segment rows is shown at 104, and the flag 106 optionally provides an opportunity to signal within a data stream where the width of the segment columns and the height of the segment rows are unevenly set. If absent, or if the flag 106 is not set, the width of the segment columns is shown individually at 108 and the height of the segment rows is shown individually at 110.

[0041] Furthermore, when the flag slice_segment_base_addr_per_tile_enable_flag 112 is set, i.e., at 114, explicit tagging of segments is achieved by explicitly signaling the segment index value tile_id_in_pps[i] for each segment 30 of the picture 18. The segment index is transmitted within the data stream, i.e., within the parameter set 70, by using the default order among the regular array of segments 30, such as the row direction indicated by the dashed arrow 116 in FIG. 1.

[0042] The semantics of the syntax elements in FIG. 4 are outlined in detail below.

[0043] A slice_segment_base_addr_per_tile_enable_flag equal to 0 specifies that the variable CtbAddrInRs of a slice segment is derived solely from the slice_segment_address. A slice_segment_base_addr_per_tile_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on the slice_segment_address and a tile-dependent base address. When the slice_segment_base_addr_per_tile_enable_flag does not exist, it is inferred to be equal to 0.

[0044] tile_id_in_pps[i] specifies the identification number of a tile in bitstream order. The value of tile_id_in_pps[i] ranges from 0 to 255, and when i is not equal to j, tile_id_in_pps[i] shall not have the same value as tile_id_in_pps[j]. When tile_id_in_pps does not exist, it is inferred to be equal to 0.

[0045] When the slice_segment_base_addr_per_tile_enable_flag is equal to 1, it can be a bitstream consistency constraint that an access unit delimiter NAL unit exists within the bitstream and the tiles_fixed_structure_flag is equal to 1.

[0046] The following pseudo-code shows how a decoder such as decoder 38 or 52 calculates a dimension of segment 30 and a base address of segment 30 based on information available within, for example, a data stream.

[0047] The following describes the coding tree block raster and tile scan conversion process.

[0048] The list colWidth[i] for i in the range from 0 to num_tile_columns_minus1, including both end values, which specifies the width of the i-th tile column in coding tree block (CTB) units, is derived as follows in (Pseudo-code 1): if(uniform_spacing_flag) for(i = 0; i <= num_tile_columns_minus1; i++) colWidth[i]=((i + 1)*PicWidthInCtbsY) / (num_tile_columns_minus1 + 1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1 + 1) else{ colWidth[num_tile_columns_minus1]=PicWidthInCtbsY for(i = 0; i < num_tile_columns_minus1; i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0049] colWidth[i] is the width of the i-th segment column among columns 0…num_tile_columns_minus1 measured within coding tree block CTB64num_tile_columns_minus1. num_tile_columns_minus1 is the number of segment columns minus 1.

[0050] The list rowHeight[j] for j in the range from 0 to num_tile_rows_minus1, including both end values, which specifies the height of the j-th tile row in CTB units, is derived as follows in (Pseudo-code 2): if(uniform_spacing_flag) for (j = 0; j <= num_tile_rows_minus1; j++) rowHeight[j] = ((j + 1) * PicHeightInCtbsY) / (num_tile_rows_minus1 + 1) - (j * PicHeightInCtbsY) / (num_tile_rows_minus1 + 1) else { rowHeight[num_tile_rows_minus1] = PicHeightInCtbsY for (j = 0; j < num_tile_rows_minus1; j++) { rowHeight[j] = row_height_minus1[j] + 1 rowHeight[num_tile_rows_minus1] -= rowHeight[j] } }

[0051] rowHeight[i] is the height of the i-th segment row within rows 0…num_tile_rows_minus1 measured within the coding tree block CTB64num_tile_rows_minus1. num_tile_rows_minus1 is obtained by subtracting 1 from the number of segment rows.

[0052] The list TileId[ctbAddrTs] ranging from 0 to PicSizeInCtbsY - 1, including both end values, specifying the conversion from CTB address to tile ID in tile scanning, is derived as follows in the following (pseudo-code 3): for (j = 0, tileIdx = 0; j <= num_tile_rows_minus1; j++) for (i = 0; i <= num_tile_columns_minus1; i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]] = tileIdx for(y = rowBd[j]; y < rowBd[j + 1]; y++) for(x = colBd[i]; x < colBd[i + 1]; x++) TileId[CtbAddrRsToTs[y * PicWidthInCtbsY + x]] = tileIdx

[0053] tileIdxTileIdMap[] is interpreted as mapping the N segment index values contained within the vector tile_id_in_pps[] to the tileIdx of that segment. tile_id_in_pps[] is a vector containing components 0…N - 1, where N is the number of segments 30, and the i-th component indicates the segment index value associated with the i-th segment in raster scan segment order 116. tileIdx indexes segment 30 in raster scan segment order 116. TileId[i] is interpreted as mapping the CTB address i, i.e., the address of block 64, measured in coding order or along coding path 22, to the tileIdx of the segment 39 in which this CTB is located. CtbAddrRsToTs[i] is a function that maps the rank i of block 64, which is the i-th block 64 of picture 18 in picture raster scan order leading horizontally through block 64 of picture 18, to its address measured in coding order 22. colBd[i] is a vector containing components 0…N, where N is the number of segment columns minus 1, i < N, the i-th component indicates the left position of the i-th segment column measured left to right within block 64, and the N-th component indicates the picture width in number of blocks.

[0054] All parameters that have been used in the pseudo-code so far, i.e., for picture 18, have been described as being used to determine the base address in the case of complete picture decoding. However, it should be noted that these parameters can also be used in the same way to determine the base address regarding the onset position of segments within sub-picture 54. Here, the parameters N, colWidth[i], num_tile_columns_minus1, rowHeight[i], num_tile_rows_minus1, tileIdxTileIdMap[], tile_id_in_pps[], CtbAddrRsToTs[i], colBd[i] are specific to this sub-picture 54, as well as its composition by segment 30 and its subdivision into corresponding blocks 64 respectively. Similarly, all references to coding path 22 and raster scan segment order 116 are applied to each sub-picture. tileIdxTileIdMap[] on the one hand guarantees the maintenance of the association between the segment index value tile_id_in_sh that assumes any one of tile_id_in_pps[] and, on the other hand, their indexing in the segments of sub-picture 54 and the raster scan segment order 116, i.e., tileIdx. As a less important note, the subscript "_pps" of the syntax element is chosen here to reflect an example that conveys the segment index value for segment association within PPS. However, as outlined above, this is just an example, and it should be noted that signaling can also be achieved elsewhere, such as within SPS or VPS. Along the lines of the latter description, alternatively, the subscript "_ps" is used, and the corresponding replacement is, of course, also applicable to this embodiment. Similarly, the following example reveals that there may be a waiver of the 1:1 correspondence between tiles and segments. This is also outlined with reference to FIG. 1. Therefore, the corresponding subsequent embodiments will use the name component "Area" or "area_" or "_area" instead, which will become clear from the following description.Furthermore, as also stated in the description of FIG. 1, the explicit assignment of tags or segment index values to segments may be excluded in alternative embodiments, for example, by implicitly deriving the mapping between the segment index values in segment information 48 and the segments in sub-picture 54 by instead using the index region tileIdx. In this case, the transmission of such an association in parameter set 36 for picture 18 may be excluded. However, the explicit assignment of segment index values to segments 30 of picture 18 reduces the mixing of tiles of two separate data streams.

[0055] A list TileSizeInCtbsY[k] for k ranging from 0 to ((num_tile_rows_minus1 + 1) * (num_tile_columns_minus1 + 1) - 1), inclusive, specifying the size of the k-th tile in the picture in CTB units, is derived as follows in (Pseudo-code 4). for (j = 0, k = 0; j <= num_tile_rows_minus1; j++) for (i = 0; i <= num_tile_columns_minus1; i++, k++) TileSizeInCtbsY[k] = colWidth[i] * rowHeight[j]

[0056] TileSizeInCtbsY[i] is a vector containing components 0... N - 1, where N is the number of segments 30, and the i-th component indicates the number of blocks 64 contained within the i-th segment in raster scan segment order 116.

[0057] A slice segment address offset in CTB units in the tile scan of the first slice segment of the k-th tile in the picture in bitstream order, for k ranging from 0 to ((num_tile_rows_minus1 + 1) *The list TileSliceSegBaseAddr[k] for k ranging from 0 to (num_tile_columns_minus1 + 1) - 1 is derived as per the following (Pseudo-code 5). TileSliceSegBaseAddr[0] = 0 for (k = 1; k <= ((num_tile_rows_minus1 + 1) * (num_tile_columns_minus1 + 1) - 1); k++) TileSliceSegBaseAddr[k] = TileSliceSegBaseAddr[k - 1] + TileSizeInCtbsY[k - 1]

[0058] TileSliceSegBaseAddr[i] is a vector containing components 0…N - 1, where N is the number of segments 30, and the i-th component indicates the address of the offset position 44 of the i-th segment in raster scan segment order 116, measured as the number of blocks 64 along the coding path from picture offset position 24 to segment offset position 44.

[0059] Here, "tile scan" refers to scan 22 as shown in FIG. 1. TileSliceSegBaseAddr, according to this example, indicates the aforementioned base address of the segment offset position 44, which is equal to the tile. The calculations shown above can be used respectively by decoders 38 and 52 to calculate the base address. When sub-picture decoding is applied, the number of segments and their arrangement shall be adaptable.

[0060] Figure 6 shows how a slice segment header, i.e., the aforementioned slice header 34, can be. Specifically, here, the slice header 34 includes a flag 120, i.e., first_slice_segment_in_area_flag, which forms start position information 42 together with the syntax element slice_segment_address122. When the flag 120 is simply not set, the slice_segment_address exists and indicates the offset of the slice start position 40 from the onset position 44 of the corresponding segment / tile 30. The latter is indicated by segment information 48 including simply one syntax element 124, i.e., tile_id_in_sh, which assumes a value outside the value range defined within the parameter set 70 in FIG. 4, i.e., by the syntax element 114.

[0061] In particular, the semantics can be shown as follows: A first_slice_sement_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the picture in decoding order when the slice_segment_base_addr_per_tile_enable_flag is equal to 0. Otherwise, when the slice_segment_base_addr_per_tile_enable_flag is equal to 1, a first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the tile of the picture in decoding order. A first_slice_segment_in_pic_flag equal to 0 specifies that the slice segment is not the first slice segment of the picture or the tile of the picture in decoding order, respectively. Therefore, the slice_segment_base_addr_per_tile_enable_flag functions as an example regarding the flag 60 discussed with respect to FIG. 2.

[0062] Note 1. This syntax element may be used for the detection of the boundaries between coded pictures that are consecutive in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of an IDR picture may cause problems in the detection of the boundaries between coded pictures. This may occur, for example, when transmitting all intra-coded video in an error-prone environment. This problem may be mitigated by using two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) alternately for two consecutive IDR pictures. The use of the temporal sublayer zero index SEI message may also be useful as its value contains different syntax elements irap_pic_id for IRAP pictures that are consecutive in decoding order. Some system environments may also have other provisions that are similarly useful for picture boundary detection, such as the use of presentation time stamps within the Rec. ITU-T H.222.0|ISO / IEC 13818-1 system, access unit framing in the ISO base media file format of ISO / IEC 14496-12, or marker bits in the IETF RFC 3550 real-time transport protocol header.

[0063] A dependent_slice_segment_flag equal to 1 specifies that the value of each slice segment header syntax element that does not exist is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When it does not exist, the value of the dependent_slice_segment_flag is inferred to be equal to 0.

[0064] The variable SliceAddrRs is derived as follows: - If dependent_slice_segment_flag is equal to 0, SliceAddrRs is set equal to CtbAddrInRs - Otherwise, SliceAddrRs is set equal to the SliceAddrRs of the preceding slice segment that contains the coding tree block whose coding tree block address for it is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs]-1], where the variable CtbAddrInRs is defined within the semantics of slice_segment_address.

[0065] tile_id_in_sh defines the index of the tile to which the slice segment belongs. The value of area_id_in_sh shall be in the range from 0 to 255. When area_id_in_sh does not exist, it is inferred to be equal to 0. Two or more tiles within a picture shall not have the same value of tile_id_in_sh.

[0066] slice_segment_address defines the address of the first coding tree block within the slice segment as follows, in the coding tree block raster scan of the picture (when slice_segment_base_addr_per_tile_enable_flag is equal to 1), or otherwise, in the tile scan of the picture.

[0067] The variable maxNumCtbY is defined as (slice_segment_base_addr_per_tile_enable_flag? TileSizeInCtbsY[tileIdxTileIdMap[tile_id_in_sh]] : PicSizeInCtbsY), where tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the currently decoded / encoded slice included by its segment information 48 onto the correct tileIdx, i.e., refers to the correct segment within picture 18 or 54, and TileSizeInCtbsY is, for this segment, the maximum value of each slice address that can be represented by the slice_segment_address of its start position information 42, i.e., results in maxNumCtbsY - 1. The decoder and encoder can calculate this information using pseudo - code 1 through 4.

[0068] The length of the slice_segment_address syntax element is then Ceil(Log2(maxNumCtbsY)) bits. The value of slice_segment_address ranges from 0 to maxNumCtbsY - 1, inclusive. When slice_segment_base_addr_per_tile_enable_flag is equal to 0, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same coded picture. When slice_segment_base_addr_per_tile_enable_flag is equal to 1, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit belonging to the same tile within the same coded picture. When slice_segment_address does not exist, it is inferred to be equal to 0.

[0069] In the coding tree block raster scan of a picture, the variable CtbAddrInRs that defines the coding tree block address is set equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_base_addr_per_tile_enable_flag?TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]]:0)]. In the tile scan, the variable CtbAddrInTs that defines the coding tree block address is set equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal that defines the difference between the luma quantization parameter and its prediction for a coding unit including cu_qp_delta_abs is set equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr that define the values to be used when determining the respective values of the quantization parameters Qp'Cb and Qp'Cr for a coding unit including cu_chroma_qp_offset_flag are both set equal to 0. Thus, in the case of relative indication mode 46, tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the currently decoded / encoded slice included by its segment information 48 onto the correct tileIdx, i.e., refers to the correct segment within picture 18 or 54, TileSliceSegBaseAddr provides the base address for this segment, and this base address is then added to the relative slice address slice_segment_address of its start position information 42 to provide the address of this slice. CtbAddrTsToRs simply converts the address measured along coding path 22 between blocks 64 to the address measured in raster scan block order. The decoder and encoder can use pseudo-code 5 to calculate the base address of segment onset position 44, or in the case of explicit information exemplified below, simply the encoder can do this.

[0070] Regarding the order of VCL NAL units and their association with the coded pictures, the following can be said.

[0071] Specifically, the following defines the order of VCL NAL units and their association with the coded pictures.

[0072] Each VCL NAL unit is a part of the coded picture.

[0073] The order of VCL NAL units within a coded picture is constrained as follows. - The first VCL NAL unit of the coded picture shall have a first_slice_segment_in_pic_flag equal to 1. - Let sliceSegAddrA and sliceSegAddrB be the values of CtbAddrTsToRs[slice_segment_address(slice_segment_base_addr_per_tile_enable_flag?TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]:0)]] for any two coded slice segment NAL units A and B within the same coded picture. When any of the following conditions is true, the coded slice segment NAL unit A shall precede the coded slice segment NAL unit B. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]], and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0074] Decoders 38 and 52 can monitor these conditions to determine when the new access unit 36 starts within the inbound data stream 10 or 12, or to determine that some NAL unit is missing.

[0075] The variant form A described so far may be modified in different ways, for example, to enable easier implementation and processing.

[0076] For example, in variant form A, as already described above with respect to FIG. 1, segment 30 may not be strictly a tile. For this reason, the slice segment base address calculation does not have to occur exclusively for tile granularity. Multiple tiles may be joined in one area and then, following the description presented above, segment 30 may be formed, and then, with respect to that segment 30, the slice segment base addresses may be defined together, that is, the slice address in the slice header is given relative to the first CTU belonging to a given area of the multiple tiles forming segment 30 in bitstream order.

[0077] Furthermore, additionally or alternatively, according to variant form B described below, it becomes clearer that instead of burdening the client side or the decoder with the calculation of the slice segment base address for each segment 30, explicit signaling of the slice segment base address may be provided within the data stream.

[0078] Furthermore, additionally or alternatively, for example, when the slice_segment_base_addess_per_tile_enable_flag is equal to 1, it is ensured that the tile_fixed_structure is equal to 1, and at the same time, the burden of repeating tile structure signaling within each PPS of the video bitstream can be reduced by enabling the tiling syntax to be conveyed either within the SPS or within the PPS. The PPS instance of the tiling syntax may invalidate the tiling syntax settings conveyed within the SPS.

[0079] The following drawings show, by highlighting, the changes to variant A described above. For example, FIG. 7 shows the tiling syntax portion that can be used either within the PPS or within the SPS. When the tiling syntax of FIG. 7 is transmitted within the PPS, it overrides the tiling syntax provided by the corresponding SPS, i.e., that tiling syntax has a higher precedence. FIG. 7 shows the possibility that the parameter set 36 of FIG. 1 conveys the base address for the segment onset position 44 by the base address data field 130.

[0080] The semantics can be as follows.

[0081] A slice_segment_base_addr_per_area_enable_flag equal to 0 specifies that the variable CtbAddrInRs of the slice segment is derived solely from the slice_segment_address. A slice_segment_base_addr_per_area_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on the slice_segment_address and the tile-dependent offset. When the slice_segment_base_addr_per_area_enable_flag does not exist, it is inferred to be equal to 0.

[0082] area_id_in_ps[i] defines the tile id in bitstream order. The value of area_id_in_ps[i] ranges from 0 to 255, and area_id_in_ps[i] shall not have the same value as area_id_in_ps[j] when i is not equal to j. When area_id_in_sh does not exist, it is inferred to be equal to 0.

[0083] slice_segment_base_address[i] defines the base slice segment address in tile scan order for all slice segments belonging to the area having the same value of area_id_in_ps[i]. When it does not exist, the value of slice_segment_base_address[i] is inferred to be equal to 0.

[0084] When area_id_enable_flag is equal to 1, it is a bitstream conformity requirement that an access unit delimiter NAL unit exists in the bitstream.

[0085] For any two values of i, it is a bitstream conformity requirement that when area_id_in_ps[i] does not have equal values, the values of slice_segment_base_address[i - 1] are not equal.

[0086] Figures 8 and 9 provide an example regarding SPS syntax and PPS syntax with individual usable flags for tiling syntax signaling. Here, the following semantics are used.

[0087] tiles_enabled_in_sps_flag equal to 1 defines that the SPS conveys the default tiling syntax that describes the default structure of the CVS.

[0088] The tiles_enabled_in_pps_flag equal to 1 stipulates that the PPS conveys the tiling syntax that describes the actual tiling structure of the picture of the CVS that refers to the PPS and overwrites the tiling syntax in the referred SPS.

[0089] The coding tree block raster and the tiling scan conversion process are as follows in this case.

[0090] The list colWidth[i] for i ranging from 0 to num_tile_columns_minus1, including both end values, which defines the width of the i-th tile column in coding tree block (CTB) units, is derived as follows in (pseudo-code 1'). if(uniform_spacing_flag ) for(i = 0; i <= num_tile_columns_minus1; i++) colWidth[i]=((i + 1)*PicWidthInCtbsY) / (num_tile_columns_minus1 + 1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1 + 1) else{ colWidth[ num_tile_columns_minus1]=PicWidthInCtbsY(6 3) for(i = 0; i < num_tile_columns_minus1; i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0091] The list rowHeight[j] for j ranging from 0 to num_tile_rows_minus1, including both end values, which defines the height of the j-th tile row in CTB units, is derived as follows in (pseudo-code 2'). if(uniform_spacing_flag) for(j = 0; j <= num_tile_rows_minus1; j++) rowHeight[j] = ((j + 1) * PicHeightInCtbsY) / (num_tile_rows_minus1 + 1) - (j * PicHeightInCtbsY) / (num_tile_rows_minus1 + 1) else{ rowHeight[num_tile_rows_minus1] = PicHeightInCtbsY (6 4) for(j = 0; j < num_tile_rows_minus1; j++){ rowHeight[j] = row_height_minus1[j] + 1 rowHeight[num_tile_rows_minus1] -= rowHeight[j] } }

[0092] The list TileId[ctbAddrTs] ranging from 0 to PicSizeInCtbsY - 1 including both end values, which defines the conversion from CTB address to tile ID in tile scanning, is derived as follows (pseudo-code 3'). for(j = 0, tileIdx = 0; j <= num_tile_rows_minus1; j++) for(i = 0; i <= num_tile_columns_minus1; i++, tileIdx++) for(y = rowBd[j]; y < rowBd[j + 1]; y++) for(x = colBd[i]; x < colBd[i + 1]; x++) TileId[CtbAddrRsToTs[y * PicWidthInCtbsY + x]] = tileIdx

[0093] The meaning of the variable remains the meaning taught above, but N, however, indicates the number of tiles which may be greater than the number M of segments 30 by collecting tiles for segment 30.

[0094] The list TileSizeInCtbsY[k] for k ranging from 0 to ((num_tile_rows_minus1 + 1)*(num_tile_columns_minus1 + 1) - 1), including both end values, which defines the size of the k-th tile in the picture in CTB units, is derived as in the following (pseudo-code 4'). for(areaId = 0; areaId <= 255; areaId++) AreaSizeInCtbsY[areaId] = 0 for(j = 0, k = 0; j <= num_tile_rows_minus1; j++) for(i = 0; i <= num_tile_columns_minus1; i++, k++) if(area_id_in_ps[j*(num_tile_columns_minus1 + 1) + i] == areaId) AreaSizeInCtbsY[areaId] += (colWidth[i] * rowHeight[j])

[0095] area_id_in_ps[] is a vector containing components 0…N - 1, where N is the number of tiles, and the i-th component indicates the segment index value associated with the segment in which the i-th tile is located in tile raster scan segment order. Assume that the values entered within the components are values between 0…256, or in other words, values within the area of possible values for the segment index in segment information 48. Recall the picture decoding option / target dependency, i.e., whether the base address of the segments in picture 18 or the base address of the segments in sub-picture 54 should be calculated for N to be the number of tiles in picture 18 or sub-picture 54 respectively. AreaSizeInCtbsY[] is interpreted as a vector containing components for each member of the area of possible values regarding the segment index in segment information 48, indicating the number of blocks 64 covered thereby for each segment index value occurring within picture 18 or sub-picture 54, regardless of the current decoding target / option.

[0096] The mapping from AreaId to AreaIdx is derived as follows. NumTilesInArea[0...255]=0; NumAreas=1 for(k=0;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 If(NumTilesInArea[area_id_in_ps[k]]==1) AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas++;

[0097] NumTilesInArea[] is interpreted as a vector containing components for each member of the area of possible values regarding the segment index in segment information 48, indicating the number of tiles covered by each segment having this segment index value for each segment index value occurring within picture 18 or sub-picture 54, regardless of the current decoding target / option. AreaIdToAreaIdx[] is interpreted as a vector containing components for each member of the area of possible values regarding the segment index in segment information 48, indicating its rank or index when assigned or measured along the raster scan segment order leading in the segment row direction from top left to bottom right for each segment index value occurring within picture 18 or sub-picture 54, regardless of the current decoding target / option.

[0098] In bitstream order, the slice segment address offset of the first slice segment of the k-th area in the picture, defined in CTB units and including both end values, for k in the range from 0 to (NumAreas - 1), the list AreaSliceSegBaseAddr[k] is derived as follows in the following (pseudo-code 5'). AreaSliceSegBaseAddr[0]=0 for(k=1;k<( NumAreas);k++) AreaSliceSegBaseAddr[k]= slice_segment_base_address [k-1]

[0099] AreaSliceSegBaseAddr[i] is interpreted as a vector containing components 0...M - 1, where M is the number of segments 30, equal to NumAreas, and in the i-th component, the base address, i.e., the AreaIdx that can be associated with the area_id_in_sh of the slice segment information 48 of the slice by the decoder and encoder using AreaIdToAreaIdx, indicates the segment onset position 44 for the i-th segment in the aforementioned raster scan segment order.

[0100] Therefore, if explicit transmission is also applied to the sub-picture decoding option, the explicit transmission of the base address for the first segment 30 of picture 18 or sub-picture 54 in raster scan segment order is excluded because it will be zero anyway, but the base address data field 130 defined by the syntax element slice_segment_base_address defines, for each segment 30, the base address of the offset position 44 of each segment that addresses the offset position 44 relative to the picture onset position 24. Note that the definition that this base address by the base address data field is zero is a result of the convention that the explicit base address transmission starts at the second segment in raster scan segment order. Naturally, alternative embodiments may be contemplated if explicit base address transmission is used instead for all segments.

[0101] If implicit derivation of the base address is used instead of explicit base address signaling, the same can be done by the decoder or encoder as follows. The following calculations are used by the encoder for base address calculation even in the case of explicit base address signaling.

[0102] A list TileSliceSegBaseAddr[k] for k in the range from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), inclusive, specifying the slice segment address offset of the first slice segment of the k-th tile in the picture in bitstream order in CTB units, and a list AreaSliceSegBaseAddr[k] for k in the range from 0 to (NumAreas-1), inclusive, specifying the slice segment address offset of the first slice segment of the k-th area in the picture in bitstream order in CTB units, are derived as follows (pseudo-code 5"). TileSliceSegBaseAddr[0]=0 AreaSliceSegBaseAddr[area_id_in_ps[0]]=0 NumTilesInArea[0...255]=0; NumTilesInArea[area_id_in_ps[0]]=1 AreaIdToAreaIdx[area_id_in_ps[0]]=0 NumAreas=1 for(k=1;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 TileSliceSegBaseAddr[k]=TileSliceSegBaseAddr[k - 1]+TileSizeInCtbsY[k-1] If(NumTilesInArea[area_id_in_ps[k]]==1) AreaSliceSegBaseAddr[NumAreas]=TileSliceSegBaseAddr[k] AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas; NumAreas++;

[0103] In addition, FIG. 10 shows how the slice header according to Variant B can be, and the following shows the possible semantics.

[0104] The first_slice_segment_in_area_flag equal to 1 specifies that when tiles_enabled_flag is equal to 0, the slice segment is the first slice segment of the picture in decoding order. Otherwise, when tiles_enabled_flag is equal to 1, the first_slice_segment_in_pic_flag equal to 1 specifies that the slice segment is the first slice segment of the picture tile in decoding order. The first_slice_segment_in_pic_flag equal to 0 specifies that it is not the first slice segment of the picture in decoding order.

[0105] Note 1. This syntax element may be used for the detection of the boundary between consecutive coded pictures in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of an IDR picture may cause problems in the detection of the boundary between coded pictures. This may occur, for example, when transmitting all intra-coded video in an error-prone environment. This problem can be mitigated by alternately using two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) for any two consecutive IDR pictures. The use of the temporal sublayer zero index SEI message may also be useful as its value contains different syntax elements irap_pic_id for consecutive IRAP pictures in decoding order. Some system environments may also have other provisions that are similarly useful for picture boundary detection, such as the use of presentation time stamps in the Rec. ITU-T H.222.0|ISO / IEC 13818-1 system, access unit framing in the ISO base media file format of ISO / IEC 14496-12, or marker bits in the IETF RFC 3550 real-time transport protocol header.

[0106] A dependent_slice_segment_flag equal to 1 specifies that the value of each slice segment header syntax element that does not exist is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When it does not exist, the value of the dependent_slice_segment_flag is inferred to be equal to 0.

[0107] The variable SliceAddrRs is derived as follows. - If dependent_slice_segment_flag is equal to 0, SliceAddrRs is set equal to CtbAddrInRs. - Otherwise, SliceAddrRs is set equal to the SliceAddrRs of the preceding slice segment that contains the coding tree block whose coding tree block address for it is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs] - 1], where the variable CtbAddrInRs is defined within the semantics of slice_segment_address.

[0108] area_id_in_sh defines the index of the tile to which the slice segment belongs. The value of area_id_in_sh ranges from 0 to 255. When area_id_in_sh does not exist, it is inferred to be equal to 0.

[0109] slice_segment_address (when slice_segment_base_addr_per_tile_enable_flag is equal to 1) specifies the first coding tree block in the slice segment in the coding tree block raster scan of the picture and, optionally, in the tile scan of the picture as follows. The variable maxNumCtbY is defined as (slice_segment_addr_offset_per_tile_enable_flag? AreaSizeInCtbsY[area_id_in_sh]: PicSizeInCtbsY).

[0110] The length of the slice_segment_address syntax element is Ceil(Log2(maxNumCtbsY)) bits. The value of slice_segment_address shall be in the range from 0 to maxNumCtbsY-1, inclusive of both endpoints. When slice_segment_addr_offset_per_tile_enable_flag is equal to 0, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same coded picture. When slice_segment_addr_offset_per_tile_enable_flag is equal to 1, the value of slice_segment_address shall not be equal to the value of slice_segment_address of any other coded slice segment NAL unit of the same area within the same coded picture. When slice_segment_address does not exist, it is inferred to be equal to 0.

[0111] In the coding tree block raster scan of a picture, the variable CtbAddrInRs that specifies the coding tree block address is set equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_addr_offset_per_tile_enable_flag?TileSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]]:0)]. In the tile scan, the variable CtbAddrInTs that specifies the coding tree block address is set equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal that specifies the difference between the luma quantization parameter and its prediction for the coding unit including cu_qp_delta_abs is set equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr that specify the values to be used when determining the respective values of the quantization parameters Qp'Cb and Qp'Cr for the coding unit including cu_chroma_qp_offset_flag are both set equal to 0.

[0112] Regarding the order of VCL NAL units and their association with the coded picture and access unit 36, it can be said as follows. Each VCL NAL unit is a part of the coded picture.

[0113] The order of VCL NAL units within a coded picture is restricted as follows. - The first VCL NAL unit of the coded picture shall have a first_slice_segment_in_pic_flag equal to 1. - Set sliceSegAddrA and sliceSegAddrB to the values of CtbAddrTsToRs[slice_segment_address + CtbAddrRsToTs[(slice_segment_base_addr_per_tile_enable_flag? AreaSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]] : 0)]] of any two coded slice segment NAL units A and B within the same coded picture. When any of the following conditions is true, the coded slice segment NAL unit A shall precede the coded slice segment NAL unit B. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]], and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0114] The description of the possible HEVC codec extensions for the efficient sub-picture extraction presented above is thus extended preliminarily with respect to the mode already shown with respect to FIG. 1, by which not only is the extraction process reduced by the advantageous relative positioning of the slice start position with respect to the onset position of the current segment, but also, in addition, information regarding the decoding of the whole picture / video is provided, and the sub-picture extraction process can actually be limited simply to activities regarding the exclusion or discarding of slices not required according to the sub-picture decoding option, or alternatively, by providing information regarding the decoding of some sub-picture decoding options. Stated another way, this extension of the HEVC modification described above means that the interest in simply decoding sub-pictures, i.e., the selection of sub-picture decoding options, is treated as if layers were missing in a multi-layer data stream by selecting all corresponding layers corresponding to the decoding of the full picture area of the original data stream. It will be noted that the latter mode may later be used independently of the relative slice start point indication discussed above.

[0115] Accordingly, the extensions described next enable an easier extraction process without the need to rewrite the parameter sets. As in the extraction process of the layer bit stream, the removal of NAL units carrying slice data and parameter sets is sufficient to create a conforming bit stream, and a decoder receiving such a bit stream can easily determine the actual operating point represented in the extracted bit stream. In this context, the operating point exists in the space consumed, at least by a subset of the segment grid or tiling grid when using tiles as segments, and optionally, by scalability dimensions such as spatial resolution, temporal resolution, SNR scalability, etc.

[0116] According to this embodiment, a parameter set such as a VPS or an SPS defines several operating points in the context of tiling, hereinafter referred to as an output tile set (OTS), and describes several characteristics thereof. Here, assuming exemplarily that the parameters regarding the OTS are included in the SPS, several embodiments are shown in FIG. 11.

[0117] As shown in FIG. 11, a parameter set, where, by way of example, SPS shows the decoding of a complete picture 18 (compared to FIG. 1) using a tiling syntax 140, an example of which is presented in FIG. 7, the subdivision of picture 18 into segments, optionally, respectively, the association of segment index values to these segments, and optionally, the base address of the onset position of these segments, as well as the size of picture 18, i.e., the horizontal size at 142 and the vertical size at 144, and, at 146, when i = 0, the decoder capability level for decoding picture 18. Since num_output_tile_format at 148 where the syntax element is transmitted is greater than 1, the parameter set of FIG. 11 additionally signals at 146 information 142 regarding further sub-picture decoding options showing the associated decoder capability levels for other picture sizes and picture 54. According to the example of FIG. 11, segment 30 resulting in corresponding picture sizes as shown by 142 and 144 for the output format, where, by way of example, the various configurations of the tiles are also signaled individually. Num_output_tile_sets transmitted at 150 is within the syntax element and indicates the number of search configurations. Each configuration is associated by an index 152 to one of the output tile formats. The number of tiles constituting each picture 18 or picture 54 is transmitted at 154, and the segment index is transmitted at 156 for each individual tile configuration. Finally, for each of the number of tile configurations indicated by 150, the parameter set shows at 158 additional information regarding decoding, i.e., coded picture buffer with decoding timestamp and / or motion time and / or parameters regarding the buffer size required to decode each segment configuration (or tile configuration) from the data stream.

[0118] When using the embodiment of FIG. 11, the decoding process performed within decoder 38 or 52 utilizes an additional input parameter, sometimes called a target output tile set, which indicates which of the defined OTSs is selected for decoding, and this input parameter represents signal 58 discussed above with respect to FIG. 1.

[0119] The parameter TargetOutputTileSet can be determined by various means. · External means such as an indication by the system layer through signaling in the file format, transport stream, SDP, or others. · The presence of an SEI message in the bitstream that indicates which tiles or OTSs are present in the bitstream, i.e., a description of the previous extraction process the bitstream has undergone. Thus, the extraction information can be included in a data stream indicating one or more decodable OTSs. · Analyze the value of the syntax element tile_id_sh in the slice header of the NAL unit within the access unit to determine which of the defined OTSs may be decodable using the content of the bitstream.

[0120] For example, priorities can be assigned to the defined OTSs as follows to assist in determining the OTS to be decoded. · Explicit signaling within a loop via num_output_tile_sets or an SEI message that assigns priorities to the OTSs · Ordinal units within a loop via num_output_tile_sets · A present tile / removed tile SEI message having an indication of all OTSs included in the bitstream

[0121] Thus, parameter set 36 may each include a priority indication indicating the priority with which a particular sub-picture configuration is selected for the decoding targets by decoders 38 and 52 for each decoding option signaled by 148 and for each segment collection as defined by 150 and 158. The priority indication may indicate the priority either overall among all of the segment collection options signaled by 150 or individually among one common sub-picture option, i.e., a group belonging to the same sub-picture size and corresponding decoder capability level.

[0122] The TileSliceSegBaseAddr derivation needs to be adjusted for Transform Type A as follows (highlighted by Turquoise). The list TileId[ctbAddrTs] for ctbAddrTs in the range from 0 to PicSizeInCtbsY - 1, including both end values, which specifies the conversion from the CTB address to the tile ID in tile scanning, is derived as follows. for(j = 0, tileIdx = 0; j <= num_tile_rows_minus1; j++) for(i = 0; i <= num_tile_columns_minus1; i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]] = tileIdx for(y = rowBd[j]; y < rowBd[j + 1]; y++) (6 9) for(x = colBd[i]; x < colBd[i + 1]; x++) TileId[CtbAddrRsToTs[y*PicWidthInCtbsY + x]] = tileIdx

[0123] The list TileSizeInCtbsY[k] for k ranging from 0 to ((num_tile_rows_minus1 + 1)*(num_tile_columns_minus1 + 1) - 1), inclusive, which specifies the size of the k-th tile in the picture in CTB units, is derived as follows. for(j = 0, k = 0; j <= num_tile_rows_minus1; j++) for(i = 0; i <= num_tile_columns_minus1; i++, k++) TileSizeInCtbsY[k]=colWidth[i]*rowHeight[j]

[0124] For the i-th target OTS, the list TileSliceSegBaseAddr[k] for k ranging from 0 to ((num_tile_rows_minus1 + 1)*(num_tile_columns_minus1 + 1) - 1), inclusive, which specifies the tile scan slice segment address offset of the first slice segment of the k-th tile in the picture in bitstream order in CTB units, is derived as follows. TileSliceSegBaseAddr[0]=0 cnt = 1 for(k = 1; k <= ((num_tile_rows_minus1 + 1)*(num_tile_columns_minus1 + 1) - 1); k++) if(tileIdxInTargetOTS) { TileSliceSegBaseAddr[cnt]=TileSliceSegBaseAddr[cnt - 1]+TileSizeInCtbsY[k - 1] cnt++ / / with cnt derivable from tile_id_in_sh with a mapping table to tile_id_in_sps }

[0125] tileIdxInTargetOTS is true for all tiles belonging to TargetOutputTileSet and false otherwise. Note that in the above, the tile array in the decoded picture of TargetOutputTileSet remains in the same state as the tile array in the original bitstream. Raster and its inverse for tile scan conversion are done considering only the tiles within targetOutputTileSet.

[0126] As already indicated above, the extension just described with respect to FIG. 11, considering the parameter set 36 and its signaling of the picture size, as well as the corresponding decoder capability level signaling not only for the original picture 18 but also for the sub-picture 54, can be used, for example, in a framework where the aforementioned slice start position information 42 does not exist. For example, by convention, a slice may be restricted to start not within segment 30 but simply at the segment onset position. In that case, for example, the segment information 48 may be sufficient. Other examples may also be possible. In any case, the decoder 38 or 52 that receives the stream 10 or 12 (regardless of the presence or absence of the start position information 42) uses the information within the parameter set 36 equal to 36' as follows. Specifically, the decoder decodes the parameter set 36 from the inbound data stream, which may be 10 or 12, and derives a decoding option indication 58 as to whether picture decoding or sub-picture decoding should be performed on that inbound data stream. Picture decoding is targeted at decoding picture 18 from data stream 10, i.e., indicates the composition of picture 10 from among all segments 30. Thus, to achieve this purpose, all slices 26 need to be within data stream 10. The sub-picture decoding purpose is targeted at decoding picture 54 from data stream 12. "Picture 54" means that two or more sub-picture decoding options can be signaled within the parameter set 36 that have the same differences in picture size and shape as well as decoder capability level. For example, in the example of FIG. 1, different sub-picture decoding options can arise from forming the sub-picture 54 by composing it from two adjacent segments 30, composing the sub-picture 54 one by one from two segments 30, or composing the sub-picture 54 from only one segment 30, and these examples can be extended, for example, by composing the sub-picture from among three segments 30.

[0127] When picture decoding is to be performed on the data stream, the decoder derives the size of picture 18 from parameter set 36 via, for example, 142 and 144 when i = 0, and derives from the data stream an indication of the decoder capability level required to decode picture 18 via, for example, 146 when i = 0. The decoder derives from parameter set 36 information regarding the segmentation of picture 18 into segments 30 that are encoded in the data stream without coding interdependence, i.e., the first set of all segments, from, for example, tiling syntax 140. Further, the decoder decodes picture 18 from the data stream in units of slices 26 into which picture 18 is partitioned along coding path 22. If the individual slices 26 contain start position information, reference is made to the above description regarding the placement of the decoded picture content of the individual slices 26. The decoder may check the decoder capability level to determine whether that decoder capability level is met by decoder 38.

[0128] However, in the case of a decoding option indication 58 suggesting that sub-picture decoding is to be performed on the inbound data stream, the decoder derives from parameter set 36 an indication of the further picture size of sub-picture 54 and, if i≠0, the decoder capabilities level required to decode the sub-picture from the data stream, such as via 142, 144, and 146, and derives from parameter set 36 information regarding the subdivision of sub-picture 54 into a second set of segments, which is a subset of the set of segments of picture 18, in which the segments are encoded into the data stream without coding interdependence, and decodes sub-picture 54 from the data stream in those slice 26 units in which sub-picture 54 is encoded. Here too, the decoder uses or follows coding path 22'. At this time, coding path 22' crosses sub-picture 54 for other target purposes. Like coding path 22 that crosses picture 18, coding path 22' continuously crosses sub-picture 54 in segment units, and each slice 26 of the slices regarding any fragment of sub-picture 54 is from within the second set of segments, and one fragment of the segments forming the composition of sub-picture 54, or one or more segments, are fully encoded. Note that the coding path or coding order 22' may cross a subset of segments 30 participating in the composition of sub-picture 54 in an order different from the order in which the subset is crossed by coding path 22 within the original picture 18. However, since segment coding independence does not prevent coding dependence across segment boundaries in any case, this does not affect the decoding result. However, within each segment 30, paths 22 and 22' coincide, which is important for maintaining decoding synchronization and even the placement of the slice content when the use of start position information 42 is not affected. In this case too, the decoder may check the decoder capabilities level regarding the sub-picture decoding option as to whether its decoder capabilities level is met by decoder 38 / 52.

[0129] As discussed above, the decoder can derive the decoding option indication 58 by various means. For example, the decoder 52 analyzes the inbound data stream, i.e., the segments covered by the slices present within 10 or 12, to determine, for example, whether slices and corresponding segments have been removed by exclusion by the network device 50, and if so, which slices and corresponding segments have been removed. Further, additionally or alternatively, for determining the decoding option or target to be used, external signaling as shown by 58 in FIG. 1 can be used by the decoder 38 or 52. The external signaling can already uniquely identify the coding option to be used, or that it should not be used, or can simply exclude some coding options that are not available, for example, by the intermediate extraction process 14. In that case, this external signaling can originate from the network device 50, or based on further information, can reliably identify a set of coding options from among which the decoder should select one coding option. Further, additionally or alternatively, for determining the decoding option or target to be used, side information within the inbound data stream, such as explicit information regarding the options still decodable, can be used by the decoder 38 or 52. Similar to the external information, the latter information may be included by the extraction entity, i.e., the device 50. Further, the decoder can examine the decoder capability levels associated with the decoding options such as those present within the parameter set 54, and when they conflict with the capabilities of the decoder, can exclude some from the list of possible decoding option candidates. Some or all combinations of these hints can still form the decoding option indication. If any ambiguity remains, the decoder can use the aforementioned priorities to determine one of the remaining possible decoding option candidates with the highest priority.

[0130] Although several aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of corresponding method steps or features of a method if a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of method steps also represent a description of corresponding blocks or items or features of a corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0131] The data stream of the present invention may be stored on a digital storage medium or transmitted on a transmission medium, such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0132] Depending on some implementation requirements, the implementation form of the present invention may be implemented in hardware or software. This implementation form may be executed using a digital storage medium, such as a floppy disk, DVD, Blu-Ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH (registered trademark) memory, which stores an electronically readable control signal that cooperates with (or is capable of cooperating with) a programmable computer system so that each method is executed. Therefore, the digital storage medium may be computer-readable.

[0133] Some embodiments according to the present invention comprise a data carrier having an electronically readable control signal that can cooperate with a programmable computer system so that one of the methods described herein is executed.

[0134] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code operable to execute one of these methods when the computer program product is run on a computer. The program code can be stored, for example, on a machine-readable carrier.

[0135] Other embodiments include a computer program stored on a machine-readable carrier for executing one of the methods described herein.

[0136] In other words, embodiments of the method of the present invention are thus computer programs having program code for executing one of the methods described herein when the computer program is run on a computer.

[0137] Further embodiments of the present invention are thus a data carrier (or digital storage medium, or computer-readable medium) having recorded thereon a computer program for executing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is generally tangible and / or non-transitory.

[0138] Further embodiments of the method of the present invention are thus a sequence of data streams or signals representing a computer program for executing one of the methods described herein. The sequence of data streams or signals can be configured to be transferred, for example, via a data communication connection, for example, via the Internet.

[0139] Further embodiments include processing means, such as a computer, or a programmable logic device, configured or adapted to execute one of the methods described herein.

[0140] Further embodiments include a computer having installed thereon a computer program for executing one of the methods described herein.

[0141] A further embodiment according to the present invention includes an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for executing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system includes, for example, a file server for transferring the computer program to the receiver.

[0142] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to execute some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array may cooperate with a microprocessor to execute one of the methods described herein. Generally, these methods are preferably executed by any hardware device.

[0143] The apparatus described herein may be implemented using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0144] The apparatus described herein, or any component of the apparatus described herein, may be at least partially implemented in hardware and / or in software.

[0145] The method described herein may be executed using a hardware device, or using a computer, or using a combination of a hardware device and a computer.

[0146] The method described herein, or any component of the apparatus described herein, may be at least partially executed by hardware and / or by software.

[0147] The embodiments described above merely illustrate the principles of the present invention. It will be understood that modifications and variations of the arrangements and details described herein will be apparent to other skilled persons. Accordingly, it is intended to be limited only by the scope of the following claims, and not by the specific details presented in the description and explanation of the embodiments herein.

Explanation of Signs

[0148] 10 Data stream 12 Data stream 14 Sub-picture extraction 16 Encoder 18 Picture 20 Video 22 Coding path 22' Coding path 24 Picture onset position 26 Slice 28 Slice 30 Segment 32 Slice payload 34 Slice header 36 Parameter set, access unit 36' Parameter set 38 Decoder 40 Start position 42 Start position information 44 Onset position 46 Relative start position 48 Segment information 50 Network device 52 Decoder 54 Picture, sub-picture 54" Picture 56 Extracted video 58 Decoding option indication 60 Start position coding mode flag 62 Absolute start position indication 64 Coding block 70 Picture parameter set 72 Sequence parameter set 74 video parameter set 76 VCL NAL unit 78 VCL NAL unit 80 NALU header 86 tiling syntax 88 tile 94 base slice segment address 96 offset slice segment address 98 slice segment address 106 flag 112 start position coding mode flag 114 syntax element, 116 raster scan segment order 120 flag 122 first syntax element 124 second syntax element 130 base address data field 140 tiling index 142 information on subpicture decoding option 152 index

Claims

1. Dividing a picture (18) into slices (28) and segments (30), providing a slice header (34) including start position information (42) corresponding to each slice (26) of the slices (28), wherein the start position information (42) indicates the start position (40) of the slice (26) with respect to the segment onset position (44) of one of the segments (30), determining an absolute address measured from the start of the picture (18), the determining including performing a sum corresponding to the start position (40) and the segment onset position (44), setting a start position coding mode flag (60) to a first state, the start position coding mode flag (60) being switchable between the first state and a second state, the first state indicating that the start position (40) is relative to the segment onset position (44), and the second state indicating that the start position (40) is relative to the picture onset position (24), encoding the start position coding mode flag (60), the segment onset position (44), the slice header (34), and the respective slices (26) into a data stream (10).

2. The method according to claim 1, wherein the picture (18) is encoded into the segments (30) without encoding interdependence.

3. The method according to claim 1, wherein the segment (30) includes one or more tiles.

4. The method according to claim 1, wherein the segment onset position (44) is indicated by a parameter set (36).

5. An encoder (16) comprising a processor that executes instructions for implementing the method according to any one of claims 1 to 4.

6. Receiving a picture (18) divided into slices (28) and segments (30) via a data stream (10), decoding from the data stream (10) a segment onset position (44) of one of the segments (30) and a slice header (34) including start position information (42) corresponding to each slice (26) of the slice (28); decoding from the data stream (10) a start position coding mode flag (60), the start position coding mode flag (60) being switchable between a first state and a second state, the first state indicating that the start position (40) is relative to the segment onset position (44), and the second state indicating that the start position (40) is relative to the picture onset position (24); identifying, based on the start position coding mode flag (60) having the first state, the start position (40) of the slice (26) relative to the segment onset position (44) using the start position information (42); determining an absolute address measured from the start of the picture (18), the determining including performing a sum corresponding to the start position (40) and the segment onset position (44); a decoding method comprising: Claims 7 The method according to claim 6, wherein the picture (18) is encoded into the segment (30) without encoding interdependence. Claims 8 The method according to claim 6, wherein the segment (30) includes one or more tiles. Claims 9 The method according to claim 6, wherein the segment onset position (44) is indicated by a parameter set (36). Claims 10 A decoder (38) comprising a processor that executes instructions for implementing the method according to any one of claims 6 to 9.

Images