Efficient sub-picture extraction

Abstract

To provide a data stream, a decoder, and / or an encoder that reduce complexity in sub-picture extraction.SOLUTION: A picture onset position 24 where a coding path 22 of an encoder 16 starts crossing of a segment 30 where its starting position 40 is located, is provided with starting position information 42 indicating a starting position 40 where a picture 18 thereafter is encoded into respective slices 26, 28 along the coding path 22, and thereby complexity in sub-picture extraction 14 is reduced. Alternatively or additionally, a sub-picture extraction process is processed similarly to selecting one layer from a multi-layer data stream.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This application relates to the concept of efficient subpicture extraction.

Background Art

[0002] Subpicture extraction is a process in which a data stream in which a video picture is encoded is adjusted according to a subpicture area without the need for re-encoding. HEVC DAM3 MCTS extraction, for example, continuously divides a picture into tiles that are independently coded with each other, and groups those tiles into a tile set that is restricted so that motion compensation prediction does not cross the tile set boundary. Thus, as far as motion compensation prediction and residual coding are concerned, without any need for re-encoding, it is possible to extract a subpicture-specific data stream from, for example, the original full-picture data stream. This MCTS extraction, however, requires adjustment of each NAL unit carrying slice data in order to adjust the slice segment address of the carried slice header.

[0003] Therefore, although the subpicture extraction process is known, it would be advantageous to have a concept that reduces the complexity of the required tasks at hand.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present invention is to provide a data stream, a decoder and / or an encoder in accordance with a concept for reducing the complexity of subpicture extraction.

Means for Solving the Problems

[0005] This object is achieved by the subject matter of the independent claims of this application.

[0006] According to a first aspect of this application, the complexity of subpicture extraction is reduced by providing each slice with start point information indicating the start position from which the picture will be encoded in each slice along the coding path, relative to the onset position where the coding path begins traversing the segment in which its start position is located. The picture is subdivided into segments that are continuously traversed by the coding path in segment units, and each slice contains one or more segments fully encoded, with the picture being encoded into segments without coding interdependence. The picture is coded into a data stream along the coding path in slice units into which the picture is subdivided. In addition, each slice includes segment information indicating the segment in which the start position is located. Thus, both the start position information and the segment information make it possible to determine in which segment the start position of each slice is located and where within that segment it is located. By encoding segments without coding interdependence, it is possible to remove one or more slices relating to a single segment without affecting the decodeability of other segments. Furthermore, even when the segments of a picture are used to compile a new subpicture by shuffling or rearranging those segments and / or discarding some of the segments of the picture, it is possible to form a data stream specific to each subpicture based on the original data stream by discarding slices that do not encode any segments forming the subpicture and adopting the undiscarded slices that still refer to the correct segments, and by relative position indication, the correct position of the starting position of each slice within each segment. Thus, according to a first aspect of this application, the data stream enables easy subpicture extraction without the need to correct the starting position information and segment information, regardless of the rearrangement of the segments of the picture in the subpicture data stream and / or the exclusion of some segments.In other words, it simplifies the subpicture extraction process by allowing the remaining slices to be used without needing to correct the starting position information, and simply removing or discarding any slices that are not related to any segments contained within the subpicture that should be extracted.

[0007] According to one embodiment, the start position information includes a variable-length coded start address syntax element. Since the start position information does not need to be reversed in any way during subpicture extraction, variable-length coding can be used during subpicture extraction without any penalty.

[0008] In further embodiments, the data stream can be switched between start position information indicating the start position of each slice relative to the segment's onset position, or absolute relative to the picture onset position where the coding path begins traversing the picture, via a start position coding mode flag. This option allows existing codecs that previously used absolute start position indications to be extended to utilize the relative position indications discussed herein. A decoder may be able to understand both types of start position indications provided by the start position information, or simply one of them.

[0009] In further embodiments, the data stream further includes a base address data field that defines the base address of the onset location for each segment, which addresses the onset location along the coding path to the picture onset location for each segment. The transmission of these base addresses reuses the computational overhead for calculating the base address based on the individual subdivision of the picture into segments.

[0010] According to one embodiment, the segment information includes a segment syntax element in which a segment index is encoded to index the segment in which the starting position is located. The syntax element may be encoded using a fixed-length code. The data stream may include an index data field that defines the association of segment index values ​​for each segment. In other words, the index data field can explicitly associate tags, i.e., segment index values, with segments, and the segment information of a slice is set to its tag, or the segment index value associated with the segment in which the starting position of each slice is located. This tagging can be easily modified during subpicture extraction.

[0011] Another aspect of the present application, which may be combined with the first aspect, is to facilitate the subpicture extraction process by treating the subpicture extraction process similarly to selecting one layer from a multilayer data stream. That is, according to the second aspect, the extractable data stream includes a set of parameters that indicate not only the size of the picture in the data stream and a first decoder capability level required to decode the picture from the data stream, but also at least one subpicture decoding option, and with respect to this at least one subpicture decoding option, a reduced picture size and a second decoder capability level required to decode the subpicture from the extracted version of the data stream. This subpicture decoding option is treated similarly to a sublayer to a higher layer corresponding to a complete picture, and the extracted version of the data stream arises from the data stream by removing or discarding slices from the data stream in which segments encoded are spatially offset with respect to the segment cluster in which the subpicture is composed. These slices may or may not include start position information and segment information as discussed above with respect to the first aspect. However, it may suffice for each slice to contain segment information indicating the segment it covers. Therefore, according to the second embodiment, the subpicture extraction process is similar to the transitions between several layers of a multilayer data stream, requiring only the exclusion of slices that are not related to the intended layer; that is, slices that do not cover any of the segments contained within the subpicture are extracted. No "transformation" or "modification" of the data stream slices that will be adopted into the extracted data stream is necessary.

[0012] A favorable embodiment is the subject of the dependent claim. A preferred embodiment of this application is described below with reference to the drawings. [Brief explanation of the drawing]

[0013] [Figure 1] This schematic diagram illustrates the concept of an extractable data stream using a relative start position indication to a slice to mitigate the subpicture extraction process, showing the data stream that has not yet been extracted, as well as the extraction process and the extracted data stream, with participating entities including encoders, decoders, and optionally, network devices. [Figure 2] This schematic diagram shows pictures and fragments of a data stream to illustrate an embodiment in which the data stream can be switched between relative and absolute slice start position indications. [Figure 3] Figure 1 is a schematic diagram illustrating an implementation of a data stream, which is a variation of the HEVC data stream. [Figure 4] Figure 3 shows a parameter set that can be used in the implementation, and here, pseudocode illustrating the content transmitted within the picture parameter set as an example. [Figure 5] Figure 3 is a schematic diagram of a picture and its segmentation into segments, along with the parameters used in the pseudocode provided for the example implementation. [Figure 6] Figure 3 shows pseudocode illustrating possible content transmitted within a slice header, based on an example implementation. [Figure 7] This figure shows pseudocode illustrating the tiling syntax that could be used to implement the codec example in Figure 3. [Figure 8] Figure 7 shows a parameter set using the tiling syntax, here illustrating pseudocode that exemplifies the possible contents of a sequence parameter set. [Figure 9] Figure 7 shows a parameter set using the tiling syntax, here illustrating pseudocode that exemplifies possible content for a picture parameter set. [Figure 10]This figure shows pseudocode illustrating possible contents of a slice header to implement the example in Figure 3, which differs from Figure 6 in that a segment can consist of one or more tiles. [Figure 11] To demonstrate that the parameter set may convey information not only about the complete picture decoding options but also about at least one additional sub-picture decoding option, here is a figure of pseudocode illustrating the possible contents of a sequence parameter set, illustrating the picture parameter set. [Modes for carrying out the invention]

[0014] A first embodiment of this application will be described with reference to Figure 1. Figure 1 shows a data stream 10 and a sub-picture specific data stream 12 derived therefrom by a sub-picture extraction 14. The data stream 10 may be generated by an encoder 16. Specifically, the encoder 16 has encoded a picture 18 within the data stream 10. Figure 1 shows the picture 18 as one picture in a video 20, but it should be noted that embodiments of this application are not limited to video data streams. Rather, embodiments described herein can be easily adapted to picture codecs. The following description, however, shows embodiments relating to video coding.

[0015] When encoding picture 18 into data stream 10, encoder 16 follows or uses a coding path 22 that traverses picture 18, for example, from a picture onset position 24 located at the upper left corner of picture 18, to the opposite picture corner, such as the lower right corner, to continue traversing to another picture in video 20, for example. Following or using the coding path 22 determines the availability of spatially adjacent portions of picture 18, such as syntax elements, samples, or parameters derived from them by encoder 18, in order to code the current portion of picture 18. For example, encoder 16 may use predictive coding. For this purpose, encoder 16 may predict the current portion, such as its sample content or syntax elements describing its sample content, based on the spatially adjacent portions of picture 18, provided that these adjacent portions precede the current portion along the coding path 22. Additionally or alternatively, encoder 16 may use other spatial coding dependencies on the current portion, such as deriving an entropy context for coding syntax elements describing the current portion of picture 18 based on adjacent portions of picture 18. Furthermore, encoder 16 uses a coding path 22 to partition or packetize the data being coded in picture 18. To this end, encoder 16 subdivides the coded data into slices along the coding path 22. Thus, each slice 26 in the data stream 10 containing the data of picture 18 covers or encodes a corresponding portion, i.e., a slice 28 of picture 18, which is traversed sequentially by the coding path 22 without interleaving the slices 28. In other words, the coding path 22 traverses each of the subdivided slices 28 of picture 18 only once.

[0016] As shown in FIG. 1, encoder 16 also follows the subdivision of picture 18 into segments 30 when encoding picture 18 into data stream 10. FIG. 1 illustratively shows the subdivision of a picture into four segments 30 arranged in a 2x2 array. However, the number of segments is not critical and may vary. However, the subdivision of picture 18 into segments 30 may be such that segments 30 are arranged in rows and columns. The subdivision of picture 18 into segments 30 may be such that the picture 18 is completely covered without gaps, with segments within one segment row having the same height and segments within one segment column having the same width. However, thereby, the rectangular segments 30 may have different sizes in that segment columns and / or segment rows may differ from each other in terms of width and height. Following the subdivision of picture 18 into segments 30 may relate to encoding dependencies. Encoder 16 encodes picture 18 into data stream 10 such that the encoding dependencies do not cross segment boundaries. For this, no part within segment 30 of picture 18 is encoded depending on parts of picture 18 outside this segment. In other words, encoder 16 encodes segments 30 independently of each other. As long as the relationship between one being slice 28 and the other being segment 30 is considered, a slice 28 completely covers one or more segments 30 or is present only within a single segment 30. For the sake of mere illustration, FIG. 1 shows that each segment 30 is composed of two slices 28. This means that coding path 22 traverses segment 30 continuously, i.e., each segment 30 is completely traversed by coding path 22 before coding path 22 traverses the next segment 30 in coding order.

[0017] In this way, the encoder 16 encodes the picture 18 in units of slices 28 within the data stream 10. Each slice 26 in the data stream 10 therefore has an encoded corresponding picture slice 28. For example, each slice 26 has payload data 32 that encodes the content of the corresponding slice portion 28 of the picture 18 in the form of one or more of the following: prediction parameters such as a prediction mode that distinguishes between intra-coding mode and inter-coding mode, motion parameters for intra-prediction blocks, intra-prediction submodes for intra-prediction blocks, subdivision information for subdividing the picture 18 into blocks, and residual data such as conversion coefficients that represent prediction residuals.

[0018] Before proceeding to the description of the slice header 34 of slice 26 and its components or syntax elements, we will briefly continue with a description of how the encoder 16 codes the picture 18 into the data stream 10. As previously mentioned, the segments 30 are segments coded independently of each other. However, the description so far has focused only on the coding of picture 18. However, if picture 18 is a picture in video 20, the encoder 16 may employ the subdivision of the picture into segments 30 for the sequence of pictures in video 20, that is, subdivision of the pictures in the picture sequence in the same way, i.e., other pictures are segmented into the same number of segments 30, and the segments 30 are of equal size at spatially coincidental segment boundaries between the pictures. This is the situation shown in Figure 1, for example, where the pictures preceding and succeeding picture 18 are shown as being subdivided in the same way as picture 18, i.e., into four segments 30. In addition to the mutual independence of the coding segments 30 within a single picture, the encoder 16 may encode each segment of picture 18 such that the encoding of each segment 30 of picture 18 does not depend on a portion of another picture called a reference picture that lies outside or is spatially offset from the segment collated to each segment of picture 18. In other words, the collated segments 30 of pictures in a picture sequence of video 20 may form a spatiotemporal region within video 20 in which the encoder 16 encodes independently of the spatiotemporal region formed by other segments of these pictures. Thus, the aforementioned inter-prediction, i.e., motion-compensated prediction, is restricted so that a block within a segment 30 of picture 18 coded using inter-prediction does not, by motion vector, refer to a portion of the reference picture that lies outside the segment of the reference picture collated to the segment in which the current block is located. Rather, the encoder 16 might choose intra-prediction for that block instead.

[0019] Furthermore, in addition to the slice header 34 of slice 26, which will be described in more detail below, it should be noted that the data stream 10 includes further information containing higher-level coding parameters, hereinafter referred to as a parameter set. This parameter set 36 may be distributed over the data stream 10 and may be hierarchically defined in terms of scope. In other words, different parts of the parameter set 36 may have different scopes. For this reason, some coding parameters may be valid for only one picture, such as picture 18, while other coding parameters may represent coding parameters valid for a sequence of pictures, including for example picture 18, and still other coding parameters may be valid for the entire video 20. The coding parameters defined within the parameter set 36 will suffice for the moment with a short discussion for the purpose of explanation below. The coding parameters contained within the parameter set 36 may indicate, for example, the picture size of the pictures of the video 20, the subdivision of the pictures into segments 30, an indication of the decoder capability level required to decode picture 18 and video 20 respectively from the data stream 10, and / or decoding time stamps, coded picture buffer extraction times, and / or buffer sizes that will be used to decode the data stream 10.

[0020] In addition to the details described above, it should be noted that the encoder 16 may form the data stream 10 such that a slice 26 contained within the data stream 10 for one picture, such as picture 18, is not interleaved with other slices 26 within the data stream 10 for another picture of the video 20. Rather, these form a continuous portion of the data stream 10 called an access unit 36. An access unit 36, therefore, aggregates slices 26 for one picture, such as picture 18. In addition to slices 26, an access unit 36 ​​may also contain other data, such as a portion of a parameter set 36. The data within an access unit 36 ​​may faithfully adhere to some predefined order or ordering rule so that a receiver of the data stream 10, such as a decoder 38, can recognize the start of the next access unit in the data stream 10 based on monitoring the order of the data in the data stream 10. Even slices within an access unit 36 ​​may be ordered with respect to each other according to the order in which the coding path 22 traverses the corresponding slices 28 within picture 18.

[0021] After describing the general framework for coding the data stream 10, the description moves on to describing the slice headers 34 and their contents. Specifically, the encoder 16 provides each slice header 34 with starting position information indicating the starting position 40 from which the picture 18 will be encoded along the coding path 22 into each slice 26, which includes each slice header 34. Knowledge of the starting position 40 enables the decoder 38 to accurately position the picture content encoded in each slice 26 for the preceding slice portion 28 along the coding path 22, i.e., the picture content for the corresponding slice portion 28 of the picture 18, even if any preceding slice for the picture 18 is missing in the data stream 10.

[0022] However, preferably, the start position information 42 indicates the start position 40 of the slice 26 in the slice header 34 containing the start position information 42 relative to the onset position 44 where the coding path 22 begins, crossing the segment 30 where the start position 40 is located, as illustrated by the arrow 46 in Figure 1. The relative indication of the start position 40 reduces subpicture extraction 14, as will be further explained below.

[0023] To compensate for the relative indication of the starting position 40 by the starting position information 42 and the associated ambiguity, the slice header 34 additionally includes segment information 48 indicating the segment 30 to which the starting position 40 of slice 26 is located, to which the segment information 48 belongs. In the example in Figure 1, for example, slice 28, where the starting position 40 is indicated by arrow 46, is located within segment 30 indexed by capital B in Figure 1, and accordingly, the segment information 48 for that slice will indicate that this slice 26 belongs to segment B. Thus, together, the starting position information 42 and the segment information 48 are sufficient to determine the absolute position of the starting position 40 in picture 18, i.e., the origin, i.e., the relative position of the starting position 40 with respect to the picture onset position 24.

[0024] Upon receiving the data stream 10, the decoder 38 decodes the start position information 42 for each slice 26 from the data stream 10 and uses the start position information 42 to locate the start position 40 of each slice relative to the onset position 44 of the segment in which the start position 40 is located. Furthermore, the decoder 38 decodes the corresponding segment information 48 for each slice 26 from the data stream 10 so that it knows about the segment 30 to which the onset position 44 to which the start position information 42 indicates the start position 40 belongs. The decoder 38 can therefore locate the onset position 44 based on a parameter set 36 indicating the subdivision of the picture 18 into segments 30 and, accordingly, combine the relative position indication of the start position 40 shown for the onset position 44 with the onset position 44 in the picture 18, which results in the absolute position of the start position 40 in the picture 18 relative to the picture onset position 24. To reduce the computational overhead for the decoder 38, the more detailed embodiments described below demonstrate the possibility that some other part of the data stream 10, such as the parameter set 36 or supplemental extension information, may expressively indicate the onset position 44 for each segment 30, so that the decoder 38 can use this instruction rather than calculating the onset position of the segment 30 itself.

[0025] As outlined above, a preferred situation is the relative indication of the start position 40 to the onset position of the corresponding segment 30. This will be discussed in more detail below. Specifically, the preferred situation becomes more apparent when examining the subpicture extraction process 14. The latter process may be performed by some entity that processes the data stream 10. For example, the extraction process may be performed by a network device 50 connected to the transmission path between the encoder 16 and the decoder 52 that receives the extracted data stream 12. The decoder 52 may be identical to the decoder 38, i.e., a decoder that would also have been capable of decoding the complete or original data stream 10, or a decoder that is capable of decoding the extracted data stream 12 but, for example, cannot decode the complete data stream 10 due to a lower decoder capability level than that of the decoder 38. The extraction process 14 may, however, be performed within the decoder 52 / 38 itself.

[0026] The extraction process 14 aims to form a data stream 12 in which picture 54 is encoded, which is simply composed of an appropriate subset of segments 30 of picture 18, and / or differs in the relative arrangement of segments 30 within the picture areas of picture 18 and picture 54, respectively. Figure 1 shows an example in which picture 54 of data stream 12 is simply composed of one of the four segments 30 of picture 18 of the original data stream 10, namely segment B, but this is just one example, and picture 54 may be composed of two or more of the segments 30 of picture 18, as long as a rectangular picture area of ​​picture 54 is produced.

[0027] Notably, with respect to slices 26 included in the data stream 10, the extraction process 14 only needs to discard or exclude slices 26 relating to segments 30 of picture 18 that are not included in picture 54, but slices 26 relating to any segment 30 that contribute to the composition of picture 54 are adopted or left in the extracted data stream 12 without any modification, specifically without any modification to the start position information 42 and segment information 48. Specifically, the start position information 42 indicates that the start position 40 of a slice 26 whose start position corresponds to a segment 30 that contributes to picture 54 is shown relative to the onset position of that segment, so this relative indication remains accurate in the reduced data stream 12. Furthermore, the segment information 48 indicates, for all slices 26 adopted in the extracted data stream 12, which segment the corresponding slice 26 belongs to, i.e., which segment its start position is in. This segment information 48 is also used in the extraction process 14 to identify slices 26 that will be excluded or discarded.

[0028] The only information present in the extracted data stream 12, which has not yet been discussed with respect to the original data stream 10, is the coding parameters associated with the modified configuration of the picture 54 of the extracted data stream 12 based on a selected set of segments 30. With respect to this topic, several possibilities exist. According to the first alternative, the extraction process 14 may require modification of the aforementioned parameter set 36' so that the corresponding parameter set 36' in the extracted data stream 12 is modified to reflect the modified picture size of the picture 54 of the extracted video 56, the subdivision of the picture 54 into segments, the decoder capability level required to decode the extracted data stream 12, as well as / or the modified decode timestamp, coded picture buffer retrieval time and / or buffer size for decode the extracted data stream 12, which may be lower than the decoder capability level required to decode the complete video 20 or picture 18 from the data stream 10. However, according to an alternative embodiment, the use of relative start position indications in start position information 42 is combined with one aspect of the present application in which, in other words, both information items are provided to the parameter set 36 of the original data stream 10, thereby allowing even the parameter set 36' of the extracted data stream 12 to remain immutable. According to this alternative, the parameter set 36 will represent all coding parameters for decoding the entire picture 18 from the data stream 10, and at the same time, all coding parameters for decoding the picture 54 from the data stream 10 or 12. The coding parameters represented by the parameter set 36 with respect to the extracted data stream 12 and its picture 54 may be represented in the parameter set 36, or in a different way, at least partially, with respect to the coding parameters represented by the parameter set 36 for decoding the picture 18 from the data stream 10.For example, as outlined in more detail below, parameter set 36 may indicate the subdivision of picture 18 into segment 30, such that the size of segment 30 is evident from this part of parameter set 36. The coding parameters of parameter set 36 for picture 54 may depend on this knowledge and may simply indicate which segments contribute to picture 54 and which relative arrangements of these contributing segments within picture 54 apply. Even the aforementioned base address indication for onset position 44 within picture 18 may be repeated within data stream 10 for picture 54 of the extracted data stream 12. Furthermore, as will be discussed further below, parameter set 36 may thus indicate not just one extraction or sub-picture decoding option for data stream 10, but several extraction or sub-picture decoding options that differ in terms of the composition of picture 54 based on segment 30 of picture 18. Therefore, in the case of an alternative in which the parameter set 36' remains invariant with respect to the parameter 36 in the original data stream 10, the decoder 52 will receive a decoding option instruction 58 indicating, for example, whether picture decoding, i.e., decoding of picture 18, should be performed on the data stream 12, or sub-picture decoding, i.e., decoding of picture 54, should be performed on the data stream 12. If the unused slice 26 has already been removed as part of a previously performed extraction process 14, this decoding option instruction should not necessarily indicate picture decoding as the slice is missing. However, it should be recalled that the extraction process 14 may be performed by the decoder 52 itself, and the decoder 52 may be equivalent to the decoder 38. For this reason, the decoder 52 may receive the complete original data stream 10 and then decide to perform the extraction process 14 in which, based on the decoding option instruction 58, remove the unused slice 26 itself, i.e., discard it. The decoding option instruction may be, for example, an external signal provided by the application or user.Alternatively, the decoding option instruction may be a specific instruction of the decoder capability level of decoder 52, which excludes some of the decoding options of data stream 10 from being decodeable by decoder 52 in any case. If picture decoding is performed on the complete picture 18, the decoding is performed as discussed with respect to decoder 38. If the subpicture decoding option is applied, i.e., if decoder 52 decodes picture 54 from data stream 12, the decoding process is substantially the same as the following difference: decoder 52 will use coding parameters from a specific parameter set 36' for this subpicture decoding of picture 54. Furthermore, if the calculation is performed by decoder 52 itself, decoder 52 will calculate a different base address for the onset position 44 of segment 30 in picture 54. The different calculation results are a result of different arrangements of segment 30 in picture 54. The start position information 42, which indicates the relative position of the slice start position 40, is the same as that indicated by the start position information 42 for the onset position 44 of the segment. However, if the onset position 44 of the segment is indicated in parameter set 36' for the placement of the segment in picture 54, the decoder 52 may use these specific base addresses for subpicture decoding to obtain picture 54, instead of performing its own calculations.

[0029] Further details regarding how decoders 38 / 52 may operate when decoding the inbound data stream, whether it could be 10 or 12, are provided at the end of the explanation of Figure 11.

[0030] The following description provides a specific example for implementing the embodiment outlined above. It is assumed that the data stream includes a start position coding mode flag 60 in the parameter set 36 that switches between a relative start position indication 46 for the slice start position 40 relative to the segment onset position 44, as described with respect to Figure 1, and an absolute start position indication 62 for the slice start position 40 relative to the picture onset position 24. In other words, the flag 60 signals whether the start position indication 42 in the slice 26 of the data stream 10 indicates the start position 40, either relative to the segment onset position 44 or absolutely relative to the picture onset position 24, as outlined with respect to Figure 1. The flag 16 can then be used to switch on the presence of a segment indication 48, as will be further discussed below. In the case of a simple relative start position indication 46, the segment indication 48 may be present in the slice 26.

[0031] A further detail that will become apparent from the embodiments described below is the fact that the slice location indication 42 can actually be signaled by a combination of flags and a conditionally signaled slice address. In the case of the relative indication option 46, the flags included in the start location information 42 indicate whether the corresponding slice is the first slice in the segment, i.e., whether its start location 40 coincides with the onset location 44 of the corresponding segment. In this case, the start location information 42 does not include any start address syntax elements. Otherwise, however, the address syntax elements point to the start location 40 of the slice relative to the onset location 44 of the corresponding segment. In the case of the absolute indication option 62, the flags in the start location information 42 indicate whether the corresponding slice starts at the picture onset location 24, i.e., whether it is the first slice of the picture, and if so, the slice location information does not include any address syntax elements. However, otherwise, the address syntax elements point to the start location 40 of each slice relative to the picture onset location 24. The flags included by the start position information 42, as described above, can be used by the decoder 38 / 52 to access unit boundary detection, i.e., detection of a new access unit 36, in absolute instruction mode. To compensate for the lack of ability to use this detectability in relative instruction mode, the encoder 16 may use alternative access unit boundary mechanisms, such as access unit delimiters at the boundaries between consecutive access units, i.e., the placement of several packets. In that case, the decoder may use these delimiters in relative instruction mode.

[0032] Before commencing a more detailed description of the embodiments, it should be noted that there are different possibilities for indicating the starting position 40 by the starting position information 42. For example, picture 18 may be regularly subdivided into an array of blocks, as shown in the upper left corner of picture 18 in Figure 1. The coding path 22 may traverse these coding blocks 64 continuously without interleaving, i.e., one coding block 64 is completely traversed before proceeding to the next coding block 64 of picture 18. The starting position information 42 may indicate the position 42 measured in units of traversed coding blocks 64 along the coding path 22 up to the starting position 40. When indicated relatively 46, the starting position information 42 would indicate the number of coding blocks 64 along the coding path 22 that separate the onset position 44 from the slice starting position 40. In the absolute indication option 62, the starting position information 42 indicates the starting position 40 in units of coding blocks 64 traversed along the coding path 22 when starting from the picture onset position 24. Therefore, in this case, the start position indicator 42 would use a linear, one-dimensional addressing region, and the base address of the segment onset position 44 described above may be measured by the number of coding blocks 64 traversed when starting from the picture onset position 24 to each segment onset position 44. The sum of the base address 44 and the slice start position 40 would result in an absolute address measured first from the start position 24. However, alternative methods of signaling the start position 40 by the start position information 42 would also be feasible. For example, the relative position of the start position 40 to the corresponding segment onset position 44 could be represented as the x and y coordinates of a vector pointing from the onset position 44 to the slice start position 40, where the components x and y are measured in samples or units of coding blocks 64, respectively.Furthermore, addressing may be performed according to the strictly linear addressing of the coding block 64 described above, that is, regardless of segmentation into segment 30, such that the addresses of coding blocks within one complete line of picture 18 follow each other before the address increases in the first coding block of the next line in the picture. Internally, decoders 38 and 52 can be switched between different addressing systems or between domains as needed.

[0033] Another possibility that becomes apparent from the following explanation is as follows: We have described segments 30 so far as possible spatial cross-sections of spatiotemporal domains where coding interdependencies do not cross the boundaries of these spatiotemporal domains. These segments 30 can be defined, for example, as a set of one or more tiles into which picture 18 can be subdivided and coded independently of each other as far as coding of a single picture 18 is concerned. Thus, a segment 30 can be a tile if a single segment 30 consists of one tile, which is the case in variant A described later, but variant B assumes that a segment 30 can consist of one or more tiles. Furthermore, as will be explained later, the aforementioned coding blocks 64 are CTUs (coding tree units), meaning that these blocks 64 are tree root blocks that have been hierarchically further subdivided into coding blocks, prediction blocks, and / or transformation blocks by multi-tree subdivision. This subdivision of the CTUs 64 into coding units, prediction blocks, and transformation blocks can be signaled as part of the corresponding slice payload 32.

[0034] Therefore, the following describes an implementation of an embodiment with respect to Figure 1 as a possible modification of the HEVC framework. As described in the introduction to this specification, HEVC DAM3 MCTS extraction requires adjustment of the slice segment address of each slice header, which can even affect the byte alignment of each slice header due to the variable-length coding of the slice address. In particular, when using HEVC DAM3 MCTS, the slice addresses of slices adopted in the MCTS extracted data stream must be corrected by their absolute pointing option, meaning these slice addresses must be modified to point to the new picture onset position of picture 54, which, due to the variable-length coding, can result in slice addresses of different lengths and, therefore, different byte alignments. The modification of the HEVC framework described later overcomes this. Specifically, a relatively simple MCTS substream extraction provides an implicit or explicit association between slices and tiles / segments through signaling of a portion of the slice segment address.

[0035] The following describes the first variant of the HEVC framework modification, variant A. As outlined above with respect to Figure 1, according to this variant, the slice segment address signals to the first CTU of each tile or tile configuration, i.e., the current segment 30, rather than to the first CTU of the picture. "First" means the leftmost and topmost CTU when using the HEVC coding order.

[0036] The difference between the two references, namely the first CTU of the current segment 30 or the first CTU of the picture, is derived by the decoder, i.e., decoder 38 or 52, by combining the picture and tile instruction information from parameter sets 36 and 36' with the association of slice 28 / 26 to this segment 30, which may be a tile or a set of tiles, respectively, and this association may be explicitly transmitted in the data stream as outlined below.

[0037] An array containing information about the segment size and slice address offset for each segment 30 within the CTB may be determined on the decoder side. Access units 36 initiate detection, which will be performed by decoders 38 or 52, and this detection may be performed based on a flag included in the start position information 42, in the case of absolute instruction option 62, as this flag indicates the first slice of access unit 36 ​​and picture 18, respectively; in the case of relative instruction option 46, this may be performed by the use of an access unit delimiter NAL unit in the data stream 10, i.e., a special packet that separates consecutive access units in the data stream 10, used to signal intersections between consecutive access units. As described above, first_slice_in_pic_flag may be included by the start position information 42 and may change to a flag indicating the start of a segment 30, which may be a tile or a tile configuration.

[0038] Figure 3 is a diagram illustrating an embodiment that arises using the HEVC technology framework. As shown, the parameter set 36 is distributed into a picture parameter set 70, a sequence parameter set 72, and a video parameter set 74, which have different ranges. Figure 3 simply shows two VCL NAL units 76 and 78, which are fragments from the data stream 10, i.e., each containing a NALU header 80 indicating that, after slice information, it contains the corresponding slice 26 consisting of a slice header 34 and a slice payload 32. The slice header 34 points to the corresponding picture parameter set 70, as indicated by arrow 80; the picture parameter set 70 points to the corresponding sequence parameter set, as indicated by arrow 82; and the sequence parameter set 72 points to the corresponding active video parameter set, as indicated by arrow 84. The parameter set 36 suggests, for example, that the picture parameter set 70 or the sequence parameter set 72 contains a tiling syntax 86, i.e., a syntax that defines the subdivision of the picture 14 into tiles 88 that form the segment 30 discussed above in Figure 1, according to one alternative form. Therefore, based on the tiling syntax 86, the decoder can calculate the base slice segment address, for example, as indicated by arrow 90. The sum of the base slice segment address 94 and the offset slice segment address 96, transmitted as part of the start position information 42 and the slice header 34, 92 gives the corresponding slice segment address 98 of the corresponding slice 26, i.e., the slice segment address measured absolutely with respect to the picture onset position 24. Figure 3 shows two slices 26 drawn as belonging to one tile of tile 88, i.e., tile number 2, which can be derived for the decoder based on this segment information 48, which is also included by the slice header 34, as will become clear from the following description.

[0039] Specifically, according to the embodiment shown in Figure 4, the tiling structure or tiling syntax 86 may be included within the picture parameter set 70. An alternative embodiment could similarly be realized in which, for example, the tiling syntax 86 resides within the sequence parameter set 72 instead of the picture parameter set.

[0040] Specifically, as shown in Figure 4, part 100 of the parameter set 70 indicates the subdivision of the picture 18 into segments 30, in this case into tiles. The number of segment columns is indicated in 102, the number of segment rows is indicated in 104, and flag 106 optionally provides an opportunity to signal in a data stream where the width of the segment columns and the height of the segment rows are set unevenly. If not present or flag 106 is not set, the width of the segment columns is indicated individually in 108, and the height of the segment rows is indicated individually in 110.

[0041] Furthermore, if the flag slice_segment_base_addr_per_tile_enable_flag112 is set, i.e., in 114, explicit tagging of segments is achieved by explicitly signaling the segment index value tile_id_in_pps[i] for each segment 30 of picture 18. The segment index is transmitted within the data stream, i.e., within the parameter set 70, by using a default order among the canonical array of segments 30, such as in the row direction indicated by the dashed arrow 116 in Figure 1.

[0042] The semantics of the syntax elements in Figure 4 will be outlined in detail below.

[0043] A slice_segment_base_addr_per_tile_enable_flag equal to 0 specifies that the variable CtbAddrInRs of a slice segment are derived solely from the slice_segment_address. A slice_segment_base_addr_per_tile_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on the slice_segment_address and the tile-dependent base address. When a slice_segment_base_addr_per_tile_enable_flag does not exist, it is inferred to be equal to 0.

[0044] tile_id_in_pps[i] specifies the tile's identifier in bitstream order. The value of tile_id_in_pps[i] is in the range of 0 to 255, and if i is not equal to j, then tile_id_in_pps[i] does not have the same value as tile_id_in_pps[j]. If tile_id_in_pps does not exist, it is inferred to be equal to 0.

[0045] When slice_segment_base_addr_per_tile_enable_flag is equal to 1, the presence of an access unit delimiter NAL unit in the bitstream and the condition that tiles_fixed_structure_flag is equal to 1 may be constraints on bitstream consistency.

[0046] The following pseudocode illustrates how a decoder, such as decoder 38 or 52, calculates a certain dimension of segment 30 and the base address of segment 30 based, for example, on information available in the data stream.

[0047] The following describes the coding tree block raster and tile traversal transformation process.

[0048] The list colWidth[i] for i, ranging from 0 to num_tile_columns_minus1, including both endpoints, which specifies the width of the i-th tile column in a coding tree block (CTB), is derived as follows (pseudocode 1): if(uniform_spacing_flag) for(i=0;i<=num_tile_columns_minus1;i++ ) colWidth[i]=((i+1)*PicWidthInCtbsY) / (num_tile_columns_minus1+1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1+1) else colWidth[num_tile_columns_minus1]=PicWidthInCtbsY for(i=0;i <num_tile_columns_minus1;i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0049] colWidth[i] is the width of the i-th segment column in column 0…num_tile_columns_minus1, measured within the coding tree block CTB64num_tile_columns_minus1. num_tile_columns_minus1 is the number of segment columns minus 1.

[0050] The list rowHeight[j], which specifies the height of the j-th tile row in CTB units, including both endpoints and ranging from 0 to num_tile_rows_minus1 for j, is derived as follows (pseudocode 2): if(uniform_spacing_flag) for(j=0;j<=num_tile_rows_minus1;j++) rowHeight[j]=((j+1)*PicHeightInCtbsY) / (num_tile_rows_minus1+1)- (j*PicHeightInCtbsY) / (num_tile_rows_minus1+1) else rowHeight[num_tile_rows_minus1]=PicHeightInCtbsY for(j= 0;j <num_tile_rows_minus1;j++){ rowHeight[j]=row_height_minus1[j]+1 rowHeight[num_tile_rows_minus1]-=rowHeight[j] } }

[0051] rowHeight[i] is the height of the i-th segment row in row 0…num_tile_rows_minus1, measured within the coding tree block CTB64num_tile_rows_minus1. num_tile_rows_minus1 is the number of segment rows minus 1.

[0052] The list TileId[ctbAddrTs], which specifies the conversion from CTB address to tile ID during tile scanning and includes both end values ​​for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, is derived as follows (pseudocode 3): for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]]=tileIdx for (y = rowBd[j]; y < rowBd[j + 1]; y++) for (x = colBd[i]; x < colBd[i + 1]; x++) TileId[CtbAddrRsToTs[y * PicWidthInCtbsY + x]] = tileIdx

[0053] tileIdxTileIdMap[] is interpreted as mapping the N segment index values included in the vector tile_id_in_pps[] to the tileIdx of that segment. tile_id_in_pps[] is a vector containing components 0...N - 1, where N is the number of segments 30, and the i-th component indicates the segment index value associated with the i-th segment in raster scan segment order 116. tileIdx indexes segment 30 in raster scan segment order 116. TileId[i] is interpreted as mapping the CTB address i, i.e., the address of block 64, measured in coding order or along coding path 22, to the tileIdx of the segment 39 in which this CTB is located. CtbAddrRsToTs[i] is a function that maps the rank i of block 64, which is the i-th block 64 of picture 18 in picture raster scan order leading in the row direction through block 64 of picture 18, to its address measured in coding order 22. colBd[i] is a vector containing components 0...N, where N is the number of segment columns minus 1, i < N, the i-th component indicates the left position of the i-th segment column from left to right measured within block 64, and the N-th component indicates the picture width in terms of the number of blocks.

[0054] It should be noted that all parameters used in the pseudocode so far, namely for picture 18, are used to determine the base address in the case of complete picture decoding, but these parameters can also be used to determine the base address with respect to the onset position of a segment within subpicture 54. Here, the parameters N, colWidth[i], num_tile_columns_minus1, rowHeight[i], num_tile_rows_minus1, tileIdxTileIdMap[], tile_id_in_pps[], CtbAddrRsToTs[i], and colBd[i] are specific to this subpicture 54, as well as its composition by segment 30 and its subdivision into the corresponding block 64, respectively. Similarly, all references to the coding path 22 and the raster scan segment order 116 apply to their respective subpictures. tileIdxTileIdMap[] ensures, on the one hand, the maintenance of the association between tileIdx and the segment index value tile_id_in_sh, which assumes one of tile_id_in_pps[], and on the other hand, their indexing in the subpicture 54 and raster scan segment order 116, i.e., the association between tileIdx. As a less important note, the subscript "_pps" of the syntax element is chosen here to reflect an example of conveying segment index values ​​for segment associations within a PPS, but as already outlined above, this is merely an example, and it should be noted that the signaling can also be realized elsewhere, such as within an SPS or VPS. In line with the latter description, or if the subscript "_ps" is used and the corresponding substitution is, of course, also applicable to this embodiment. Similarly, the following example illustrates that a one-to-one correspondence between tiles and segments may be abandoned, which is also outlined with reference to Figure 1. Therefore, the corresponding subsequent embodiments will use the name component "Area" or "area_" or "_area" instead, as will become clear from the following description.Furthermore, as also stated in the description of Figure 1, the explicit assignment of tags or segment index values ​​to segments may be excluded in alternative embodiments by implicitly deriving a mapping between segment index values ​​in segment information 48 and segments in subpicture 54, for example by using the index area tileIdx instead, in which case the transmission of such associations in parameter set 36 for picture 18 may be excluded. However, the explicit assignment of segment index values ​​to segments 30 of picture 18 reduces the mixing of tiles from two separate data streams.

[0055] Specifies the size of the kth tile in the picture in CTB units, including both end values, from 0 to ((num_tile_rows_minus1+1) * The list TileSizeInCtbsY[k] for k in the range (num_tile_columns_minus1+1)-1) is derived as shown in the following (pseudocode 4). for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++) TileSizeInCtbsY[k]=colWidth[i] * rowHeight[j]

[0056] TileSizeInCtbsY[i] is a vector containing components 0...N-1, where N is the number of segments 30, and the i-th component indicates the number of blocks 64 contained within the i-th segment of raster scan segment order 116.

[0057] In bitstream order, specifies the slice segment address offset in CTB units during the tile scan of the first slice segment of the k-th tile in the picture, including both endpoints, from 0 to ((num_tile_rows_minus1+1) *The list TileSliceSegBaseAddr[k] for k in the range (num_tile_columns_minus1+1)-1) is derived as shown in the following (pseudocode 5). TileSliceSegBaseAddr[0]=0 for(k=1;k<=((num_tile_rows_minus1+1) * (num_tile_columns_minus1+1)-1);k++) TileSliceSegBaseAddr[k]=TileSliceSegBaseAddr[k-1]+TileSizeInCtbsY[k-1]

[0058] TileSliceSegBaseAddr[i] is a vector containing components 0…N-1, where N is the number of segments 30, and the i-th component indicates the address of the onset position 44 of the i-th segment of the raster scan segment order 116, measured as the number of blocks 64 along the coding path from picture onset position 24 to segment onset position 44.

[0059] Here, "tile scan" refers to a scan 22 as shown in Figure 1. TileSliceSegBaseAddr, according to this example, indicates the aforementioned base address of the segment's onset position 44, which is equal to the tile. The calculations shown above can be used by decoders 38 and 52, respectively, to calculate the base address. If subpicture decoding is applied, the number of segments and their arrangement are to be adaptable.

[0060] Figure 6 shows what a slice segment header, i.e., the slice header 34 described above, can look like. Specifically, here the slice header 34 includes a flag 120, i.e., first_slice_segment_in_area_flag, which together with the syntax element slice_segment_address 122, forms the starting position information 42. If the flag 120 is simply not set, then slice_segment_address exists, indicating the offset of the slice starting position 40 from the onset position 44 of the corresponding segment / tile 30. The latter is indicated by segment information 48 containing tile_id_in_sh, which is simply a single syntax element 124, i.e., a value outside the range of values ​​defined in the parameter set 70 in Figure 4, i.e., assumed by syntax element 114.

[0061] In particular, the semantics can be shown as follows: A first_slice_sement_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the picture in decryption order, when slice_segment_base_addr_per_tile_enable_flag is equal to 0. Otherwise, when slice_segment_base_addr_per_tile_enable_flag is equal to 1, a first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the picture tile in decryption order. A first_slice_segment_in_pic_flag equal to 0 specifies that the slice segment is not the first slice segment of the picture or tile in decryption order, respectively. Thus, slice_segment_base_addr_per_tile_enable_flag serves as an example for flag 60 discussed with respect to Figure 2.

[0062] Note 1. This syntax element can be used to detect boundaries between coded pictures that are consecutive in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of the IDR picture may cause problems in detecting boundaries between coded pictures. This can occur, for example, when transmitting all intra-coded video in an error-prone environment. This problem can be mitigated by using two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) alternately for two consecutive IDR pictures. The use of a temporal sublayer zero-index SEI message may also be useful, as its SEI message contains a different syntax element irap_pic_id for consecutive IRAP pictures in decoding order. Some system environments have other provisions that may also be useful for picture boundary detection, such as presentation timestamps in the Rec.ITU-T H.222.0|ISO / IEC13818-1 system, access unit framing in the ISO / IEC 14496-12 ISO-based media file format, or the use of marker bits in the IETF RFC 3550 real-time transport protocol header.

[0063] A `dependent_slice_segment_flag` value equal to 1 specifies that the value of each missing slice segment header syntax element is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When none exists, the value of `dependent_slice_segment_flag` is inferred to be equal to 0.

[0064] The variable SliceAddrRs is derived as follows: -If dependent_slice_segment_flag is equal to 0, SliceAddrRs will be set to be equal to CtbAddrInRs. - Otherwise, SliceAddrRs is set to be equal to the SliceAddrRs of the preceding slice segment containing the coding tree block whose coding tree block address is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs]-1], in which case the variable CtbAddrInRs is defined within the semantics of slice_segment_address.

[0065] `tile_id_in_sh` specifies the index of the tile to which the slice segment belongs. The value of `area_id_in_sh` ranges from 0 to 255. If `area_id_in_sh` does not exist, it is inferred to be equal to 0. Two or more tiles in a picture are not supposed to have the same value of `tile_id_in_sh`.

[0066] slice_segment_address defines the address of the first coding tree block in a slice segment as follows (when slice_segment_base_addr_per_tile_enable_flag is equal to 1) in a coding tree block raster scan of a picture, or otherwise in a tile scan of a picture:

[0067] The variable maxNumCtbsY is defined as (slice_segment_base_addr_per_tile_enable_flag ? TileSizeInCtbsY[tileIdxTileIdMap[tile_id_in_sh]]:PicSizeInCtbsY), where tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the currently decoded / encoded slice, contained by its segment information 48, onto the correct tileIdx, i.e., referring to the correct segment in picture 18 or 54, and TileSizeInCtbsY gives the maximum value of each slice address that will be presented / representable for this segment by its start position information 42 slice_segment_address, i.e., maxNumCtbsY-1. Decoders and encoders can compute this information using pseudocodes 1 to 4.

[0068] The length of the `slice_segment_address` syntax element is then `Ceil(Log2(maxNumCtbsY))` bits. The value of `slice_segment_address` is in the range from 0 to `maxNumCtbsY-1`, including both endpoints. When `slice_segment_base_addr_per_tile_enable_flag` is equal to 0, the value of `slice_segment_address` is not equal to the value of `slice_segment_address` of any other coded slice segment NAL unit in the same coded picture. When `slice_segment_base_addr_per_tile_enable_flag` is equal to 1, the value of `slice_segment_address` is not equal to the value of `slice_segment_address` of any other coded slice segment NAL unit to which the same tile in the same coded picture belongs. If `slice_segment_address` does not exist, it is inferred to be equal to 0.

[0069] In a picture coding tree block raster scan, the variable CtbAddrInRs, which defines the coding tree block address, is set to equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_base_addr_per_tile_enable_flag?TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]]:0)]. In a tile scan, the variable CtbAddrInTs, which defines the coding tree block address, is set to equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal, which defines the difference between the Luma quantization parameter for a coding unit containing cu_qp_delta_abs and its prediction, is set to equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr, which specify the values ​​to be used when determining the respective values ​​of the quantization parameters Qp'Cb and Qp'Cr for a coding unit containing cu_chroma_qp_offset_flag, are both set to equal 0. Thus, in relative instruction mode 46, tileIdxTileIdMap[tile_id_in_sh] maps the tile_id_in_sh of the slice currently being decoded / encoded, contained by its segment information 48, onto the correct tileIdx, i.e., referring to the correct segment in picture 18 or 54, TileSliceSegBaseAddr provides the base address for this segment, and this base address is then appended to the relative slice address slice_segment_address of its starting position information 42 to obtain the address of this slice. CtbAddrTsToRs simply converts the address measured along the coding path 22 to the address measured along the raster scan block order between blocks 64. The decoder and encoder can use pseudocode 5 to calculate the base address of segment onset position 44, or, in the case of explicit information as illustrated below, the encoder can simply do this.

[0070] The order of VCL NAL units and their association with coded pictures can be described as follows:

[0071] Specifically, the following defines the order of VCL NAL units and their association with coded pictures.

[0072] Each VCL NAL unit is a part of a coded picture.

[0073] The order of VCL NAL units within a coded picture is constrained as follows: - The first VCL NAL unit of the coded picture shall have a first_slice_segment_in_pic_flag equal to 1. - Set sliceSegAddrA and sliceSegAddrB to the values ​​of CtbAddrTsToRs[slice_segment_address(slice_segment_base_addr_per_tile_enable_flag?TileSliceSegBaseAddr[tileidxTileidMap[tile_id_in_sh]:0)]] of any two coded slice segment NAL units A and B in the same coded picture. Coded slice segment NAL unit A precedes coded slice segment NAL unit B if any of the following conditions are true. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]], and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0074] Decoders 38 and 52 can monitor these conditions to determine when a new access unit 36 ​​is initiated within the inbound data stream 10 or 12, or to determine if any NAL units are missing.

[0075] The modified form A described above may be modified in different ways, for example, to enable easier implementation and processing.

[0076] For example, in variant form A, as already explained above with respect to Figure 1, segment 30 does not have to be strictly a tile. Therefore, slice segment base address calculation does not have to occur exclusively with respect to tile granularity, and multiple tiles may be joined into one area, then form segment 30 according to the explanation presented above, and then a slice segment base address may be defined with respect to that segment 30, that is, the slice address in the slice header is given relative to a first CTU belonging to a given area of ​​multiple tiles that form segment 30 in bitstream order.

[0077] Furthermore, as an additional or alternative, according to variant form B described below, instead of burdening the client-side or decoder with the calculation of the slice segment base address for each segment 30, explicit signaling of the slice segment base address can be provided within the data stream.

[0078] Additionally or alternatively, for example, the burden of repeating tile structure signaling within each PPS of a video bitstream, while ensuring that tile_fixed_structure is equal to 1 when slice_segment_base_addess_per_tile_enable_flag is equal to 1, can be reduced by allowing the tiling syntax to be transmitted either within the SPS or within the PPS, with the PPS instance of the tiling syntax potentially overriding the tiling syntax settings transmitted within the SPS.

[0079] The following diagrams, highlighted, show the changes from the modified form A described above. For example, Figure 7 shows a portion of the tiling syntax that may be used either within the PPS or the SPS. When the tiling syntax of Figure 7 is transmitted within the PPS, it overrides the tiling syntax provided by the corresponding SPS, i.e., its tiling syntax has a higher advantage. Figure 7 shows the possibility that the parameter set 36 of Figure 1 transmits the base address to the segment onset position 44 by the base address data field 130.

[0080] The semantics can be as follows:

[0081] A slice_segment_base_addr_per_area_enable_flag equal to 0 specifies that the slice segment variable CtbAddrInRs is derived solely from slice_segment_address. A slice_segment_base_addr_per_area_enable_flag equal to 1 specifies that the derivation of CtbAddrInRs is based on slice_segment_address and tile-dependent offsets. When slice_segment_base_addr_per_area_enable_flag does not exist, it is inferred to be equal to 0.

[0082] area_id_in_ps[i] defines the tile's ID in bitstream order. The value of area_id_in_ps[i] ranges from 0 to 255, and if i is not equal to j, area_id_in_ps[i] will not have the same value as area_id_in_ps[j]. If area_id_in_sh does not exist, it is inferred to be equal to 0.

[0083] slice_segment_base_address[i] defines the base slice segment address in the tile scan order for all slice segments belonging to the area having the same value as area_id_in_ps[i]. If it does not exist, the value of slice_segment_base_address[i] is inferred to be equal to 0.

[0084] When area_id_enable_flag is equal to 1, the presence of an access unit delimiter NAL unit within the bitstream is a requirement for bitstream consistency.

[0085] For any two values ​​of i, if area_id_in_ps[i] does not have equal values, then the values ​​of slice_segment_base_address[i-1] must not be equal; this is a requirement for bitstream consistency.

[0086] Figures 8 and 9 provide an example of SPS and PPS syntax with individual available flags for tiling syntax signaling. The following semantics are used:

[0087] A tiles_enabled_in_sps_flag equal to 1 specifies that SPS communicates the default tiling syntax that describes the default structure of CVS.

[0088] A tiles_enabled_in_pps_flag equal to 1 specifies that the PPS will communicate a tiling syntax that describes the actual tiling structure of the CVS picture, overriding the tiling syntax in the referenced SPS.

[0089] The coding tree block raster and tiling scan transformation process in this case is as follows:

[0090] The list colWidth[i] for i, ranging from 0 to num_tile_columns_minus1, including both endpoints, which defines the width of the i-th tile column in a coding tree block (CTB), is derived as follows (pseudocode 1'): if(uniform_spacing_flag ) for(i=0;i<=num_tile_columns_minus1;i++) colWidth[i]=((i+1)*PicWidthInCtbsY) / (num_tile_columns_minus1+1)-(i*PicWidthInCtbsY) / (num_tile_columns_minus1+1) else colWidth[ num_tile_columns_minus1]=PicWidthInCtbsY(6 3) for(i=0;i <num_tile_columns_minus1;i++){ colWidth[i]=column_width_minus1[i]+1 colWidth[num_tile_columns_minus1]-=colWidth[i] } }

[0091] The list rowHeight[j] for j ranging from 0 to num_tile_rows_minus1, including both endpoints, which defines the height of the j-th tile row in CTB units, is derived as follows (pseudocode 2'): if(uniform_spacing_flag) for(j=0;j<=num_tile_rows_minus1;j++) rowHeight[j]=((j+1)*PicHeightInCtbsY) / (num_tile_rows_minus1+1)- (j*PicHeightInCtbsY) / (num_tile_rows_minus1+1) else rowHeight[num_tile_rows_minus1]=PicHeightInCtbsY (6 4) for(j=0;j <num_tile_rows_minus1;j++){ rowHeight[j]=row_height_minus1[j]+1 rowHeight[num_tile_rows_minus1]-=rowHeight[j] } }

[0092] The list TileId[ctbAddrTs] for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, including both endpoints, which defines the conversion from CTB address to tile ID in tile scanning, is derived as follows (pseudocode 3'). for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) for(y=rowBd[j];y <rowBd[j+1];y++) for(x=colBd[i];x <colBd[i+1];x++) TileId[CtbAddrRsToTs[y*PicWidthInCtbsY+x]]=tileIdx

[0093] The meaning of the variables remains as explained above, however, N represents the number of tiles that can be greater than the number M of segment 30, due to the collection of tiles for segment 30.

[0094] The list TileSizeInCtbsY[k] for k, which includes both endpoints and defines the size of the k-th tile in the picture in CTB units, is derived as shown in the following (pseudocode 4'). for(areaId=0;areaId<=255;areaId++) AreaSizeInCtbsY[areaId]=0 for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++) if(area_id_in_ps[j*(num_tile_columns_minus1+1)+i]==areaId) AreaSizeInCtbsY[areaId]+=(colWidth[i]*rowHeight[j])

[0095] area_id_in_ps[] is a vector containing components 0…N-1, where N is the number of tiles, and the i-th component represents the segment index value associated with the segment where the i-th tile is located in the tile raster scan segment order. The values ​​entered in the components are assumed to be between 0 and 256, or in other words, within the range of possible values ​​for the segment index in segment information 48. Recall the dependency on the picture decoding option / target, i.e., for N to be the number of tiles in picture 18 or sub-picture 54, respectively, the base address of the segment in picture 18 or the base address of the segment in sub-picture 54 should be calculated. AreaSizeInCtbsY[] is interpreted as a vector containing components for each member of the area of ​​possible values ​​for the segment index in segment information 48, indicating the number of blocks 64 covered by each segment index value occurring within picture 18 or sub-picture 54, whatever the current decoding target / option may be.

[0096] The mapping from AreaId to AreaIdx is derived as follows: NumTilesInArea[0...255]=0; NumAreas=1 for(k=0;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 If(NumTilesInArea[area_id_in_ps[k]]==1) AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas++;

[0097] NumTilesInArea[] is interpreted as a vector containing components for each member of the region of possible values ​​for the segment index in segment information 48, indicating the number of tiles covered by each segment having this segment index value, regardless of the current decoding target / option, for each segment index value occurring within picture 18 or subpicture 54. AreaIdToAreaIdx[] is interpreted as a vector containing components for each member of the region of possible values ​​for the segment index in segment information 48, indicating the rank or index when assigned or measured along the raster scan segment order leading from top left to bottom right in the segment row direction, whatever the current decoding target / option may be, for each segment index value occurring within picture 18 or subpicture 54.

[0098] The list AreaSliceSegBaseAddr[k] for k, ranging from 0 to (NumAreas-1), including both endpoints, which defines the slice segment address offset of the first slice segment of the k-th area in the picture in CTB units in bitstream order, is derived as follows (pseudocode 5'): AreaSliceSegBaseAddr[0]=0 for(k=1;k<( NumAreas);k++) AreaSliceSegBaseAddr[k]= slice_segment_base_address [k-1]

[0099] AreaSliceSegBaseAddr[i] is interpreted as a vector containing components 0…M-1, where M is the number of segments 30, equal to NumAreas, and in the i-th component, it indicates the segment onset position 44 for the i-th segment in the aforementioned raster scan segment order, using AreaIdx, which can be associated with the area_id_in_sh of the slice's segment information 48 by decoders and encoders using AreaIdToAreaIdx, i.e., AreaIdToAreaIdx.

[0100] Therefore, if explicit transmission is still applied to the subpicture decoding option, explicit transmission of the base address for the first segment 30 of picture 18 or subpicture 54 in the raster scan segment order is excluded as it will be zero in either case. However, note that the base address data field 130, defined by the syntax element slice_segment_base_address, defines the base address of the onset position 44 of each segment for each segment 30, addressing the onset position 44 for the picture onset position 24. The definition that this base address by the base address data field is zero is a result of the convention that explicit base address transmission begins with the second segment in the raster scan segment order. Naturally, if explicit base address transmission is used instead for all segments, alternative embodiments may be considered.

[0101] If explicit base address signaling is not used, and instead implicit derivation of the base address is used, the same thing can be done by the decoder or encoder as follows. The following calculation is used by the encoder for calculating the base address even in the case of explicit base address signaling.

[0102] The list TileSliceSegBaseAddr[k], which specifies the slice segment address offset of the first slice segment of the k-th tile in the picture in CTB units, including both endpoints, for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1), and the list AreaSliceSegBaseAddr[k], which specifies the slice segment address offset of the first slice segment of the k-th area in the picture in CTB units, including both endpoints, for k ranging from 0 to (NumAreas-1), are derived as follows (pseudocode 5"): TileSliceSegBaseAddr[0]=0 AreaSliceSegBaseAddr[area_id_in_ps[0]]=0 NumTilesInArea[0...255]=0; NumTilesInArea[area_id_in_ps[0]]=1 AreaIdToAreaIdx[area_id_in_ps[0]]=0 NumAreas=1 for(k=1;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) NumTilesInArea[area_id_in_ps[k]]+=1 TileSliceSegBaseAddr[k]=TileSliceSegBaseAddr[k - 1]+TileSizeInCtbsY[k-1] If(NumTilesInArea[area_id_in_ps[k]]==1) AreaSliceSegBaseAddr[NumAreas]=TileSliceSegBaseAddr[k] AreaIdToAreaIdx[area_id_in_ps[k]]=NumAreas; NumAreas++;

[0103] In addition, Figure 10 shows what the slice header can look like with deformation form B, and the possible semantics are shown below.

[0104] A first_slice_segment_in_area_flag equal to 1 specifies that the slice segment is the first slice segment of the picture in decoding order, when tiles_enabled_flag is equal to 0. Otherwise, when tiles_enabled_flag is equal to 1, a first_slice_segment_in_pic_flag equal to 1 specifies that the slice segment is the first slice segment of the picture's tiles in decoding order. A first_slice_segment_in_pic_flag equal to 0 specifies that it is not the first slice segment of the picture in decoding order.

[0105] Note 1. This syntax element can be used to detect boundaries between coded pictures that are consecutive in decoding order. However, when IDR pictures are consecutive in decoding order and have the same NAL unit type, the loss of the first slice of the IDR picture may cause problems in detecting boundaries between coded pictures. This can occur, for example, when transmitting all intra-coded video in an error-prone environment. This problem can be mitigated by using two different IDR NAL unit types (IDR_W_RADL and IDR_N_LP) alternately for any two consecutive IDR pictures. The use of a temporal sublayer zero-index SEI message may also be useful, as its SEI message contains a different syntax element irap_pic_id for consecutive IRAP pictures whose value is different in decoding order. Some system environments have other provisions that may also be useful for picture boundary detection, such as presentation timestamps in the Rec.ITU-T H.222.0|ISO / IEC13818-1 system, access unit framing in the ISO / IEC 14496-12 ISO-based media file format, or the use of marker bits in the IETF RFC 3550 real-time transport protocol header.

[0106] A `dependent_slice_segment_flag` value equal to 1 specifies that the value of each missing slice segment header syntax element is inferred to be equal to the value of the corresponding slice segment header syntax element in the slice header. When none exists, the value of `dependent_slice_segment_flag` is inferred to be equal to 0.

[0107] The variable SliceAddrRs is derived as follows: -If dependent_slice_segment_flag is equal to 0, SliceAddrRs will be set to be equal to CtbAddrInRs. - Otherwise, SliceAddrRs is set to be equal to the SliceAddrRs of the preceding slice segment containing the coding tree block whose coding tree block address is CtbAddrTsToRs[CtbAddrRsToTs[CtbAddrInRs]-1], where the variable CtbAddrInRs is defined within the semantics of slice_segment_address.

[0108] `area_id_in_sh` specifies the index of the tile to which the slice segment belongs. The value of `area_id_in_sh` ranges from 0 to 255. If `area_id_in_sh` does not exist, it is inferred to be equal to 0.

[0109] The slice_segment_address (when slice_segment_base_addr_per_tile_enable_flag is equal to 1) defines the first coding tree block in a slice segment during a coding tree block raster scan of a picture, and possibly during a tile scan of a picture, as follows: The variable maxNumCtbY is defined as (slice_segment_addr_offset_per_tile_enable_flag?AreaSizeInCtbsY[area_id_in_sh]:PicSizeInCtbsY).

[0110] The length of the `slice_segment_address` syntax element is `Ceil(Log2(maxNumCtbsY))` bits. The value of `slice_segment_address` is in the range from 0 to `maxNumCtbsY-1`, including both endpoints. When `slice_segment_addr_offset_per_tile_enable_flag` is equal to 0, the value of `slice_segment_address` is not equal to the value of `slice_segment_address` of any other coded slice segment NAL unit in the same coded picture. When `slice_segment_addr_offset_per_tile_enable_flag` is equal to 1, the value of `slice_segment_address` is not equal to the value of `slice_segment_address` of any other coded slice segment NAL unit in the same area within the same coded picture. If `slice_segment_address` does not exist, it is inferred to be equal to 0.

[0111] In a picture coding tree block raster scan, the variable CtbAddrInRs, which specifies the coding tree block address, is set to equal to CtbAddrTsToRs[slice_segment_address+(slice_segment_addr_offset_per_tile_enable_flag?TileSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]]:0)]. In a tile scan, the variable CtbAddrInTs, which specifies the coding tree block address, is set to equal to CtbAddrRsToTs[CtbAddrInRs]. The variable CuQpDeltaVal, which specifies the difference between the Luma quantization parameter for a coding unit containing cu_qp_delta_abs and its prediction, is set to equal to 0. The variables CuQpOffsetCb and CuQpOffsetCr, which specify the values ​​used when determining the respective values ​​of the Qp'Cb and Qp'Cr quantization parameters for coding units containing cu_chroma_qp_offset_flag, are both set to 0.

[0112] Regarding the order of the VCL NAL units and their association with the coded picture and access units 36, it can be said that: Each VCL NAL unit is a portion of a coded picture.

[0113] The order of VCL NAL units within a coded picture is constrained as follows: - The first VCL NAL unit of the coded picture shall have a first_slice_segment_in_pic_flag equal to 1. - Set sliceSegAddrA and sliceSegAddrB to the value of CtbAddrTsToRs[slice_segment_address+CtbAddrRsToTs[(slice_segment_base_addr_per_tile_enable_flag?AreaSliceSegBaseAddr[AreaIdToAreaIdx[area_id_in_sh]]:0)]] of any two coded slice segment NAL units A and B in the same coded picture. Coded slice segment NAL unit A precedes coded slice segment NAL unit B if any of the following conditions are true. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is less than TileId[CtbAddrRsToTs[sliceSegAddrB]]. - TileId[CtbAddrRsToTs[sliceSegAddrA]] is equal to TileId[CtbAddrRsToTs[sliceSegAddrB]], and CtbAddrRsToTs[sliceSegAddrA] is less than CtbAddrRsToTs[sliceSegAddrB].

[0114] The description of possible HEVC codec extensions for efficient subpicture extraction presented above is a preliminary extension to the embodiment already shown with respect to Figure 1, which reduces the subpicture extraction process not only by providing a favorable relative representation of the slice start position to the onset position of the current segment, but also by providing information on the subpicture decoding option, or alternatively, information on decoding several subpicture decoding options, so that the extraction process can be limited in practice to activities related to the exclusion or discarding of slices that are not required according to the subpicture decoding option. In other words, with this extension of the HEVC modification described above, the interest in simply decoding subpictures, i.e., the selection of a subpicture decoding option, is handled so that layers of a multilayer data stream are missing by selecting all corresponding layers that correspond to decoding the complete picture area of ​​the original data stream. It will be noted later that the latter embodiment may be used independently of the relative slice start point indication discussed above.

[0115] Therefore, the extension described below enables a simpler extraction process that does not involve rewriting the parameter set. As in the layer bitstream extraction process, the removal of NAL units that convey slice data and parameter sets is sufficient to create a consistent bitstream, which a decoder receiving can easily determine the actual operating point represented within the extracted bitstream. In this context, the operating point exists in the space spent, at least by a subset of the segment grid or tiling grid when tiles are used as segments, and optionally by scalability dimensions such as spatial resolution, temporal resolution, and SNR scalability.

[0116] According to this embodiment, a parameter set such as VPS or SPS defines several operating points in the context of tiling, which is called the output tile set (OTS), and some characteristics thereof are described below. Hereinafter, assuming illustratively that the parameters relating to the OTS are contained within the SPS, several embodiments are shown in Figure 11.

[0117] As shown in Figure 11, the parameter set, where SPS exemplifies the decoding of a complete picture 18 (compared to Figure 1) using the tiling syntax 140, an example of which is presented in Figure 7, shows the subdivision of picture 18 into segments, each optionally associated with a segment index value for these segments, and optionally the base address of the onset position of these segments, as well as the size of picture 18, i.e., the horizontal size in 142 and the vertical size in 144, and in 146, the decoder capability level for decoding picture 18, if i=0. Since num_output_tile_format, to which the syntax elements are sent in 148, is greater than 1, the parameter set in Figure 11 signals in 142 to 146 information about further subpicture decoding options, which additionally show other picture sizes and associated decoder capability levels for picture 54. As shown in the example in Figure 11, segment 30, which yields corresponding picture sizes as shown by 142 and 144 for the output format, here, exemplary, various tile configurations are also signaled individually. The num_output_tile_sets sent at 150 is within the syntax element and indicates the number of configurations to search. Each configuration is associated with one of the output tile formats by index 152. The number of tiles that make up each picture 18 or picture 54 is sent at 154, and the segment index is sent at 156 for each individual tile configuration. Finally, for each tile configuration of the number shown by 150, the parameter set at 158 ​​indicates additional information regarding decoding, namely the coded picture buffer with a decoding timestamp and / or travel time, and / or parameters regarding the buffer size required to decode each segment configuration (or tile configuration) from the data stream.

[0118] When using the embodiment of Figure 11, the decoding process performed within decoder 38 or 52 utilizes an additional input parameter, sometimes called the target output tile set, which indicates which of the defined OTS is selected for decoding, and this input parameter represents the signal 58 discussed above with respect to Figure 1.

[0119] The parameter TargetOutputTileSet can be determined by various means. External means such as system layer instructions via signaling in file formats, transport streams, SDP, or other means. • The presence of SEI messages within the bitstream that indicate which tiles or OTS are present in the bitstream, i.e., a description of the previous extraction process that the bitstream underwent. Thus, the extraction information may be contained within a data stream that indicates one or more decodeable OTS. • The value of the syntax element tile_id_sh in the slice header of the NAL unit within the access unit is analyzed to determine which of the defined OTS may be decryptable using the bitstream content.

[0120] For example, priorities may be assigned to defined OTS to help determine which OTS should be decrypted, as follows: • Explicit signaling within the loop via num_output_tile_sets or SEI messages assigning priority to OTS • Ordered units within a loop via num_output_tile_sets • Present / removed tile SEI messages containing instructions for all OTS included in the bitstream.

[0121] Therefore, the parameter set 36 may include, for each decoding option signaled by 148, and for each segment collection as defined by 150 and 158 with respect to that decoding option, priority instructions indicating the priority of which segment collection, defining a particular subpicture configuration, is selected as the decoding target by decoders 38 and 52. The priority instructions may indicate priority overall among all segment collection options whose number is signaled by 150, or simply individually among groups belonging to a single common subpicture option, i.e., the same subpicture size and corresponding decoder capability level.

[0122] The TileSliceSegBaseAddr derivation needs to be adjusted with respect to variant form A, as shown below (highlighted in turquoise). The list TileId[ctbAddrTs] for ctbAddrTs ranging from 0 to PicSizeInCtbsY-1, including both endpoints, which specifies the conversion from CTB address to tile ID during tile scanning, is derived as follows: for(j=0, tileIdx=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, tileIdx++) tileIdxTileIdMap[tile_id_in_pps[tileIdx]]=tileIdx for(y=rowBd[j];y <rowBd[j+1];y++) (6 9) for(x=colBd[i];x <colBd[i+1];x++) TileId[CtbAddrRsToTs[y*PicWidthInCtbsY+x]]=tileIdx

[0123] The list TileSizeInCtbsY[k], which specifies the size of the k-th tile in the picture in CTB units, is derived as follows, including both endpoint values, for k ranging from 0 to ((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1. for(j=0, k=0;j<=num_tile_rows_minus1;j++) for(i=0;i<=num_tile_columns_minus1;i++, k++ ) TileSizeInCtbsY[k]=colWidth[i]*rowHeight[j]

[0124] For the i-th target OTS, the list TileSliceSegBaseAddr[k], which includes both endpoints and specifies the slice segment address offset in the tile scan of the first slice segment of the k-th tile in the bitstream-ordered picture in CTB units, is derived as follows: TileSliceSegBaseAddr[0]=0 cnt=1 for(k=1;k<=((num_tile_rows_minus1+1)*(num_tile_columns_minus1+1)-1);k++) if(tileIdxInTargetOTS){ TileSliceSegBaseAddr[cnt]=TileSliceSegBaseAddr[cnt-1]+TileSizeInCtbsY[k-1] cnt++ / / with cnt derivable from tile_id_in_sh with a mapping table to tile_id_in_sps }

[0125] tileIdxInTargetOTS is true for all tiles belonging to TargetOutputTileSet, and false otherwise. Note that the tile array in the decoded picture of TargetOutputTileSet remains in the same state as the tile array in the original bitstream. Raster transformations for tile traversal and vice versa are performed considering only the tiles in targetOutputTileSet.

[0126] As already shown above, the extension described with respect to Figure 11, which takes into account the signaling of the parameter set 36 and the picture size, as well as the signaling of the corresponding decoder capability level not only with respect to the original picture 18 but also with respect to the subpicture 54, can be used, for example, in a framework where the aforementioned slice start position information 42 is not present. For example, by convention, a slice may be constrained not to start within segment 30, but simply to start at the segment onset position. In that case, for example, segment information 48 may be sufficient. Other examples may also be possible. In any case, a decoder 38 or 52 receiving stream 10 or 12 (with or without start position information 42) uses the information in parameter set 36 equal to 36' as follows: Specifically, the decoder decodes parameter set 36 from the inbound data stream, which may be 10 or 12, and derives a decoding option instruction 58 indicating whether picture decoding or subpicture decoding should be performed on that inbound data stream. Picture decoding targets the decoding of picture 18 from data stream 10, i.e., the configuration of picture 10 from all segments 30. Therefore, to achieve this objective, all slices 26 must be within data stream 10. Subpicture decoding targets the decoding of picture 54 from data stream 12. "Picture 54" means that two or more subpicture decoding options having the same differences in picture size and shape as well as decoder capability level can be signaled within parameter set 36. For example, in the example in Figure 1, different subpicture decoding options may arise from configuring to form a subpicture 54 from two adjacent segments 30, configuring one subpicture 54 from each of two segments 30, or configuring a subpicture 54 from only one segment 30, and these examples can be extended, for example, by configuring a subpicture from three segments 30.

[0127] If picture decoding is to be performed on the data stream, the decoder derives the size of the picture 18 from the parameter set 36, for example via 142 and 144 if i=0, and an indication of the decoder capability level required to decode the picture 18 from the data stream, for example via 146 if i=0. The decoder derives from the parameter set 36 information about the subdivision of the picture 18 into a first set of segments 30 that are encoded into the data stream without encoding interdependence, i.e., from the tiling syntax 140. Furthermore, the decoder decodes the picture 18 from the data stream in units of slices 26, in which the picture 18 is partitioned along the coding path 22. If each slice 26 contains start position information, see the above description for the arrangement of the decoded picture content of each slice 26. The decoder may check the decoder capability level to see if its decoder capability level is met by the decoder 38.

[0128] However, in the case of decoding option instruction 58 which suggests that subpicture decoding is performed on the inbound data stream, the decoder derives from parameter set 36 instructions for further picture sizes of subpicture 54 and the decoder capability level required to decode the subpicture from the data stream, such as via 142, 144, and 146 if i≠0, and from parameter set 36 information regarding the subdivision of subpicture 54 into a second set which is a subset of the set of segments of picture 18, and which is a second set of segments whose segments are encoded into the data stream without encoding interdependence, and decodes subpicture 54 from the data stream in units of those slices 26 in which subpicture 54 is encoded. Even here, the decoder uses or follows coding path 22', in which case coding path 22' traverses other target purposes, namely subpicture 54. Like the coding path 22 that traverses picture 18, the coding path 22' traverses subpicture 54 segment by segment, and each slice 26 of the slices relating to any fragment of subpicture 54 contains one or more segments from a second set of segments that make up subpicture 54, all fully encoded. Note that the coding path or coding sequence 22' may traverse a subset of segments 30 that participate in the composition of subpicture 54 in a different order than the order in which the subset is traversed by the coding path 22 within the original picture 18. However, this does not affect the decoding result, as segment coding independence prevents coding dependency across segment boundaries in any case. However, within each segment 30, paths 22 and 22' must match, and this is important for maintaining decoding synchronization and even the arrangement of slice content, when it does not affect the use of start position information 42. In this case as well, the decoder may check the decoder capability level with respect to the subpicture decoding option to see if its decoder capability level is satisfied by decoder 38 / 52.

[0129] As discussed above, the decoder can derive the decoding option instruction 58 by various means. For example, the decoder 52 can analyze the inbound data stream, i.e., segments covered by slices present in 10 or 12, to determine, for example, whether slices and corresponding segments have been removed by exclusion by the network device 50, and if so, which slices and corresponding segments have been removed. In addition, or alternatively, an external signaling pathway, such as the one shown by 58 in Figure 1, may be used by the decoder 38 or 52 to determine the decoding option or target to be used. The external signaling pathway may already uniquely identify the coding option to be used, or simply exclude several coding options that should not be used or are not available, for example, by the intermediate extraction process 14. In this case, the external signaling pathway may originate from the network device 50 or, based on further information, reliably identify a set of coding options from which the decoder should select one coding option. Furthermore, additionally or alternatively, the decoder 38 or 52 may use supplementary information within the inbound data stream, such as explicit information about still decryptable options, to determine which decoding option or target to use. Similar to external information, the latter information may be included by the extracted entity, i.e., device 50. Additionally, the decoder may examine the decoder capability levels associated with the decoding options, such as those present in the parameter set 54, and eliminate some from the list of possible decoding option candidates if they conflict with the decoder's capabilities. Some or all combinations of these hints may also form a decoding option indication. If any ambiguity remains, the decoder may use the aforementioned priority order to determine one of the remaining possible decoding option candidates with the highest priority.

[0130] While several embodiments have been described in the context of the apparatus, it is clear that when a block or device corresponds to a method step or a feature of a method step, these embodiments also represent a description of the corresponding method. Similarly, embodiments described in the context of a method step also represent a description of the corresponding block, item, or feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) a hardware device, such as a microprocessor, programmable computer, or electronic circuit. In some embodiments, one or more of the most important method steps may be performed by such a device.

[0131] The data stream of the present invention may be stored on a digital storage medium, or transmitted over a transmission medium such as a wireless transmission medium like the Internet or a wired transmission medium.

[0132] Depending on certain implementation requirements, the implementations of the present invention may be implemented in hardware or software. These implementations may be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH® memory, which stores electronically readable control signals that cooperate (or are capable of cooperating) with a programmable computer system so that each method can be performed. Thus, the digital storage medium may be computer-readable.

[0133] Some embodiments of the present invention include a data carrier having an electronically readable control signal that can cooperate with a programmable computer system so that one of the methods described herein is performed.

[0134] In general, embodiments of the present invention may be implemented as a computer program product having program code that is operable to perform one of these methods when the computer program product is executed on a computer. The program code may be stored, for example, on a machine-readable carrier.

[0135] Other embodiments include a computer program stored on a machine-readable carrier for performing one of the methods described herein.

[0136] In other words, embodiments of the method of the present invention are, therefore, computer programs having program code for performing one of the methods described herein when the computer program is executed on a computer.

[0137] A further embodiment of the present invention is a data carrier (or digital storage medium, or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier, digital storage medium, or recorded medium is generally tangible and / or non-temporary.

[0138] A further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program for performing one of the methods described herein. The data stream or sequence of signals may be configured to be transmitted, for example, over a data communication connection, for example, over the Internet.

[0139] Further embodiments include processing means configured or adapted to perform one of the methods described herein, such as a computer or a programmable logic device.

[0140] Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0141] Further embodiments of the present invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system includes, for example, a file server for transferring the computer program to the receiver.

[0142] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, the field-programmable gate array may work with a microprocessor to perform one of the methods described herein. Generally, these methods are preferably performed by any hardware device.

[0143] The apparatus described herein may be implemented using hardware devices, a computer, or a combination of hardware devices and a computer.

[0144] The apparatus described herein, or any component of the apparatus described herein, may be implemented in hardware and / or at least partially in software.

[0145] The methods described herein may be carried out using hardware devices, using a computer, or using a combination of hardware devices and a computer.

[0146] Any of the methods or components of the apparatus described herein may be implemented by hardware and / or software, at least in part.

[0147] The embodiments described above merely illustrate the principles of the present invention. Modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. Therefore, the invention is intended to be limited only by the scope of the following claims and not by any specific details presented in the description and explanation of the embodiments herein. [Explanation of Symbols]

[0148] 10 Data Streams 12 Data Streams 14. Extracting sub-pictures 16 encoders 18 Pictures 20 Videos 22 Coding Path 22' Coding Path 24 Picture onset position 26 slices 28 slices 30 segments 32-slice payload 34 slice header 36 parameter sets, access units 36' Parameter Set 38 Decoders 40 starting position 42 Starting position information 44 Onset position 46 Relative starting position 48 Segment Information 50 network devices 52 Decoders 54 Pictures, Sub-pictures 54" Picture 56 extracted videos 58 Decryption Option Instructions 60 Starting position coding mode flag 62 Absolute starting position indication 64 coding blocks 70 Picture Parameter Sets 72 Sequence Parameter Sets 74 Video Parameter Sets 76 VCL NAL Unit 78 VCL NAL Unit 80 NALU Header 86 Tiling Syntax 88 tiles 94 Base slice segment address 96 Offset slice segment address 98 slice segment addresses 106 Flags 112 Starting position coding mode flag 114 Syntax elements, 116 Raster scan segment order 120 Flags 122 The first syntactic element 124 The second syntactic element 130 Base Address Data Field 140 Tiling Index 142 Information on sub-picture decoding options 152 Index

Claims

1. Dividing the picture (18) into slices (28) and segments (30), To provide a slice header (34) including start position information (42) corresponding to each slice (26) of the slice (28), wherein the start position information (42) indicates the start position (40) of the slice (26) relative to the segment onset position (44) of one of the segments (30) Determining the absolute address measured from the beginning of the picture (18), the determination includes performing a sum corresponding to the start position (40) and the segment onset position (44), Setting the start position coding mode flag (60) to a first state, wherein the start position coding mode flag (60) is switchable between the first state and a second state, the first state indicating that the start position (40) is relative to the segment onset position (44), and the second state indicating that the start position (40) is absolute with respect to the picture onset position (24). An encoding method comprising encoding the start position coding mode flag (60), the segment onset position (44), the slice header (34), and each of the slices (26) into a data stream (10).

2. The method according to claim 1, wherein the picture (18) is encoded into the segment (30) without encoding interdependence.

3. The method according to claim 1, wherein the segment (30) includes one or more tiles.

4. The method according to claim 1, wherein the segment onset position (44) is indicated by a parameter set (36).

5. An encoder (16) comprising a processor that executes instructions for carrying out the method according to any one of claims 1 to 4.

6. The picture (18), which is divided into slices (28) and segments (30), is received via the data stream (10), Decoding from the data stream (10) the segment onset position (44) of one of the segments (30) and the slice header (34) which includes start position information (42) corresponding to each slice (26) of the slice (28), Decoding a start position coding mode flag (60) from the data stream (10), wherein the start position coding mode flag (60) is switchable between a first state and a second state, the first state indicating that the start position (40) is relative to the segment onset position (44), and the second state indicating that the start position (40) is absolute with respect to the picture onset position (24). Based on the start position coding mode flag (60) having the first state, the start position information (42) is used to locate the start position (40) of the slice (26) relative to the segment onset position (44), A decoding method comprising determining an absolute address measured from the beginning of the picture (18), wherein determining includes performing a sum corresponding to the start position (40) and the segment onset position (44).

7. The method according to claim 6, wherein the picture (18) is encoded into the segment (30) without encoding interdependence.

8. The method according to claim 6, wherein the segment (30) includes one or more tiles.

9. The method according to claim 6, wherein the segment onset position (44) is indicated by a parameter set (36).

10. A decoder (38) comprising a processor that executes instructions for carrying out the method described in any one of claims 6 to 9.

Images