Encoders, decoders, and data streams used for progressive decoder refresh decoding and scalable decoding.

Abstract

This invention relates to methods, encoders, decoders, and data streams for decoding images, and particularly for decoding consecutive sequences of images. Some embodiments may utilize a so-called Progressive Decoder Refresh (GDR) decoding scheme for decoding images. Some embodiments may propose scalable decoding and progressive decoder refresh improvements.

Description

Technical Field

[0001] This invention relates to decoding images, and more particularly to decoding consecutive sequences of images. Some embodiments may utilize scalable decoding and / or so-called Progressive Decoder Refresh (GDR) decoding schemes to decode images. Some embodiments may propose improvements to scalable decoding and Progressive Decoder Refresh. Background Technology

[0002] In modern video decoding, some scenarios require very low latency transmission. Whenever low end-to-end latency is required, bitrate variations varying from image to image are undesirable. Typically, video is encoded in a way that results in the following: the size of the encoded images varies not only due to the complexity of the content but also due to the prediction structure used. More specifically, typically, video is encoded using a prediction structure based on some images being encoded as intra-frame slices (not dependent on other images), and other images being encoded as B or P slices (using other images as references). Clearly, images encoded without predicting other images result in a larger size than when temporal correlation is used and images are encoded as B or P slices.

[0003] There are some techniques that slightly shift this typical coding structure, where each element in an image is encoded, with some mix of blocks encoded using only intra-frame prediction, and some blocks encoded using inter-frame prediction. In such a case, there is no image encoded using only intra-frame prediction, and therefore, all image sizes remain similar (if the ratio of intra-frame predicted blocks to inter-frame predicted blocks remains similar across all images).

[0004] Achieving lower end-to-end latency is ideal because the maximum image size is smaller compared to other decoding structures, and therefore the time to transmit it is also shorter.

[0005] Typically, such structures are labeled Progressive Decode & Refresh (GDR) because they differ from typical decoding structures in that, to achieve a clean image, several images need to be decoded, and the video area is progressively decoded and refreshed until the content can be displayed properly; whereas in typical decoding structures, only a specific type of intra-frame (Random Access Point - RAP) is required to be rendered, and the content can be displayed instantaneously without decoding more access units.

[0006] In accessing frames or images within a sequence of images, there may be a trade-off between so-called loading time and decoding efficiency. Ideally, a mechanism should be available that allows for shorter loading times (on average or in the worst case) without compromising decoding efficiency. Furthermore, images can be subdivided into regions (e.g., tiles) that can be refreshed over time in a so-called refresh cycle RP. However, identifying which regions are clean (refreshed) and which are not is ambiguous, and thus incurs penalties, such as intra-frame prediction not being easily constrained between unclean (dirty) and clean regions. Summary of the Invention

[0007] Therefore, the purpose of this invention is to improve existing GDR encoders, decoders, and data streams.

[0008] The first aspect relates to a video data stream comprising a sequence of images, the sequence including at least one image decoded by a progressive decoder refresh (GDR) and one or more subsequent images in a refresh cycle (RP). The video data stream also includes a parameter set (SPS) defining multiple image configurations that subdivide an image region (e.g., an entire frame) into a first sub-region and a second sub-region, wherein one corresponds to a refreshed sub-region (e.g., a set of image regions) comprising one or more refreshed image regions (e.g., tiles), and the other corresponds to an unrefreshed sub-region comprising one or more unrefreshed image regions. For each image within a refresh cycle, the video data stream also includes an image configuration identifier (reg_conf_idx) for identifying a corresponding image configuration from the multiple image configurations.

[0009] Furthermore, a decoder is proposed for decoding at least one image from a data stream, representing a sequence of images including at least one progressive decoder refresh (GDR) decoded image and one or more subsequent images in a refresh cycle (RP). The decoder is configured to read a parameter set (SPS) from the data stream, the SPS defining multiple image configurations that subdivide an image region (e.g., an entire frame) into a first sub-region and a second sub-region, wherein one corresponds to a refreshed sub-region (e.g., a set of image regions) including one or more refreshed image regions (e.g., tiles), and the other corresponds to an unrefreshed sub-region including one or more unrefreshed image regions. For each image within a refresh cycle, the decoder is further configured to read an image configuration identifier (reg_conf_idx) from the data stream to identify a corresponding image configuration from the multiple image configurations in order to decode at least one image.

[0010] Furthermore, an encoder is proposed for encoding at least one image from a sequence of images comprising at least one progressive decoder refresh (GDR) decoded image and one or more subsequent images in a refresh cycle (RP) into a data stream. The encoder is configured to write a parameter set (SPS) into the data stream, the SPS defining multiple image configurations that subdivide an image region (e.g., an entire frame) into a first sub-region and a second sub-region, wherein one corresponds to a refreshed sub-region (e.g., a set of image regions) comprising one or more refreshed image regions (e.g., tiles), and the other corresponds to an unrefreshed sub-region comprising one or more unrefreshed image regions. For each image within a refresh cycle, the encoder is further configured to set an image configuration identifier (reg_conf_idx) in the data stream to identify a corresponding image configuration from the multiple image configurations.

[0011] The second aspect relates to a video data stream comprising a sequence of images including at least one progressive decoder refresh (GDR) decoded image and one or more subsequent images in a refresh cycle, wherein each image in the sequence is sequentially decoded into the video data stream in units of blocks (e.g., CTUs) into which the corresponding image is subdivided. The video data stream includes implicit signaling, wherein refresh sub-regions of the corresponding images are implicitly signaled in the video data stream based on the block decoding order. Additionally or alternatively, for each block, the video data stream includes a syntax element (e.g., a flag) indicating whether the following is true:

[0012] a) A block is the last block located in the first subregion of the corresponding image and the last block to be decoded (e.g., flag: last_ctu_of_gdr_region), and / or

[0013] b) The block is the first block located in the first subregion of the corresponding image and that is decoded first (e.g., flag: first_ctu_of_gdr_region), and / or

[0014] c) Block adjacency defines the boundary of the first sub-region, and / or

[0015] d) The block is located inside the first subregion (e.g., flag: gdr_region_flag).

[0016] Furthermore, a decoder is proposed for decoding from a data stream at least one image from a sequence of images comprising at least one progressive decoder refresh (GDR) decoded image and one or more subsequent images in a refresh cycle (RP), wherein each image in the sequence of images is sequentially decoded from the video data stream in units of blocks (e.g., CTUs) into which the corresponding image is subdivided. The decoder is configured to implicitly deduce the refresh sub-regions of at least one image from the data stream based on the block decoding order. Additionally or alternatively, for each block, the decoder is configured to read from the data stream syntax elements (e.g., flags) indicating whether the following is true:

[0017] a) A block is the last block located in the first subregion of the corresponding image and the last block to be decoded (e.g., flag: last_ctu_of_gdr_region), and / or

[0018] b) The block is the first block located in the first subregion of the corresponding image and that is decoded first (e.g., flag: first_ctu_of_gdr_region), and / or

[0019] c) Block adjacency defines the boundary of the first sub-region, and / or

[0020] d) The block is located inside the first subregion (e.g., flag: gdr_region_flag).

[0021] Furthermore, an encoder is proposed for encoding at least one image from a sequence of images comprising at least one progressive decoder refresh (GDR) decoded image and one or more subsequent images in a refresh cycle (RP) into a data stream, wherein each image in the sequence of images is sequentially encoded into the video data stream in units of blocks (e.g., CTUs) into which the corresponding image is subdivided. For each block, the encoder is configured to write a syntax element (e.g., a flag) indicating whether it is the following into the data stream:

[0022] a) A block is the last block located in the first subregion of the corresponding image and the last block to be decoded (e.g., flag: last_ctu_of_gdr_region), and / or

[0023] b) The block is the first block located in the first subregion of the corresponding image and that is decoded first (e.g., flag: first_ctu_of_gdr_region), and / or

[0024] c) Block adjacency defines the boundary of the first sub-region, and / or

[0025] d) The block is located inside the first subregion (e.g., flag: gdr_region_flag).

[0026] The third aspect relates to a multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), wherein the second sequence of images in the second layer includes at least one progressive decoder refresh (GDR) image as a starting image and one or more subsequent images in a refresh cycle. The multi-layer scalable video data stream includes a signal notification carrying information about the possibility of inter-layer prediction from samples in the first layer for unrefreshed sub-regions of the GDR images in the second layer. Additionally, the multi-layer scalable video data stream includes the following information:

[0027] In unrefreshed sub-regions of one or more subsequent images included in a refresh cycle, motion vector prediction is either disabled, or motion vector prediction is implemented non-temporally, or

[0028] In unrefreshed sub-regions of a GDR image, motion vector prediction is either disabled or implemented non-temporally.

[0029] Furthermore, a decoder is proposed for decoding at least one image from a multi-layer scalable video data stream, the multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), wherein the second sequence of images in the second layer includes at least one progressive decoder refresh (GDR) image as a starting image and one or more subsequent images in a refresh cycle. The decoder is configured to read from the multi-layer scalable video data stream a signal notification carrying information about the possibility of inter-layer prediction from samples of the first layer for an unrefreshed sub-region of the GDR image of the second layer. The decoder is further configured to, in response to the signal notification,:

[0030] In unrefreshed sub-regions of one or more subsequent images included in a refresh cycle, disable motion vector prediction or implement motion vector prediction non-temporally, or

[0031] In unrefreshed sub-regions of GDR images, disable motion vector prediction or implement motion vector prediction non-temporally.

[0032] Furthermore, an encoder is proposed for encoding at least one image into a multi-layer scalable video data stream, the multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), wherein the second sequence of images in the second layer comprises at least one progressive decoder refresh (GDR) image as a starting image and one or more subsequent images in a refresh cycle. The encoder is configured to write a signal notification carrying information about the probability of inter-layer prediction from samples of the first layer for an unrefreshed sub-region of the GDR image of the second layer into the multi-layer scalable video data stream and the following information:

[0033] In unrefreshed sub-regions of one or more subsequent images included in a refresh cycle, motion vector prediction is either disabled, or motion vector prediction is implemented non-temporally, or

[0034] In unrefreshed sub-regions of a GDR image, motion vector prediction is either disabled or implemented non-temporally.

[0035] The fourth aspect relates to a multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), each of the first and second layers comprising multiple temporal sublayers. The scalable video data stream also includes a signaling notification (e.g., vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers of the second layer (e.g., the enhancement layer) are decoded via inter-layer prediction.

[0036] Furthermore, a decoder is proposed for decoding at least one image from a multi-layer scalable video data stream, the multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), each of the first and second layers comprising multiple temporal sublayers. The decoder is configured to decode one or more of the temporal sublayers using inter-layer prediction based on signaling notifications derived from the scalable video data stream, the signaling notifications (e.g., vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers of the second layer (e.g., the enhancement layer) will be decoded via inter-layer prediction.

[0037] Furthermore, an encoder is proposed for encoding at least one image into a multi-layer scalable video data stream, the multi-layer scalable video data stream comprising a first sequence of images in a first layer (e.g., a base layer) and a second sequence of images in a second layer (e.g., an enhancement layer), each of the first and second layers comprising multiple temporal sublayers. The encoder is configured to encode one or more of the temporal sublayers by using inter-layer prediction and writing signaling notifications (e.g., vps_sub_layer_independent_flag[i][j]) into the scalable video data stream, the signaling notifications indicating which temporal sublayers of the second layer (e.g., the enhancement layer) are decoded via inter-layer prediction.

[0038] The fifth aspect relates to a video data stream comprising at least one image subdivided into tiles and a tile reordering flag, wherein,

[0039] a) If the tile reordering flag in the data stream (e.g., sps_enforce_raster_scan_flag) is in the first state, then signal the tiles of the image to be decoded using the first decoding order according to the tile traversal of the image, and / or

[0040] b) If the tile reordering flag in the data stream has a second state, then signal the tiles of the picture to be decoded using a second decoding order that traverses the picture along the raster scan order.

[0041] Furthermore, a decoder is proposed that is configured to decode an image from a data stream, wherein:

[0042] a) If the tile reordering flag in the data stream (e.g., sps_enforce_raster_scan_flag) is in the first state, the decoder is configured to decode the tiles of the image from the data stream using the first decoding order according to the tile traversal of the image, and / or

[0043] b) If the tile reordering flag in the data stream has a second state, the decoder is configured to decode the tiles of the image from the data stream using a second decoding order that traverses the image along the raster scan order.

[0044] Furthermore, an encoder is proposed that is configured to encode images into a data stream, wherein:

[0045] a) The encoder is configured to set a tile reordering flag (e.g., sps_enforce_raster_scan_flag) in the data stream to a first state, thereby indicating that the tiles of the image will be decoded using the first decoding order according to the tile traversal of the image, and / or

[0046] b) The encoder is configured to set the tile reordering flag in the data stream to a second state, thereby indicating that the tiles of the image will be decoded using a second decoding order that traverses the image along the raster scan order. Attached Figure Description

[0047] In the following, embodiments of the present disclosure are described in more detail with reference to the accompanying drawings, wherein:

[0048] Figure 1 This diagram illustrates the progressive decoding refresh concept according to an embodiment.

[0049] Figure 2 This diagram illustrates the progressive decoding refresh concept according to an embodiment.

[0050] Figure 3 This diagram illustrates the progressive decoding refresh concept of a column using an image region according to an embodiment.

[0051] Figure 4 This diagram illustrates the progressive decoding refresh concept of using a row of an image region according to an embodiment.

[0052] Figure 5 This diagram illustrates a progressive decoding refresh concept using grouped image regions according to an embodiment.

[0053] Figure 6 A schematic diagram illustrating the progressive decoding refresh concept using scalable multi-layer video bitstreams according to an embodiment is shown, and

[0054] Figure 7 This illustration shows a schematic diagram of a progressive decoding refresh concept using a scalable multi-layer video bitstream with multiple temporal sub-layers, according to an embodiment. Detailed Implementation

[0055] Equivalent or equivalent elements, or elements having equivalent or equivalent functionality, are referred to by equivalent or equivalent reference numerals in the following description.

[0056] Method steps depicted and described with reference to block diagrams may also be performed in a different order than those depicted and / or described. Furthermore, method steps involving specific features of the apparatus may be replaced by those features of the apparatus, and vice versa.

[0057] Furthermore, in this disclosure, terms, frames, and images can be used interchangeably.

[0058] Figure 1This example illustrates a Progressive Decode & Refresh (GDR) decoding structure, which differs from conventional decoding structures in that, to achieve clean images, several images need to be decoded, and the video area is progressively decoded and refreshed until the content can be displayed appropriately. In contrast, conventional decoding structures only require the presentation of a specific type of intra-frame (Random Access Point - RAP) and can display content instantaneously without decoding additional access units.

[0059] Figure 1 Display images 1011, 1012, ..., 101 in sequential order. n The sequence is 100. Each image is numbered 1011, 1012, ..., 101. n The image can be divided into image regions 102, for example, into tiles. Tiles 102 can be intra-frame decoded 102a or inter-frame decoded 102b. Intra-frame decoded image regions 102a provide access, from which the decoder can begin accessing the bitstream and refresh the entire image according to the Progressive Decoding Refresh (GDR) principle. Therefore, a frame or image including such intra-frame decoded image regions 102a can also be called an Access Unit (AU).

[0060] The image including the intra-decoded image region 102a can also be referred to as GDR image 103. In this example, every two images can be GDR images 103. Therefore, in this example, the GDR increment (delta) is two. The image region that has been intra-decoded (i.e., the intra-decoded image region 102a) can also be referred to as a refreshed image region or a clean image region, respectively. The image region that has not been decoded after being accessed can be referred to as an unrefreshed image region or a dirty image region, respectively.

[0061] Variant A: Based on full MCTS

[0062] The refresh cycle (RP) is the time interval during which the process continues until all image regions 102 are refreshed and a clean image can be displayed. Different configurations exist for this bitstream. Figure 1 In the diagram, (nine tiles x tiles refreshed every two frames - 1) = 17 frames is the refresh cycle (RP), which is the number of frames until all image areas 102 are refreshed.

[0063] Furthermore, in this example, there can be a GDR image 103 for every two images, that is, an image from which the decoder can access the bitstream and decode the complete image after the complete RP. This can be achieved by encoding all image regions 102 independently over time (e.g., referred to as MCTS in HEVC) and by extending the intra-frame decoded blocks 102a at a distance of two frames between the image regions 102 in the example above.

[0064] Variant B: Constrained inter-frame tiles (see also mentioned above) Figure 2 )

[0065] Figure 2 The following configuration is shown, in which the GDR structure can be implemented as follows: GDR frames 103 are displayed every 18 frames, and dependencies between regions are allowed (without MCTS), but only dependencies on previously refreshed regions are permitted. In this case, only one region is refreshed using intra-blocks, and other regions can reference regions that are refreshed in a timely manner. This allows for better efficiency as the first configuration shown, since regions are not encoded using full MCTS.

[0066] exist Figure 2 In the non-restrictive example shown, there is one GDR image 103 for every 18 images, and the refresh cycle (RP) consists of 17 frames.

[0067] Variant C: Column-based

[0068] Figure 3 Another non-limiting example is shown, in which one GDR image 103 is rendered for every 18 images, and 17 frames constitute a refresh cycle (RP). As can be seen, the intra-frame decoded image region 102a in this example may include image columns rather than image tiles as discussed above.

[0069] Variant D: Row-based

[0070] Figure 4 Another non-limiting example is shown, in which one GDR image 103 is rendered for every 18 images, and 17 frames constitute a refresh cycle (RP). As can be seen, the intra-frame decoded image region 102a in this example may include image rows rather than image tiles as discussed above.

[0071] In summary, for all the non-restrictive example variants of GDR discussed above, as shown in the table below, RP can be 17, and the periodicity of GDR can be Δ = 2 or Δ = 18.

[0072]

[0073] M = Number of tiles

[0074] RP = Refresh Cycle

[0075] Δ = Incremental GDR, the distance between GDR images, i.e., the possible starting point for decoding.

[0076] Another important aspect of evaluating GDR technology is the loading time required to display the image, which consists of the RP plus the time needed to wait until the GDR image 103 is found. The table above shows the loading time under average and worst-case scenarios.

[0077] Problems in different GDR scenarios:

[0078] 1) As discussed above, configuration A is the worst-case decoding efficiency for the same RP because the region is encoded as MCTS. However, the loading time for such a configuration is much shorter than that for any configuration BD. A mechanism that allows for a reduction in loading time (on average or in the worst case) without compromising decoding efficiency is ideal.

[0079] 2) As can be seen in BC, regions may not be defined statically, thus reducing signaling overhead and efficiency penalties associated with using independent regions (such as tiles). However, identifying which regions are clean (refreshed) and which are not yet refreshed may be ambiguous, and thus incur some penalties, such as intra-frame prediction not being easily constrained between unclean (dirty) and clean regions.

[0080] 1. Dynamic GDR area signaling

[0081] To address the issue of regions varying depending on the image (i.e., changing from an unrefreshed image area to a refreshed image area), two inventive approaches can be envisioned to avoid the burden of having to send the updated PPS with every single image:

[0082] In the first inventive approach, several configurations of the image region (e.g., in the form of tiles) can be signaled within the SPS (Sequence Parameter Set) or PPS (Image Parameter Set), and the tile header can point to an index indicating which tile configuration is being used for a given AU (Access Unit). Thus, this inventive approach can provide dynamic signaling for refreshed and unrefreshed image regions.

[0083]

[0084] In this disclosure, the image area may include the entire frame 101 or image 101. The image area may be divided into image sub-areas. For example, in... Figure 2As exemplarily shown in the fourth frame, the first image sub-region 101r may only include the refreshed image region 102r. Therefore, this first image sub-region 101r can also be referred to as the refreshed image sub-region 101r. On the other hand, the second image sub-region 101u may only include the image region 102u that has not yet been refreshed. Therefore, this second image sub-region 101u can also be referred to as the unrefreshed image sub-region 101u. Image region 102 generally includes image tiles (...). Figure 1 and Figure 2 ), Image tile column ( Figure 3 ), image tile lines ( Figure 4 The refreshed image region 102r may include at least one of the following: a decoded block (e.g., a CTU), a decoded block line, a decoded block row, a decoded block diagonal, a sample, a sample row, or a sample column. Figure 1 and Figure 2 ), refreshed image tile column ( Figure 3 ), refreshed image tile lines ( Figure 4 The refreshed decoded block (e.g., CTU), refreshed decoded block line, refreshed decoded block row, refreshed decoded block diagonal, refreshed sample, refreshed sample row, or refreshed sample column are at least one of these. The unrefreshed image region 102u may include unrefreshed image tiles (…). Figure 1 and Figure 2 ), Image tile column that has not yet been refreshed ( Figure 3 ), Image tile lines that have not yet been refreshed ( Figure 4 At least one of the following: an unrefreshed decoded block (e.g., CTU), an unrefreshed decoded block line, an unrefreshed decoded block row, an unrefreshed decoded block diagonal, an unrefreshed sample, an unrefreshed sample row, or an unrefreshed sample column.

[0085] According to an embodiment of the first aspect of the present invention, a video data stream can be provided, the video data stream including images 1011, 1012, ..., 101 nThe sequence 100 includes at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images in a refresh period RP. The video data stream also includes a parameter set (e.g., SPS or PPS) defining multiple image configurations that subdivide image region 101 (e.g., the entire frame) into first subregions 101r (e.g., one or more first image regions 102 including tiles, rows, columns, etc.) and second subregions 101u (e.g., one or more second image regions 102 including tiles, rows, columns, etc.), wherein one subregion corresponds to a refreshed subregion 101r including one or more (i.e., a set) refreshed image regions (e.g., tiles), and another subregion 101u corresponds to an unrefreshed subregion including one or more (= a set) unrefreshed image regions. According to the inventive principles, for each image 1011, 1012, ..., 101 within the refresh period RP... n The video data stream includes an image configuration identifier (e.g., region_configuration_idx) used to identify a corresponding image configuration from multiple image configurations.

[0086] According to another embodiment, a corresponding decoder is proposed for analyzing the data stream output from images 1011, 1012, ..., 101. n At least one image of sequence 100 is decoded, sequence 100 including at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images 1012, ..., 101 in refresh cycles (RP). n The decoder is configured to read a parameter set (e.g., PPS or SPS) from the data stream. This parameter set defines multiple picture configurations that subdivide a picture region (e.g., the entire frame) 101 into a first sub-region 101r and a second sub-region 101u. One sub-region corresponds to a refreshed sub-region (e.g., a set of refreshed picture regions) 101r that includes one or more refreshed picture regions (e.g., refreshed tiles) 102r, and the other sub-region corresponds to an unrefreshed sub-region 101u that includes one or more unrefreshed picture regions 102u. For each picture 1011, 1012, ..., 101 within a refresh cycle (RP). n The decoder can be further configured to read from the data stream an image configuration identifier (region_configuration_idx) used to identify a corresponding image configuration from multiple image configurations, so as to identify at least one image 1011, 1012, ..., 101 n Decode it.

[0087] According to another embodiment, a corresponding encoder is provided for encoding images 1011, 1012, ..., 101... n At least one image of sequence 100 is encoded into a data stream, sequence 100 including at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images 1012, ..., 101 in refresh cycles (RP). n The encoder is configured to write a parameter set into a data stream. This parameter set defines multiple image configurations that subdivide image region 101 into a first sub-region 101r and a second sub-region 101u. One sub-region 101r corresponds to a refreshed sub-region 101r that includes one or more refreshed image regions 102r, and the other sub-region 101u corresponds to a non-refreshed sub-region 101u that includes one or more unrefreshed image regions 102u. For each image 1011, 1012, ..., 101 within a refresh cycle (RP). n The encoder is further configured to set a configuration identifier in the data stream for each image to identify a corresponding image configuration from multiple image configurations, so as to identify at least one image 1011, 1012, ..., 101 n Decode it.

[0088] According to another embodiment, a corresponding method is proposed for analyzing data from images 1011, 1012, ..., 101... n At least one image of sequence 100 is decoded, sequence 100 including at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images 1012, ..., 101 in refresh cycles (RP). n The method includes the following steps: reading a parameter set from a data stream, the parameter set defining multiple image configurations that subdivide image region 101 into a first sub-region 101r and a second sub-region 101u, wherein one corresponds to a refreshed sub-region 101r including one or more refreshed image regions 102r, and the other corresponds to a non-refreshed sub-region 101u including one or more unrefreshed image regions 102u. The method further includes the following steps: for each image 1011, 1012, ..., 101 within a refresh cycle (RP). n The image configuration identifier is read from the data stream to identify a corresponding image configuration from multiple image configurations, so as to identify at least one image 1011, 1012, ..., 101. n Decode it.

[0089] According to another embodiment, a corresponding method is proposed, which is used to process images 1011, 1012, ..., 101... nAt least one image of sequence 100 is encoded into a data stream, sequence 100 including at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images 1012, ..., 101 in refresh cycles (RP). n The method includes the following steps: writing a parameter set into a data stream, the parameter set defining multiple image configurations that subdivide image region 101 into a first sub-region 101r and a second sub-region 101u, wherein one corresponds to a refreshed sub-region 101r including one or more refreshed image regions 102r, and the other corresponds to an unrefreshed sub-region 101u including one or more unrefreshed image regions 102u. The method further includes the following steps: for each image 1011, 1012, ..., 101 within a refresh cycle (RP). n In the data stream, an image configuration identifier is set to identify the corresponding image configuration from multiple image configurations.

[0090] As mentioned above, images 1011, 1012, ..., 101 n The sequence 100 can be decoded in different ways. For example, images 1011, 1012, ..., 101 n Sequence 100 can be decoded in a manner that prevents intra-frame prediction from crossing the boundary between the first sub-region 101r and the second sub-region 101u. Alternatively, images 1011, 1012, ..., 101... n Sequence 100 can be decoded in a way that the temporal prediction of the refreshed sub-region 101r does not refer to the unrefreshed sub-region 101u. Alternatively, images 1011, 1012, ..., 101... n Sequence 100 can be decoded in a way that the context model derivation does not cross the boundary between the first sub-region 101r and the second sub-region 101u.

[0091] According to an advantageous embodiment, a corresponding image configuration indicator is included in the currently decoded images 1011, 1012, ..., 101 of the image sequence 100. n The refreshed image area 102r and the unrefreshed image area 102u are shown in the image.

[0092] According to another advantageous embodiment, each of the multiple image configurations may include a set of region indices for signaling which image regions 102 are refreshed image regions 102r and which image regions 102 are not refreshed image regions 102u. This provides explicit signaling for refreshed image regions 102r and not-refreshed image regions 102u.

[0093] For example, as shown and previously referenced Figure 1 and Figure 2 The images 1011, 1012, ..., 101 included in the image sequence 100 are discussed. n It can be subdivided into one or more tiles 102. In this case, each picture configuration from multiple picture configurations may include a set of tile indices for signaling which picture tiles 102 are refreshed picture tiles 102r and which picture tiles 102 are not refreshed picture tiles 102u. Thus, in one embodiment, the region configuration may contain a set of tile indices.

[0094] According to another embodiment, as referenced Figure 3 As an example, the images 1011, 1012, ..., 101 included in the sequence of images 100 are discussed. n It can be further subdivided into image tile columns, wherein each image configuration, derived from multiple image configurations, includes at least one column index for signaling which image columns are refreshed (image column 102r) and / or which image columns are not refreshed (image column 102u). Therefore, a region configuration can contain tile column indexes.

[0095] According to another embodiment, as referenced Figure 4 As an example, the images 1011, 1012, ..., 101 included in the sequence of images 100 are discussed. n It can be subdivided into image tile rows, wherein each image configuration from multiple image configurations includes at least one row index for signaling which image rows are refreshed image rows 102r and / or which image rows are not refreshed image rows 102u. Therefore, a region configuration can contain tile row indexes.

[0096] Image regions can also be represented by decoded blocks, such as CTUs (decode tree units).

[0097] According to another embodiment, the images 1011, 1012, ..., 1013 included in the image sequence 100 are... n It can be subdivided into decoded block (e.g., CTU) rows, wherein each of the multiple picture configurations may include at least one row decoded block index for signaling which decoded block rows are refreshed decoded block rows 102r and / or which decoded block rows are not refreshed decoded block rows 102u. Therefore, the region configuration may contain CTU row indexes.

[0098] According to another embodiment, the images 1011, 1012, ..., 1013 included in the image sequence 100 are... nIt can be further subdivided into decoded block (e.g., CTU) columns, wherein each of the multiple image configurations may include at least one column decoded block index for signaling which decoded block columns are refreshed decoded block columns 102r and / or which decoded block columns are not refreshed decoded block columns 102u. Therefore, the region configuration may contain a CTU column index.

[0099] According to another embodiment, the images 1011, 1012, ..., 1013 included in the image sequence 100 are... n The area configuration can be subdivided into decoder block (e.g., CTU) diagonals, and each of the multiple area configurations can include at least one diagonal decoder block index for signaling which decoder block diagonals are refreshed (102r) and / or which decoder block diagonals are not refreshed (102u). Therefore, the area configuration can contain one or more indices of the CTU diagonals.

[0100] Image regions can also be represented by samples.

[0101] According to another embodiment, the images 1011, 1012, ..., 1013 included in the image sequence 100 are... n It can be subdivided into sample rows, wherein each image configuration from multiple image configurations may include at least one sample row index for signaling which sample rows are refreshed sample rows 102r and / or which sample rows are not refreshed sample rows 102u. Therefore, a region configuration may contain one or more sample row indices.

[0102] According to another embodiment, the images 1011, 1012, ..., 1013 included in the image sequence 100 are... n It can be further subdivided into sample columns, wherein each image configuration from multiple image configurations may include at least one sample column index for signaling which sample columns are refreshed sample columns 102r and / or which sample columns are not refreshed sample columns 102u. Therefore, a region configuration may contain one or more sample column indexes.

[0103] According to yet another embodiment, a corresponding image configuration can be signaled in the slice header and / or in the access unit delimiter of the video data stream.

[0104] Then, the slice header will indicate which configuration to use:

[0105]

[0106] Alternatively, information about the region configuration used is included in the Access Module Delimiter (AUD).

[0107] According to another embodiment, a video data stream is provided, which includes images 1011, 1012, ..., 101... n Sequence 100 includes at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images 1012, ..., 101 in a refresh cycle (RP). n Images 1011, 1012, ..., 101 n Each image in sequence 100 can be sequentially decoded into the video data stream in units of blocks 102 (e.g., CTUs) into which the corresponding image is subdivided. The video data stream may include implicit signaling, wherein the corresponding images 1011, 1012, ..., 101... n The refreshed sub-region 102r is implicitly signaled in the video data stream based on the block decoding order.

[0108] Furthermore, a corresponding decoder is proposed, namely, for analyzing the data stream from images 1011, 1012, ..., 101. n A decoder that decodes at least one image of sequence 100, sequence 100 including at least one image 103 decoded by progressive decoder refresh (GDR) and one or more subsequent images in refresh cycle RP. Images 1011, 1012, ..., 101 n Each image in sequence 100 can be sequentially decoded from the video data stream in units of blocks 102 (e.g., CTUs) into which the corresponding image is subdivided. The decoder can be configured to implicitly deduce the refreshed sub-regions 102r of at least one image from the data stream based on the block decoding order.

[0109] Furthermore, a corresponding encoder is proposed, namely, for encoding images 1011, 1012, ..., 101... n At least one image of sequence 100 is encoded into an encoder in a data stream. Sequence 100 includes at least one progressive decoder refresh (GDR) decoded image 103 and one or more subsequent images in refresh cycles RP. Images 1011, 1012, ..., 101 n Each image in sequence 100 can be sequentially encoded into the video data stream in units of blocks 102 (e.g., CTUs) into which the corresponding image is subdivided. The encoder can be configured to implicitly derive a refreshed sub-region 101r of at least one image from the data stream based on the block decoding order.

[0110] Therefore, in this approach, CTU-based signaling of region boundaries can be used. Each CTU (image region) 102 can contain a flag indicating whether it is the last CTU 102 of the GDR region 103 (which can be CABAC decoded). This signaling affects, for example, the availability of samples for intra-frame prediction and / or CABAC resetting. The advantage of such an approach is that it is more flexible and not limited to a fixed grid defined in the parameter set.

[0111] Therefore, according to the embodiment, the syntax element is used to indicate that the image originates from picture 1011, 1012, ..., 101 n The boundary between the refreshed sub-region 101r and the unrefreshed sub-region 101u of the image 101 in sequence 100 and / or used to indicate which sub-region is the refreshed sub-region 101r and which sub-region is the unrefreshed sub-region 101u.

[0112]

[0113] In cases where each CTU 102 indicates whether it is the last CTU in GDR region 103, it is beneficial to identify whether the last CTU 102 in the region means the last CTU 102 in terms of row or column. Such an indication can be made in a parameter set, for example, in SPS.

[0114]

[0115]

[0116] For example, a flag `region_horizontal_flag` equal to 1 can indicate that the last CTU flag in a CTU indicates a horizontal segmentation. In other cases, it indicates a vertical segmentation.

[0117] Therefore, according to another embodiment, a video data stream, an encoder, and a decoder are proposed, wherein,

[0118] In the case indicated by the syntax element:

[0119] a) Block 102 is the last block located in the refreshed sub-region 101r of the corresponding image 101 and is the last block to be decoded.

[0120] The video data stream includes additional syntax elements (e.g., region_horizontal_flag) used to indicate whether it is the latter:

[0121] a1) Block 102 is one or more lines of the refreshed sub-region 101r, and the last decoded block of block 102, or

[0122] a2) Block 102 is the last decoded block of one or more columns of the refreshed sub-region 101r.

[0123] According to another embodiment, additional syntax elements may indicate whether the last block is derived from a horizontal segmentation (e.g., region_horizontal_flag=1) of the decoded split tree upon which the corresponding image 101 is subdivided into blocks 102, or from a vertical segmentation.

[0124] In another embodiment, it can be indicated whether the region is an intra-prediction interruption, i.e., the neighboring region of another region is not available for prediction or CABAC, etc. Thus, the video data stream can include an intra-prediction interruption indication for indicating the latter: for example, if a neighboring picture region 102u is contained in a sub-region 101u that has not yet been refreshed, then the neighboring block 102 of the neighboring picture region 102u is not available for prediction.

[0125] In both cases defined above, the grid used in the region can be aligned with the CTU size. In other words, the refreshed sub-region 101r may include one or more refreshed image regions 102r arranged in a grid aligned with the size of the blocks 102 into which the corresponding image 101 is subdivided.

[0126] In the embodiments described so far, it is not necessarily known which areas are refreshed (cleaned) and which are not refreshed (dirty). In this case, all areas are considered to be "independent" of each other in all or some of the following aspects:

[0127] • Intra-frame prediction interruption, meaning that the nearest neighbors of another region cannot be used for prediction.

[0128] Spatial / temporal MV prediction

[0129] ·CABAC

[0130] Alternatively, the signaling implicitly indicates that the leftmost region 101r is a clean region, and that the availability of intra-frame blocks is constrained for this region, i.e., for all or some of the following aspects, a block 102 in the leftmost region cannot use a block from another (non-leftmost) region:

[0131] • Intra-frame prediction interruption, meaning that the nearest neighbors of another region cannot be used for prediction.

[0132] Spatial / temporal MV prediction

[0133] ·CABAC

[0134] Therefore, according to the embodiment, implicit signaling can signal that the first block 102 at a predetermined location in the block decoding order (e.g., the first CTU in the upper left corner) is part of the refreshed sub-region 101r.

[0135] As an alternative to the implicit derivation described above for refreshed region 102r and unrefreshed region 102u, some embodiments of the present invention may provide explicit signaling, wherein a video data stream, a corresponding encoder, and a corresponding decoder are proposed, wherein for each block 102, the video data stream may include a syntax element indicating whether it is the latter:

[0136] a) Block 102 is the last block located in the first subregion 101r of the corresponding image and the last block to be decoded (e.g., flag: last_ctu_of_gdr_region), and / or

[0137] b) Block 102 is the first block located in the first subregion 101r of the corresponding image and is the first block to be decoded (e.g., flag: first_ctu_of_gdr_region), and / or

[0138] c) Block 102 adjacent to the boundary of the first sub-region 101r, and / or

[0139] d) Block 102 is located inside the first subregion 101r (e.g., flag: gdr_region_flag).

[0140] In other words, as an alternative to the implicit derivation described above for the refreshed region 102r and the unrefreshed region 102u, it is proposed to explicitly indicate which region is the clean region 102r and which region is not the clean region 102r, as discussed in the following embodiments.

[0141] In an embodiment, in addition to indicating the end of GDR region 103, the beginning of GDR region 103 can also be indicated at the CTU level, for example, by using a flag (CABAC decoded).

[0142]

[0143]

[0144] This will help with the MCTS-style refresh approach (see...) Figure 1 In this context, the refreshed region 101r is always decoded independently.

[0145] In another embodiment, if horizontal region segmentation is enabled, the CTU-based region start and / or end markers can only be signaled in the first CTU column of tile 102, and if vertical region segmentation is enabled, the CTU-based region start and / or end markers can only be signaled in the first CTU row of tile 102.

[0146] Therefore, according to the embodiment, the image region of the corresponding image 101 can be vertically subdivided into one or more slices 102, wherein for each slice 102, a syntax element (e.g., flag: last_ctu_of_gdr_region / / flag: first_ctu_of_gdr_region) is signaled.

[0147] According to another embodiment, the image region of the corresponding image 101 can be horizontally subdivided into one or more rows of blocks 102, wherein syntax elements (e.g., flags: last_ctu_of_gdr_region / / flags: first_ctu_of_gdr_region)

[0148] i. Signal only in the first row, or

[0149] ii. Use signals to indicate each row.

[0150] In another embodiment, a CTU-based (CABAC-decoded) flag can be signaled to indicate whether CTU102 is part of GDR refresh area 103.

[0151]

[0152] In another embodiment, the CTU start and / or end index of the GDR refresh region 103 can be signaled in the slice header.

[0153] One benefit of the embodiments described above is that the image region can be decoupled from the use of tile 102, and the scan order may therefore remain unaffected. In most applications using GDR, low-latency transmission is desired. To achieve low-latency transmission, all packets sent should be of the same size, not just all AUs. Typically, in those low-latency scenarios, each AU can be divided into multiple packets, and to ensure that all packets are of the same size (or very similar), each packet should have the same number of refreshed (belonging to clean area 101r) blocks 102r and the same number of unrefreshed (belonging to dirty area 101u) blocks 102u.

[0154] Figure 5 Displaying images 1011, 1012, ..., 101 nA non-limiting example of a sequence 100, wherein each image 1011, 1012, ..., 101 n It can be divided into multiple groups 501a, 501b, ..., 501n. In order to ensure that all groups have the same size (or are very similar), each group 501a, 501b, ..., 501n can have the same number of refreshed (belonging to the clean area 101r) blocks 102r and the same number of unrefreshed (belonging to the dirty area 101u) blocks 102u.

[0155] If the tiles were used for this purpose, the tile scan order will be used, and therefore groups 501a, 501b, ..., 501n may not have the same number of refreshed (belonging to clean area 101r) blocks 102r and the same number of unrefreshed (belonging to dirty area 101u) blocks 102u.

[0156] In another embodiment, tiles are used, but syntax elements are added to the parameter set that enforces raster scan instead of tile scan. For example, `sps_enfoce_raster_scan_flag`. In this case, raster scan will be used, and byte alignment will not occur within the bitstream to allow the CTU to start a new tile.

[0157] Therefore, according to an embodiment, a video data stream is proposed, which includes at least one image 101 subdivided into tiles 102 and a tile reordering flag (e.g., sps_enforce_raster_scan_flag), wherein,

[0158] a) If the tile reordering flag in the data stream has a first state, then signal that tile 102 of image 101 will be decoded using the first decoding order according to the tile traversal of image 101, and / or

[0159] b) If the tile reordering flag in the data stream has a second state, then the tile 102 of picture 101 is signaled to decode using a second decoding order that traverses picture 101 along the raster scan order.

[0160] Another embodiment proposes a corresponding decoder that can be configured to decode image 101 from a data stream, wherein:

[0161] a) If the tile reordering flag in the data stream (e.g., sps_enforce_raster_scan_flag) is in the first state, the decoder is configured to decode tiles 102 of image 101 from the data stream according to the first decoding order of tile traversal of image 101, and / or

[0162] b) If the tile reordering flag in the data stream has a second state, the decoder is configured to decode the tile 102 of the image 101 from the data stream using a second decoding order that traverses the image 101 along the raster scan order.

[0163] As referenced above Figure 5 The decoder can be configured to use a Progressive Decode-Refresh (GDR) approach to decode image 101, where image 101 can be image 1011, 1012, ..., 101. n As part of sequence 100, sequence 100 includes at least one GDR-decoded image 103 and one or more subsequent images, wherein image 101 is decoded block by block and divided into multiple groups 501a, 501b, ..., 501n, wherein two or more groups (and preferably, each group) include the same amount of refreshed blocks 102r and / or the same amount of unrefreshed blocks 102u.

[0164] Another embodiment proposes a corresponding encoder configured to encode image 101 into a data stream, wherein:

[0165] a) The encoder can be configured to set a tile reordering flag (e.g., sps_enforce_raster_scan_flag) in the data stream to a first state, thereby instructing tiles 102 of image 101 to be decoded using the first decoding order according to the tile traversal of image 101, and / or

[0166] b) The encoder is configured to set the tile reordering flag in the data stream to a second state, thereby instructing the tiles 102 of image 101 to be decoded using a second decoding order that traverses image 101 along the raster scan order.

[0167] 2. Scalable GDR constraints

[0168] In the case of GDR for scalable bitstreams, it may be possible to have the highest quality RP as discussed in this paper, while at the same time being able to achieve a smaller low-quality RP (LQRP), in which the unrefreshed region 101u at the highest quality can be replaced by samples of lower-quality content from a lower layer.

[0169] Figure 6This illustrates a non-limiting example of a GDR approach using a scalable bitstream 600 having a first layer (e.g., base layer - BL) 601 and a second layer (e.g., enhancement layer - EL) 602, wherein a missing refreshed region (e.g., an unrefreshed or non-refreshed image region) 102u in an unrefreshed image sub-region 101u of the second layer (e.g., enhancement layer - EL) 602 can be replaced by an upsampled sample of the refreshed image sub-region 201r of the refreshed image sub-region 202r of the first layer (e.g., base layer - BL) 601.

[0170] In one embodiment, the decoding process of EL 602 manages the state of the defined GDR regions (whether they have been refreshed since the GDR) and indicates for each region whether it has been initialized for each layer. If a region is not initialized, a resampling process is performed for the reference layer 601 of that region, and the sample values ​​are replaced. Thus, when decoding begins at the access unit containing the EL GDR image 103, images of higher layers can be immediately presented to the user, thereby progressively updating to EL quality during the RP process.

[0171] However, constraints in the bitstream are necessary for the operation of the procedures described above.

[0172] Therefore, according to an embodiment, a multi-layer scalable video data stream 600 is proposed, which includes images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 601. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 602 n The second sequence 100. Images 1011, 1012, ..., 101 in the second layer 602. n The second sequence 100 may include at least one progressive decoder refresh (GDR) image 103 as a starting image and one or more subsequent images in a refresh cycle (RP), wherein the multi-layer scalable video data stream 600 may include a signal notification carrying information about the probability that an unrefreshed sub-region 101u of the GDR image 103 of the second layer 602 will be inter-layer predicted from samples 202r of the first layer 601. The signal notification may further carry the following information:

[0173] • In a sub-region 101u that has not yet been refreshed and is included in one or more subsequent images in a refresh cycle (RP), motion vector prediction is disabled, or motion vector prediction is implemented non-temporally, or

[0174] • In the unrefreshed sub-region 101u of GDR image 103, motion vector prediction is disabled, or motion vector prediction is implemented non-temporally.

[0175] In one embodiment, for the unrefreshed region 101u in GDR image 103, TMVP (Temporal Motion Vector Prediction) or sub-block TMVP (i.e., Syntax-Based Motion Vector Prediction) can be disabled, such that when sample 102u is replaced by upsampled BL sample 202r, subsequent EL images 1011, ..., 101... n It is possible to use alternative sample 202r for prediction, which significantly reduces encoder / decoder drift compared to using erroneous motion vectors that would occur under unconstrained conditions.

[0176] Therefore, the basic principle of this aspect proposes to provide a multi-layer scalable video data stream 600, wherein the inter-layer predictions performed from samples 202r of the first layer 601 may include one or more samples 102u of the unrefreshed sub-region 101u of the GDR image 103, replaced by upsampled versions of samples 202r refreshed from the first layer 601.

[0177] Another embodiment proposes that all samples 102u of the entire unrefreshed sub-region 101u of GDR image 103 can be replaced by an upsampled version of the refreshed sample 202r of the first layer 601, so that the images 1011, 1012, ..., 101 of the second sequence 100 of the decoded images from the second layer 602 n It can be presented to the user immediately.

[0178] Another embodiment proposes one or more subsequent images 1012, ..., 101 of the second layer 100. n The unrefreshed subregion 101u can be refreshed by in-layer prediction (e.g., inside the second layer 602) using upsampled alternative samples 202r from the first layer 601, which progressively update the refreshed sample 102r of the second layer 602.

[0179] In another embodiment, motion vector candidates for combinations affected by TMVP or sub-block TMVP candidates in the merge list are disabled, so that when sample 101u is replaced by upsampled BL sample 202r, the subsequent EL image does not depend on incorrect motion vectors on the decoder side.

[0180] In another embodiment, the same constraint applies to decoder-side motion vector refinement (DMVR) (i.e., motion vector refinement based on reference sample values), which will also result in cut-off artifacts in other cases.

[0181] Therefore, according to the embodiment, at least one of the following decoding concepts is disabled in order to decode one or more subsequent images 1012, ..., 101 contained in the refresh cycle (RP).n Decode the unrefreshed sub-region 101u:

[0182] • Temporal Motion Vector Prediction (TMVP)

[0183] Advanced Temporal Motion Vector Prediction (ATMVP)

[0184] • Candidates affected by TMVP, such as motion vector candidates in the merge list that are affected by TMVP or sub-block TMVP.

[0185] • Decoder-side motion vector refinement (DMVR)

[0186] According to another embodiment, at least one of the following decoding concepts is disabled in order to decode the unrefreshed sub-region 101u of the GDR image 103:

[0187] • Temporal Motion Vector Prediction (TMVP)

[0188] Advanced Temporal Motion Vector Prediction (ATMVP)

[0189] • Candidates affected by TMVP, such as motion vector candidates in the merge list that are affected by TMVP or sub-block TMVP.

[0190] • Decoder-side motion vector refinement (DMVR)

[0191] Furthermore, DMVR is disabled to allow for the processing of one or more subsequent images 1012, ..., 101 contained in the refresh cycle RP. n Decode the unrefreshed sub-region 101u.

[0192] In another embodiment, the layer with GDR decoding is decoded independently of other layers, and the expression of sample substitution in the bitstream can be implemented using the indicated other layers with sufficient content for sample substitution. Thus, according to this embodiment, the second layer 602 can be decoded independently of the first layer 601 or independently of any other layer, and wherein, if images 1011, 1012, ..., 101 are randomly accessed at GDR image 103... n If the second sequence 100, then the signal indicates that the unrefreshed sub-region 101u of the GDR image 103 of the second layer 602 will be predicted interlayer from the sample 202r of the first layer 601 or any predetermined (indicated) other layer with sufficient content.

[0193] In another embodiment, the constraints described above may take the form of bitstream requirements that depend on the identified refreshed and unrefreshed regions.

[0194] Furthermore, based on this aspect, it is proposed to provide corresponding encoders, decoders, methods for encoding, and methods for decoding.

[0195] According to an embodiment, a decoder is provided for decoding at least one image from a multi-layer scalable video data stream 600, the multi-layer scalable video data stream 600 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 601. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 602 n The second sequence 100. Images 1011, 1012, ..., 101 in the second layer 602. n The second sequence 100 may include at least one progressive decoder refresh (GDR) image 103 as the starting image and one or more subsequent images 1012, ..., 101 in the refresh cycle (RP). n The decoder can be configured to read from a multi-layer scalable video data stream 600 a signal notification carrying information about the possibility of inter-layer prediction from samples 202r of the first layer 601 for an unrefreshed sub-region 101u of the GDR image 103 of the second layer 602. The decoder is further configured to, in response to the signal notification,:

[0196] • In one or more subsequent images 1012, ..., 101 contained in a refresh cycle (RP) n Disable motion vector prediction or implement motion vector prediction non-temporally in the unrefreshed sub-region 101u, or

[0197] • Disable motion vector prediction or implement motion vector prediction non-temporally in the unrefreshed sub-region 101u of GDR image 103.

[0198] According to another embodiment, an encoder is provided for encoding at least one image into a multi-layer scalable video data stream 600, the multi-layer scalable video data stream 600 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 601. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 602 n The second sequence 100, wherein images 1011, 1012, ..., 101 in the second layer 602. n The second sequence 100 includes at least one Progressive Decoder Refresh (GDR) image 103 as the starting image and one or more subsequent images 1012, ..., 101 in the refresh cycle (RP). nThe encoder can be configured to write a signal notification carrying information about the possibility of interlayer prediction from refreshed samples 202r of the first layer 601 for an unrefreshed sub-region 101u of the GDR image 103 of the second layer 602. The signal notification can further carry the following information:

[0199] • In one or more subsequent images 1012, ..., 101 contained in a refresh cycle (RP) n In the unrefreshed sub-region 101u, motion vector prediction is disabled, or motion vector prediction is implemented non-temporally, or

[0200] • In the unrefreshed sub-region 101u of GDR image 103, motion vector prediction is disabled, or motion vector prediction is implemented non-temporally.

[0201] According to another embodiment, a method is provided for decoding at least one image from a multi-layer scalable video data stream 600, the multi-layer scalable video data stream 600 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 601. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 602 n The second sequence 100, wherein images 1011, 1012, ..., 101 in the second layer 602. n The second sequence 100 includes at least one Progressive Decoder Refresh (GDR) image 103 as the starting image and one or more subsequent images 1012, ..., 101 in the refresh cycle (RP). n The method includes the step of reading a signal notification from a multi-layer scalable video data stream 600, carrying information about the possibility of inter-layer prediction from refreshed samples 202r of the first layer 601 for an unrefreshed sub-region 101u of a GDR image 103 of the second layer 602. The method may also include the step of performing at least one of the following actions in response to the signal notification:

[0202] • In one or more subsequent images 1012, ..., 101 contained in a refresh cycle (RP) n In the unrefreshed sub-region 101u, disable motion vector prediction or implement motion vector prediction non-temporally, or

[0203] • In the unrefreshed sub-region 101u of GDR image 103, disable motion vector prediction or implement motion vector prediction non-temporally.

[0204] According to another embodiment, a method is provided for encoding at least one image into a multi-layer scalable video data stream 600, the multi-layer scalable video data stream 600 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 601. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 602 n The second sequence 100, wherein images 1011, 1012, ..., 101 in the second layer 602. n The second sequence 100 includes at least one Progressive Decoder Refresh (GDR) image 103 as the starting image and one or more subsequent images 1012, ..., 101 in the refresh cycle (RP). n The method includes writing a signal notification carrying information about the possibility of interlayer prediction from samples 202r of the first layer 601 for an unrefreshed sub-region 101u of the GDR image 103 of the second layer 602. The signal notification may further include information about the possibility of interlayer prediction from samples 202r of the first layer 601 for one or more subsequent images 1012, ..., 101 included in a refresh cycle (RP). n In the unrefreshed sub-region 101u of the GDR image 103, motion vector prediction is disabled or implemented non-temporally.

[0205] 3. Regarding layer dependencies in scalable video

[0206] Scalable video offers numerous benefits for mass streaming systems. For example, by using different versions of content and / or different resolutions, overall compression efficiency is significantly improved compared to several independent bitstreams with different versions and / or different resolutions (simulcast). This can lead to substantial savings in storage at servers and CDNs, thereby reducing the deployment cost of streaming services.

[0207] However, the decoding efficiency of the scalable bitstream transmitted to the terminal device is lower than the corresponding decoding efficiency if it were a single-layer bitstream. That is, inter-layer prediction incurs an efficiency loss due to some signaling overhead. Ideally, a joint optimization should be implemented that reduces the required storage capacity of several versions, i.e., the overall bit rate of all versions, without significantly increasing the size of each version compared to a single-layer version. This can be achieved by evaluating the described optimization problem and labeling it for each AU, regardless of whether inter-layer prediction is used.

[0208] In previous standards, this was done by using a discardable flag in a lower layer to mark those images as discardable. Alternatively, slices can also be marked as not using intermediate layers.

[0209] However, those mechanisms are not very flexible and fail to promote independence in terms of layer dependencies of some images in the stream (e.g., the ability to...). Figure 7 The example shown in the figure can efficiently discard layers to hinder the use of efficient file formats, where L0 TiD1 can be discarded when the user is interested in L1 because L1 TiD1 images do not use inter-layer prediction.

[0210] Figure 7 Non-limiting examples illustrating other aspects of the invention. A multi-layer scalable bitstream 700 may include a first layer (e.g., a base layer) 701 and a second layer (e.g., an enhancement layer) 702. The bitstream 700 may include more layers than the two exemplarily depicted layers 701, 702.

[0211] Each of the first layer 701 and the second layer 702 may include two or more time-related sublayers. For example, the first layer 701 may include a first time-related sublayer 701a and a second time-related sublayer 701b. The second layer 702 may include a first time-related sublayer 702a and a second time-related sublayer 702b.

[0212] As illustrated by arrow 710, some images contained in layers 701, 702 can be intra-frame decoded. For example, image 2012 of temporal sublayer 701b of layer 701 can be intra-frame decoded by referring to one or more images 2011, 2013 of different temporal sublayers 701b of the same layer 701.

[0213] Alternatively, as exemplarily indicated by arrow 711, some other images contained in layers 701, 702 can be inter-frame decoded. For example, image 1011 of temporal sublayer 702a of layer 702 can be inter-frame decoded by referring to image 2011 of temporal sublayer 701a of a different layer 701. For example, image 1011 of first temporal sublayer 702a of second layer (e.g., enhancement layer) 702 can be inter-frame decoded by referring to image 2011 of first temporal sublayer 701a of first layer (e.g., base layer) 701.

[0214] According to an embodiment, an image contained in a temporal sublayer of a layer can refer only to an image contained in temporal sublayers of different layers but having the same temporal sublayer hierarchy. For example, an image 2011 contained in a first temporal sublayer 702a of a second layer 702 can refer only to an image 1011 that is also contained in a first temporal sublayer 701a but belongs to the first layer 701. That is, first temporal sublayers 701a and 702a have the same temporal sublayer hierarchy.

[0215] exist Figure 7 In the non-limiting examples shown, only the images contained in the first sub-layer 702a of the second layer 702 (here: having odd indices) 1011, 1013, 1015, ..., 101 n This can be seen by referring to the image of the first sub-layer 701a of the first layer 701 (here: with odd indices) 2011, 2013, 2015, ..., 201. n It is then decoded frame by frame.

[0216] Any temporal sublayer that may not be used for inter-frame decoding can be discarded. In particular, any temporal sublayer with a higher temporal sublayer level than the predetermined temporal sublayer used for inter-frame decoding can be discarded.

[0217] Therefore, such as Figure 7 As shown in the non-limiting example, if the user is interested in the second layer 702, it can be seen that inter-frame decoding is performed only on the images contained in the first temporal sublayer 702a of the second layer 702. That is, only images 2011, 2013, ..., 201 contained in the first temporal sublayer 701a of the first layer 701 are referenced. n Therefore, only the first temporal sublayer 701a of the first layer 701 is of interest. Furthermore, images 2012, 2014, ..., 201 contained in the second temporal sublayer 701b of the first layer 701 are not referenced. n-1 Furthermore, the second temporal sublayer 701b of the first layer 701 is not of interest and can be discarded.

[0218] Typically, it can be seen that signaling overhead is more detrimental to higher temporal sublayers in bitstream 700. For example, an efficient way to implement the described features would be, for example, to encode every two images as dependent, or, for example, to encode every four images using inter-layer dependencies (Note: if an image is decoded with inter-layer dependencies, it can be called dependent; and if an image is decoded without inter-layer dependencies, it can be called independent).

[0219] Therefore, in the embodiments, layer dependency further indicates whether a temporal sublayer within a dependent layer depends on a lower layer.

[0220] This can be done, for example, in a VPS (or SPS) as shown below:

[0221]

[0222]

[0223] A value of 1 for `vps_sub_layer_independent_flag[i][j]` indicates that the (temporal) sublayer with index `j` contained within the layer with index `i` does not use inter-layer prediction. A value of 0 for `vps_sub_layer_independent_flag[i][j]` indicates that the (temporal) sublayer with index `j` contained within the layer with index `i` can use inter-layer prediction.

[0224] Therefore, according to an embodiment, a multi-layer scalable video data stream 700 is proposed, which includes images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 701. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 702 n The second sequence 100, each of the first layer 701 and the second layer 702 includes multiple temporal sublayers 701a, 701b; 702a, 702b. The scalable video data stream 700 may include signal notifications (e.g., vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers 702a, 702b of the second layer (e.g., enhancement layer) 702 can be decoded by inter-layer prediction.

[0225] According to another embodiment, a corresponding decoder is provided for decoding at least one image from a multi-layer scalable video data stream 700, the multi-layer scalable video data stream 700 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 701. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 702 nThe second sequence 100, each of the first layer 701 and the second layer 702 includes multiple temporal sublayers 701a, 701b; 702a, 702b. The decoder can be configured to decode one or more of the temporal sublayers 701a, 701b; 702a, 702b using inter-layer prediction based on signal notifications (e.g., vps_sub_layer_independent_flag[i][j]) derived from the scalable video data stream 700, the signal notifications indicating which temporal sublayers 702a, 702b of the second layer 702 will be decoded via inter-layer prediction.

[0226] According to another embodiment, an encoder is provided for encoding at least one image into a multi-layer scalable video data stream 700, the multi-layer scalable video data stream 700 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 701. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 702 n The second sequence 100, each of the first layer 701 and the second layer 702 includes multiple temporal sublayers 701a, 701b; 702a, 702b. The encoder can be configured to encode one or more of the temporal sublayers 701a, 701b; 702a, 702b by using inter-layer prediction, and is configured to write a signal notification (e.g., vps_sub_layer_independent_flag[i][j]) to the scalable video data stream 700, the signal notification indicating which temporal sublayers 702a, 702b of the second layer 702 are decoded by inter-layer prediction.

[0227] According to another embodiment, a corresponding method is provided for decoding at least one image from a multi-layer scalable video data stream 700, the multi-layer scalable video data stream 700 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 701. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 702 nThe second sequence 100, each of the first layer 701 and the second layer 702 includes multiple temporal sublayers 701a, 701b; 702a, 702b. The method includes the step of decoding one or more of the temporal sublayers 702a, 702b using inter-layer prediction based on signaling notifications derived from a scalable video data stream 700, the signaling notifications (e.g., vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers 702a, 702b of the second layer 702 will be decoded via inter-layer prediction.

[0228] According to another embodiment, a method is provided for encoding at least one image into a corresponding multi-layer scalable video data stream 700, the multi-layer scalable video data stream 700 including images 2011, 2012, ..., 201 in a first layer (e.g., a base layer) 701. n Images 1011, 1012, ..., 101 in the first sequence 200 and the second layer (e.g., enhancement layer) 702 n The second sequence 100, each of the first layer 701 and the second layer 702, includes multiple temporal sublayers 701a, 701b; 702a, 702b. The method includes the step of encoding one or more of the temporal sublayers 702a, 702b using inter-layer prediction based on signaling notifications derived from a scalable video data stream 700, the signaling notifications (e.g., vps_sub_layer_independent_flag[i][j]) indicating which temporal sublayers 702a, 702b of the second layer 702 are decoded via inter-layer prediction.

[0229] As previously mentioned, a temporal sublayer may include a temporal sublayer hierarchy, such as a first temporal sublayer, a second temporal sublayer, a third temporal sublayer, and so on. The temporal sublayer hierarchy of a layer can be indicated by means of temporal identifiers (e.g., syntax elements) in the signal notifications mentioned above contained in the video data stream 700.

[0230] Further references are provided above. Figure 7This explains why uninteresting temporal sublayers above a predetermined temporal sublayer level can be discarded (because they are not used for inter-frame prediction). Therefore, there may be a threshold indicating the level of temporal sublayers that can be used for inter-frame prediction. This means that temporal sublayers above and below this threshold can be used for inter-frame prediction, while temporal sublayers above this threshold cannot. The latter can be discarded. This threshold can be indicated by a predetermined temporal sublayer. For example, the video data stream 700 can signal that inter-frame prediction can be used only for the second temporal sublayer up to the specified level. Thus, the second temporal sublayer forms the threshold for inter-frame prediction. Therefore, the first and second temporal sublayers can be used for inter-frame prediction, while any additional temporal sublayers with a temporal sublayer level above the predetermined second temporal sublayer (threshold) can be discarded.

[0231] The aforementioned temporal identifier can indicate a threshold, i.e., a temporal sub-layer level that can be reached using inter-layer prediction. Conversely, a temporal identifier can indicate a threshold, i.e., a temporal sub-layer level that can be reached without using inter-layer prediction.

[0232] Therefore, according to the embodiment, the signal notification in the video data stream 700 may include a predetermined time identifier (threshold), which can be used to decode the time sub-layers 702a and 702b of the second layer 702 without inter-layer prediction.

[0233] This means that the temporal sub-layers 702a and 702b of the second layer 702, which include temporal identifiers with values ​​higher than a predetermined temporal identifier (threshold), are decoded without inter-layer prediction. Figure 7 In the example shown, the predetermined temporal identifier references the first temporal sublayer 702a because only the first temporal sublayer 702a is decoded via inter-layer prediction. Therefore, all temporal sublayers above the first temporal sublayer 702a (here, the second temporal sublayer 702b) and any higher temporal sublayers (if they exist) are decoded without inter-layer prediction.

[0234] Furthermore, the second layer 702, including those temporal sublayers 702a, 702b with temporal identifiers having values ​​higher than or lower than a predetermined temporal identifier (threshold), utilizes inter-layer prediction for decoding. Figure 7 In the example shown, the predetermined temporal identifier is referenced to the first temporal sublayer 702a because only the first temporal sublayer 702a is decoded by inter-layer prediction.

[0235] Therefore, if a signal notification (e.g., vps_sub_layer_independent_flag[i][j]) can instruct the temporal sublayer 702b of the second layer 702 not to use the temporal sublayer 701b of the lower layer 701 for interlayer prediction, then the temporal sublayer 701b of the lower layer 701 can be marked as discardable from or dropped from the multi-layer scalable video data stream 700.

[0236] In another embodiment, an enable flag is included in the VPS to indicate whether the sub_layer_independent_flag is included in the VPS or whether all sublayers depend on the lower layer by default.

[0237] Therefore, according to the embodiment, the multi-layer scalable video data stream 700 may also include syntax elements (e.g., enable_flag in VPS) for indicating whether signal notifications (e.g., vps_sub_layer_independent_flag[i][j]) are included in the multi-layer scalable video data stream 700. If not included, all temporal sub-layers 702a and 702b of the second layer 702 may depend on the lower layer 701 by default.

[0238] As mentioned above, an efficient way to achieve the described features would be, for example, to encode every two images as dependencies or, for example, to encode every four images using inter-layer dependencies.

[0239] Therefore, according to the embodiment, the encoder can be configured to process consecutive images (e.g., every two images) 1012, 1014, ..., 101 of the second sequence 100 of images. n-1 The first predetermined row is encoded as dependent, or the second predetermined row of the second sequence 100 of consecutive images (e.g., every four images) is encoded as using inter-layer dependencies.

[0240] Although some aspects have been described in the context of the device, these aspects clearly also represent descriptions of the corresponding methods, where blocks or devices correspond to method steps or features of method steps. Similarly, aspects described in the context of method steps also represent descriptions of corresponding blocks, items, or features of the corresponding device.

[0241] Some or all of the method steps can be performed by (or using) hardware devices such as, for example, microprocessors, programmable computers, or electronic circuits. In some embodiments, one or more of the most important method steps can be performed by such devices.

[0242] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware, software, or at least partially in hardware or at least partially in software. Implementations can be carried out using digital storage media (e.g., floppy disks, DVDs, Blu-ray discs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory) having electronically readable control signals stored thereon that cooperate with (or are capable of cooperating with) a programmable computer system to perform the corresponding methods. Therefore, the digital storage medium can be computer-readable.

[0243] Some embodiments of the invention include a data carrier having electronically readable control signals that are capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0244] Generally, embodiments of the present invention can be implemented as a computer program product having program code that is operational so that, when the computer program product is run on a computer, one of the methods is performed. The program code may, for example, be stored on a machine-readable medium.

[0245] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0246] In other words, therefore, an embodiment of the inventive method is a computer program having program code for performing one of the methods described herein when the computer program is run on a computer.

[0247] Therefore, another embodiment of the inventive method is a data carrier (or digital storage medium or computer-readable medium) including a computer program recorded thereon for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium is typically tangible and / or non-transitory.

[0248] Therefore, another embodiment of the inventive method is a data stream or signal sequence representing a computer program for performing one of the methods described herein. This data stream or signal sequence can, for example, be configured to be transferred via a data communication connection (e.g., via the Internet).

[0249] Other embodiments include processing components, such as computers or programmable logic devices, configured or adapted to perform one of the methods described herein.

[0250] Another embodiment includes a computer having a computer program installed thereon for performing one of the methods described herein.

[0251] Further embodiments of the invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program for carrying out one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0252] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to implement some or all of the functionality of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to implement one of the methods described herein. Generally, the method is preferably implemented by any hardware device.

[0253] The device described in this article can be implemented using hardware devices, a computer, or a combination of hardware devices and a computer.

[0254] The methods described in this article can be implemented using hardware devices, computers, or a combination of hardware devices and computers.

[0255] While this disclosure has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments, as well as other embodiments of this disclosure, will become apparent to those skilled in the art upon reference to this description. Therefore, the appended claims are intended to encompass any such modifications or embodiments.

Claims

1. A decoder for decoding at least one image from a multi-layer scalable video data stream (700), the multi-layer scalable video data stream (700) including images (2011, 2012, ..., 2013) in a first layer (701). n The first sequence (200) and the second layer (702) of the images (1011, 1012, ..., 101) n The second sequence (100) of ) is, where, Each of the first layer (701) and the second layer (702) comprises a plurality of temporal sub-layers (701a, 701b, 702a, 702b), wherein the decoder is configured to use inter-layer prediction for decoding one or more of the plurality of temporal sub-layers (702a, 702b) of the second layer (702) based on a signaling derived from the scalable video data stream (700), the signaling indicating which temporal sub-layers (702a, 702b) of the second layer (702) are to be decoded using inter-layer prediction, wherein the signaling comprises a predetermined temporal identifier to indicate which temporal sub-layers (702a, 702b) of the second layer (702) are to be decoded using inter-layer prediction, wherein the temporal sub-layer (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier is decoded without using inter-layer prediction, and wherein, if the decoder derives from the signaling that a temporal sub-layer (702b) of the second layer (702) does not use a temporal sub-layer (701b) of the first layer (701) for inter-layer prediction, the decoder is configured to discard and not use the temporal sub-layer (701b) of the first layer (701) when decoding the temporal sub-layer (702b) of the second layer (702), wherein the second layer (702) is higher than the first layer (701), wherein the multi-layer scalable video data stream further comprises a syntax element to indicate whether the signaling is included in the multi-layer scalable video data stream (700) or whether all temporal sub-layers (702a, 702b) of the second layer (702) are to be dependent on the first layer (701) by default.

2. The decoder of claim 1, wherein the temporal sub-layer (702a) of the second layer (702) comprising a temporal identifier having a value equal to or lower than the predetermined temporal identifier is decoded using inter-layer prediction.

3. An encoder for encoding at least one picture into a multi-layer scalable video data stream (700) comprising a first sequence (200) of pictures (2011, 2012,..., 201 n) in a first layer (701) and a second sequence (100) of pictures (1011, 1012,..., 101 m) in a second layer (702), wherein n the first sequence (200) of pictures (2011, 2012,..., 201 n) in the first layer (701) is encoded in a first picture order count (POC) order and the second sequence (100) of pictures (1011, 1012,..., 101 m) in the second layer (702) is encoded in a second POC order, n the first POC order is different from the second POC order, and Each of the first layer (701) and the second layer (702) comprises a plurality of temporal sub-layers (701a, 701b, 702a, 702b), wherein the encoder is configured to encode one or more of the plurality of temporal sub-layers (702a, 702b) of the second layer (702) using inter-layer prediction and to signal into the scalable video data stream (700), the signaling indicating which temporal sub-layers (702a, 702b) of the second layer (702) are encoded using inter-layer prediction, wherein the signaling comprises a predetermined temporal identifier to indicate which temporal sub-layers (702a, 702b) of the second layer (702) are to be encoded using inter-layer prediction, wherein the temporal sub-layer (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier is encoded without using inter-layer prediction, and wherein, if the encoder signals that a temporal sub-layer (702b) of the second layer (702) does not use a temporal sub-layer (701b) of the first layer (701) for inter-layer prediction, the encoder is configured to encode the temporal sub-layer (702b) of the second layer (702) without using the temporal sub-layer (701b) of the first layer (701), wherein the second layer (702) is higher than the first layer (701), wherein the multi-layer scalable video data stream further comprises a syntax element to indicate whether the signaling is included in the multi-layer scalable video data stream (700) or whether all temporal sub-layers (702a, 702b) of the second layer (702) are to be dependent on the first layer (701) by default. wherein the temporal sub-layers (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier are encoded without using inter-layer prediction, and wherein, if the encoder determines that the temporal sub-layers (702b) of the second layer (702) do not use the temporal sub-layers (701b) of the first layer (701) for inter-layer prediction, the encoder is configured to discard and not use the temporal sub-layers (701b) of the first layer (701) when encoding the temporal sub-layers (702b) of the second layer (702), wherein the second layer (702) is higher than the first layer (701), wherein the multi-layer scalable video data stream further comprises a syntax element for indicating whether the signaling is included in the multi-layer scalable video data stream (700) or whether all temporal sub-layers (702a, 702b) of the second layer (702) are by default dependent on the first layer (701).

4. The encoder of claim 3, wherein the temporal sub-layers (702a) of the second layer (702) comprising a temporal identifier having a value equal to or lower than the predetermined temporal identifier are encoded using inter-layer prediction.

5. A method for decoding at least one image from a multi-layer scalable video data stream (700), the multi-layer scalable video data stream (700) including images (2011, 2012, ..., 2013) in a first layer (701). n The first sequence (200) and the second layer (702) of the images (1011, 1012, ..., 101) n The second sequence (100) of ) is, where, each of the first layer (701) and the second layer (702) comprises a plurality of temporal sub-layers (701a, 701b, 702a, 702b), wherein the method comprises the steps of: decoding one or more of the plurality of temporal sub-layers (702a, 702b) of the second layer (702) using inter-layer prediction based on a signaling derived from the scalable video data stream (700), the signaling indicating which temporal sub-layers (702a, 702b) of the second layer (702) are to be decoded using inter-layer prediction, wherein the signaling comprises a predetermined temporal identifier to indicate which temporal sub-layers (702a, 702b) of the second layer (702) are to be decoded using inter-layer prediction, wherein the temporal sub-layers (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier are decoded without using inter-layer prediction, and wherein, if the signaling indicates that the temporal sub-layers (702b) of the second layer (702) do not use the temporal sub-layers (701b) of the first layer (701) for inter-layer prediction, the method further comprises the step of discarding and not using the temporal sub-layers (701b) of the first layer (701) when decoding the temporal sub-layers (702b) of the second layer (702), wherein the second layer (702) is higher than the first layer (701), wherein the temporal sub-layers (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier are decoded without using inter-layer prediction, and wherein, if the signaling indicates that the temporal sub-layers (702b) of the second layer (702) do not use the temporal sub-layers (701b) of the first layer (701) for inter-layer prediction, the method further comprises the step of discarding and not using the temporal sub-layers (701b) of the first layer (701) when decoding the temporal sub-layers (702b) of the second layer (702), wherein the second layer (702) is higher than the first layer (701), wherein the multi-layer scalable video data stream further comprises a syntax element for indicating whether the signaling is included in the multi-layer scalable video data stream (700) or whether all temporal sub-layers (702a, 702b) of the second layer (702) by default depend on the first layer (701).

6. A method of encoding at least one picture into a multi-layer scalable video data stream (700) comprising a first sequence (200) of pictures (2011, 2012,..., 201 n) in a first layer (701) and a second sequence (100) of pictures (1011, 1012,..., 101 m) in a second layer (702), wherein, n the first sequence (200) of pictures (2011, 2012,..., 201 n) in the first layer (701) is encoded in a first picture order count (POC) order and the second sequence (100) of pictures (1011, 1012,..., 101 m) in the second layer (702) is encoded in a second POC order, n the first POC order is different from the second POC order, and Each of the first layer (701) and the second layer (702) comprises a plurality of temporal sub-layers (701a, 701b; 702a, 702b), wherein the method comprises the steps of: encoding one or more of the plurality of temporal sub-layers (702a, 702b) of the second layer (702) by using inter-layer prediction and signaling into the scalable video data stream (700) which temporal sub-layers (702a, 702b) of the second layer (702) are encoded using inter-layer prediction, wherein the signaling comprises a predetermined temporal identifier to indicate which temporal sub-layers (702a, 702b) of the second layer (702) are decoded using inter-layer prediction, wherein the temporal sub-layer (702b) of the second layer (702) comprising a temporal identifier having a value higher than the predetermined temporal identifier is decoded without using inter-layer prediction, and wherein, if the signaling indicates that a temporal sub-layer (702b) of the second layer (702) does not use a temporal sub-layer (701b) of the first layer (701) for inter-layer prediction, the method further comprises the step of discarding the temporal sub-layer (701b) of the first layer (701) and not using the temporal sub-layer (701b) of the first layer (701) when encoding the temporal sub-layer (702b) of the second layer (702), wherein the second layer (702) is higher than the first layer (701), wherein the multi-layer scalable video data stream further comprises a syntax element for indicating whether the signaling is included in the multi-layer scalable video data stream (700) or whether all temporal sub-layers (702a, 702b) of the second layer (702) by default depend on the first layer (701).

Images