Coding concept allowing parallel processing, transport demultiplexer and video bitstream

Abstract

To provide a coding concept which makes more effect low delay coding possible in a parallel processing environment, a transport demultiplex concept, a decoder and an encoder for providing video bitstreams.SOLUTION: In a decoder, a raw byte sequence payload describing a picture in slices, wavefront parallel processing (WPP) substreams 92 or tiles and coded using context-adaptive binary arithmetic coding is subdivided or chopped into tranches 98a and 98b with continuation of the context-adaptive binary arithmetic coding probability adaptation across tranche boundaries 92. Tranche boundaries additionally introduced within slices, WPP substreams or tiles do not lead to a reduction in the entropy coding efficiency of these elements. However, the tranches are smaller than the original slices, WPP substreams or tiles and accordingly they may be transmitted earlier, i.e. with lower delay, than the un-chopped original entities, i.e. slices, WPP substreams or tiles.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to coding concepts that allow parallel processing, for example in the evolving High Efficiency Video Coding (HEVC), transport demultiplexers and video bitstreams. [Background technology]

[0002] Parallelization of the encoder and decoder is of great importance not only due to the increased processing demands of the HEVC standard, but also due to the expected increase in video resolution. Multi-core architectures are becoming available in a variety of modern electronic devices. Thus, an efficient method is needed to enable the utilization of multiple multi-core architectures.

[0003] The encoding or decoding of the largest coding unit (LCU) occurs in a raster scan, whereby the CABAC (Context based Adaptive Binary Arithmetic Coding) probabilities are adapted to the characteristics of each image. Spatial dependencies exist between adjacent LCUs. For different components, such as motion vectors, prediction, intra-prediction, and others, each LCU depends on its left, above, top-left, and top-right neighboring LCUs. To allow for parallelization of the decoding, these dependencies generally need to be broken or are broken in modern applications.

[0004] Several concepts of parallelization have been proposed, namely wave-field processing using entropy slicing [3], wave-field parallel processing (WPP) operations using substreams [2][4],

[11] , or tiling [5]. The latter does not necessarily have to be combined with wave-field processing to allow parallelization at the decoder or encoder. In this respect, tiles are similar to WPP substreams. Our initial motivation for further investigation of the entropy slicing concept is to implement techniques that reduce the coding efficiency loss and ease the burden on the bitstream for encoder and decoder parallelization methods.

[0005] To provide a better understanding, one can first have a look at the structure of H.264 / AVC[1], in particular the use of LCUs.

[0006] An H.264 / AVC coded video sequence consists of a series of access units collected in a Network Abstruction Layer (NAL) unit stream, which use only one sequence parameter set. Each video sequence can be decoded independently. A coded sequence consists of a series of coded pictures. A coded frame can be a full frame or a single field. Each picture is divided into fixed-size macroblocks (in HEVC [5]: LCUs). Several macroblocks or LCUs can be merged together into a slice. An image is thus a collection of one or more slices. The purpose of this data separation is to allow independent decoding of samples of an area of ​​an image, which is represented by a slice, without using data from other slices.

[0007] A technique often referred to as "entropy slicing" [3] is the division of a conventional slice into additional sub-slices. Specifically, it refers to the division of the entropy coded data of a single slice. The arrangement of entropy slices in a slice can be of different kinds. The simplest is to use each row of LCU(s) / macroblocks in a frame as one entropy slice. Alternatively, columns or separate regions can be used as entropy slices, which can even be interrupted by each other and switched with each other, e.g., slice 1 in FIG. 1.

[0008] The obvious purpose of the entropy slicing concept is to be able to take advantage of parallel CPU / GPU and multi-core architectures to improve the decoding process time, i.e. to speed up the process. Although the current slices can be divided into partitions that can be analyzed and reconstructed without regard to other slice data, and some advantages can be achieved by the entropy slicing method, this leads to some disadvantages.

[0009] As proposed in [2],

[10] ,

[11] and partially incorporated in [5], the entropy slicing concept has been further extended to substream wavefront processing (WPP), where a substream iterative scheme is defined, which has improved line-by-line entropy state initialization compared to entropy slicing.

[0010] The tile concept allows separation of image information to be coded while each title has its own raster scan order. Tiles are defined by a common structure, which is repeated within a frame. Tiles can also have specific column widths and line heights for LCUs or CUs. Titles can also be coded independently, or in a way that does not require joint processing with other tiles, so that a decoder thread can process tiles of an access unit completely, or at least for some coding operation steps in an independent manner, i.e., entropy coding and transform coding.

[0011] Thus, in the latter case, e.g., for the filter stage of the HEVC codec, tiles largely enable the tile encoder and decoder to operate fully or partially independently, as well as running in parallel.

[0012] To make full use of parallelism techniques in the capture, encoding, transmission, decoding, and presentation chain of a video communication system, or similar systems, the transfer and access of data between communication participants is a significant and time-consuming step due to the full end-to-end delay injection. This is especially problematic when using parallelism techniques such as tiles, substreams, or entropy slicing.

[0013] The WPP substream data approach means that there is no data locality when the coded data of a partition is processed, i.e. a single thread decoding an access unit must jump over a potentially large portion of memory to access the data of the next WPP substream line. In order to operate in a fully parallelized manner, a multi-threaded decoding system must wait for the transmission of a particular data, i.e. a WPP substream, and utilizes wavefront processing.

[0014] In video streaming, allowing higher resolutions (Full HD, Quad HD, etc.) leads to a larger amount of data to be transmitted. For time-sensitive scenarios, so-called low-latency uses, e.g. videoconferencing (<145 ms), or gaming applications (<40 ms), very low end-to-end delays are required. Thus, transmission time becomes a critical factor. Consider an ADSL upload link for a videoconferencing application. Here, so-called random access points of the stream, which usually refer to I-frames, would be candidates for creating bottlenecks during transmission.

[0015] HEVC allows so-called wavefront processing as well as tile processing at the encoder as well as on the decoder side. This is enabled using entropy slices, WPP substreams or a combination of both. Parallel processing is also enabled by parallel tile encoding and decoding.

[0016] In the "non-parallelized targeting" case, the data for an entire slice is delivered at once, so the last CU of the slice is accessible by the decoder when it is sent. If you have a single-threaded decoder, this is not a problem.

[0017] In the case of multi-threading, where multiple CPUs or cores may be used, the decoding process however would like to start as soon as the encoded data arrives at the wavefront decoder or tile decoder thread.

[0018] Thus, it would be advantageous to have a concept at hand that allows reducing the coding delay in a parallel processing environment without significantly reducing coding efficiency. Summary of the Invention [Problem to be solved by the invention]

[0019] It is therefore an object of the present invention to provide an encoding concept, a transmission demultiplexing concept and a video bitstream that allows more efficient, low-delay encoding in a parallel processing environment. [Means for solving the problem]

[0020] This object is achieved by the subject matter of the attached independent claims.

[0021] According to a first aspect of the present application, a raw byte sequence payload representing an image in a slice, WPP substream or tile and coded using context-adaptive binary arithmetic coding is subdivided or truncated into slices across a tranche boundary while continuing the context-adaptive binary arithmetic coding probability adaptation. With this approach, the additionally introduced slice boundaries within a slice, WPP substream or tile do not result in a decrease in the entropy coding efficiency of these elements. However, on the other hand, the slices are smaller than the original slice, WPP substream or tile, and therefore they can be transmitted earlier, i.e. with less delay, than the original non-truncated entities, i.e. slices, WPP substreams or tiles.

[0022] According to another aspect that can be combined with the first aspect, a sub-stream marker NAL unit is used within a sequence of NAL units of a video bitstream such that a transport demultiplexer can assign data of a slice in the NAL unit to a corresponding sub-stream or tile and provide a multi-threaded decoder on the corresponding sub-streams or tiles in parallel.

[0023] Advantageous embodiments are the subject matter of the dependent claims. Further preferred embodiments of the invention are explained in more detail below with reference to the figures. [Brief description of the drawings]

[0024] [Figure 1]FIG. 1 shows a schematic diagram illustrating possible compositions of entropy slices. [Diagram 2] FIG. 2 shows a schematic diagram illustrating three tiles spanning three slices. [Diagram 3] FIG. 3 shows a schematic diagram illustrating an example of segment interleaving of four variable length segment cycle interleaving schemes. [Figure 4] FIG. 4 shows a schematic diagram illustrating the encoding, segmentation, interleaving and decoding of entropy slice data. [Diagram 5] FIG. 5 shows a schematic diagram illustrating an example of a four variable length slice interleaving scheme that always uses marker codes and periodically interleaves the actual slice data over multiple NAL units. The marker codes are used even when no partitions are present. This can be further enhanced by using a slice identifier, following the marker, to indicate the slice number. This reduces the need to always send markers, as is required for the periodic mode. [Figure 6A] FIG. 6A shows a pseudocode table illustrating NAL unit syntax. [Figure 6B] FIG. 6B shows a pseudocode table illustrating NAL unit syntax. [Figure 7] FIG. 7 shows a table of pseudocode illustrating the sequence parameter set syntax. [Figure 8] FIG. 8 shows a pseudocode table illustrating the low delay slice layer RBSP syntax. [Figure 9] FIG. 9 shows a pseudocode table illustrating slice header syntax. [Figure 10] FIG. 10 shows a pseudocode table illustrating the sub-stream marker syntax. [Figure 11] FIG. 11 shows a schematic diagram illustrating an example for a simple encapsulation of entropy slice data (AF is an MPEG-2 TS Adaptation Field). [Figure 12] FIG. 12 shows a schematic diagram illustrating another embodiment for single Elementary Stream (ES) encapsulation of entropy slice data. [Figure 13] FIG. 13 shows a diagram illustrating another example for packed multi-ES encapsulation of entropy slice data. [Figure 14] FIG. 14 shows a schematic block diagram illustrating a transport demultiplexer for a single ES. [Figure 15] FIG. 15 shows a schematic block diagram illustrating a transport demultiplexer for multiple ESs. [Figure 16] FIG. 16 shows a schematic block diagram illustrating the encoder. [Figure 17] FIG. 17 shows a schematic block diagram illustrating the decoder. [Figure 18] FIG. 18 shows a flow chart of the steps performed by the decoder. [Figure 19] FIG. 19 shows a schematic diagram illustrating an embodiment for multi-ES using RTP. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] To save time, parallel decoder threads can be started and the data for a frame can be finalized; the embodiments described below use segmentation of data, structuring for parallelization, such as splitting the data into one or more tiles or one or more WPP substreams of data into smaller slices using a low latency interleaving approach.

[0026] Thus, the encoder may deliver data to the decoder via a transmission path from the encoder to the decoder in pieces corresponding to at least a byte-aligned portion or tile or portions thereof of a particular set of LCUs or sub-streams.

[0027] Because the slices may be smaller than a complete WPP sub-stream or tile and / or fit into the practical maximum transfer unit (MTU) of the transmission path, multiple WPP streams or slices can be placed in a transfer unit between the encoder and decoder before finalization of the complete access unit, and decoding on the decoder side can begin much earlier than when using sequential transmission of complete WPP sub-streams or tiles of an access unit.

[0028] This obviously results in a faster transmission of the slices in the decoder and an earlier start of the parallel decoding process. The method can also be applied through frame boundaries, for example in wavefront methods, where the slice(s) or entropy slice(s) of the next frame can already be decoded based on the knowledge that the necessary information for decoding the entropy slice(s) of the next frame depends on the availability of interframe references. Those already decodable data of the following frames in the decoding order can be derived from the maximum allowed / signaled motion vector length or additional information in the stream indicating the dependency of the data parts on the previous frame(s) or a fixed reference scheme used signaled in a fixed position in the order, such as a parameter set.

[0029] An image can be coded by Largest Coding Unit (LCU)-one entropy slice per row(s), or with WPP substreams, or even a combination as one WPP substream per row can be further included in a separate entropy slice. This kind of data structure is necessary to exploit wavefront processing techniques at the decoder side. Alternatively, tiles can be used to allow parallel processing.

[0030] During the encoding process, the bitstream of each slice containing the WPP stream or tile data can be split into variable-sized pieces to fit the maximum transmission unit size between the encoder and decoder. The resulting pieces are then interleaved and put into MTU-sized packets that can be passed along for transmission.

[0031] To allow processing at the decoder side, a marker code can be inserted before or after each slice. A suitable marker code for HEVC can be "0x00 00 02", which even passes the start code emulation prevention. After receiving a packet containing multiple slices, the receiver or decoder can analyze the bitstream actually contained during the start code emulation prevention process so that no additional analysis steps are required. For example, there can be two modes for slice identification. There can always be a cyclic sequence of slices, starting from the slice with tranche_id (slice identifier) ​​equal to 1, up to the slice with tranche_id equal to n. This allows safe data signaling in the second general scheme. An alternative scheme could be a specific header following the marker indicating the tranche_id, for example as an 8-bit value.

[0032] Deinterleaving the interleaved slice data can be applied based on knowledge of the number of slices per packet, which may be a NAL unit packet. Thus, there can also be a mapping of WPP substreams or tiles to slices. This mapping can be derived implicitly from the number of tiles / number of WPP substreams, or can be directly signaled in the SPS. The mapping is important for the deinterleaving process so that the data for a particular WPP substream or tile can be identified and made available to the wavefront or parallel decoder thread responsible for decoding the WPP substream or tile in question.

[0033] A low_delay_flag can be present in the NAL unit header to inform the decoder about the use of an interleaving scheme for low-delay encapsulation.

[0034] Other modes may involve interleaving and deinterleaving on the transport layer, i.e. outside the decoding process at the RTP[8][9]

[13] or MPEG-2 Transport Stream[7] layer:

[0035] Thus, a header can be placed in front of the packet and the presence of a segment can be indicated by a flag containing size information in bytes for the current segment. Since the transport layer is separated from the decoding process, there may be no need to accumulate marker codes, since the additional information of the transport layer always needs to be removed before passing those data to the decoder. The transport layer then also reorders the data to send the bitstream to the decoder.

[0036] A variable length header can be used for an extra multiplexing layer, which can be part of the codec or can be introduced before the actual Raw Byte Sequence Data (RBSP) access of the decoder. One header scheme can be seen in Fig. 3. However, a header as well as the indicator can also be directly in front of each slice indicating its length. As already mentioned above, there is still a need to map the indicator into the bitstream structure.

[0037] The slice size may be a constant size, for example x bytes per slice. This results in a simple multiplexing scheme, for example as shown in FIG.

[0038] The constant size of the segments can pose a problem at the end of the bitstream due to their variable length.

[0039] There are two general solutions that are feasible: the first one is to generate periodic x-byte segments (usually the bitstream representation of the slices is byte-aligned) and control the consumption of each byte by each decoder engine, i.e. the decoder detects the completion of the entropy slice or indicates a marker code.

[0040] As shown, if the segments in the header are of variable length, the second method signals the segment length.

[0041] The size of the segments and the interleaving modes can be signaled in one Supplemental Enhancement Information (SEI)-message or in the Sequence Parameter Set (SPS).

[0042] The transmission scheme is shown in FIG.

[0043] Another interesting method is to use a finalization or marker code at the end of a set of segments of a packet, e.g. a NAL or slice packet. In this case, variable length segments are possible and therefore a complete analysis of the bitstream is required. This additional analysis method for multiplexing can be combined with the start code emulation prevention analysis to limit memory accesses now and is necessary as a first step before accessing the RBSP data contained in the NAL unit. This kind of marker scheme is shown in Figure 5.

[0044] The idea here is to split higher level structures, e.g. real slices, entropy slices and the like, into contained lower level data structures, e.g. WPP substreams, tiles, etc., in an interleaved manner while interleaving data into slices. These slices, each belonging to a lower level structure, e.g. a specific WPP substream or tile, are interleaved in low-delay packets, which may be specific NAL units, or even slices, additionally signaled by a low-delay interleaving flag, or a lightweight slice header indicating the low-delay interleaving approach by a flag or slice type, as shown in the figure as "NAL unit #1", so that the decoder informs the "single" threaded decoder to apply the reordering function, which uses sequential processing of the slices in the original / deinterleaved order in the decoder. To split the data of the real slice as interleaved pieces over multiple packets to obtain low-delay characteristics, the transport layer may fragment the NAL units containing the low-delay interleaved data into network packets of maximum MTU (Maximum Transmission Unit) size. The fragmentation of the real slice data into multiple NAL units may also be directly applied by the coding layer, and therefore, it is necessary to signal such a type of NAL unit that contains the continuation of the slice, as shown in "NAL unit #2" in Figure 5. In order to detect the finalization of the interleaved data in multiple packets such as NAL units, there may be a need for a specific finalization code or a flag indicating completion in the slice or NAL header, as also shown in "NAL unit #2" in the figure.

[0045] In case of NAL packet loss, there is also a need to detect the loss. This can apply to additional information in the header, e.g., a lightweight slice header, e.g., the first MB of the included slice, or only in the specific slice #1. If there is information, e.g., offsets for the actual size of the WPP substream or slice, then anyone can use these size values ​​(offset values ​​for a specific WPP substream or tile) to do a sanity check after receiving the NAL unit with the finalization code and the previous NAL unit.

[0046] That is, as described, the slices may be packetized into packets 300 in such a way that each packet 300 is composed of one slice T# of each WPP substream or tile of the image arranged in a defined order # among the WPP substreams or tiles or a subset of the WPP substreams or tiles of the image (because, for example, a certain WPP substream or tile has already been completely carried by a previous packet), each packet having a header 302 with information indicating the position and / or length of the slice T# packed in the respective packet 300, or a marker 304 separating the slices T# from each other within the respective packet 300, and the decoder may be configured to use the information composed by the header 302 or marker 304 to access the slices in the packet upon receiving the raw byte sequence payload. The packet 300a with the first slice of the WPP substream or tile of the image - according to the defined order among the WPP substreams or tiles - may have a low delay characteristic indicator 306. And, a packet 300b having a second or subsequent WPP substream or tile of an image - according to a defined order among the WPP substreams or tiles - having slice T♯ may have a continuity indicator 308. The packet 300 may be a NAL unit or slice.

[0047] Below, an example is provided for signaling syntax and semantics for low-delay interleaving into slices.

[0048] Nevertheless, as mentioned above, splitting the slice data, such as WPP substream or tile data, can also be applied at or below the slice level.

[0049] Now, a method is presented, which can be combined with the analysis for start code emulation prevention to reduce additional processing steps. Thus, interleaving is applied at the RBSP level of the HEVC codec.

[0050] A slice can be considered as a division of the split RBSP data into sections to be interleaved in the NAL unit payload for low latency data access. Finalization of a slice can be indicated by the code 0x000002 and can be followed by an 8-bit slice identifier tranche_id. The slices can be interleaved in a cyclic manner, so that the slice end code is not followed by an implicitly derived tranche_id. The Rbsp data in a single slice corresponds to the data of a tile, data of a substream, data of a slice or data of an entropy slice.

[0051] In the NAL unit syntax, two modes are allowed for low-delay interleaving, indicated by the "low delay encapsulation_flag": a cyclic arrangement of the slices, and also the indication of the slices by an additional identifier "tranche_id" following the marker coder by a flag, e.g. "low delay cyclic_flag", in the NAL unit header. These two flags can be present in the Sequence Parameter Sets or even in the APS. For a cyclic slice arrangement, there can still be a need to know the number of slices during analysis, e.g. provided as "num_low_delay_tranches" in the SPS.

[0052] In a NAL unit, the interleaved "LD_rbsp_bytes" are read by the parser and rearranged into the actual sequential RBSP order in the final for loop in the NAL syntax: for ( i= 0, i++, i < num_low_delay_tranches)[ for ( j= 0, j++, j < NumBytesInRBSP[i] )[ rbsp_byte[NumBytesInRBSP++] = LD_rbsp_byte[j][i] ] There can also be explicit signaling in SPS or APS for a fixed size of circularly arranged slices as indicated by "low_delay_tranche_length_minus1". The latter is not used in the exemplary NAL unit syntax, but is straightforward when keeping in mind the packetization as shown in Figure 4. In the NAL unit syntax of Figure 6, the packetization as shown in Figure 5 and discussed above was fundamental.

[0053] To enable this interleaving property of fragments on multiple packets, e.g., slices and / or NAL units, a requirement for a global buffer, e.g., an array of LD_rbsp_byte for a fragment, can exist to repeatedly access the RBSP data of already received NAL units.

[0054] To allow for error recovery, after receiving a deterministic code or when the sum of the received bytes of the slice is equal to the slice size, this can be derived from an offset value provided in the included slice data, e.g., data for the respective WPP sub-stream or tile of which the slice in question is part.

[0055] An important requirement for WPP sub-streams arranged in interleaved low-delay segments is that only data from segment n is accessed by segment n+1, which is already provided in segment n and is already stored or available at the decoder.

[0056] The low-delay slice layer RBSP syntax for reordering / deinterleaving on slice level can be designed as follows. In particular, the syntax should then have almost the same behavior as on the NAL unit layer, but the reordering must be defined on the slice level. Figure 8 shows the low-delay slice layer RBSP syntax.

[0057] When using slice headers to packetize interleaved slices, there may be a need to indicate the codec level, e.g., that entropy coding of a slice of a WPP substream should not be interrupted when a new slice is received, rather than resetting the CABAC state. Not resetting CABAC for a slice is indicated as "no_cabac_reset_flag" in the slice header. The indicated slice header is suitable for low-delay slices, and therefore the entropy_slice feature must also be present. The corresponding slice header syntax is shown in Figure 9.

[0058] The transport layer allows optimizing the scheduling of data transferred to the decoder units based on the fact that a number of substreams / tiles / slices (which assume an abstract entity on the transport layer that can be represented by substreams, or tiles, or parts of substreams or tiles, or parts of bitstreams, allowing similar functions, i.e. parallel decoding or gradual decoder refresh) in the coding layer can be handled independently of each other. One possibility is to start sending slices in parallel to several decoding devices with minimum delay. The bitstream consists of a sequence of NAL units, which are the smallest items that can be handled independently on the transport layer. Therefore, the next way of handling on the transport layer is based on the substreams / tiles / slices contained in separate slice or entropy slice NAL units.

[0059] The transport layer should also optimize decoder performance and error resilience based on the fact that the coding layer uses gradual decoder refresh. One option is to discard irrelevant parts of the bitstream if previous parts of the bitstream were not received correctly, e.g., due to transmission errors, or were not received at all, e.g., due to a switch between transmission channels.

[0060] To allow this type of development / optimization, various information is signaled to the transport layer.

[0061] General side information is signaled using descriptors. - The number of substreams / tiles, where "1" means that there is only one stream / tile that contains all video frames. - Information common to all substreams / tiles, e.g., whether all substreams / tiles are the same size or have the same buffer requirements - Independent information about each substream / tile, e.g. whether the substreams / tiles are of different sizes or have different buffer requirements - the number of sequential decoder refresh steps, where "1" means that sequential decoder refresh requests are not used -One flag indicating whether these sub-streams / tiles are capable of low-latency parallel processing

[0062] If the number of substreams / tiles is >1, syntax elements are inserted before each data block in the stream that contains a substream / tile. These syntax elements follow the NAL unit syntax but use a unique NAL unit type that is not used by the coding layer (e.g. nal_unit_type=0x19 or nal_unit_type=0x1F) and are referred to below as substream markers.

[0063] These syntax elements are used as markers and at least to carry information about the data block following the data field identified with the substream / tile.

[0064] If the number of gradual decoder refresh steps > 1, these syntax elements also flag that the substream / tile is intra-coded (enabling incremental decoder refresh).

[0065] The corresponding syntax is shown in Figure 10. The following constraints apply: forbidden_zero_bit is equal to 0. nal_ref_flag is equal to 0. nal_unit_type is equal to 0x19. substream_ID: counter value starting from 0 for the first slice belonging to a picture, incremented with each further slice or entropy slice belonging to the same picture. is_intra: if '1', the following NAL unit contains an intra-coded slice or an intra-coded entropy slice.

[0066] A method for encapsulation of a video stream in a transport multiplex is shown in Figure 11, where each slice or entropy slice is transmitted separately in an integer number of transport stream packets. If the size of the payload does not exactly match the available bytes of a fixed size TS packet, the last TS packet contains an adaptation field.

[0067] It should also be noted that a similar behavior of elementary streams of an MPEG-2 Transport Stream can be provided by an RTP session or RTP stream of the Real-time Transport Protocol, as shown in Figure 19. In RTP [8], an RTP stream (identified by media type and payload type, as shown in SDP

[12] ) can be included in its own RTP session, where an RTP session is identified by an (IP) network address, a (UDP) port, and also a source identifier (SSRC). A media session shown in the SPD can include multiple RTP sessions, each containing a different media type. However, it is also possible to transport the same media stream (e.g. video) in different RTP streams, where the RTP streams can be included in the same RTP session (similar to 1 below) or in their own RTP session (similar to 2 below). Figure 19 illustrates case 2.

[0068] The RTP payload format [9]

[13] has a decoding order number (DON) that allows the receiver to recover the decoding order of NAL units when they are intentionally transmitted out of the decoding order for error resilience purposes as described in [9]

[13] . An additional marker MKR is therefore not required. When transferring slices of a WPP substream or tile, in the order in which they become available from the encoding process, the DON can also be used to recover the decoding order of the slices before providing them to a single decoder. However, in this case, an additional delay is introduced at the decoder due to the separate deinterleaving process before the decoding process. The system described herein can provide the encoded slices directly to the decoding processes of different WPP substreams or tiles while the data arrives at the receiver. As indicated by the RTP sequence number in the RTP header, the identity of the slice associated with the WPP substream or tile can be derived from the slice address of the slice segment header of the slice segment and the transmission order of the packets. In this scenario, DON is used only for applications without backward compatibility, i.e., without the improved ability of the decoder to decode WPP substream or tile slices transmitted out of decoding order when they arrive. The transmission of slice data out of decoding order is just applied on the WPP substream and tile level, i.e., in the transmitted data, slices of a single WPP substream or tile are transmitted in decoding order and data of different WPP substreams or tiles is interleaved.

[0069] There are two possible options: 1. All slices and entropy slices are contained in the same elementary stream, i.e. the same PID is assigned to all TS packets of that video stream; In the following text, this method will be referred to as single ES encapsulation. 2. Different PIDs are assigned to slices and entropy slices of the same video bitstream; in the following text, this method is referred to as multi-ES encapsulation.

[0070] If the first option is treated as a special case of a more general structure by setting the same PID for all ESs, then Figure 11 is valid for both options.

[0071] A more efficient method for encapsulation in a single ES is shown in Figure 12, where at most one adaptation field per image is needed.

[0072] A more efficient method for encapsulation of multiple ESs is shown in Figure 13. Here, the adaptation field is avoided. Instead, different slices, e.g. the concatenated tiles in the following figure, start immediately in the same Transport Stream packet.

[0073] A possible structure of a transport demultiplexer for encapsulation with one single elementary stream (ES) targeted to a multi-threaded decoder is shown in Figure 14. The entropy slice in the figure may contain data for a particular WPP sub-stream or tile.

[0074] The Transport Buffer (TB) collects data belonging to transport packets and transfers it to the Multiplex Buffer (MB). At the output of the MB, the NAL unit headers are evaluated and the substream markers are discarded, while the data carried in the substream markers is stored. The data of each slice or entropy slice is stored in a separate Slice Buffer (SB), from where it is retrieved by the multithreaded decoder once a decoder thread is available.

[0075] A possible structure of a transport demultiplexer for an encapsulation with multiple elementary streams targeting a multi-threaded decoder is shown in FIG.

[0076] The concepts outlined above are restated in other words below: The following descriptions may therefore be combined with the additional details of the above descriptions individually.

[0077] Fig. 16 shows the general structure of an encoder according to an embodiment of the present application. The encoder 10 can be implemented in a multi-threaded manner or not, i.e. simply capable of operating in a single thread. That is, the encoder 10 can be implemented, for example, using multiple CPU kernels. In other words, the encoder 10 can support parallel processing, but does not have to do so. The encoding concept of the present application allows a parallel processing encoder to efficiently apply parallel processing without, however, reducing the compression efficiency. With regard to parallel processing capabilities, a similar statement applies to the decoder, which is described later with regard to Fig. 17.

[0078] Encoder 10 is a video encoder, although in general encoder 10 may be an image encoder. Images 12 of a video 14 are shown as inputting encoder 10 at input 16.

[0079] The encoder 10 is a hybrid encoder, i.e. the image 12 is predicted by a predictor 18 and the prediction residual 20 obtained by a residual determiner 22, e.g. a subtractor, undergoes a transformation, e.g. a spectral decomposition such as a DCT, and a quantization in a transform / quantization module 24. The quantized residual 26 thus obtained undergoes an entropy coding, i.e. a context-adaptive binary arithmetic coding, in an entropy coder 28. A reconstructed version of the residual available to the decoder, i.e. an inverse quantized and retransformed residual signal 30, is recovered by a retransform and requantization module 31 and combined by a combiner 33 with a prediction signal 32 of the predictor 18, thereby obtaining a reconstruction 34 of the image 12. However, the encoder 10 operates on a block basis. The reconstructed signal 34 therefore has discontinuities at the block boundaries and therefore a filter 36 can be applied to the reconstructed signal 34 in order to generate a reference image 38 for the predictor 18 to predict the subsequent coded image. As indicated by the dashed line in Figure 16, the predictor 18, however, also uses the reconstructed signal 34 directly, without the filter 36 or intermediate version. In the case of coding an image, the filter 36 can be left out.

[0080] The predictor 18 may select between different prediction modes to predict a certain block of the image 12. There can be temporal prediction modes depending on which block is predicted on the basis of a previously coded image, spatial prediction modes depending on which block is predicted on the basis of previously coded blocks of the same image, inter-layer prediction modes depending on which block of an image showing a scene at a higher layer, e.g. at a higher spatial resolution or from a further viewpoint, is predicted on the basis of a corresponding image showing this scene at a lower layer, e.g. at a lower spatial resolution or from another viewpoint.

[0081] A syntax is used to compile the quantized residual data 26, i.e., the transform coefficient levels and other residual data, as well as the coding mode data including, for example, the prediction modes and prediction parameters for individual blocks of the image 12 as determined by the predictor 18, and these syntax elements are subject to entropy coding by the entropy coder 28. The resulting data stream output by the entropy coder 28 is therefore referred to as the raw byte sequence payload 40.

[0082] The elements of encoder 10 of FIG. 16 are interconnected as shown in FIG.

[0083] Fig. 17 shows a decoder compatible with the encoder of Fig. 16, i.e. capable of decoding raw byte sequence payloads. The decoder of Fig. 17 is generally indicated by reference numeral 50 and consists of an entropy decoder 52, a retransform / inverse quantization module 54, a synthesizer 56, a filter 58 and a predictor 60. The entropy decoder 42 receives the raw byte sequence payload 40 and performs an entropy decoding using context-adaptive binary arithmetic decoding to recover a residual signal 62 and coding parameters 64. Said retransform / inverse quantization module 54 performs an inverse quantization and a retransform of the residual data 62 and forwards the residual signal thus obtained to a synthesizer 56. The synthesizer 56 also receives a prediction signal 66 from a predictor 60 which in turn forms said prediction signal 66 using the coding parameters 64 on the basis of a reconstructed signal 68 determined by synthesizing the prediction signal 66 and the residual signal 65 by the synthesizer 56. 16, the predictor 60 may alternatively or additionally use a filtered version of the reconstructed signal 68 or some intermediate versions thereof. The image to be finally reconstructed and output at output 70 of the decoder 50 may likewise be determined on an unfiltered version of the composite signal 68 or some filtered version thereof.

[0084] According to the tile concept, the image 12 is subdivided into tiles, and the prediction of at least blocks within these tiles is restricted to using only data related to the same tile as the basis for spatial prediction. This measure allows at least prediction to be performed in parallel for each tile. For illustrative purposes only, FIG. 16 shows the image 12 as being subdivided into nine tiles. The subdivision of each tile into nine blocks as shown in FIG. 16 also serves only as an example. Furthermore, for completeness, it should be noted that the method of separately coding tiles cannot be restricted to spatial prediction (intra-prediction). Rather, any prediction of the coding parameters of each tile across tile boundaries and any dependency of the context selection in the entropy coding of each tile across tile boundaries can be forbidden, since it is restricted to rely only on data of the same tile. Thus, the decoder can perform the just-mentioned operations in parallel, i.e., on a tile-by-tile basis.

[0085] For transmission over several transmission channels, the syntax elements have to be entropy coded slice by slice by the entropy coder 28. To this end, the entropy coder 28 scans the blocks of a tile, first traversing the blocks of the first tile, then proceeding to the blocks of the next tile in tile order, etc. A raster scan order can be used, for example, to scan the blocks tile by tile within a tile. The slices are then packed into NAL units, which are the smallest units of transmission. Before entropy coding a slice, the entropy coder 28 initializes its CABAC probabilities, i.e. the probabilities used to arithmetically code the syntax elements of that slice. The entropy decoder 52 initializes the same, i.e. the probabilities at the slice start. Each initialization, however, has a negative impact on the entropy coding efficiency, since the probabilities are continuously adapted to the real symbol probability statistics of various contexts, and thus a reset of the CABAC probabilities indicates a deviation from the adapted state. As known to those skilled in the art, entropy coding leads to optimal compression only if the probabilities match the actual symbol probability statistics.

[0086] Thus, the decoder operates as shown in FIG. 18 in accordance with an embodiment of the present application. The decoder receives, in step 80, a raw byte sequence payload that describes an image 12 in a tile 82 within a slice in the tile. In FIG. 18, the first tile 82 in a tile order 84 is exemplarily shown as being truncated or divided into two slices 86a and 86b, each exemplarily covering a subsequence of the sequence of blocks in the tile. Then, in step 82, slices 86a and 86b are entropy decoded. However, in the entropy decoding of slices 86a and 86b, the CABAC probability adaptation is continuous across slice boundaries. That is, while decoding slice 86a, the CABAC probabilities are continuously adapted to the actual symbol statistics, and the state at the end of entropy decoding slice 86a is adapted to the start of entropy decoding slice 86b. In step 90 , the raw byte sequence payload, and thus entropy decoded, is decoded to obtain the image 12 .

[0087] By continuing the CABAC probability adaptation across slice boundaries 92 located inside tiles 82, these slice boundaries do not adversely affect the entropy coding efficiency across subdivisions of image 12 into tiles 82. On the other hand, tile parallelism is still possible. Moreover, since it is possible to transmit slices individually and the slices are smaller than the complete tile 82, the decoding of each tile in step 90 can start as soon as the first slice of the respective tile has been received and entropy decoded.

[0088] The description of figures 16 to 18 primarily concerns the use of tiles. As mentioned above, tiles result from a spatial division of an image. Like tiles, slices also spatially sub-divide an image. Slices are therefore also a means to enable parallel encoding / decoding. Like tiles, prediction etc. is inhibited so that slices can be decoded individually. The description of figures 16 to 18 is therefore also valid for dividing slices into pieces.

[0089] The same applies when using WPP substreams. The WPP substreams also represent a spatial division of the image 12, i.e., into WPP substreams. In contrast to tiles and slices, the WPP substreams do not impose restrictions on prediction and do not touch selections beyond the WPP substream. The WPP substreams extend along block rows, e.g., LCU rows, as shown in FIG. 4, and in order to allow parallel processing, a compromise is simply made between the WPP substreams (see FIG. 4) 92 as defined and for each WPP substream 92 except the first WPP substream 92 in sequence with respect to CABAC entropy coding, in which the CABAC probability is not completely reset but introduced or set equal, and the CABAC probability occurs for each WPP substream on the same side of the image 12, e.g., the left side as shown in FIG. 4, as the LCU order starts and after entropy decoding the immediately preceding WPP substream up to the second LCU 94. Thus, subject to some encoding delay between sequences of WPP substreams, these WPP substreams 92 may be decoded in parallel such that portions of image 12 that are decoded in parallel, i.e. simultaneously, form a sort of wavefront 96 that moves across the image in a slanted manner from left to right.

[0090] That is, when transmitting the descriptions of Figures 16 to 18 to the WPP sub-streams, any WPP sub-stream 92 (Figure 4) can also be subdivided into segments 98a and 98b without interrupting the CABAC probability adaptation at the boundary 100 between these segments 98a and 98b within each WPP sub-stream 92, thereby avoiding the penalty in terms of entropy coding efficiency due to the individual transmission capabilities of both segments 98a and 98b, but maintaining the ability to use wavefront parallelism, which can start earlier because the segments are smaller than the complete WPP sub-stream 92.

[0091] As mentioned above with respect to Figures 1 to 15, there are several possibilities to packetize and transmit the slices in NAL units. Reference is made to Figure 3, where tiles or substreams or slices of the slice or substream are divided into slices in the arithmetic coding domain with a header preceding the n-th slice of each substream or tile and indicating information allowing the slice boundary to be localized. Another embodiment is the one shown in Figure 9, where the subdivision of a tile or WPP substream into slices is performed by slightly modifying the slice structure: a slice starts at a tile or WPP substream boundary, i.e. at the beginning of a tile or WPP substream. No_cabac_reset_flag is set to zero, which causes the normal CABAC probabilities to be initialized / reset. However, slices carrying slices starting inside a tile or WPP substream have no_cabac_reset_flag set to 1, which causes the continuation of the aforementioned CABAC probability adaptation.

[0092] As far as de-interleaving is concerned, it is performed in a reception step 80, e.g., for each slice it is determined which WPP substream or tile the respective slice belongs to. Different possibilities are mentioned above, e.g., a round-robin cycle through the number of WPP substreams or tiles of the current picture. Alternatively, in case of using a slice header to carry the slices, the slice header may contain an indication that may identify the start of the respective slice within the current picture 12.

[0093] In this regard, it is pointed out that the decomposition of a slice, a WPP substream or a tile into slices is performed along a decoding order defined within each slice, WPP substream or tile. That is, within each slice, WPP substream or tile, the part of the image spatially covered by the respective slice, WPP substream or tile is coded / decoded to the respective slice, WPP substream or tile in its decoding order, and each slice, WPP substream or tile, covers a consecutive part of the respective slice, WPP substream or tile along its decoding order. Thus, an order, i.e. an encoding / decoding order, between slices belonging to the same slice, WPP substream or tile is defined, and each slice has a rank within the order. The subdivision of the image into WPP substreams or tiles is signaled to the decoder, so that said decoder knows about the subdivision. Thus, to associate each slice with its respective WPP substream or tile, it is sufficient to have a starting address that identifies the starting position at which each slice successively covers the image, for example using the encoding / decoding order of the tile / WPP substream of which it is a part. Even the order between slices belonging to a given tile or WPP substream can be reconstructed by the transport demultiplexer or decoder, for example using the starting positions. However, for re-sorting, information in the transport packet headers of the lower OSI (Open Systems Interconnection) layer can be used, for example as the decoding order number, i.e. DON, as described above for RTP transmission. A transport demultiplexer of the above type can be configured similarly to the transport demultiplexer described above, to store data of slices of an equal WPP substream or tile in one slice buffer and data of slices of WPP substreams or tiles associated with different WPP substreams or tiles in different slice buffers. As described above, the slice structure, i.e. the slice header, can be used to carry the slices.

[0094] Reference is now made to the embodiments of figures 11 to 15 for restating again. As described in these figures, slices S i are packetized into NAL units, each NAL unit 110 (see figure 11) having a NAL unit header 112. It should be noted that slices S i can be normal slices or slices carrying slices according to figure 9. They therefore only carry data for one WPP substream or tile of the current image, i.e. data or tiles of the i-th WPP substream, respectively. For fragmentation, said NAL units 110 are carried by transport stream (TS) packets 114, i.e. in their payload section 116. To do so, each NAL unit 110 and said corresponding slice S i is preceded by a respective substream marker MKR indicating i, i.e. the WPP substream or tile to which the directly succeeding slice directly succeeding the NAL unit 110 belongs.

[0095] NAL units 110 carrying slices belonging to different WPP substreams or tiles can be distributed over one or more elementary streams ES or over the same elementary stream, as described in Figures 11 to 13. As mentioned above, a "elementary stream" may also identify a separate RTP stream in its own RTP session.

[0096] As explained with respect to figures 14 and 15, the transport demultiplexer may comprise a multiplex buffer MB, a slice buffer SB and a transport buffer TB. The slice buffer SB is populated by a multi-threaded decoder MTD capable of parallel decoding of the images of a WPP sub-stream or tile. The transport buffer TB is adapted to collect data belonging to TS packets of a given elementary stream of the video bitstream and to transfer said data to the multiplex buffer MB. The transport demultiplexer is configured to evaluate the NAL unit headers of the NAL units of the NAL unit sequence packetized into TS packets at the output of the multiplex buffer MB, to accumulate the substream marker data carried in the substream marker NAL units and to discard the substream marker NAL units MKR, and to accumulate the data of the slices or tiles of the substream in the NAL units following the substream marker NAL units, data fields that refer to one, i.e. an equal WPP substream or tile in the same slice buffer SB, and the data of the slices of the WPP substream or tile in the NAL units following the substream marker NAL units, data fields that refer to different WPP substreams or tiles in different slice buffers SB. As shown in Fig. 15, the transport demultiplexer consists of a demultiplexer, called TS demux in Fig. 15, which receives a video bitstream and splits the TS packets of the video bitstream into different elementary streams, i.e. distributes the TS packets of the video bitstream into different elementary streams. The demultiplexer performs this division or distribution according to multiple PIDs (referred to as PIDs in Figure 15) contained in the TS header of the TS packets, so that each basic stream consists of TS packets with a PAD different from the PAD of the TS packets of the other basic streams.

[0097] That is, if a slice corresponds to a section in the sense of the embodiment of Figure 9, the MTD, i.e. the multi-threaded decoder, may start processing one or more WPP sub-streams or tiles of the current image as soon as the corresponding slice buffer SB of each WPP sub-stream or tile has data contained therein, resulting in reduced delay.

[0098] Although some aspects are described in relation to an apparatus, it will be apparent that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in relation to a method step represent a description of the corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some or more of the most important method steps can be performed by such an apparatus.

[0099] The inventive encoded bitstream can be stored on a digital storage medium or transmitted over a transmission medium, such as a wireless transmission medium or a wired transmission medium, such as the Internet.

[0100] This above mentioned contribution therefore describes a method for structuring, in particular in tiles, wavefront parallel processing (WPP) substreams, slices or entropy slices, etc., a method for the transmission of structured video data as provided by the low-latency encapsulation and the new HEVC coding standard. Techniques are provided that enable low-latency transmission in a parallel processing encoder-transmitter-receiver-decoder environment, in particular via interleaved transmission of entropy slices / slices / tiles / substreams. To solve the bottleneck problem outlined in the introduction of the specification and to minimize the transmission delay and the decoding time, i.e. the end-to-end delay, techniques are provided, in particular for an interleaved entropy slice scheme for parallel transmission and processing.

[0101] Depending on specific implementation requirements, the embodiments of the present invention can be implemented in hardware or in software. The embodiments can be implemented using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or FLASH memory, having electronically readable control signals stored thereon, which cooperate (or can cooperate) with a programmable computer system so that the respective methods are executed. The digital storage medium may therefore be computer readable.

[0102] Some embodiments of the present invention include a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0103] Typically, the embodiments of the present invention may be implemented as a computer program product having program code which, when the computer program product runs on a computer, is operated to perform one of the methods. The program code may, for example, be stored on a machine readable carrier.

[0104] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0105] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0106] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium, or computer readable medium) comprising recorded thereon the computer program for performing one of the methods described herein. The data carrier, digital storage medium or recorded medium is generally tangible and / or non-transitory.

[0107] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein, for example a data stream or a sequence of signals being adapted to be transmitted via a data communications connection, for example the Internet.

[0108] A further embodiment comprises a processing means, for example a computer, or a programmable logic circuit, configured to or adapted to perform one of the methods described herein.

[0109] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0110] Further embodiments according to the present invention include an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory element, etc. The apparatus or system may, for example, include a file server for transferring the computer program to the receiver.

[0111] In some embodiments, a programmable logic circuit (e.g., a field programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

[0112] The above-described embodiments merely represent examples of the principles of the present invention. It is understood that modifications and variations of the devices and details described herein will be apparent to others skilled in the art. It is therefore intended to be limited only by the scope of the appended claims and not by the detailed description of the specification as set forth in the description and illustration of the present invention.

[0113] literature [1]Thomas Wiegand, Gary J. Sullivan, Gisle Bjontegaard, Ajay Luthra, "Overvie w of the H.264 / AVC Video Coding Standard", IEEE Trans. Circuits Syst. Vide o Technol., vol. 13, N7, July 2003. [2]JCTVC-E196,"Wavefront Parallel Processing", 5th JCT-VC Meeting, Geneva 2011. [3]JCTVC-D070, "Lightweight slicing for entropy coding", 4th Meeting, Daegu, 2011. [4]JCTVC-D073, "Periodic initialization for wavefront coding functionality", 4th Meeting, Daegu, 2011. [5]HEVC WD5: Working Draft 5 of High-Efficiency Video Coding JTCVC-G1103, 5th JCT-VC Meeting, Geneva Meeting November 2011. [6]JTCVC-D243, "Analysis of entropy slices approaches", 4th Meeting, Daegu, 2011. [7]ISO / IEC13818-1 / 2011, MPEG-2 Transport Stream including AMDs 1-6. [8]IETF Real-time transport protocol, RTP RFC 3550. [9]IETF RTP Payload Format, IETF RFC 6184.

[10] JCTVC-F275, Wavefront and Cabac Flush: Different Degrees of Parallelism Without Transcoding, , Torino Meeting

[11] JCT-VC-F724, Wavefront Parallel Processing for HEVC Encoding and Decoding,Torino Meeting** at end of description

[12] IETF Session Description Protocol (SDP), RFC 4566

[13] IETF RTP Payload Format for High Efficiency Video Coding, draft-schierl- payload-h265

Claims

1. a receiving module configured to receive from an encoder describing an image with WPP substreams, one WPP substream per LCU row of the image, and a raw byte sequence payload encoded using CABAC in slices into which the WPP substreams are segmented and thus slice boundaries are introduced; an entropy decoding module configured to entropy decode the slices and continue CABAC probability adaptation beyond the slice boundaries introduced in the WPP sub-stream; a decoding module configured to decode the raw byte sequence payload to obtain the image; A decoder comprising:

2. 2. The decoder of claim 1, wherein the slices are packetized using slice headers, and wherein the decoder is configured, upon receiving the slices, to interrupt the CABAC probability adaptation by resetting CABAC probabilities or to continue the CABAC probability adaptation upon receiving a new slice in response to a flag in the slice header of the new slice, or a slice type of the new slice, or a NAL unit type of a NAL unit containing the new slice.

3. 2. The decoder of claim 1, wherein the decoder is configured to deinterleave the slices upon receiving the slices by checking, for each slice, to which WPP sub-stream the slice belongs.

4. 2. The decoder of claim 1, wherein the slices are packetized into packets arranged in a defined order among the WPP substreams, with each packet containing one slice of each WPP substream of the image, or a subset of the WPP substreams of the image, and each packet containing a header indicating the position and / or length of the slices packed in the respective packet or a marker separating the slices from one another within the respective packet, and wherein the decoder is configured to access the slices in the packets using information contained in the header or marker when receiving the raw byte sequence payload.

5. 5. The decoder of claim 4, characterized in that a packet of the WPP substream of the image including a first slice according to the order defined between the WPP substreams includes a low delay characteristic indicator, and a packet of the WPP substream of the image including a second or subsequent slice according to the order defined between the WPP substreams includes a continuity indicator.

6. The decoder of claim 4, wherein the packets are NAL units or slices.

7. a receiving module configured to receive from an encoder a raw byte sequence payload describing an image in a WPP substream and encoded using CABAC in slices, in which the WPP substream is segmented and thus slice boundaries are introduced; an entropy decoding module configured to entropy decode the slices and to continue CABAC probability adaptation across the slice boundaries introduced within the WPP substream by adopting a last CABAC probability of entropy decoding of another slice of the WPP substream when starting entropy decoding of the slice of the WPP substream; a decoding module configured to decode the raw byte sequence payload to obtain the image; A decoder comprising:

8. 1. An encoder comprising: a coding module configured to form a raw byte sequence payload by coding an image to describe the image in WPP substreams, one WPP substream per LCU row of the image, entropy code the raw byte sequence using CABAC, transmit the raw byte sequence in slices in which the WPP substreams are segmented and thus slice boundaries are introduced, and continue CABAC probability adaptation in the entropy coding across the slice boundaries introduced in the WPP substream.

9. 9. The encoder of claim 8, wherein the encoder is configured to form the raw byte sequence such that the slices conform to a maximum transfer unit size.

10. A data transmission method, comprising: transmitting a video bitstream including a raw byte sequence payload that describes an image with WPP substreams, one WPP substream for each LCU row of the image, and that is encoded using CABAC, the video bitstream being decomposed into slices of the WPP substreams, where the WPP substreams are segmented and thus slice boundaries are introduced, and CABAC probability adaptation continues beyond the slice boundaries introduced in the WPP substreams, where each slice includes an explicit indication of rank among the slices along which the WPP substream to which it belongs is decomposed in turn. A method comprising:

11. 11. The method of claim 10, wherein the slices are packetized into packets such that each packet contains one slice of each WPP substream of the image, or a subset of the WPP substreams of the image arranged in a defined order among the WPP substreams, and each packet contains a header indicating the position and / or length of the slices packed in the respective packet or a marker separating the slices within the respective packet from each other.

12. 11. The method of claim 10, characterized in that a packet of the WPP substream of the image including a first slice according to the defined order between the WPP substreams includes a low latency characteristic indicator, and a packet of the WPP substream of the image including a second or subsequent slice according to the defined order between the WPP substreams includes a continuity indicator.

13. The method according to claim 10, characterized in that the packets are NAL units or slices.

14. - describing an image in a WPP stream with one WPP sub-stream per LCU row of the image and receiving raw byte sequence payloads encoded using CABAC in slices of the WPP sub-stream, where the WPP sub-stream is segmented and thus slice boundaries are introduced; continuing CABAC probability adaptation across slice boundaries introduced in the WPP substream and entropy decoding the slices; decoding the raw byte sequence payload to obtain the image; A method of decoding, comprising:

15. 2. A method for transport demultiplexing comprising: receiving a video bitstream describing an image with WPP substreams, one for each LCU row of the image, and including raw byte sequence payloads encoded using CABAC, the video bitstream being decomposed into slices of the WPP substreams where the WPP substreams are segmented and thus slice boundaries are introduced, and CABAC probability adaptation is continued across the slice boundaries introduced in the WPP substreams, the video bitstream including, for each slice, information identifying which WPP substream the respective slice belongs to; and associating the slices with the WPP substreams using the information.

16. 1. A method for encoding an image, comprising the steps of: forming a raw byte sequence payload by encoding an image to describe the image with WPP substreams, one for each LCU row of the image, and entropy coding the raw byte sequence using CABAC; transmitting the raw byte sequence in slices into which the WPP substreams are segmented and thus slice boundaries are introduced; and continuing CABAC probability adaptation beyond the slice boundaries introduced in the WPP substreams in the entropy coding step.

17. A computer-readable digital storage medium storing a computer program having a program code for carrying out the method according to any one of claims 14 to 16, when the computer program is executed on a computer.

Images