Sample array coding for low-delay

Abstract

To provide a decoder and a method for improving efficiency of entropy coding.SOLUTION: Entropy coding of a current part of a predetermined entropy slice is based on, not only, respective probability estimations of the predetermined entropy slice as adapted using a previously coded part of the predetermined entropy slice, but also probability estimations as used in the entropy coding of a spatially neighboring, in previous entropy slice of the entropy slice order at a neighboring part thereof. Thereby, the probability estimations used in entropy coding are adapted to actual symbol statistics more closely, thereby lowering of the coding efficiency decreases, the lowering of the coding efficiency is normally caused by lower-delay concepts. Temporal interrelationships are exploited additionally or alternatively.SELECTED DRAWING: None

Description

[Technical field]

[0001] This application relates to sample array coding, such as image or video coding. [Background technology]

[0002] Due to the increased processing demands of the HEVC standard as well as the expected increase in video resolution, parallelization of the encoder and decoder becomes very important. Multi-core architectures are becoming available in a wide range of modern electronic devices. As a result, efficient methods are needed to enable the use of multi-core architectures.

[0003] The coding or decoding of LCUs occurs in a raster scan, whereby CABAC probabilities are adapted to the characteristics of each image. Spatial dependencies exist between adjacent LCUs. Each LCU (largest coding unit) depends on its left, upper, upper left and upper right neighboring LCUs for different components, e.g., motion vectors, prediction, intra prediction, etc. These dependencies usually need to be or are broken in state-of-the-art applications to allow parallelization in decoding.

[0004] Several concepts of parallelization, in other words wavefront processing, have been proposed. The impetus for further research is to implement techniques that reduce the loss of coding efficiency and therefore the bitstream burden for parallelization approaches in the encoder and decoder. Furthermore, low-latency processing has not been possible with the available technologies. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2011 / 042645 [Patent Document 2] US Patent Application Publication No. 2008 / 001796 [Non-patent literature]

[0006] [Non-Patent Document 1] Henry et al, Wavefront Parallel Processing, 96. MPEG MEETING, 21-3-2011-25-3-2011, GENEVA, no. m19714, 19 March 2011, XP030048281 Summary of the Invention

[0007] It is therefore an object of the present invention to provide a coding concept for sample arrays that allows lower delay with relatively little penalty in coding efficiency.

[0008] This object is achieved by the subject matter of the independent claims.

[0009] If the entropy coding of the current part of the predetermined entropy slice is based not only on the respective probability estimates of the predetermined entropy slice adapted using the previously coded part of the predetermined entropy slice, but also on the probability estimates used in the entropy coding of the spatially adjacent, previous entropy slice in the entropy slice order, the probability estimates used for the entropy coding are more closely adapted to the actual symbol statistics, thereby reducing the decrease in coding efficiency that is usually caused by the low delay concept. Temporal correlations can additionally or instead be exploited.

[0010] For example, the dependency on the probability estimates used in the entropy coding of a spatially adjacent entropy slice preceding it in the entropy slice order can include a predetermined initialization of the probability estimates at the beginning of the entropy coding of the entropy slice. Usually, the probability estimates are initialized to values ​​adapted to the symbol statistics of a representative mix of sample array material. To avoid transmission of the initialization values ​​of the probability estimates, they are known to the encoder and the decoder by convention. However, such predetermined initialization values ​​will naturally deviate - more or less - from the actual statistics of the currently coded sample array material, and are therefore naturally just a compromise between the bit rate of the side information on the one hand and the coding efficiency on the other hand. Probability adaptation during the process of coding of an entropy slice adapts the probability estimates to the actual symbol statistics. This process is accelerated by initializing the probability estimates at the beginning of the entropy coding of the current / predetermined entropy slice with the already adapted probability estimates of the previous entropy slice in the just mentioned spatially adjacent entropy slice order, since the later values ​​are to some extent adapted to the actual symbol statistics of the currently at hand sample array. Nevertheless, by using the probability estimates used in the adjacent part in the initialization of the probability estimates for the predetermined / current entropy slice, rather than appearing at the end of the entropy coding of the previous entropy slice, low delay coding can be enabled. With this measure, wavefront processing is still possible.

[0011] Furthermore, the above-mentioned dependency on the probability estimates used in the entropy coding of the spatially adjacent, previous entropy slice in the entropy slice order, can include an adaptation process that adapts the probability estimates used in the entropy coding of the current / predetermined entropy slice itself. The adaptation of the probability estimates includes the use of the just coded part, i.e. the just coded symbol, to adapt the current state of the probability estimates to the actual symbol statistics. By this measure, the initialized probability estimates are adapted to the actual symbol statistics with some adaptation rate. This adaptation rate is increased by performing the just-mentioned adaptation of the probability estimates not only based on the currently coded symbol of the current / predetermined entropy slice, but also by the probability estimates appearing in the spatially adjacent, adjacent part of the previous entropy slice in the entropy slice order. Also, by appropriately selecting the spatial neighborhood of the current part of the current entropy slice and the adjacent part of the previous entropy slice, wavefront processing is still possible. The advantage of combining the probability estimate adaptation along the current entropy slice with the probability adaptation of the previous entropy slice is that the rate at which adaptation to the actual symbol statistics occurs is increased, since the number of symbols traversed in the current and previous entropy slices contributes to the adaptation, rather than simply the symbols of the current entropy slice. [Brief description of the drawings]

[0012] Advantageous implementations of embodiments of the invention are the subject matter of the dependent claims. Further preferred embodiments are described with reference to the following drawings, in which: [Figure 1] 1 shows a block diagram of an exemplary encoder. [Diagram 2] 1 shows a schematic diagram of a division of an image into slices and slice parts (i.e. blocks or coding units) along with a coding order defined therein. [Diagram 3]2 shows a functional flow diagram of an example encoder such as the one of FIG. 1; [Figure 4] 2 shows a schematic diagram illustrating the functioning of an exemplary encoder such as the one of FIG. 1; [Diagram 5] 1 shows a schematic diagram for implementing parallel operation of an encoder and a decoder. [Figure 6] 1 shows a block diagram of an exemplary decoder. [Figure 7] 7 shows a flow diagram of the functionality of an exemplary decoder such as the one of FIG. 6. [Figure 8] FIG. 7 shows a diagrammatic view of an exemplary bitstream resulting from the encoding schemes of FIGS. [Figure 9] We show diagrammatically an example of how probabilities are calculated with the help of other LCUs. [Figure 10] A graph illustrating RD results for Intra (4 threads) in comparison with HM3.0 is shown. [Figure 11] A graph illustrating RD results for low latency (1 thread) in comparison with HM3.0 is shown. [Figure 12] A graph illustrating RD results for random access (1 thread) in comparison with HM3.0 is shown. [Figure 13] A graph illustrating RD results for low latency (4 threads) in comparison with HM3.0 is shown. [Figure 14] 1 illustrates diagrammatically and exemplarily a possible combination of entropy slices; [Figure 15] 13 illustrates diagrammatically and exemplarily a possible signaling of entropy slices; [Figure 16] 1 diagrammatically and exemplarily illustrates encoding, segmenting, interleaving and decoding of entropy slice data by chunks; [Figure 17] 1 illustrates diagrammatically and exemplarily possible compositing between frames; [Figure 18] 1 illustrates diagrammatically and exemplarily possible uses of sequenced information; [Figure 19]13A-B show diagrammatically the possibility of wavefronts running diagonally in spatial / temporal space across a continuous sample array. [Figure 20] 13A and 13B diagrammatically illustrate another embodiment of subdividing an entropy slice into chunks; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] To facilitate understanding of the strategies outlined below that improve low-delay achievement with less penalty in terms of coding efficiency, the encoder of Fig. 1 will first be described in more general terms, without a prior discussion of advantageous concepts of embodiments of the present application and how they can be incorporated into the embodiments of Fig. 1. However, it should be mentioned that the structure shown in Fig. 1 merely serves as an illustrative environment in which embodiments of the present application can be used. Generalizations and variations of the encoder and decoder according to embodiments of the present invention will be briefly described.

[0014] Figure 1 shows an encoder that encodes a sample array 10 into an entropy encoded data stream 20. As shown in Figure 1, the sample array 10 may be one of a sequence of sample arrays 30, and the encoder may be configured to encode the sequence 30 into the data stream 20.

[0015] The encoder of Figure 1 is generally designated by reference numeral 40 and comprises a precoder 42 followed by an entropy encoding stage 44 that outputs the data stream 20. The precoder 42 is configured to receive and operate on the sample array 10 to describe its content in terms of syntax elements of a predefined syntax, each syntax element being a respective one of a set of predefined syntax element types and then associated with a respective semantic.

[0016] In describing the sample array 10 using syntax elements, the precoder 42 can subdivide the sample array 10 into coding units 50. The term "coding unit" is instead called "coding tree unit" (CTU) for reasons that will be described in more detail below. One possibility of how the precoder 42 can subdivide the sample array 10 into coding units 50 is exemplarily shown in FIG. 2. According to this embodiment, the subdivision is a regular subdivision of the sample array 10 into coding units 50, so that the latter are arranged in rows and columns to cover the entire sample array 10 gaplessly without overlap. In other words, the precoder 42 can be configured to describe each coding unit 50 by syntax elements. Some of these syntax elements can form subdivision information for further subdivision of the respective coding units 50. The subdivision information may describe the subdivision of each coding unit 50 into prediction blocks 52 by the precoder 42, which associates a prediction mode to associated prediction parameters for each of these prediction blocks 52, for example by multi-tree subdivision. This prediction subdivision may allow prediction blocks 52 that differ in size, as illustrated in FIG. 2. The precoder 42 may also associate residual prediction information to the prediction blocks 52 in order to further subdivide the prediction blocks 52 into residual blocks 54 to describe the prediction residual for each prediction block 52. The precoder may thus be configured to generate a syntax description of the sample array 10 according to the hybrid coding scheme. However, as already mentioned above, the just-mentioned manner in which the precoder 42 describes the sample array 10 by syntax elements is presented merely for illustrative purposes and may be implemented differently.

[0017] The precoder 42 can exploit spatial interrelationships between the contents of adjacent coding units 50 of the sample array 10. For example, the precoder 42 can predict syntax elements for a particular coding unit 50 from syntax elements determined for previously coded coding units 50 that are spatially adjacent to the currently coded coding unit 50. In Figs. 1 and 2, for example, the top and left hand neighbors serve for prediction, as illustrated by arrows 60 and 62. Moreover, the precoder 42 can extrapolate already coded content of adjacent coding units 50 to the current coding unit 50 in intra-prediction mode to obtain a prediction of the samples of the current coding unit 50. As shown in Fig. 1, beyond exploiting spatial interrelationships, the precoder 42 can temporally predict samples and / or syntax elements for the current coding unit 50 from previously coded sample arrays, as illustrated by arrow 64 in Fig. 1 for illustration purposes. That is, motion compensated prediction may be used by the precoder 42, and the motion vectors themselves may follow temporal prediction from motion vectors of previously coded sample arrays.

[0018] That is, the precoder 42 can describe the contents of the sample array 10 per coding unit, and for this purpose can use spatial prediction. The spatial prediction is limited for each coding unit 50 to spatially adjacent coding units of the same sample array 10, so that when following the coding order 66 in the coding units 50 of the sample array 10, the adjacent coding units serving as prediction references for the spatial prediction are traversed by the coding order 66 before the current coding unit 50. As illustrated in FIG. 2, the coding order 66 defined in the coding units 50 can be, for example, a raster scan order in which the coding units 50 traverse row by row from top to bottom. Optionally, the subdivision of the array 10 into an array of tiles can make the scan order 66 - in a raster scan order - traverse the coding units 50 that first comprise one tile before preceding the next in a tile order that can also be of the raster scan type. For example, spatial prediction may simply include adjacent coding units 50 in a row of coding units above the row of coding units in which the current coding unit 50 resides, and coding units in the same row of coding units but to the left of the current coding unit. As will be explained in more detail below, this restriction on spatial correlation / spatial prediction enables parallel wavefront processing.

[0019] The precoder 42 forwards the syntax elements to the entropy coding stage 44. As just outlined, some of these syntax elements are predictively coded, i.e., they represent prediction residuals. The precoder 42 can therefore be considered as a predictive coder. Alternatively, the precoder 42 can be a transform coder configured to transform the predictive coding residuals of the content of the coding unit 50.

[0020] An exemplary internal structure of the entropy coding stage 44 is also shown in Figure 1. As shown, the entropy coding stage 44 optionally encodes each of the syntax elements received from the precoder 42 into a symbol s of the symbol alphabet on which the entropy coding engine 44 operates, where the number of possible stages exceeds the cardinality of the symbol alphabet. i In addition to the optional symbolizer 70, the entropy coding engine 44 may include a context selector 72 and initializer 74, a probability estimation manager 76, a probability estimation adapter 78, and an entropy coding core 80. The output of the entropy coding core forms the output of the entropy coding stage 44. The entropy coding core 80 also receives two inputs, one for the symbols s of a sequence of symbols, i and the other receives the probability estimate p i The input has an input for receiving the signal.

[0021] Due to the nature of entropy coding, the coding efficiency in terms of compression rate increases with improvement of the probability estimates; the better the probability estimates match the actual symbol statistics, the better the compression rate.

[0022] In the embodiment of FIG. 1, the context selector 72 selects each symbol s i , the corresponding context c i It should be noted that the selection of the context, however, merely forms an optional feature and can be omitted, for example, by using the same context for each symbol. However, when using context selection, the context selector 72 can be configured to perform the selection of the context based at least in part on information about coding units outside the current coding unit, in other words about neighboring coding units within the aforementioned limited neighborhood.

[0023] Manager 76 includes a storage device that stores, for each available context, an associated probability estimate. For example, the symbol alphabet may be a binary alphabet, so that simply one probability value is stored for each available context.

[0024] The initializer 74 can continually initialize or reinitialize the probability estimates stored within the manager 76 for the available context. Possible moments at which such initialization can be performed are discussed further below.

[0025] Adapter 78 is symbol s i and the corresponding probability estimate p i , and adapts the probability estimates in the manager 76 accordingly. That is, for each symbol s i Each time a probability estimate is applied to the data stream 20 by the entropy coding core 80 to entropy code this current symbol s i We can vary this probability estimate by the value of p i is better adapted to the actual symbol statistics when encoding the next symbol (by its context) associated with that probability estimate. That is, the adapter 78 applies a probability estimate for a selected context to the corresponding symbol s i from the manager 76 and hence the probability estimate p i and so adapt it to the same context c i The next symbol s i An adapted probability estimate is used for

[0026] The entropy coding core 80 is configured to operate, for example, by an arithmetic coding scheme or in a probability-interval-split-entropy coding scheme. In arithmetic coding, the entropy coding core 80 continuously updates its state, for example when encoding a sequence of symbols, with the state being determined by a probability interval determined by a probability interval width value and a probability interval offset value. When operating in the pipe concept, the entropy coding core 80 subdivides the domain of possible values ​​of the probability estimate into different intervals, for example by performing a fixed-probability entropy coding on each of these intervals, so that a sub-stream is obtained for each of the sub-intervals whose coding efficiency is respectively adapted to the associated probability interval. In the case of entropy coding, the output of the data stream 20 is a signaling of the arithmetically coded data stream to the information of the decoding side, which allows to emulate or redo the interval subdivision process.

[0027] Of course, for the entropy coding state 44, all the information about the sample array 10, i.e., all the syntax elements / symbols s i can be entropy coded by initializing the probability estimates only once at the beginning, and then continuously updating the probability estimates by the adapter 78. This, however, results in a data stream 20 that must be decoded sequentially at the decoding side. In other words, there is no possibility for any decoder to subdivide the resulting data stream into several sub-parts and decode the sub-parts in parallel. This in turn precludes any low-latency efforts.

[0028] It is therefore preferred, as outlined in more detail below, to subdivide the amount of data describing the sample array 10 into so-called entropy slices, each of which therefore covers a different set of syntax elements relating to the sample array 10. However, if the entropy coding stage 44 entropy codes each entropy slice, which is completely independent of one another, by first initializing the probability estimates once and then successively updating the probability estimates separately for each entropy slice, the coding efficiency decreases due to an increasing percentage of the data describing the sample array 10 for which the probability estimates used are not (yet) accurately adapted to the actual symbol statistics.

[0029] To overcome the just mentioned problems corresponding to the desire for low coding delay on the one hand and high coding efficiency on the other hand, the following coding scheme can now be used, which is described with reference to FIG.

[0030] First, the data describing the sample array 10 is subdivided into portions, referred to in the following as "entropy slices". The subdivision 80 does not have to be overlap-free. On the other hand, this subdivision can at least partially correspond to a spatial subdivision of the sample array 10 into different portions. That is, according to the subdivision 80, the syntax elements describing the sample array 10 can be distributed on different entropy slices depending on the position of the coding unit 50 to which the respective syntax element pertains. See, for example, FIG. 2. FIG. 2 shows an exemplary subdivision of the sample array 10 into different portions 12. Each portion corresponds to a respective entropy slice. As exemplarily shown, each portion 12 corresponds to a row of the coding unit 50. Other subdivisions are, however, possible as well. However, it is beneficial if the subdivision of the sample array 10 into portions 12 follows the coding order 66 described above, so that the portions 12 cover consecutive runs of coding units along the coding order 66. Even so, however, the beginning and ending locations of portions 12 along coding order 66 need not coincide with the left and right edges, respectively, of a row of coding units 50. Even coincidences with boundaries of coding units 50 that follow one another directly, and coding order 66 need not be mandatory.

[0031] By subdividing the sample array 10 in such a way, an entropy slice order 66 is defined among the portions 12 in which the portions 12 follow one another along the coding order 66. Moreover, for each entropy slice, a respective entropy coding pass 14 is defined, in other words a fragment of the coding pass 66 running towards each portion 12. In the embodiment of Fig. 2, in which the portions 12 correspond to the rows of the coding units 50, the entropy coding passes 14 of each entropy slice point parallel to one another along the row direction, i.e. here from the left-hand side to the right-hand side.

[0032] It should be noted that it is possible to limit the spatial prediction performed by the precoder 42 and the derivation of the context performed by the context selector 72 so as not to cross slice boundaries, i.e. so that the spatial prediction and the context selection are not determined by data corresponding to other entropy slices. In this way, an "entropy slice" corresponds to the usual definition of a "slice" in H.264, which are decodable completely independently of each other, except for the probability initialization / adaptation dependency outlined below. However, since the WPP process still allows the reversal of the precoding, i.e. the restoration based on syntax elements, as far as the entropy context selection is concerned, it is also possible to cross slice boundaries in order to allow spatial prediction and context selection, i.e. generally speaking dependencies, and to exploit local / spatial mutual dependencies. In that respect, an entropy slice corresponds in some way to a "dependent slice".

[0033] The subdivision 80 may be performed, for example, by the entropy coding stage 44. The subdivision may be fixed or may vary within the array of sequences 30. The subdivision may be fixed per convention or may be signaled within the data stream 20.

[0034] Based on the entropy slices, the actual entropy coding, i.e., 82, can take place. For each entropy slice, the entropy coding can be structured into a start phase 84 and a continue phase 86. The start phase 84 includes, for example, the initialization of probability estimates as well as the triggering of the actual entropy coding process for the respective entropy slice. The actual entropy coding is then performed during a continue phase 86. The entropy coding during phase 86 is performed along a respective entropy coding path 14. The start phase 84 for each entropy slice is controlled such that the entropy coding of the multiple entropy slices starts sequentially using the entropy slice order 16.

[0035] Here, in order to avoid the penalties outlined above resulting from the entropy coding of each entropy slice being completely independent of each other, the entropy coding process 82 is controlled such that the current part, e.g., the current coding unit of the current entropy slice, is entropy coded based on the respective probability estimates of the current entropy slice adapted using the part coded before the current entropy slice, i.e., the part of the current entropy slice to the left of the current coding unit 50 in the case of Figure 2, and the probability estimates used in its adjacent part, i.e., the adjacent coding unit, in the entropy coding of the previous entropy slice which is spatially adjacent in the entropy slice order 16.

[0036] To more clearly describe the above dependencies, reference is made to Figure 4, which indicates the n-1, n and n+1 entropy slices in the entropy slice order 16 by reference numeral 90. Each entropy slice 90 contains a sequence of syntax elements that describe the portion 12 of the sample array 10 to which the respective entropy slice 90 pertains. Along the entropy coding path 14, the entropy slice 19 is segmented into a sequence of segments 92, each of which corresponds to a respective one of the coding units 50 of the portion 12 to which the entropy slice 90 pertains.

[0037] As mentioned above, the probability estimates used in the entropy coding of the entropy slices 90 are continuously updated along the entropy coding path 14 during the continuation phase 86, so that the probability estimates become increasingly better adapted to the actual symbol statistics of each entropy slice 90 - i.e., the probability estimates are associated with each entropy slice. In FIG. 4, the probability estimates 94 used in the entropy coding of the entropy slices 90 during the continuation phase 86 are continuously updated, but only the states of the probability estimates 94 that arise at the beginning and end positions of the segments 92 are illustrated and referred to further below. In particular, the state prior to the entropy coding of the first segment 92, initialized during the start phase 84, is illustrated at 96, the state that appears after coding the first segment is illustrated at 98, and the state that appears after coding the first two segments is illustrated at 100. The same elements are illustrated in FIG. 4 for the n-1 entropy slice and the subsequent entropy slice, i.e., entropy slice n+1, in the entropy slice order 16.

[0038] Here, to achieve the dependency outlined above, the initial state 96 for the entropy coding of the nth entropy slice 90 is set by any intermediate state of the probability estimate 94 that appears during the coding of the previous entropy slice n-1. "Intermediate state" refers to any state of the probability estimate 94, except for the initial state 96 and the final state that appears after the entropy coding of the entire entropy slice n-1. By doing so, the entropy coding of the sequence of entropy slices 90 along the entropy slice order 16 can be parallelized with a degree of parallelism determined by the ratio of the number of previous segments 92, i.e., a, to the number of subsequent segments 92, i.e., b, to the state used for the entropy coding of the next entropy slice. In particular, in FIG. 4, a is exemplarily set to be equal to the adaptation of state 100 by initialization, as illustrated by arrow 104, to set the state 96 of the current entropy slice equal to the state 100 of the previous entropy slice.

[0039] With this approach, the entropy coding of any segment 92 that follows state 100 in the entropy coding pass order 14 depends on the probability estimates 94 adapted based on the previous segment of the same entropy slice during the continuation phase 86 as well as the probability estimates used in the entropy coding of the third segment 92 of the previous entropy slice 90.

[0040] Thus, the entropy coding of the entropy slices 90 can be performed in parallel in a pipelined scheduling. The only restriction imposed on the time scheduling is that the entropy coding of some entropy slices can start only after finishing the entropy coding of the a-th segment 92 of the previous entropy slice. Entropy slices 90 that directly follow each other in the entropy slice order 16 are not subject to any other restrictions regarding the time ordering of the entropy coding procedures during the continuation phase 86.

[0041] However, according to other embodiments, stronger coupling is additionally and / or alternatively used. In particular, as illustrated by the representative arrow 106 in Fig. 4, the adaptation of the probability estimates during the continuation phase 86 causes the data of the coding unit corresponding to a particular segment 92 to change the probability estimates 94 from the state at the beginning of the respective segment 92 to the end of this segment 92, thereby improving the approximation of the actual symbol statistics shown above. That is, the adaptation 106 is performed for entropy slice n-1 simply with the data of entropy slice n-1, and the same applies for the adaptation of the probability estimates 106 for entropy slice n, etc. For example, it is possible to perform the initialization described above with respect to the arrow 104 by performing the adaptation of the probability estimates 106 without further interference between the entropy slices 90. However, to facilitate approximation of the probability estimates of the actual symbol statistics, the adaptations 106 of the probability estimates of successive entropy slices can be combined, so that the adaptations 106 of the probability estimates of the previous entropy slice n-1 also influence or take into account when adapting the adaptations of the probability estimates of the current entropy slice n. This is illustrated in FIG. 4 by the arrow 108 pointing from the state 100 of the probability estimate 94 for the entropy coding of the n-1 entropy slice 90 to the state 110 of the spatially adjacent probability estimate 94 for the entropy coding of the n-1 entropy slice 90. When using the initialization of states 96 outlined above, the probability adaptation combination 108 can use, for example, any b probability estimation states that appear after the entropy coding of the b segments 92 of the previous entropy slice. More precisely, the probability estimates appearing after the entropy coding of the first segment 92 of the current entropy slice may be the result of taking into account 108 the probability estimation state resulting from the normal probability adaptation 106 and the adaptation 106 of the probability estimates during the entropy coding of the a+1-th segment 92 of the previous entropy coding. "Taking into account" may include, for example, some averaging operations. An embodiment is outlined further below.In other words, the state 98 of the probability estimates 94 for the entropy coding of the nth entropy slice 90 at the beginning of that entropy coding segment may be the result of an averaging operation between the previous state 96 of the probability estimates 94 for the entropy coding of the current entropy slice n adapted using the adaptation 106 and the previous state before the entropy coding of the a+1th segment 92 of the previous entropy slice n-1 as modified by the probability adaptation 106. Similarly, the state 100 may be the result of an averaging operation between the result of the adaptation 106 performed during the entropy coding of the current entropy slice n and the result of the probability adaptation during the entropy coding of the a+2th segment 92 of the previous entropy slice n-1, etc.

[0042] For more information, Let p(n) → {i,j}, where i,j denote the position of any coding unit (where (0,0) is the top-left position and (I,J) is the bottom-right position), i ∈ {1...I}, j ∈ {1...J}, I is the column number, J is the row number, and p() defines the pass order 66. P [i,j] denotes the probability estimate used in entropy coding unit {i,j}, T(P [i,j] ) is the P [i,j] 4 shows the result of probability adaptation 106 of Then, the probability estimates 106 of successive entropy slices 90 are combined to produce an internal adaptation of the normal entropy slice P by p(n-1) =T(P p(n) ) can be substituted according to

[0043] JPEG2025041597000001.jpg8134

[0044] where N can be 1 or ≥ 1, and {i,j}1… N, each is selected from any previous (in entropy slice order 16) entropy slice 90 and its associated part 12. The function "average" can be a weighted sum, a median function, or any other one. p(n)={i,j} is the current coding unit, and p(n+1) follows according to coding orders 14 and 66 respectively. In the proposed embodiment, p(n+1)={i+1,j}.

[0045] Preferably, {i,j}1… N For each k∈{1…N}, {i,j}1… N ={i k ,j k} and i k <i + 3, j k <j is satisfied. Here, p(n)={i,j} is the current coding unit (i.e., either the second of the current entropy slice or the following coding unit), that is, they do not exist beyond the wavefront.

[0046] In the latter variant, the time scheduling of entropy coding is more closely coupled when entropy coding entropy slices 90 in parallel. That is, during the continuation phase 86, the next segment 92 in the line of the current entropy slice can start simply by the end of the corresponding segment of the previous entropy slice at a further upper position in the entropy coding path order 14.

[0047] In other words, the above description illustrates an embodiment in which the decoder 40, and in particular the entropy encoding stage 44, is configured to perform initialization of the probability estimate 94 for an entropy slice 90, such as the nth, before decoding along the respective encoding pass 14 the first segment 92 corresponding to the first coding unit / first block 50 of the portion 12 corresponding to the nth entropy slice, using the probability estimate that appears after entropy decoding along the respective encoding pass 14 of the second coding unit / block of the portion 12 corresponding to the previous entropy slice in the entropy slice order 16. Additionally or alternatively, the decoder 40, and in particular the entropy encoding stage 44, may be configured to perform entropy decoding and probability estimation adaptation such that, for each entropy slice 90, after the current part / block / coding unit 50 of the current entropy slice is entropy decoded based on the respective probability estimates 94 of the current entropy slice 90, the respective probability estimates 94 of the current entropy slice are adapted by the probability estimates appearing in the entropy decoding of this current part of the current entropy slice and adjacent parts / blocks / coding units 50 of the previous entropy slice that are spatially adjacent, such as in the upper adjacent row in the second column to the right of the current part / block / coding unit of the current entropy slice.

[0048] As is evident from the above description, the probability estimates must be adapted / managed separately for each entropy slice 50. This can be done by sequentially processing the entropy slices and storing the probability estimate states in the respective probability estimate stores 120 (see FIG. 1), as exemplarily shown and mentioned with respect to FIG. 4, i.e. 96, 98, 100, 110 and 102. Alternatively, multiple entropy coding stages 44 can be provided to decode the entropy slices in parallel. This is illustrated in FIG. 5, where multiple instantiations of the entropy coding stage 44 are shown, each associated with a respective one of the entropy slices and a corresponding portion 12 of the sample array 10. FIG. 5 also illustrates the decoding process and its possible implementation by using parallel instantiations of the respective entropy decoding stages 130. Each of these entropy decoding stages 130 is fed by a respective one of the entropy slices conveyed by the data stream 20.

[0049] 5 illustrates that the entropy encoding stage 44 and the respective decoding stages 130 do not operate completely independently and in parallel with one another. Rather, the stored probability estimation states as stored in storage device 120 are passed from one stage corresponding to a respective entropy slice to the other stages associated with subsequent entropy slices according to entropy slice order 16.

[0050] 5 also shows, for ease of explanation, possible traversal orders for traversing possible subdivisions of the coding units 50, such as traversal orders 140 defined among the prediction blocks 52 within one coding unit 50. For all these prediction blocks 52, corresponding syntax elements are included in the respective segments 92 corresponding to the values ​​of these syntax elements, and the probability estimates 94 are adapted during the traversal of the pass 140, by adaptation during the traversal of the coding units 50 defining "T" as described above. In CABAC as per H.264 and HEVC, "T" is performed table-based, i.e. by a "table walk" that defines a transition from a current state of the probability estimates for a current context to a next state determined by the current symbol value associated with that context.

[0051] Before proceeding to FIG. 6, which shows an embodiment of a decoder corresponding to the embodiment of the encoder of FIG. 1, it should be noted that the predictive nature of the precoder 42 serves only as an illustrative embodiment. According to a further alternative embodiment, the precoder 42 can be left out by the syntax elements on which the entropy coding stage 44 operates, for example the original sample values ​​of the sample array 10. As a further alternative, the precoder 42 can be configured to form a subband decomposition of the sample array 10 as in JPEG. The optional nature of the context selector 72 has already been described above. The same applies with respect to the initializer 74. The same may be equipped differently.

[0052] Figure 6 shows a decoder 200 corresponding to the encoder 40 of Figure 1. It is clear from Figure 6 that the structure of the decoder 200 substantially mirrors the structure of the encoder 40. That is, the decoder 200 comprises an input 202 for receiving the data stream 20, followed by the concatenation of an entropy decoding stage 204 and a constructor 206. The entropy decoding stage 204 entropy decodes the entropy slices conveyed in the data stream 202 and then produces the decoded symbols si , and syntax elements, respectively, to a constructor 206, which then requests the syntax elements from the entropy decoding stage 204 by respective requests 208. In other words, the constructor 206 is also responsible for parsing the syntax element stream generated by the precoder 42 in the encoder. Thus, the constructor 206 sequentially requests the syntax elements from the entropy decoding stage 204. The entropy decoding stage 204 is constructed in substantially the same way as the entropy encoding stage 44. Thus, the same references of the internal blocks of the entropy decoding stage 204 are reused. The symbolizer 70, if present, turns the requests for syntax elements into requests for symbols, and the entropy decoding core 80 converts the requests for the syntax elements into requests for symbols, respectively, and the entropy decoding core 80 converts the requests for the syntax elements into requests for the symbols, respectively, to the respective symbol values ​​s i, the symbolizer 70 maps the sequence of received symbols forming a valid symbol word to syntax elements and forwards it to the constructor 206. The constructor 206 recovers the sample array 10 from the stream of syntax elements received from the entropy decoding stage 204, such as by using the predictive decoding outlined above. More precisely, the constructor 206 performs the decoding for each coding unit by performing the predictions 60, 62 and 64, also using the coding order 66. One or more predictions for the syntax elements or for the sample values ​​are combined additively, optionally by using prediction residuals obtained from the syntax elements of the syntax element stream. The entropy decoding core 80, just like the entropy coding core 80, can follow an arithmetic decoding concept or a probability interval partitioning entropy decoding concept. For arithmetic decoding, the entropy decoding core 80 may continuously update its internal state in the form of values ​​such as a sub-interval width value and an offset value pointing to this sub-interval. The updates are done using the arriving data stream. The current sub-interval is determined for each symbol s by the context selector 72 together with the probability estimation manager 76. i Probability estimates provided for p i The adapter 78 is subdivided similarly to the entropy coding core 80 using the symbol s i Related Context i The probability estimate of p i To update the decoded symbol value s i The adaptation of the probability estimates is performed using: Initialization by the initializer 74 is done on the same instances in the same way as on the encoding side.

[0053] The decoder of FIG. 6 operates in a manner very similar to the function of the encoder described above with respect to FIG. 3. In step 230, the sample array 10 is subdivided into entropy slices. See for example FIG. 8, which shows the data stream 20 arriving at the input 202 and in particular the entropy slices 90 contained therein. In step 230, each of these entropy slices 90 is associated with the portion 12 to which it relates, so that the above-mentioned initialization of probability estimates and adaptation of probability estimates based on the respective previous entropy slices can be performed. The subdivision of the sample array, or more precisely the association of the entropy slices 90 with their respective portions 12, can be performed by the constructor 206. The association can be achieved by different measures, such as side information contained in the data stream 20 in the portions that are not entropy coded, or by convention.

[0054] The entropy slices 90 are then entropy decoded in an entropy decoding process 232 in a manner that mirrors the encoding process 82, in other words, by performing a start phase 234 and a continue phase 236 for each entropy slice 90 in the same manner and at the same instance as the encoding procedure, with initialization and adaptation of probability estimates.

[0055] The same parallelism as described above for encoding is possible on the decoding side. The entropy decoding stage instantiations 130 shown in Figure 5 can each be embodied as shown for the entropy decoding stage 204 in Figure 6. The probability estimate storage device 240 can be used to store the state of the probability estimates for use in the entropy decoding stage 130 responsible for entropy decoding of subsequent entropy slices in the entropy coding order 16.

[0056] After the embodiments of the present application are described, the concepts described so far are described again below using different wording. Below, some further aspects of the present application are described. In the following, the above-mentioned coding unit 50 is called LCU (Largest Coding Unit), thereby complying with the terminology for the upcoming HEVC standard.

[0057] First, the probability adaptation 106 mentioned above is briefly described again with respect to FIG.

[0058] The current LCU uses probability estimates, such as CABAC probabilities, available after encoding the previous left-hand LCU. For example, the LCU in FIG. 9 denoted by x is assumed to be entropy coded using the adapted probability estimate p1 for the left-hand side of LCU x until the end of the entropy coding of the left-hand LCU. However, better results can be achieved in probability adaptation if information is used from one or more other LCUs that have already been processed and are available, not just the one on the left.

[0059] As mentioned above, in the entropy decoding and entropy coding of the entropy slice, before the coding or decoding of each LCU, a new probability estimate is calculated with the help of already existing probabilities (probability estimates) from other LCUs. More precisely, the probability adaptation can be performed not only from any previous LCU of the current entropy slice, but also from LCUs of previous entropy slices in the entropy slice order. This approach is again represented in FIG. 9. Thus, the initial probability of the current LCU, indicated by X in FIG. 9, can be calculated as follows:

[0060] JPEG2025041597000002.jpg8147

[0061] where a1,…a k is the weighting factor of the LCU.

[0062] It is tested which weight probability provides the best results. In this experiment, only the neighboring LCUs are used. The study reveals that using 75% weights from the left LCU and 25% from the upper right LCU. In Figures 10-13 the results are provided. The graphs given with "Prob.Adap" use the probability adaptation described above.

[0063] However, in adapting the probability estimates, we cannot use only the neighboring blocks. Each nearest LCU has its own special neighborhood that can be of significant use for the probability optimization. In other words, we cannot apply only the LCUs from the nearest row above. In FIG. 9, the derivation of the probability estimates is first taken from the neighborhood and then each upper right LCU (p5 and p k An example can be seen here, which can be taken from

[0064] It should be appreciated that the probability recalculation or adaptation of the probability estimates outlined above introduces some complexity. The new computation of the probability estimates occurs, for example, in three steps: First, the probability estimates for each candidate must be obtained from the state of each context. This can be done by storing them in storage devices 120 and 240, respectively, or by steering parallel entropy slice n / decoding processes such that these states are simultaneously available. Second, the optimized probabilities (p new ) is created. That is, to combine the adapted probability estimates of the different entropy slices, we can use, for example, some averaging operations. As a final step, the state of the new context is denoted by p newand replaces the old one; i.e., the probability estimation manager 76 adopts the new probability estimation thus obtained. This performance for each syntax element can greatly increase the complexity, especially with multiplication operations. The only way to reduce this penalty is to try to avoid these three steps. If the number of candidates and their weights are determined, a pre-computed table for each state can be approximated. Thus, only one simple access to the table date with the help of the index of the candidate (state of the context) is required.

[0065] It is claimed that this technique can provide good results for both applications, with and without entropy slicing. The first application uses only one slice per frame, so the probability adaptation is optimized without any other changes. In the case of entropy slicing, the probability adaptation occurs within each slice independent of the other slices. This allows for fast learning of the probability of the current LCU.

[0066] In the above description, the use of the second LCU of the upper line, i.e. the use of the second LCU for the initialization of the probability estimation, was also provided. If some conditions of the bitstream (entropy slice) mentioned above are achieved, the parallelization of the encoding and decoding is possible. The dependency of the CABAC probability between LCUs must be broken. It is important to make the first LCU of each line independent of the last LCU of the previous line by the wavefront parallel processing. That can be achieved, for example, if the CABAC probability is reinitialized at the beginning of each line of LCUs. However, this method is not optimal, since each reinitialization loses the achieved CABAC probability adapted to the specificity of the image. This penalty can be reduced if the initialization of the CABAC probability of the first LCU of each line occurs by the probability obtained after the second LCU of the previous line.

[0067] As mentioned above, gains in probability adaptation speed can be achieved by combining the probability adaptation of spatially adjacent entropy slices. In particular, in other words, the above description relates to a decoder such as the one of FIG. 6 for recovering a sample array (10) from an entropy coded data stream, configured to entropy decode (performed by an entropy decoding stage) a number of entropy slices in the entropy coded data stream, performing, for each entropy slice, entropy decoding along a respective entropy coding pass (14) using a respective probability estimate, adapting the respective probability estimate along the respective entropy coding pass using a previously decoded part of the respective entropy slice, and sequentially performing the entropy decoding of the multiple entropy slices using an entropy slice order (16). and respectively recovering different portions (12) of the sample array associated with the entropy slice by performing entropy decoding of a current part (x) of a predetermined entropy slice based on a respective probability estimate of the predetermined entropy slice adapted using a previously decoded part (e.g., including p1) of the predetermined entropy slice in the entropy decoding of the predetermined entropy slice, and a probability estimate used for an adjacent part (such as p4) of a spatially adjacent entropy slice in the entropy decoding of a spatially adjacent previous entropy slice in the entropy slice order (e.g., the slice including X).

[0068] The different parts can be rows of blocks (e.g. LCUs or macroblocks) of a sample array. The latter can be images of a video. The entropy coding pass can extend row-wise. The entropy coding, and therefore the probability adaptation as well, can be context-adaptive. The entropy slice order can generally be selected such that along the entropy slice order the different parts follow each other in a direction (16) that is angled with respect to the entropy coding pass (14) of the entropy slice, and then extend substantially parallel to each other. By this measure, the "wavefront" of the currently decoded parts of the entropy slice (such as p1, p4, p5+1 and pk in the figure) can generally be positioned along a line that forms a smaller angle with respect to the entropy slice pass than the direction of the sequence of parts. For all threads processing the slice in parallel, the wavefront can have a slope of 1y for every 2x block positions, so that the top-left reference is always there.

[0069] The decoder may be configured to perform entropy decoding along a respective entropy coding pass for each entropy slice in units of parts of the sample array portion of the respective entropy slice, such that each entropy slice consists of the same number of parts, and the sequences of parts of the parts are arranged horizontally with respect to each other along the entropy slice path. The current part of the portion of the predetermined entropy slice belongs to the resulting grid of parts (e.g., LCU or macroblock). In performing entropy decoding along a respective entropy coding pass for each entropy slice, the decoder may store an offset / shift between the start times of directly successive entropy slices in the entropy slice order, such that the wavefront of the currently decoded part of the entropy slice forms a diagonal, such as up to a line with a slope of 0.5x block position with respect to the entropy slice path and entropy slice order direction. The offset / shift may correspond to two parts for every pair of directly successive entropy slices. Alternatively, the decoder can simply avoid that the distance of the currently decoded part of the entropy slice that is directly consecutive (and directly adjacent as far as their sample array portions 12 are concerned) is less than two parts. See the figure above: as soon as a part / block p4 is decoded, the part / block to its right is decoded according to the pass order 16, and either X or the part / block before it, if any, is decoded at the same time. By this strategy, the decoder can use the already adapted probability estimate based on the content of part / block p4, i.e. the part in the spatially adjacent portion that is arranged in the part of the portion 12 of the predetermined entropy slice that follows the current part X in the pass order 16, to determine the probability estimate to be used in the decoding of X.In the case of a fixed offset in the decoding of immediate entropy slices of two parts, the decoder can simultaneously use the already adapted probability estimates based on the contents of part / block p4 for the entropy decoding of the subsequent spatially adjacent part (i.e. the part to the right of p4).

[0070] As described above, to determine the probability estimate to be used for decoding X, a weighted sum of the already adapted probability estimates can be used.

[0071] Also, as mentioned above, the entropy slice order can cross frame boundaries.

[0072] It should be noted that the just outlined probability recruitment from the previous entropy slice can be performed for all parts of the current / predetermined entropy slice for which such neighboring parts are available in the previous entropy slice, i.e. this is also true for the first part along the path direction 16, and for this very first part / block (the left-most of each entropy slice in the figure) the recruitment is equal to the initialization described above.

[0073] For better adaptation, the two above-mentioned methods can also be combined together in this case. The results of this process with 1-thread and 4-thread, i.e. parallel used processing units, are illustrated in Figures 10-13 (graphs of 2LCU+Prob.Adap or 2LCU) (Prob.Adap stands for "probability adaptation").

[0074] To better understand the context of the above embodiments and in particular the further embodiments described below, in particular the use of LCU, one can first have a look at the structure of H.264 / AVC.

[0075] A coded video sequence in H.264 / AVC consists of a series of access units collected in a NAL unit stream, which use only one set of sequence parameters. Each video sequence can be decoded independently. A coded sequence consists of a sequence of coded pictures. A coded frame can be a whole frame or a single field. Each picture is partitioned into fixed-size macroblocks (in HEVC: LCUs). Several macroblocks or LCUs can be merged together into one slice. A picture is therefore a collection of one or more slices. The goal of this data separation is to allow independent decoding of samples in an area of ​​the picture represented by a slice, without the use of data from other slices.

[0076] A technique often referred to as "entropy slicing" is the division of a conventional slice into additional sub-slices. Specifically, it means slicing the entropy coded data of a single slice. The arrangement of entropy slices in a slice can have different kinds. The simplest one is to use each row of LCUs / macroblocks in a frame as one entropy slice. Alternatively, columns or separate regions can be used as entropy slices that can be interrupted and even switched with each other, e.g. slice 1 in FIG. 14.

[0077] The obvious purpose of the entropy slice concept is to improve the time of the decoding process, i.e. to enable the use of parallel CPU / GPU and multi-core architectures to speed up the process. The current slice can be divided into partitions that can be analyzed and restored without reference to other slice data. Although a few benefits are achieved by the entropy slice approach, it also comes with some penalties.

[0078] First of all, the main objective is to generate a bitstream suitable for parallel encoding and decoding processes. Due to the use of spatial and motion information for prediction, it must be considered that an LCU can only be coded if its neighboring LCUs (left, top, top right) are already available as coded versions. To allow parallelism by slicing, a shift between the processing of slices must be realized (e.g. a shift of two LCUs, typical for the wavefront approach). For the adaptation of CABAC probabilities, LCUs use probabilities available from the previous coded LCU. With respect to the raster scan order, the problem caused by slicing an image is that it hinders parallelism, since the first LCU of each line is determined by the last LCU of the previous line. The effect of this is that the dependency of CABAC probabilities between slices must be broken so that several slices can be started at the same time. One way to do this is the usual CABAC reinitialization, but it causes all adopted data to be lost. This can result in an increase in bitrate.

[0079] Secondly, each slice can generate its own sub-bitstream and serialize it into the main stream. However, extra information needs to be passed to the decoder so that it can correctly identify these slices and their position in the main stream. Two scenarios for signaling are supported: the position information can be stored in the picture header (slice length information) or in each slice header (pointing to a start code). The byte alignment and position information at the end of each entropy slice increase the loss.

[0080] In order to reduce the penalty introduced by the signaling for entropy slices, it is important to use good coding techniques for the signaling. A visible penalty for entropy slice signaling is introduced in a frame if a start code is used for each slice, i.e., if too many additional bytes (e.g., a minimum of 4 bytes per slice) are added to the bitstream. Indeed, the insertion of entropy slices 90 using a start code has benefits in low-delay scenarios where the encoder can output the entropy slices immediately. In such cases, there is no advance signaling of possible entry points.

[0081] However, in strict scenarios with less low delay, the opportunity to store the slice length (offset) seems more appropriate. One well-known way to code such information is variable length code (VLC) or exponential-Golomb coding. The main feature of VLC is the addition of null information (zeros) before the true information. With the help of these zeros, a code can be determined that stores the information about the offset length. We propose another technique to achieve this, the scheme of which is shown in FIG. 15, where X=EntropySliceSize is the amount of bytes contained in the entropy slice. Each next X (offset) is defined as the difference in size between the offset of the previous, already coded and signaled entropy slice and the current one. The main feature of this concept is to form the difference by size X, thus reducing the amount of data, which can be added up to 3 bits, allowing the decoder to extract the appropriate information about the size of each entropy slice. Compared to VLC, the amount of bits saved in the entropy slice header can be obtained.

[0082] That is, for example, according to the embodiment of Fig. 15, a concept for entropy coding an information signal is provided, which allows higher compression rates compared to previously available concepts, despite parallel processing capabilities. According to this embodiment, a section 20 of an entropy coded data stream in which a sample array is coded comprises entropy slices such as 90 in Fig. 8 in which different portions 12 of the sample array are entropy coded, and a header illustrated by dashed line 300 in Fig. 9 comprising information representative of a starting position 302 of the entropy slice 90, measured in the entropy decoding domain, within the entropy coded data stream, said information comprising a difference value representative of the difference between the starting position of the previous entropy slice n-1 and the starting position of a predetermined entropy slice n, for a predetermined entropy slice, said difference being included in the data stream as a VLC bit sequence.

[0083] In particular, VLC bit sequences are sequences whose difference x is in the interval [0,2 a -1], [2 a ,2 b +2 a -1], [2 b +2 a ,2 c +2 b +2 a 15], a prefix of variable length y indicating that it falls at the yth position in a sequence of other numbers z, and a PCM-encoded suffix of the yth position in a sequence of lengths a, b, c, .... If a, b, c, ... are chosen to be powers of 2, then the byte ordering can be preserved if the corresponding y is added, i.e., so that a+1, b+2, c+3, etc. are all powers of 2. The number z is not limited to the three chosen exemplarily in FIG. 15.

[0084] An encoder such as in FIG. 1 converts successive start position differences into a VLC bit sequence by first determining the prefix, i.e., the subinterval in which the difference lies (the yth one), and then adding a suffix equal to the start position difference, i.e., 0, 2, a , 2 b +2 a , minus the yth other. A decoder like that in Figure 6 is designed to derive the starting position of the current entropy slice n from the VLC bit sequence, i.e., first check the prefix to get y, then take the difference as the value of the suffix plus 0, 2 a , 2 b +2 a , the other y-th, and then adding the difference to the starting point of the previous entro slice n-1, which is then designed accordingly.

[0085] Further benefits can be achieved by segmenting the entropy slices, i.e. for lower delay transmission and faster decoding.

[0086] In video streaming, the realization of higher resolutions (Full HD, Quad HD, etc.) leads to higher transmitted data amounts. For time-sensitive scenarios, i.e. low latency use cases (<145ms), the transfer time becomes a critical factor. Consider an ADSL upload link for a video conferencing application, where the so-called random access points of the stream (usually these are called I-frames) are candidates to create a bottleneck during the transmission.

[0087] To solve the problem and minimize the transmission and decoding time delay, i.e., end-to-end delay, a new technique can be applied for the interleaved entropy slicing scheme for parallel transmission and processing.

[0088] HEVC allows so-called wavefront processing on the decoder side. It is enabled by the use of entropy slices. In the normal case, the data of the whole slice is delivered at once. The decoding process starts as soon as the encoded data arrives at the wavefront-decoder-engine. To reduce the time that the decoder can start and finish one frame, according to an embodiment of the invention, a segmentation of the entropy slices into small chunks using an interleaving approach is used. Hence, the encoder can deliver the data corresponding to a particular entropy slice to the transport layer earlier than in the normal case. It results in a faster transmission and an earlier start of the parallel decoding process at the client.

[0089] The generation of chunks of slices can be further achieved by subdividing the entropy slice in further slices (dependent slices) that preserve all dependencies. When doing so at each Largest Coding Unit (LCU) / Coding Tree Unit (CTU), these chunks can be interleaved by additionally using a system layer technique that allows the chunks to be transmitted in an interleaved format and restored by additional signaling or at least providing knowledge of the original decoding order of the chunks. Such signaling can be the Decoding Order Number (DON) as defined in the IETF RTP Payload Format for H.264 / AVC (RFC3984). Another system method can apply the chunks of the wavefront substream to different transport streams, such as in the MEPG-2 system, by assigning different PIDs to each of them and further multiplexing and thereby interleaving them on the transport channel.

[0090] The approach can also be applied across frame boundaries, if slices or entropy slices of subsequent frames can already be decoded, for example in the wavefront method, based on the knowledge that the information required for decoding the entropy slices of subsequent frames is based on the availability of interframe references. The data of these already decodable frames following in the decoding order can be derived from the maximum possible / signaled motion vector length or additional information in the stream indicating the dependency of the data part on the previous frame, or a fixed reference scheme indicating the position signaled at a fixed position in the sequence, such as a parameter set. This is outlined further below.

[0091] An image can be coded by the maximum coding unit (LCU) depicted in Figures 2 and 9 - one entropy slice per row. It is advantageous for the use of wavefront techniques at the decoder side. During the coding process, the bitstream of each slice can be divided into fixed-size segments. The resulting segments can then be interleaved and delivered for transmission. Fixed-size segments can cause problems at the end of the bitstream due to their variable length.

[0092] Here, there are two possible general solutions: The first solution is to generate 1-byte segments (usually the bitstream representation of a slice is byte-aligned) and control the consumption of bytes by each decoder engine, i.e. the decoder discovers the completion of an entropy slice.

[0093] A second solution is to use a finalization code at the end of the slice. In the last case, variable length segments are possible, but that may also lead to a higher amount of data. Another method is the signaling of the entropy slice length. One such variant is described below.

[0094] The size of the segments and the interleaving mode can be signaled either in one SEI message or in the SPS.

[0095] The transmission scheme is shown in FIG.

[0096] Thus, for example, according to the embodiment of Fig. 16, a concept of entropy coding of sample arrays is provided, which allows lower delays compared to previously available concepts. According to this embodiment, the coded data stream 20, in which an information signal is coded, comprises slices such as entropy slices 90 or just completely independently decodable slices (shown on the left hand side) in which different portions 12 of the information signal are (predictively and / or entropy) coded, the slices 90 being subdivided in the coded data stream 20 into chunks (shown on the right hand side) that are arranged in an interleaved manner, the interleaving being represented by brackets 312.

[0097] As indicated above and described with respect to other aspects, the slices may be entropy slices 90, which may in turn be a suitable subset of the slices of the frame, and the encoded data stream may thus be an entropy encoded data stream 20.

[0098] Interleaving 312 of slices 90 allows for lower latency because a decoder responsible for decoding any slice 90 does not have to wait for the duration consumed by the previous slice (per slice order 16) of other decoders. Rather, all available decoders can start decoding their associated slices as soon as their first chunk 310 is available, eliminating potential inter-slice dependencies (see wavefront approach).

[0099] The different portions 12 may be entropy coded into entropy slices 90 using probability estimates set independently within the entropy slice, or, for example, using an entropy slice-to-entropy slice adoption of the probability estimates described above, in other words, the different portions 12 are coded into entropy slices 90 along respective entropy coding passes 14, with lower entropy slices having their corresponding portions 12 coded therein using probability estimates derived from the probability estimates used in higher ranked entropy slices, as previously described, particularly in spatially adjacent parts of the corresponding portions.

[0100] The entropy coded data stream 20 may additionally comprise a header, shown as an option in FIG. 16. The header 300 may relate to the frames (sample arrays) of the sequence 30, the header 300 comprising information representing the length of the entropy slices. Information about the length of the entropy slices 90 may be coded in the header 300 described above using a VLC code. With knowledge of the length of the entropy slices it is possible to identify at the decoding side the last chunk associated with each of the entropy slices 90 and its length. However, start codes or other indication schemes may also be used. The start position of the slices may also be identified by the decoding process knowing exactly where the slices end. It may therefore be possible to just rely on the decoder indication, but this may require signalling between decoders in some cases in case the "earlier" entropy slices end later in the stream than the "later" slices. This may require "adaptive" signalling, which may be based on start codes in the stream for certain cases.

[0101] For example, the entropy slice 90 may be placed after the header 300, as illustrated in FIG.

[0102] The chunks 310 may be of equal length, at least as far as the contiguous portion at the beginning of the sequence of chunks 310 from the first one of the chunks according to the order in which the chunks are arranged in the entropy coded data stream 20. The subsequent chunks may vary in length. The subsequent chunks may be equal to the length of the contiguous portion at the beginning of the sequence or may be less than this length. The length of the subsequent chunks may be derived from the above-mentioned information in the header 300 indicating the length or starting position of the entropy slice 90. The chunks 310 may be arranged in the entropy coded data stream 20 in a cyclic manner according to the order defined in the entropy slice. Chunks that are completely within the previous cycle relative to the entropy slice may be skipped in the current and subsequent cycles.

[0103] Other information signals, then, sample array sequences, such as a video signal, can similarly be conveyed by the data stream 20. Thus, the different portions 12 need not be portions of a predefined sample array, such as an image / frame.

[0104] As mentioned above, the entropy slice 90 can have different portions 12 of the sample array 10 coded therein by predictive coding, using entropy inter-slice prediction and / or entropy coding of the prediction residual of the inter-frame prediction. That is, as mentioned above, the different portions can be spatially distinct portions of the frame 10 or multiple frames 30. The latter case applies when the slices or entropy slices of the following frames can already be decoded, for example in the wavefront method, based on knowledge due to the inter-frame references that the information required for the decoding of the entropy slices of the following frames is available. These already decodable data of the following frames in the decoding order can be derived from the maximum allowed / signaled motion vector length or additional information in the stream indicating the dependency of the data parts of the previous frames, the entropy inter-slice prediction can include intra prediction, while the inter-frame prediction can include motion compensated prediction. An example is outlined below.

[0105] The independence of probability estimate setting within an entropy slice described above can relate to both probability adaptation as well as context modeling: the context selected within an entropy slice can be selected independently from other entropy slices, and the probability estimates of a context can likewise be initialized and adapted independently from any other entropy slice.

[0106] A corresponding entropy decoder can be constructed as follows.

[0107] An entropy decoder configured to decode an entropy coded data stream 20 comprising entropy slices 90 in which different portions of a frame are entropy coded, the entropy slices being subdivided into chunks 310 arranged in an interleaved manner within the entropy coded data stream 20, may be configured as shown in FIG. 6 and may additionally comprise a deinterleaver, represented as 314 in FIG. 16, configured to deinterleave the chunks 310.

[0108] In particular, as illustrated in FIG. 5, the entropy decoder may include multiple entropy decoders 130, such as threads running on different processing kernels, and the deinterleaver may be configured to forward, for each entropy slice, its chunk 310 to the entropy decoder 44 associated with the respective entropy slice.

[0109] In other words, the entropy slices can be subdivided into chunks and then interleaved, and the decoder can be equipped with a deinterleaver that deinterleaves the chunks, and operations on the entropy slices can begin in parallel along path 16 even before reception of any entropy slice as a whole. It should be recalled that the length of the chunks is preferably measured in the entropy coding domain rather than the syntax domain (although the latter option is available as well), so as to correspond to a number of particular spatial parts / blocks in, for example, an image, etc.

[0110] Next, the use of temporal dependencies is described, which can be used in addition to or instead of the extended forms of probability estimation described so far.

[0111] The wavefront processing pattern can be extended for entropy coding, as described herein, by a new probability calculation for each LCU that also uses the inter-frame temporal dependency.

[0112] As is known, the probabilities are reinitialized at the beginning of each frame (first LCU), which causes the probabilities already obtained in the previous frame to disappear. To reduce the loss of coding efficiency, the probabilities of the end state of the image (320) or, in case of using an entropy slice, the end state of the slice (322) can be delivered from the reference frame 324 to the first LCU of the current frame 10 or entropy slice 12, respectively (FIG. 17). Since the parallel wavefront processing can also cross frame boundaries, i.e., during the coding of a slice of a frame, the coding process of the slice of the previous frame may not have been finished already, such data of the corresponding slice in the reference frame can be derived not only from the end position, but also from the previous position in the referenced slice. Therefore, signaling can be used to indicate the reference position or can be indicated by a scheme.

[0113] JPEG2025041597000003.jpg119168

[0114] The analysis process of the state at the end of the reference frame is tested by the method of probability adaptation, the results of which are illustrated on Figures 10 to 19 (temporal graphs).

[0115] Another opportunity to use dates from other frames is to exchange the obtained probabilities between aligned LCUs. The main idea is based on the assertion that the nature of the reference frame is not very different from the current frame. To speed up the learning of probabilities along LCUs in a frame, we can try to hand over the end state of each LCU in the current frame to the appropriate LCU. This proposal is illustrated in Figure 18.

[0116] Under the reference frame, different opportunities can be realized. For example, the last coded frame can be used as the reference frame. Alternatively, the last coded frame only from the same temporal layer can be applied for reference.

[0117] Moreover, this approach can be merged with the methods already proposed above, such as the use of last (slice) information from the reference frame, probability adaptation, and the use of the second LCU in the upper line.

[0118] The above spatial adaptation process can be modified as follows.

[0119] JPEG2025041597000004.jpg12170

[0120] where N can be 1 or ≥ 1, and {i,j}1… N are selected from (among) any previous (in the entropy slice order 16) entropy slice 90 in the current sample array 10 and its associated portion 12, respectively, M can be 1 or more, and {i,j}′1... M is in the previous sample array 350. CU50{i,j}'1... M At least one of CU50{i,j}1... can be arranged in the same position with respect to p(n). N Regarding the possible choices of , reference is made to the above. The function "average" can be either a weighted sum, a median function or others.

[0121] The above spatial adaptation process can be replaced by the following:

[0122] JPEG2025041597000005.jpg10170

[0123] where M can be 1 or ≥ 1, and {i,j}'1… Mis in the previous sample array. {i,j}'1... M At least one of {i,j}1... can be arranged in the same position with respect to p(n). N Regarding possible choices of {i,j}'1..., reference is made to the above. The function "average" can be a weighted sum, a median function, or any other. M At least one of may be arranged in the same position with respect to p(n).

[0124] As a particular extension of the use of aligned information, an approach can be applied that utilizes dates obtained from other blocks from one or more reference frames.

[0125] The techniques mentioned earlier use only information obtained from direct neighbors in the current frame or in the reference frame. However, that does not mean that the probability obtained in this case is the best one. Neighboring LCUs do not necessarily have the best probability model due to image segmentation (residuals). It is assumed that the best results can be achieved with the help of the block where the prediction is made. And, therefore, this suitable block can be used as a reference for the current LCU.

[0126] Thus, in the above adaptation example, {i,j}1... N and / or {i,j}'1… M can be selected by the CU, which serves as the provider of the predictor for p(n).

[0127] The proposed temporal probability adaptation / initialization scheme can also be used without entropy slicing or with a single entropy slice per frame.

[0128] According to the latter aspect, gains in probability adaptation speed are achieved by combining the probability adaptation of temporally adjacent / related frames. Described therein is a decoder as in Fig. 6, configured to recover a sequence of sample arrays from an entropy coded data stream and configured to entropy decode a current frame of the entropy coded data stream to recover a current sample array of the sequence of sample arrays, perform entropy decoding along the entropy coding path using a probability estimate, and adapt the probability estimate along the entropy coding path using a previously decoded part of the current frame, the entropy decoding stage configured to initialize or determine the probability estimate for the current frame based on the probability estimate used in decoding a previously decoded frame of the entropy coded data stream.

[0129] That is, for example, the probability estimates for the current frame are initialized based on the resulting probability estimates after finishing the decoding of a previously decoded frame of the entropy coded data stream. The buffer requirements are therefore low, since only the end states of the probability estimates have to be buffered until the beginning of the decoding of the current frame. Of course, this embodiment can be combined with the embodiment of figures 1-9 in that for the first part of each portion 12, not only the probability estimates used for the spatially adjacent parts in the previous entropy slice (if available) are used, but also the end states of the probability estimates of the corresponding (e.g. spatially) entropy slice in the previous frame, for example in a weighted manner. Since the parallel wavefront processing can also cross frame boundaries, i.e. during the encoding of a slice of a frame, the encoding process of the slice of the previous frame may not have already finished, such data of the corresponding slice in the reference frame can be derived not only from the end position, but also from the previous position in the referenced slice. Hence, signaling can be used to indicate the reference position, or can be indicated by a scheme.

[0130] Further, for example, the probability estimates used for the encoding of previously decoded parts / blocks of a frame are all buffered, not just the final state, and the decoder performs entropy decoding of the current part (X) of the predetermined entropy slice based on the respective probability estimates of the predetermined entropy slice adapted using the previously decoded parts (e.g. including p1) of the predetermined entropy slice in the entropy encoding of the predetermined entropy slice (see the above description of the probability derivation of spatial combinations) and the probability estimates used in the entropy decoding of the spatially corresponding parts of the entropy slice of the previously decoded frame, as described above, and optionally by additionally using the probability estimates used in the entropy decoding of the spatially adjacent previous entropy slice in the entropy slice order (e.g. the slice including X) in the spatially adjacent adjacent parts (such as p4) of the previous entropy slice. Also, as described above, spatial correspondence within a part and identification of previously decoded frames that are suitable for probability adoption for the current frame can be determined with the aid of motion information such as motion index, motion vector, etc. of the current part / block.

[0131] So far, the wavefront extending during the wavefront processing has been described as extending mainly diagonally through one sample array 10, with the encoding / decoding being performed in the order of the sample arrays. However, this is not necessary. Reference is now made to FIG. 19. FIG. 19 shows a portion from a sequence of sample arrays, the sample arrays of which are represented as being arranged in a sample array encoding order 380 defined therein and which may or may not coincide with the time order of presentation. FIG. 19 exemplarily shows a subdivision of the sample array 10 into four entropy slices each. The entropy slices already encoded / decoded are shown hatched. Four encoding / decoding threads (encoding / decoding stages) 382 are currently operating on the four entropy slices 12 of the sample array with index n. However, Fig. 19 shows thread number 5 on the left, and this further thread 382, ​​having number 5 in Fig. 19, can operate to encode / decode the next sample array in the line, i.e. n+1, for which it is guaranteed that in the currently encoded / decoded frame n the respective reference parts are already available, i.e. have already been processed by one of the threads 1 to 4. These parts are for example referenced in the prediction indicated by 64 in Fig. 1.

[0132] 19 exemplarily shows by a dashed line 384 a line extending through sample array n+1, which is co-located with respect to the boundary between, on the one hand, the already processed, i.e. already coded / decoded, part of sample array n, i.e. the hatched part in the sample array, and, on the other hand, the not yet processed, part of sample array n, i.e. the non-hatched part of the sample array. Fig. 19 also shows by double arrows the maximum possible length of a motion vector measured in the column and row directions, i.e. y max and x max Thus, FIG. 19 also shows, by dashed line 386, a displaced version of line 384, in other words a distance y in the column direction.max , x in row direction max 19. As shown, the coding units 50 in sample array n+1, where any reference portion in sample array n is guaranteed to be found, are located so as to be completely contained within the already processed portion of sample array n, in other words those set to half of sample array n+1, are upstream with respect to line 386. Thus, thread 5 can already operate to decode / encode these coding units, as shown in FIG. 19. Even the sixth thread can operate on the second entropy slice in the entropy slice order 16 of sample array n+1, as shown in the figure. This causes the wavefront to extend not only spatially but also temporally, by a spatial-temporal space across the sequence 30 of sample arrays.

[0133] Note that the wavefront aspect just mentioned also works in combination with the probability estimation combination outlined above across entropy slice boundaries.

[0134] Furthermore, with regard to the chunking aspects outlined above, it should also be noted that the subdivision of an entropy slice into smaller pieces, i.e. chunks, is not limited to being performed in the entropy coding domain, i.e. in the entropy compression domain. Consider the above discussion: the entropy slicing described above has the benefit of reducing the loss of coding efficiency despite the possibility of wavefront processing by deriving probability estimates from previously coded / decoded entropy slices or previously coded / decoded frames, i.e. initializing and / or adapting the probability estimates based on the probability estimates of such previous entropy slices. Each of these entropy slices is assumed to be entropy coded / decoded by one thread in the case of wavefront processing. That is, when subdividing an entropy slice, it is not necessary to render the chunks codable / decodable in parallel. Rather, the encoder merely provides an opportunity to output a sub-part of the bitstream for that entropy slice before finalizing the entropy coding, and the decoder provides an opportunity to operate on these sub-parts, i.e. chunks, before receiving the remaining chunks of the same entropy slice. Moreover, interleaving is possible at the receiving side. However, it is not necessary to perform the subdivision in the entropy coding domain to allow for later de-interleaving. In particular, by simply resetting the internal state of the probability intervals intermittently, i.e., the probability interval width value and the offset value, respectively, of the entropy coding / decoding core, the above proposed subdivision of the entropy slice into smaller chunks can be performed without severe loss of coding efficiency. However, the probability estimates are not reset. Rather, they are continuously updated / adapted from the beginning to the end of the entropy slice, respectively. With this approach, it is possible to sub-divide the entropy slice into individual chunks, with the subdivision being performed in the syntax element domain rather than in the compressed bitstream domain.The subdivision may follow the spatial subdivision outlined below to facilitate signaling of the chunk interface to the decoder: Each chunk may be provided with its own chunk header indicating its starting position in the sample array, measured with respect to the coding order 14, for example relative to the starting position of the respective entropy slice together with an index for that entropy slice, or relative to a salient position of the sample array 10, such as the upper left corner.

[0135] To more clearly describe the subdivision of the entropy slices into chunks according to the latter embodiment, reference is made to FIG. 20. FIG. 20 shows the sample array 10, which is subdivided into four entropy slices, purely for purposes of illustration. The currently encoded portion of the sample array 10 is shown hatched. Three threads are currently working on the entropy coding of the sample array 10, outputting chunks of entropy slices on a first-aid basis: see for example the first entropy slice in the entropy slice order 16, which corresponds to the portion 12 of the sample array 10. After coding the subpart 12a of the portion 12, the encoder performs some finalization procedures to finalize the arithmetic bitstream produced so far from the subpart 12a, in case the entropy coding core 80 arithmetically codes it to form the chunk 390 therefrom. The coding procedure is then resumed with respect to the subsequent subpart 12b of the entropy slice 12 in the coding order 14, starting a new entropy bitstream. This means that the internal state, such as the probability interval width and probability interval offset values ​​of the entropy coding core 80, is reset. However, the probability estimates are not reset; they are left unchanged. This is illustrated by arrow 392 in Fig. 20. In Fig. 20, it is exemplarily shown that the entropy slice or portion 12 is subdivided into two or more sub-portions, such that even the second chunk 1b follows some entropy that ends before reaching the end of the portion 12 along the coding order 14, where the next chunk in the line is started by others.

[0136] At the same time, the other thread operates on the second entropy slice or portion 12 in the entropy slice order 16. Upon finishing the first sub-portion of this second entropy slice / portion 12, chunk 2a is output, where entropy coding of the remainder of the second entropy slice begins, however, keeping the probability estimates valid at the end of chunk 2a.

[0137] Figure 20 seeks to illustrate, via a timeline 394, that chunks 390 are output as soon as they are finalized. This leads to an interleaving similar to that depicted in Figure 16. Each chunk is packetized into a packet and can be transported to the decoding side by some transport layer in any order. The transport layer is illustrated with arrow 396.

[0138] The decoder must reassign the chunks to their sub-portions 12a, 12b, etc. To this end, each chunk 390 may have a header section 398 that indicates the location of the beginning of its associated sub-portion 12a or 12b, i.e., the sub-portion whose describing syntax elements are entropy coded in the respective chunk. Using this information, the decoder can associate each chunk 390 with its entropy slice and with its sub-portion within portion 12 of the entropy slice.

[0139] For ease of explanation, Fig. 20 also exemplarily illustrates the possibility that the connection between successive sub-portions 12a, 12b of the entropy slice 12 need not coincide with the boundary between successive coding units 50. Rather, the connection can be defined at a deeper level of the exemplary multi-tree subdivision of the coding units described above. The position information included in the header 398 can indicate the start of the sub-portion associated with the current chunk 390 sufficiently precisely to identify the respective sub-block of the respective coding unit, i.e., the position within the sequence of syntax elements described with respect to the respective sub-block.

[0140] As is clear from the above discussion, little loss of coding efficiency results from the subdivision of entropy slices into chunks. The mere entropy termination process and packetization may involve some coding efficiency loss, but on the other hand the benefits of low latency are enormous.

[0141] Note also that the spatial subdivision chunk aspect just mentioned also works in combination with the probability estimation combination outlined above, both spatially and temporally, and across entropy slice boundaries.

[0142] A decoder such as the decoder of FIG. 6 can cancel chunk transmission as follows. In particular, the decoder can check to which entropy slice the current chunk belongs. This check can be made based on the position information mentioned above. It can then check whether the current chunk corresponds to a first sub-portion of the portion of the corresponding entropy slice along the entropy coding path 14. If so, the decoder can take into account the state of the respective probability estimates appearing at the end of the entropy decoding of the current chunk when entropy decoding the current chunk under adaptation of the respective probability estimates and entropy decoding other chunks corresponding to second sub-portions of the portion of the predefined entropy slice along the entropy coding path. "Taking into account" can include setting the probability estimates at the beginning of chunk 1b at the end of sub-portion 12a of chunk 1a equal to the probability estimates appearing by probability adaptation starting from the state of the probability estimates at the beginning of chunk 1a, or a combination of it with the entropy probability estimates from other entropy slices as described above. As far as the probability initialization at the beginning of the first chunk 12a is concerned, reference is made to the above discussion, as this also forms the beginning of the corresponding entropy slice. In other words, if the current slice is the second or subsequent chunk in the order 14, the decoder can entropy decode the current chunk using probability estimates that depend on the probability estimates appearing at the end of the entropy decoding of a chunk that corresponds to a sub-portion of a portion of the previous predetermined entropy slice along the entropy encoding path 14 for the sub-portion that corresponds to the current chunk.

[0143] The above description reveals different methods that can be useful for parallel encoding and decoding, as well as for optimizing the processes already present in the emerging HEVC video coding standard. A brief overview of entropy slices has been presented. It has been shown how they can be formed, what benefits can be achieved by slicing, and what penalties can result from these techniques. Many methods have been proposed that are supposed to improve the learning process of probabilities along LCUs (largest coding units) in a frame by better exploiting the local dependencies between LCUs and also the temporal dependencies between LCUs of different frames. It is claimed that different combinations offer improvements over the concept both with and without parallelization of encoding and decoding.

[0144] For example, the performance improvement in high efficiency with the best mix of the proposed approaches is -0.4% in intra, -0.78% in low latency and -0.63% in random access compared to HM3.0 without entropy slicing, or -0.7% in intra, -1.95% in low latency and -1.5% in random access compared to the entropy slicing approach with regular reinitialization.

[0145] In particular, among others, the following techniques have been presented above: Exploiting the local as well as temporal dependencies of LCUs to optimize CABAC probability adaptation before encoding each LCU (see Figures 1-9, 17 and 18). It allows for more flexibility in decoding and also makes use of entropy slicing to make certain regions in a frame independent of each other. · Parallelization, e.g. allowing minimal signaling of slice / entropy slice start positions for wavefront processing (see Fig. 15). · Interleaved transport of entropy slices / slices allows low latency transport in a parallelized encoder-transmitter-receiver-decoder environment (see Figure 16).

[0146] All the above mentioned methods have been integrated and tested on HM3.0. The obtained results, where the reference point is HM3.0 without any entropy slicing implementation, are presented in Tables 1 and 2 (where 2LCU is the use of the second LCU in the upper line, 2LCU+Prob.Adap is the use of 2LCU merged with the method of probability adaptation, and Temporal is the use of temporal dependency (final state of the reference frame) with probability adaptation for each LCU).

[0147] JPEG2025041597000006.jpg28114

[0148] JPEG2025041597000007.jpg28114

[0149] However, it is interesting to see how the proposed approach, by reinitializing the probabilities at the beginning of each line of LCUs, affects the wavefront processing. These results are illustrated in Tables 3 and 4 (where orig_neiInit is a comparison between HM3.0 without entropy slicing and the use of entropy slicing with initialization).

[0150] JPEG2025041597000008.jpg29130

[0151] JPEG2025041597000009.jpg28130

[0152] The above results show that a significant use of intra-frame and inter-frame dependencies and a rational application of already acquired information prevent average losses.

[0153] The wavefront processing approach to HEVC video encoding and decoding merges the possibility of using dependencies between adjacent LCUs as well as temporal frame dependencies through the concept of wavefront parallel processing. In this way, losses can be reduced and performance advances can be achieved.

[0154] Gains in probability adaptation speed have been achieved by computing probability adaptation of spatially adjacent entropy slices.

[0155] As already mentioned above, all the above aspects can be combined with each other, so that the mention of a particular possibility for implementation with respect to a particular aspect naturally also applies to the other aspects.

[0156] Although some aspects have been described in the context of an apparatus, it will be apparent that these aspects also represent a description of a corresponding method, where a block or device corresponds to a method step or a function of a method step. Similarly, aspects described in the context of a method step also represent a description of a corresponding block, item or function of a corresponding apparatus. Some or all of the method steps can be performed by (or using) a hardware apparatus, such as, for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some of the most important method steps can be performed by such an apparatus.

[0157] The encoded signals of the present invention described above can be stored on a digital storage medium or can be transmitted over a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0158] Depending on specific implementation requirements, embodiments of the present invention can be implemented in hardware or in software. The implementation can be carried out using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM or flash memory, having electronically readable control signals stored thereon and cooperating (or capable of cooperating) with a programmable computer system such that the respective method is performed. Hence, the digital storage medium can be computer readable.

[0159] Some embodiments of the present invention comprise a data carrier having electronically readable control signals capable of cooperating with a programmable computer system to cause one of the methods described herein to be performed.

[0160] Generally, embodiments of the present invention may be implemented as a computer program product with program code operable to perform one of the methods of the present invention when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.

[0161] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0162] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0163] A further embodiment of the inventive method is therefore a data carrier (or digital storage medium or computer readable medium) comprising a computer program recorded thereon for performing one of the methods described herein. The data carrier, digital storage medium or recording medium is typically tangible and / or non-transitory.

[0164] A further embodiment of the inventive method is therefore a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals can for example be arranged to be transmitted by a data communication connection, for example the Internet.

[0165] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.

[0166] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0167] Further embodiments according to the invention comprise an apparatus or system configured to transfer (e.g. electronically or optically) a computer program for performing one of the methods described herein to a recipient. The recipient may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may comprise, for example, a file server for transferring the computer program to the recipient.

[0168] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0169] The above described embodiments are merely illustrative of the principles of the present invention. It is understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. It is therefore intended that the present invention be limited only by the scope of the impending claims and not by the specific details expressed by the description and illustration of the embodiments herein.

Claims

1. 1. A decoder for recovering a video sample array from an entropy encoded data stream, the decoder comprising: an entropy decoder configured to entropy decode a plurality of entropy slices in the entropy encoded data stream to recover the sample array, each of the plurality of entropy slices corresponding to a row of a plurality of rows of the sample array, each of the plurality of rows including the same number of blocks. Equipped with For a current entropy slice of the plurality of entropy slices, the entropy decoder: during a starting phase of entropy decoding, initialize a first probability estimate for the current entropy slice before decoding a first block of a current row corresponding to the current entropy slice based on a second probability estimate obtained after completing entropy decoding of a previously decoded block of a previous row corresponding to a previous entropy slice of the multiple entropy slices; wherein the first block is the leftmost block of the current row, and the previously decoded block is adjacent to the first block, and the current row and previous row are successive rows of the sample array; during a continuing phase of the entropy decoding, performing entropy decoding using the first probability estimates along an entropy coding path spanning the current row from left to right, where the entropy decoding is performed by adapting the first probability estimates along the entropy coding path using only previously decoded portions of the current entropy slice. decoder.

2. 2. The decoder of claim 1, wherein the entropy decoder is configured to entropy decode the entropy slices in a sequence, the sequence being selected such that the rows follow each other along the sequence in a diagonal direction relative to the entropy coding pass of the entropy slice.

3. 2. The decoder of claim 1, wherein the entropy decoder is configured to store the second probability estimates after the previously decoded block is entropy decoded and to use the stored second probability estimates to initialize the first probability estimates.

4. 2. The decoder of claim 1, wherein the entropy decoder is configured to, when entropy decoding a current portion of the current entropy slice, adapt the first probability estimate based on a third probability estimate obtained after decoding a portion of the previous entropy slice during the continued phase of the entropy decoding.

5. 2. The decoder of claim 1, wherein the previous row is spatially above the current row, and the entropy decoding of the previous entropy slice begins before the entropy decoding of the current entropy slice.

6. 2. The decoder of claim 1 , wherein the sample array is a current sample array of a sequence of sample arrays, and the entropy decoder is configured to entropy decode a current entropy slice based on the second probability estimates, the previously decoded portion of the current entropy slice, and the first probability estimates adapted with a fourth probability estimate used in entropy decoding a previously decoded frame corresponding to one of the sample arrays other than the current sample array.

7. 10. The decoder of claim 1, wherein the decoder is at least part of a programmable logic device, a programmable gate array, a microprocessor, a computer, or a hardware apparatus.

8. 1. A method for recovering a video sample array from the video encoded entropy coded data stream, the method comprising: entropy decoding a plurality of entropy slices in the entropy encoded data stream to recover the sample array, each of the plurality of entropy slices corresponding to one of a plurality of rows of the sample array, each of the plurality of rows including the same number of blocks; during a start phase of the entropy decoding step, for a current entropy slice of the plurality of entropy slices, initializing a first probability estimate for the current entropy slice before decoding a first block of a current row corresponding to the current entropy slice based on a second probability estimate obtained after completing entropy decoding of a previously decoded block of a previous row corresponding to a previous entropy slice of the plurality of entropy slices; wherein the first block is the leftmost block of the current row, and the previously decoded block is adjacent to the first block, and the current row and previous row are successive rows of the sample array; the entropy decoding step is performed using the first probability estimates along an entropy coding path spanning the current row from left to right during a continuing phase of the entropy decoding step, where the entropy decoding step includes adapting the first probability estimates along the entropy coding path using only previously decoded portions of the current entropy slice. method.

9. 9. The method of claim 8, wherein the step of entropy decoding the entropy slices is performed in a sequence, the sequence being selected such that the rows follow each other along the sequence in a diagonal direction relative to the entropy coding pass of the entropy slices.

10. 9. The method of claim 8, further comprising: storing the second probability estimate after the previously decoded block is entropy decoded; and using the stored second probability estimate to initialize the first probability estimate.

11. 9. The method of claim 8, further comprising: when entropy decoding a current portion of the current entropy slice, adapting the first probability estimate based on a third probability estimate obtained after decoding a portion of the previous entropy slice.

12. The method of claim 8 , wherein the previous row is spatially above the current row, and the entropy decoding of the previous entropy slice begins before the entropy decoding of the current entropy slice.

13. 9. The method of claim 8, wherein the sample array is a current sample array of a sequence of sample arrays, the method comprising entropy decoding the current entropy slice based on the second probability estimate, the first probability estimate adapted with the previously decoded portion of the current entropy slice, and a fourth probability estimate used in entropy decoding a previously decoded frame corresponding to one of the sample arrays other than the current sample array.

14. 1. A method for encoding a sample array into a video encoded entropy coded data stream, comprising: entropy encoding a plurality of entropy slices into the entropy encoded data stream, each of the plurality of entropy slices corresponding to a row of a plurality of rows of the sample array, each of the plurality of rows including a same number of blocks; initializing, during a start phase of the entropy coding step, a first probability estimate for a current entropy slice of the plurality of entropy slices, before decoding a first row of the current row corresponding to the current entropy slice, based on a second probability estimate obtained after completing entropy coding of a previously coded block of a previous row corresponding to a previous entropy slice of the plurality of entropy slices; Including, wherein the first block is the leftmost block of the current row, and the previously encoded block is adjacent to the first block, and the current row and previous row are successive rows of the sample array; the entropy encoding step is performed using the first probability estimates along an entropy encoding pass spanning the current row from left to right during a continuing phase of the entropy encoding step, where the entropy encoding step includes adapting the first probability estimates along the entropy encoding pass using only previously encoded portions of the current entropy slice. method.

15. 15. The method of claim 14, wherein the step of entropy encoding the plurality of entropy slices is performed in sequence, and the entropy slice order is selected such that the different portions follow each other in the entropy slice order in a diagonal direction relative to the entropy encoding pass of the entropy slices.

16. 15. The method of claim 14, further comprising storing the second probability estimates after the previously coded block is entropy coded, the stored second probability estimates being used to initialize the first probability estimates.

17. 15. The method of claim 14, further comprising: when entropy encoding a current portion of the current entropy slice, adapting the first probability estimate based on a third probability estimate obtained after encoding a portion of the previous entropy slice.

18. The method of claim 14 , wherein the previous row is spatially above the current row, and the entropy coding of the previous entropy slice begins before the entropy coding of the current entropy slice.

19. 15. The method of claim 14, wherein the sample array is a current sample array of a sequence of sample arrays, the method comprising: entropy coding the current entropy slice based on the second probability estimate, a first probability estimate adapted using the previously coded portion of the current entropy slice, and a fourth probability estimate used in entropy coding a previously coded frame corresponding to one of the sample arrays other than the current sample array.

Images