Sample array coding for low latency

Abstract

To provide a decoder and a method for improving efficiency of entropy coding.SOLUTION: Entropy coding of a current part of a predetermined entropy slice is based on, not only, respective probability estimations of the predetermined entropy slice as adapted using a previously coded part of the predetermined entropy slice, but also probability estimations as used in the entropy coding of a spatially neighboring, in previous entropy slice of the entropy slice order at a neighboring part thereof. Thereby, the probability estimations used in entropy coding are adapted to actual symbol statistics more closely, thereby lowering of the coding efficiency decreases, the lowering of the coding efficiency is normally caused by lower-delay concepts. Temporal interrelationships are exploited additionally or alternatively.SELECTED DRAWING: None

Description

Technical Field

[0001] The present application relates to sample array coding such as the coding of images or videos.

Background Art

[0002] Due to the increased processing requirements according to the HEVC standard and the expected increase in video resolution, the parallelization of encoders and decoders has become very important. In a wide range of modern electronic devices, multi-core architectures have become available. As a result, an efficient method that enables the use of multi-core architectures is required.

[0003] The coding or decoding of LCU occurs in raster scan, whereby the CABAC probability is adapted to the characteristics of each image. Spatial dependencies exist between adjacent LCUs. Each LCU (largest coding unit) is determined by its adjacent LCUs on the left, above, upper left, and upper right for different components, such as motion vectors, prediction, intra prediction, and others. In order to enable parallelization in decoding, these dependencies usually need to be interrupted or are interrupted in state-of-the-art applications.

[0004] Some concepts of parallelization, or in other words wavefront processing, have been proposed. What is motivating further research is to implement a technique that reduces the loss of coding efficiency and thus the burden on the bitstream for parallelization approaches in encoders and decoders. Furthermore, low-latency processing was not possible with available techniques.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Non-Patent Literature

[0006]

Non-Patent Literature 1

Summary of the Invention

[0007] Therefore, an object of the present invention is to provide an encoding concept for a sample array that enables a lower delay with a relatively small penalty in encoding efficiency.

[0008] This object is achieved by the subject matter of the independent claims of the claims.

[0009] When the entropy encoding of the current part of a predetermined entropy slice is based not only on the probability estimation of each of the predetermined entropy slices adapted using the parts encoded before the predetermined entropy slice, but also on the probability estimation used in the adjacent part in the entropy encoding of the entropy slice preceding in the entropy slice order and spatially adjacent, the probability estimation used for entropy encoding is more closely adapted to the actual symbol statistics, thereby reducing the decrease in encoding efficiency usually caused by a low-delay concept. Temporal correlations can additionally or alternatively be utilized.

[0010] For example, dependencies on the probability estimate used in the entropy coding of a previous entropy slice in a spatially adjacent entropy slice sequence can include the initialization of the probability estimate at the beginning of the entropy coding of a predetermined entropy slice. Typically, the probability estimate is initialized to values ​​adapted to the symbolic statistics of a representative blend of the sample array material. To avoid transmitting the initialization values ​​of the probability estimate, they are known to the encoder and decoder by convention. However, such predetermined initialization values ​​naturally deviate—more or less—from the actual statistics of the sample array material currently being coded, and are therefore simply a compromise between the bitrate of the side information on the one hand and coding efficiency on the other. Probability adaptation during the coding process of an entropy slice causes the probability estimate to be adapted to the actual symbolic statistics. This process is accelerated by initializing the probability estimate at the beginning of entropy coding the current / predetermined entropy slice using the already adapted probability estimate of the previous entropy slice in the spatially adjacent entropy slice order just described, since the later values ​​are adapted to some extent to the actual symbol statistics of the sample array currently at hand. Nevertheless, low-latency coding can be enabled by using the probability estimate used in its adjacent part rather than appearing at the end of entropy coding the previous entropy slice in the initialization of the probability estimate for the predetermined / current entropy slice. Wavefront processing is still possible with this strategy.

[0011] Furthermore, the aforementioned dependency on the probability estimate used in the entropy coding of the previous entropy slice in a spatially adjacent entropy slice order can include an adaptation process that adapts the probability estimate used in the entropy coding of the current / predetermined entropy slice itself. The adaptation of the probability estimate involves the use of just-encoded parts, i.e., just-encoded symbols, to adapt the current state of the probability estimate to the actual symbolic statistics. Through this policy, the initialized probability estimate is adapted to the actual symbolic statistics at some adaptation rate. This adaptation rate is increased by performing the adaptation of the just-described probability estimate not only based on the currently-encoded symbols of the current / predetermined entropy slice, but also by the probability estimate appearing in the adjacent parts of the previous entropy slice in a spatially adjacent entropy slice order. Wavefront processing is still possible by appropriately selecting the spatial neighborhood of the current part of the current entropy slice and the adjacent parts of the previous entropy slice. The advantage of combining the probabilistic estimation adaptation along the current entropy slice with the probabilistic adaptation of the previous entropy slice is that the rate at which adaptation to the actual symbol statistic occurs increases, because the number of symbols traversed in the current and previous entropy slices, rather than just the symbols of the current entropy slice, contributes to the adaptation. [Brief explanation of the drawing]

[0012] Preferred embodiments of the present invention are the subject of the dependent claims. Further preferred embodiments are described with reference to the following drawings. [Figure 1] A block diagram of an exemplary encoder is shown. [Figure 2] This diagram shows a schematic representation of the division of an image into slices and slice parts (i.e., blocks or coded units) according to the coding order defined within them. [Figure 3]Figure 1 shows a flowchart illustrating the functionality of an exemplary encoder. [Figure 4] Figure 1 shows an illustrative diagram illustrating the function of an example encoder. [Figure 5] This diagram illustrates the implementation of parallel operation of an encoder and decoder. [Figure 6] An illustrative block diagram of a decoder is shown. [Figure 7] Figure 6 shows an example of a decoder's functionality flowchart. [Figure 8] Figures 1-6 illustrate exemplary bitstreams resulting from the encoding schemes shown. [Figure 9] An example of how to calculate probabilities with the help of other LCUs is shown graphically. [Figure 10] The graph shows the RD results for the intranet (4 threads) compared to HM3.0. [Figure 11] The graph shows the RD results for low latency (1 thread) compared with HM3.0. [Figure 12] The graph shows the RD results for random access (1 thread) in comparison with HM3.0. [Figure 13] The graph shows the RD results for low latency (4 threads) compared to HM3.0. [Figure 14] Possible synthesis of entropy slices is illustrated schematically and illustratively. [Figure 15] Possible signaling for entropy slicing is illustrated diagrammatically and illustratively. [Figure 16] The encoding, segmentation, interleaving, and decoding of entropy slice data using chunks are illustrated schematically and illustratively. [Figure 17] Possible combinations between frames are illustrated diagrammatically and illustratively. [Figure 18] Possible uses of the arranged information are illustrated graphically and illustratively. [Figure 19]This diagram illustrates the possibility of wavefronts running diagonally across spatial and temporal space in a continuous sample array. [Figure 20] Another example of subdividing an entropy slice into chunks is diagrammatically shown. [Modes for carrying out the invention]

[0013] To facilitate understanding of the strategies outlined below that improve the achievement of low latency with less penalty to encoding efficiency, the encoder in Figure 1 is first described in more general terms without prior discussion of the advantageous concepts of the embodiments of the present application and how they can be incorporated into the embodiment in Figure 1. However, it should be noted that the structure shown in Figure 1 serves simply as an environment for explanation in which embodiments of the present application can be used. Generalizations and modifications of the encoder and decoder according to embodiments of the present invention are briefly described.

[0014] Figure 1 shows an encoder that encodes a sample array 10 into an entropy-encoded data stream 20. As shown in Figure 1, the sample array 10 can be one of a sequence 30 of sample arrays, and the encoder can be configured to encode the sequence 30 into a data stream 20.

[0015] The encoder in Figure 1 is generally indicated by reference numeral 40 and comprises a precoder 42 followed by an entropy coding stage 44 that outputs a data stream 20. The precoder 42 is configured to receive and operate on a sample array 10, so that each syntax element is one of a predetermined set of syntax element types, and then its content is described by a predetermined syntax element of the syntax associated with each semantics.

[0016] In describing the sample array 10 using syntax elements, the precoder 42 can subdivide the sample array 10 into coding units 50. The term “coding unit” is instead referred to as “coding tree unit” (CTU) for reasons that will be described in more detail below. One possible way in which the precoder 42 can subdivide the sample array 10 into coding units 50 is illustrated in Figure 2. According to this embodiment, the subdivision subdivides the sample array 10 into coding units 50 in a regular manner, so that the latter are arranged in rows and columns to cover the entire sample array 10 gaplessly without overlap. In other words, the precoder 42 can be configured to describe each coding unit 50 by syntax elements. Some of these syntax elements can form subdivision information for further subdivision of each coding unit 50. The subdivision information can describe the subdivision of each coding unit 50 into prediction blocks 52 by a precoder 42 that associates the prediction mode with relevant prediction parameters for each of these prediction blocks 52, for example, by multi-tree subdivision. This prediction subdivision can allow prediction blocks 52 of different sizes, as illustrated in Figure 2. The precoder 42 can also associate residual prediction information with the prediction blocks 52 in order to further subdivide the prediction blocks 52 into residual blocks 54 in order to describe the prediction residuals for each prediction block 52. Thus, the precoder can be configured to generate a syntax description of the sample array 10 by a hybrid coding scheme. However, as already stated above, the method just described by which the precoder 42 describes the sample array 10 by syntax elements is presented for illustrative purposes only and can be implemented in different ways.

[0017] The precoder 42 can utilize the spatial relationships between the contents of adjacent coding units 50 in the sample array 10. For example, the precoder 42 can predict syntax elements for a particular coding unit 50 from syntax elements determined for coding units 50 that were spatially adjacent to the coding unit 50 currently being coded. In Figures 1 and 2, for example, the upper and left-hand neighbors are helpful for prediction, as illustrated by arrows 60 and 62. Furthermore, in intra-prediction mode, the precoder 42 can extrapolate the already coded content of adjacent coding units 50 to the current coding unit 50 to obtain a prediction of the samples for the current coding unit 50. As shown in Figure 1, beyond the use of spatial relationships, the precoder 42 can temporarily predict samples and / or syntax elements for the current coding unit 50 from a previously coded sample array, as illustrated by arrow 64 in Figure 1. In other words, motion-compensated predictions can be used by the precoder 42, and the motion vector itself can follow a temporal prediction from the motion vector of the previously encoded sample array.

[0018] In other words, the precoder 42 can describe the content of the sample array 10 on a coding unit basis, and for this purpose, spatial prediction can be used. The spatial prediction is limited to spatially adjacent coding units of the same sample array 10 for each coding unit 50, and as a result, when a coding order 66 is followed within a coding unit 50 of the sample array 10, adjacent coding units that serve as prediction references for the spatial prediction are traversed by the coding order 66 before the current coding unit 50. As illustrated in Figure 2, the coding order 66 defined within a coding unit 50 can be, for example, a raster scan order in which the coding unit 50 traverses row by row from top to bottom. Optionally, the subdivision of the array 10 into an array of tiles can cause the scan order 66 to traverse the coding unit 50 that has one tile preceding the next, in a tile order which can also be a raster scan type. For example, spatial prediction can simply include adjacent coding units 50 in the row of coding units above the row of coding units in which the current coding unit 50 resides, and coding units in the same row of coding units but to the left of the current coding unit. As will be explained in more detail below, this limitation on spatial relationships / spatial predictions enables parallel wavefront processing.

[0019] The precoder 42 transfers the syntax elements to the entropy coding stage 44. As just outlined, some of these syntax elements are predictively coded, i.e., represent predictive residuals. The precoder 42 can therefore be considered a predictive encoder. Alternatively, the precoder 42 can be a transform encoder configured to transform the predicted code residuals of the content of the coding unit 50.

[0020] An exemplary internal structure of the entropy coding stage 44 is also shown in Figure 1. As shown in the figure, the entropy coding stage 44 optionally takes each syntax element received from the precoder 42 and the entropy coding engine 44 operates on it to form symbols of the symbol alphabet, with the number of possible stages exceeding the base of the symbol alphabet. i The entropy coding engine 44 may include a symbolizer that converts to a sequence of symbols. In addition to the optional symbolizer 70, the entropy coding engine 44 may include a context selector 72 and an initializer 74, a probability estimation manager 76, a probability estimation adapter 78 and an entropy coding core 80. The output of the entropy coding core forms the output of the entropy coding stage 44. The entropy coding core 80 also has two inputs, in other words, one is a sequence of symbols s i The input receives one value, and the other is a probability estimate p for each of the symbols. i It has an input for receiving data.

[0021] Due to the nature of entropy coding, coding efficiency with respect to compression rate increases with improvements in probability estimation; the better the probability estimation matches the actual symbol statistics, the better the compression rate.

[0022] In the embodiment shown in Figure 1, the context selector 72 selects each symbol s i In contrast, the corresponding context c is found within the set of available contexts managed by manager 76. i It is configured to select a context. It should be noted, however, that context selection is merely an optional feature and can be omitted, for example, by using the same context for each symbol. However, when context selection is used, the context selector 72 can be configured to perform context selection based at least in part on information about coding units outside the current coding unit, in other words, on adjacent coding units within the limited neighborhood described above.

[0023] The manager 76 comprises a storage device that stores the associated probability estimates for each available context. For example, the symbol alphabet can be a binary alphabet, and as a result, for each available context, only one probability value can be stored.

[0024] The initializer 74 can intermittently initialize or re-initialize the probability estimates stored within the manager 76 for the available contexts. The possible instants at which such an initialization can be performed are discussed further below.

[0025] The adapter 78 accesses a pair of the symbol s i and the corresponding probability estimate p i and thus adapts the probability estimate within the manager 76. That is, for each symbol s i to be entropy-coded, each time the probability estimate is applied by the entropy-coding core 80 to the data stream 20, the adapter 78 can vary this probability estimate according to the value of this current symbol s i , and as a result, this probability estimate p i is better adapted to the actual symbol statistics when encoding the next symbol associated with that probability estimate (by that context). That is, the adapter 78 receives from the manager 76 the probability estimate for the selected context together with the corresponding symbol s i and thus adapts the probability estimate p i , and as a result, the adapted probability estimate is used for the next symbol s i of the same context c i .

[0026] The entropy coding core 80 is configured to operate, for example, by an arithmetic coding scheme or in a probabilistic-interval-partition-entropy coding scheme. In arithmetic coding, the entropy coding core 80 continuously updates the state of a sequence of symbols, for example, by a state defined by a probability interval defined by a probability interval width value and a probability interval offset value. When operating in a pipe concept, the entropy coding core 80 subdivides a domain of possible probabilistic estimates into different intervals, for example, by performing fixed-probabilistic entropy coding for each of these intervals, thereby obtaining a substream for each sub-interval, the coding efficiency of which is tailored to the relevant probability interval, respectively. In the case of entropy coding, the output of the data stream 20 is a signaling of the arithmetically coded data stream to decoding-side information that allows the interval subdivision process to be emulated or redone.

[0027] Naturally, for the entropy-encoded state 44, all information about the sample array 10, i.e., all syntax elements / symbols s, is included. i It is possible to entropy encode the probability estimate by simply initializing it once at the beginning, and then continuously updating the probability estimate by the adapter 78. This, however, results in a data stream 20 that must be sequentially decoded on the decoding side. In other words, for any decoder, there is no possibility of subdividing the resulting data stream into several sub-parts and decoding the sub-parts in parallel. This, in turn, prevents any low-latency effort.

[0028] Therefore, as will be outlined in more detail below, it is preferable to subdivide the amount of data describing the sample array 10 into so-called entropy slices. Each of these entropy slices, therefore, covers a different set of syntactic elements relating to the sample array 10. However, if the entropy coding stage 44 entropy codes each entropy slice, which is completely independent of the others, by first initializing the probability estimate once and then continuously updating the probability estimate individually for each entropy slice, the coding efficiency decreases due to the increasing percentage of data describing the sample array 10 for which the probability estimate used is not yet (accurately) applied to the actual symbolic statistics.

[0029] To overcome the aforementioned problem of meeting the demands for low coding delay on the one hand and high coding efficiency on the other, the following coding scheme, as described in Figure 3, can be used.

[0030] First, the data describing the sample array 10 is subdivided into parts referred to below as "entropy slices." The subdivisions 80 do not need to be overlap-free. On the other hand, this subdivision can correspond at least partially to a spatial subdivision of the sample array 10 into different parts. That is, according to the subdivision 80, the syntax elements describing the sample array 10 can be distributed onto different entropy slices depending on the location of the coding unit 50 to which each syntax element relates. See, for example, Figure 2. Figure 2 shows an exemplary subdivision of the sample array 10 into different parts 12. Each part corresponds to its respective entropy slice. As shown exemplary, each part 12 corresponds to a row of coding unit 50. Other subdivisions are, however, equally possible. However, the subdivision of the sample array 10 into parts 12 is beneficial if it follows the coding order 66 described above, and as a result, the parts 12 cover a continuous run of coding units along the coding order 66. However, even if that were the case, the start and end positions of the portion 12 along the coding order 66 do not need to coincide with the left and right edges of the rows of the coding unit 50, respectively. Even the coincidence with the boundaries of the coding unit 50 does not need to be enforced, as they follow each other directly.

[0031] By subdividing the sample array 10 in this manner, the entropy slice order 66 is determined within the parts 12 that follow each other along the coding order 66. Furthermore, for each entropy slice, a fragment of the coding path 66 running toward each part 12 is determined. In the embodiment of Figure 2, where the parts 12 coincide with the rows of the coding unit 50, the entropy coding paths 14 of each entropy slice run parallel to each other along the row direction, that is, from left to right.

[0032] It should be noted that by limiting the spatial predictions performed by the precoder 42 and the context derivation performed by the context selector 72 so as not to cross slice boundaries, it is possible to ensure that spatial predictions and context selection are not determined by data corresponding to other entropy slices. Thus, "entropy slices" correspond to the usual definition of "slice" in H.264, which are decodeable completely independently of each other, except for the dependency of probabilistic initialization / adaptation outlined below. However, since WPP processing is still possible insofar as precoding inversion, i.e., reconstruction based on syntax elements, and entropy context selection are involved, it is also possible to cross slice boundaries in order to allow spatial predictions and context selection, i.e., dependencies in general, and to leverage local / spatial interdependencies. To that extent, entropy slices correspond in some way to "dependent slices".

[0033] Subdivision 80 can be performed, for example, by the entropy coding stage 44. Subdivision can be fixed or varied within the array of sequence 30. Subdivision can be fixed per convention or signaled within the data stream 20.

[0034] Based on the entropy slices, actual entropy coding, i.e., 82, can be performed. For each entropy slice, the entropy coding can be structured into an initiation phase 84 and a continuation phase 86. The initiation phase 84 includes, for example, the initialization of probability estimation and the triggering of the actual entropy coding process for each entropy slice. The actual entropy coding is then performed during the continuation phase 86. The entropy coding during phase 86 is performed along each entropy coding path 14. The initiation phase 84 for each entropy slice is controlled so that the entropy coding of multiple entropy slices starts sequentially using the entropy slice order 16.

[0035] Here, in order to avoid the penalty outlined above resulting from the entropy coding of each entropy slice which is completely independent of each other, the entropy coding process 82 is controlled so that the current part, for example, the current coding unit of the current entropy slice, is entropy coded based on the probability estimates of each current entropy slice adapted using the part of the current entropy slice to the left of the current coding unit 50 in the case of Figure 2, and the probability estimates used in the entropy coding of the adjacent part, i.e., the adjacent coding unit, of the spatially adjacent previous entropy slice in the entropy slice order 16.

[0036] To more clearly illustrate the dependencies described above, Figure 4 is referenced. Figure 4 shows the n-1, n, and n+1th entropy slices in the entropy slice order 16, indicated by reference numeral 90. Each entropy slice 90 contains a sequence of syntax elements describing the portion 12 of the sample array 10 associated with each entropy slice 90. Along the entropy coding path 14, the entropy slice 19 is segmented into a sequence of segments 92, each corresponding to one of the coding units 50 of the portion 12 to which the entropy slice 90 relates.

[0037] As described above, the probability estimate used in entropy coding of entropy slice 90 is continuously updated along the entropy coding path 14 during the continuation phase 86, and as a result, the probability estimate gradually adapts better to the actual symbol statistics of each entropy slice 90 - that is, the probability estimate is relevant to each entropy slice. In Figure 4, the probability estimate 94 used in entropy coding of entropy slice 90 is continuously updated during the continuation phase 86, but the states of the probability estimate 94 that occur at the beginning and end positions of segment 92 are further illustrated and mentioned below. In particular, the state before entropy coding of the first segment 92 initialized during the start phase 84 is shown by 96, the state that appears after coding the first segment is shown by 98, and the state that appears after coding the first two segments is shown by 100. The same elements are also shown in Figure 4 for the n-1 entropy slice and the subsequent entropy slice, i.e., entropy slice n+1, in the entropy slice order 16.

[0038] Here, in order to achieve the dependencies outlined above, the initial state 96 for the entropy coding of the nth entropy slice 90 is set by any intermediate state of the probability estimate 94 that appears during the coding of the previous entropy slice n-1. An "intermediate state" refers to any state of the probability estimate 94 except the initial state 96 and the final state that appears after the entropy coding of all the entropy slices n-1. In this way, the entropy coding of the sequence of entropy slices 90 along the entropy slice order 16 can be parallelized to a degree of parallelism determined by the ratio of the number of previous segments 92, i.e. a, to the number of subsequent segments 92, i.e. b, which is used for the initialization of the probability estimate 94 for the entropy coding of the next entropy slice. In particular, in Figure 4, a is exemplary set by initialization to be equivalent to an adaptation of state 100, so that the state 96 of the current entropy slice, illustrated by arrow 104, is equivalent to the state 100 of the previous entropy slice.

[0039] This policy ensures that the entropy coding of any segment 92 following state 100 in the entropy coding path sequence 14 is determined during the continuation phase 86 by a probability estimate 94 adapted based on the previous segment of the same entropy slice, as well as the probability estimate used in the entropy coding of the third segment 92 of the previous entropy slice 90.

[0040] Therefore, the entropy coding of entropy slices 90 can be performed in parallel in a pipelining schedule. The only limitation imposed on the time scheduling is that the entropy coding of some entropy slices can start simply after the entropy coding of the a-th segment 92 of the previous entropy slice has been completed. Entropy slices 90 that directly follow each other in the entropy slice order 16 are not subject to any other limitations regarding the time sequence of the entropy coding procedure during the continuation phase 86.

[0041] However, according to other embodiments, stronger coupling is used additionally and / or instead. In particular, as illustrated by the representative arrow 106 in Figure 4, the adaptation of the probability estimate during the continuation phase 86 results in the data of the coding unit corresponding to a particular segment 92 changing the probability estimate 94 from the state at the beginning of each segment 92 to the end of this segment 92, thereby improving the approximation of the actual symbol statistics shown above. That is, the adaptation 106 is performed for entropy slice n-1 simply with the data of entropy slice n-1, and the same applies to the adaptation 106 of the probability estimate for entropy slice n, etc. For example, it is possible to perform the initialization described above with respect to arrow 104 by performing the adaptation 106 of the probability estimate without further interference between entropy slices 90. However, in order to facilitate the approximation of the actual symbolic statistic by probability estimation, the adaptations 106 of probability estimates of successive entropy slices can be combined, so that when adapting the adaptation of the probability estimate of the current entropy slice n, the adaptation 106 of the previous entropy slice n-1 also influences or takes into account. This is illustrated in Figure 4 by an arrow 108 pointing from the state 110 of spatially adjacent probability estimate 94 for the entropy coding of the n-1th entropy slice 90 to the state 100 of probability estimate 94 for the entropy coding of the nth entropy slice 90. When using the initialization of the state 96 outlined above, the combination of probability adaptations 108 can use, for example, any b probability estimate state that appears after the entropy coding of the b segment 92 of the previous entropy slice. More precisely, the probability estimate that appears after the entropy coding of the first segment 92 of the current entropy slice can be the result of considering 108 the probability estimate state resulting from the normal probability adaptation 106 and the adaptation 106 of the probability estimate between the entropy coding of the (a+1)th segment 92 of the previous entropy coding. "Considering" can include, for example, some averaging operations. An example is outlined further below.In other words, the state 98 of the probabilistic estimate 94 at the beginning of the entropy coding segment for the nth entropy coding of the entropy coding of the nth entropy slice 90 can be the result of averaging the state 96 immediately preceding the probabilistic estimate 94 for the entropy coding of the current entropy slice n, adapted using the adaptation 106, with the state before the entropy coding of the a+1th segment 92 of the previous entropy slice n-1, modified by the probabilistic adaptation 106. Similarly, the state 100 can be the result of averaging the result of the adaptation 106 performed during the entropy coding of the current entropy slice n with the result of the probabilistic adaptation during the entropy coding of the a+2th segment 92 of the previous entropy slice n-1, etc.

[0042] For more details, Let p(n) → {i,j}. Here, i and j represent the positions of any coding units (where (0,0) is the top-left position and (I,J) is the bottom-right position), i ∈ {1…I}, j ∈ {1…J}, I is the column number, J is the row number, and p() defines path order 66. P [i,j] This shows the probability estimation used in the entropy coding unit {i,j}. T(P [i,j] ) is based on P based on the entropy coding unit {i,j} [i,j] The results of probability adaptation 106 are shown. At that time, the probability estimates 106 of the consecutive entropy slices 90 are combined, and the internal application of the normal entropy slice is given by the following equation P p(n-1) =T(P p(n) It can be replaced according to ).

[0043] JPEG0007869289000001.jpg8134

[0044] Here, N can be 1 or 2 or more, and {i,j}1… N, respectively, are selected from any previous (in entropy slice order 16) entropy slice 90 and its associated part 12. The function "average" can be a weighted sum, a median function, or the other one. p(n) = {i,j} is the current coding unit, and p(n+1) follows according to coding orders 14 and 66, respectively. In the proposed embodiment, p(n+1) = {i+1,j}.

[0045] Preferably, {i,j}1… N For each k ∈ {1…N}, {i,j}1… N ={i k ,j k} and i k <i + 3, j k <j is satisfied. Here, p(n) = {i,j} is the current coding unit (i.e., either the second of the current entropy slice or the following coding unit), that is, they do not exist beyond the wavefront.

[0046] In the latter variant, the time scheduling of entropy coding is more closely coupled when entropy coding entropy slices 90 in parallel. That is, during the continuation phase 86, the next segment 92 in the line of the current entropy slice can start simply by the end of the corresponding segment of the previous entropy slice at a further upper position in the entropy coding path order 14.

[0047] In other words, the above description illustrates an embodiment in which the decoder 40 and, in particular, the entropy coding stage 44 are configured to initialize their probability estimates 94 by probability estimates that appear in the entropy slice order 16 after entropy decoding the second coded unit / block of the portion 12 corresponding to the previous entropy slice along the respective coding path 14, before decoding the first segment 92 corresponding to the first coded unit / first block 50 of the portion 12 corresponding to the nth entropy slice 90 along the respective coding path 14. Additionally or instead, the decoder 40 and, in particular, the entropy coding stage 44 may be configured to perform entropy decoding and probability estimation adaptation for each entropy slice 90, such that after the current part / block / coding unit 50 of the current entropy slice is entropy-decoded based on the respective probability estimation 94 of the current entropy slice 90, the respective probability estimation 94 of the current entropy slice is adapted by the probability estimation that appears in the entropy decoding of the adjacent part / block / coding unit 50 of the spatially adjacent previous entropy slice, such as in the row adjacent to the top in the second column to the right of the current part / block / coding unit of the current entropy slice.

[0048] As is clear from the above description, the probability estimation must be adapted / managed separately for each entropy slice 50. This can be done by sequentially processing the entropy slices and storing the probability estimation states in each probability estimation memory 120 (see Figure 1), as illustrated and mentioned with respect to Figure 4, namely 96, 98, 100, 110, and 102. Alternatively, multiple entropy coding stages 44 can be provided to decode the entropy slices in parallel. This is illustrated in Figure 5, where multiple instances of the entropy coding stage 44 are shown, each relating to one of the entropy slices and the corresponding portion 12 of the sample array 10, respectively. Figure 5 also illustrates the decoding process and possible embodiments thereof by using parallel instances of each entropy decoding stage 130. Each of these entropy decoding stages 130 is supplied by each of the entropy slices transmitted by the data stream 20.

[0049] Figure 5 shows that the entropy coding stage 44 and their respective decoding stages 130 do not operate completely independently and in parallel with one another. Rather, the stored probabilistic estimation states, such as those stored in the memory device 120, are passed from one stage corresponding to each entropy slice to other stages related to entropy slices that follow according to the entropy slice order 16.

[0050] Figure 5 also shows, for illustrative purposes, possible traverse orders that traverse possible subdivisions of the coding unit 50, such as the traverse order 140 defined in the prediction block 52, within a single coding unit 50. For all these prediction blocks 52, the corresponding syntax elements are included within their respective segments 92, corresponding to the values ​​of these syntax elements, and the probability estimate 94 is adapted during the traverse of path 140 by adaptation during the traverse of the coding unit 50 that defines the aforementioned "T". In CABAC for H.264 and HEVC, "T" is performed on a table basis, i.e., by a "table walk" that defines the transition from the current state of the probability estimate for the current context to the next state determined by the current symbol value associated with that context.

[0051] Before proceeding to Figure 6, which shows an embodiment of the decoder corresponding to the encoder embodiment in Figure 1, it should be noted that the predictive nature of the precoder 42 serves merely as an illustrative embodiment. Furthermore, according to an alternative embodiment, the precoder 42 can be terminated by a syntax element on which the entropy coding stage 44 operates, which is, for example, the original sample values ​​of the sample array 10. Alternatively, the precoder 42 can be configured to form a subband decomposition of the sample array 10, as in JPEG. The optional nature of the context selector 72 has already been described above. The same applies to the initializer 74. The same may be equipped in different ways.

[0052] Figure 6 shows the decoder 200 corresponding to the encoder 40 in Figure 1. It is clear from Figure 6 that the structure of the decoder 200 substantially reflects the structure of the encoder 40. That is, the decoder 200 has an input 202 that receives the data stream 20, followed by a coupling of an entropy decoding stage 204 and a constructor 206. The entropy decoding stage 204 entropy decodes the entropy slices transmitted in the data stream 202, and then the decoded symbols si The constructor 206 forwards the syntax elements to the constructor 206, which then requests the syntax elements from the entropy decoding stage 204 by request 208. In other words, the constructor 206 is also responsible for parsing the syntax element stream generated by the precoder 42 within the encoder. Thus, the constructor 206 sequentially requests the syntax elements from the entropy decoding stage 204. The entropy decoding stage 204 is constructed in substantially the same way as the entropy coding stage 44. Thus, the same reference codes in the internal blocks of the entropy decoding stage 204 are reused. The symbolizer 70, if present, converts the request for syntax elements into a request for symbols, and the entropy decoding core 80 receives the respective symbol values ​​s iIn response, the symbolizer 70 maps the received sequence of symbols that form a valid symbol word to syntax elements and forwards it to the constructor 206. The constructor 206 reconstructs the sample array 10 from the stream of syntax elements received from the entropy decoding stage 204, such as using the predictive decoding outlined above. More precisely, the constructor 206 performs decoding unit by unit by performing predictions 60, 62, and 64, also using the coding order 66. One or more predictions for syntax elements or sample values ​​are added together, optionally using prediction residuals obtained from the syntax elements in the syntax element stream. The entropy decoding core 80 can follow either the arithmetic decoding concept or the probabilistic interval-partitioned entropy decoding concept, just like the entropy coding core 80. In the case of arithmetic decoding, the entropy decoding core 80 can continuously update its internal state in the form of values ​​such as partial interval width values ​​and offset values ​​that point to these partial intervals. The updates are made using the incoming data stream. The current partial interval is determined by the context selector 72 together with the probability estimation manager 76 for each symbol s i The probability estimate p provided for i It is subdivided in a manner similar to the entropy coding core 80. The adapter 78 is defined by the context selector 72 as symbols s i Related context c i Probability estimate p i To update the decoded symbol values ​​s i The adaptation of the probability estimation is performed using this method. Initialization by initializer 74 is done on the same instance in the same way as on the encoding side.

[0053] The decoder in Figure 6 operates in a manner very similar to the encoder described above with respect to Figure 3. In step 230, the sample array 10 is subdivided into entropy slices. See, for example, Figure 8. Figure 8 shows the data stream 20 arriving at input 202, and in particular the entropy slices 90 contained therein. In step 230, each of these entropy slices 90 is associated with the portion 12 to which each entropy slice 90 is related, and as a result, the initialization and adaptation of the probability estimation described above based on each previous entropy slice can be performed. The subdivision of the sample array, or more precisely, the association of the entropy slices 90 with their respective portions 12, can be performed by the constructor 206. The association can be achieved by different measures, such as side information contained within the data stream 20 in the unentropy encoded portion, or by convention.

[0054] The entropy slice 90 is then entropy-decoded in the entropy decoding process 232 in a manner that reflects the coding process 82, in other words, for each entropy slice 90, the start phase 234 and continuation phase 236 are executed in the same manner as the coding procedure for the same instance, by initializing and adapting the probability estimation.

[0055] The same parallelization described above for encoding is possible on the decoding side. The instantiation 130 of the entropy decoding stage shown in Figure 5 can be realized as shown for the entropy decoding stage 204 in Figure 6. The probability estimation memory 240 can be used to store the state of the probability estimation for use in the entropy decoding stage 130, which is responsible for the entropy decoding of the subsequent entropy slices in the entropy coding sequence 16.

[0056] After the embodiments of the present application are described, the concepts described so far are described again below using different wording. Several further embodiments of the present application are described below. Hereinafter, the coding unit 50 described above will be referred to as the LCU (Largest Coding Unit), thereby conforming to the terminology of the upcoming HEVC standard.

[0057] Firstly, the aforementioned probability adaptation 106 is again briefly described in relation to Figure 9.

[0058] The current LCU uses a probabilistic estimation, such as a CABAC probability, which is available after encoding the previous left-hand LCU. For example, the LCU in Figure 9, denoted by x, is assumed to be entropically encoded using a probabilistic estimation p1 adapted for the left side of LCUx until the end of the entropy encoding of the left-hand LCU. However, if information is used not only from the left side but also from one or more other LCUs that have already been processed and are available, better results can be achieved in probabilistic adaptation.

[0059] As described above, in the entropy decoding and encoding of entropy slices, a new probability estimate is calculated with the help of already existing probabilities (probability estimates) from other LCUs before encoding or decoding each LCU. More precisely, probability adaptation can be performed not only from all previous LCUs of the current entropy slice, but also from the LCUs of previous entropy slices in the entropy slice order. This approach is again represented in Figure 9. Thus, the initial probability of the current LCU, indicated by X in Figure 9, can be calculated as follows:

[0060] JPEG0007869289000002.jpg8147

[0061] Here, a1,...a k This is the weighting coefficient for LCU.

[0062] The probability of which weights yields the best results has been tested. In this experiment, only neighboring LCUs are used. This study reveals that 75% of the weights come from the left LCU and 25% come from the upper right LCU. The results are presented in Figures 10-13. The graphs given using "Prob.Adap" employ the probabilistic adaptation described above.

[0063] However, in adapting the probability estimation, we cannot use only adjacent blocks. Each nearest LCU has a particular neighborhood in which its use for probability optimization can be significant. In other words, we cannot apply only the LCUs from the nearest row on the upper side. In Figure 9, the derivation of the probability estimation is taken first from the neighborhood, and then each of the upper right LCUs (p5 and p) in the next upper row. k Examples can be seen from (see reference).

[0064] It should be acknowledged that the adaptation of probability recalculation or probability estimation outlined above introduces some complexity. The new operation of probability estimation occurs, for example, in the following three steps: First, the probability estimate of each candidate must be obtained from the state of each context. This is done by storing them in memory devices 120 and 240 respectively, or by steering parallel entropy slice n / decoding processes so that these states are available simultaneously. Second, the optimized probability (p) is obtained using equation (1). new ) is created. That is, to combine the adapted probability estimates of different entropy slices, for example, several averaging operations can be used. As a final step, the state of the new context is p newIt is converted from and replaces the old one. That is, the probability estimation manager 76 adopts the new probability estimation thus obtained. This performance for each syntax element can greatly increase complexity, especially when multiplication is used. The only way to mitigate this penalty is to try to avoid these three steps. Once the number of candidates and their weights are determined, a pre-calculated table can be approximated for each state. Thus, only one simple access to the table date is required with the help of an index of candidates (the state in the context).

[0065] This technique is claimed to be able to provide good results for both applications, with and without entropy slicing. In the first application, only one slice is used per frame, and therefore probabilistic adaptation is optimized without any other changes. In the case of entropy slicing, probabilistic adaptation occurs within each slice, independent of other slices. This enables the current LCU to learn probabilities quickly.

[0066] In the above explanation, the use of the second LCU in the upper line, i.e., the use of the second LCU for the initialization of probability estimation, was also provided. Parallelization of encoding and decoding is possible if some of the above-mentioned conditions for the bitstream (entropy slice) are met. The dependency of CABAC probabilities between LCUs must be broken. It is important that the first LCU of each line is made independent of the last LCU of the previous line by wavefront parallelization. This can be achieved, for example, if the CABAC probabilities are reinitialized at the beginning of each line of the LCU. However, this method is not optimal because each reinitialization loses the achieved CABAC probabilities adapted to the specificity of the image. This penalty can be reduced if the initialization of the CABAC probability of the first LCU of each line is caused by the probability obtained after the second LCU of the previous line.

[0067] As described above, the benefit in the rate of probabilistic adaptation can be achieved by combining the probabilistic adaptations of spatially adjacent entropy slices. In particular, to put it another way, the above description is a decoder like one in Figure 6 that reconstructs a sample array (10) from an entropy-encoded data stream, configured to entropy-decode multiple entropy slices in the entropy-encoded data stream (performed by an entropy-decoding stage), performing entropy-decoding along each entropy-encoded path (14) for each entropy slice using its respective probability estimate, adapting each probability estimate along each entropy-encoded path using the previously decoded part of each entropy slice, and sequentially decoding multiple entropy slices using the entropy-slice order (16). We also anticipate a decoder that starts with and then performs entropy decoding of a predetermined entropy slice, thereby reconstructing different parts (12) of the sample array associated with the entropy slice, based on the probability estimates of each predetermined entropy slice applied in the entropy decoding of the predetermined entropy slice using parts decoded before the predetermined entropy slice (e.g., including p1), and the probability estimates used in adjacent parts (such as p4) of spatially adjacent entropy slices in the entropy decoding of spatially adjacent entropy slices (e.g., slices including X) in the entropy decoding of the previous entropy slice in the entropy slice order, thereby reconstructing different parts (12) of the sample array associated with the entropy slice.

[0068] The different parts can be rows of blocks (e.g., LCUs or macroblocks) in the sample array. The latter can be images in a video. The entropy coding path can extend row-wise. Entropy coding, and therefore probabilistic adaptation, can also be context-adaptive. The entropy slice order can generally be chosen such that, along the entropy slice order, the different parts follow each other in a direction (16) that forms an angle with respect to the entropy coding path (14) of the entropy slice, and then extend substantially parallel to each other. By this policy, the "wavefronts" of the currently decoded parts of the entropy slice (such as p1, p4, p5+1, and pk in the figure) can generally be positioned along lines that form a smaller angle with respect to the entropy slice path than the direction of the sequence of parts. For all threads processing slices in parallel, since the top-left reference is always there, the wavefronts can be made to have a slope of 1y for every 2x block positions.

[0069] The decoder performs entropy decoding along each entropy coding path for each entropy slice, in units of parts of the sample array portion of each entropy slice, so that each entropy slice consists of the same number of parts, and the sequence of parts of that portion is arranged laterally relative to each other along the entropy slice path. The current part of a predetermined portion of an entropy slice belongs to the resulting grid of parts (e.g., LCU or macroblock). In performing entropy decoding along each entropy coding path for each entropy slice, the decoder can store the offset / shift between the start times of directly consecutive entropy slices in the entropy slice order, so that the wavefront of the currently decoded part of the entropy slice forms a diagonal line with a slope of 0.5x block position with respect to the entropy slice path and the entropy slice order direction. The offset / shift can correspond to two parts for all pairs of directly consecutive entropy slices. Alternatively, the decoder can simply avoid a situation where the distance between the currently decoded parts of directly consecutive (and directly adjacent insofar as those sample array portions 12 are involved) entropy slices is less than two parts. See the figure above: as soon as part / block p4 is decoded, the part / block to its right is decoded according to the path order 16, and if any, either X or the part / block preceding it is decoded simultaneously. This policy allows the decoder to use a probabilistic estimate already adapted based on the contents of part / block p4, i.e., the parts in spatially adjacent portions arranged in parts of a predetermined entropy slice portion 12 that follows the current part X in the path order 16, in order to determine the probabilistic estimate used in decoding X.In the case of a fixed offset in the immediate entropy slice decoding of two parts, the decoder can simultaneously use a probabilistic estimate already adapted based on the content of part / block p4 for the entropy decoding of the spatially adjacent subsequent part (i.e., the part to the right of p4).

[0070] As mentioned above, the sum of the weights of the previously applied probability estimates can be used to determine the probability estimate used for decoding X.

[0071] Furthermore, as mentioned above, the entropy slice order can extend beyond the frame boundary.

[0072] It should be noted that the probabilistic adoption just outlined from a preceding entropy slice can be performed for all parts of the current / predetermined entropy slice for which such adjacent parts are available in the preceding entropy slice. That is, this is also true for the first part along path direction 16, for this very first part / block (the leftmost of each entropy slice in the diagram), the adoption is equivalent to the initialization described above.

[0073] To better adapt the process, the two methods described above can be combined in this case as well. The results of this process using one thread and four threads, i.e., processing units used in parallel, are illustrated in Figures 10-13 (graphs of 2LCU + ​​Prob.Adap or 2LCU) (Prob.Adap indicates "probabilistic adaptation").

[0074] To better understand the context of the above embodiments and, in particular, the further embodiments described below, especially the use of LCU, we can first review the H.264 / AVC configuration.

[0075] In H.264 / AVC, an encoded video sequence consists of a set of access units collected in a NAL unit stream, which use only one set of sequence parameters. Each video sequence can be decoded independently. An encoded sequence consists of a sequence of encoded images. An encoded frame can be an entire frame or a single field. Each image is divided into fixed-size macroblocks (LCUs in HEVC). Several macroblocks or LCUs can be merged together into a single slice. An image is therefore a collection of one or more slices. The goal of this data separation is to enable independent decoding of samples in areas of the image represented by a slice, without using data from other slices.

[0076] The technique, often referred to as "entropy slicing," is the division of a conventional slice into additional sub-slices. Specifically, it means slicing entropy-encoded data into a single slice. The arrangement of entropy slices within a slice can take on different forms. The simplest is to use each row of an LCU / macroblock in a frame as a single entropy slice. Alternatively, columns or separate regions can be used as entropy slices that can be interrupted and even switched with each other, such as slice 1 in Figure 14.

[0077] The obvious objective of the entropy slice concept is to improve the decoding process time, i.e., to speed up the process, by enabling the use of parallel CPU / GPUs and multi-core architectures. Current slices can be divided into partitions that can be analyzed and reconstructed without referencing other slice data. While the entropy slice approach achieves a few benefits, it also introduces some penalties.

[0078] First, the primary objective is to generate a bitstream suitable for parallel encoding and decoding processes. Because spatial and motion information is used for prediction, it must be considered that an LCU can only be encoded if adjacent LCUs (left, top, top right) are already available as encoded versions. To enable parallelism through slicing, a shift between slice processing (e.g., a shift of two LCUs, typical for the wavefront approach) must be implemented. For CABAC probability adaptation, LCUs use probabilities available from the previously encoded LCU. Regarding the raster scan order, a problem arising from slicing the image is that it hinders parallelism, as the first LCU of each line is determined by the last LCU of the previous line. This effect requires that the dependency of CABAC probabilities between slices be broken so that several slices can be started simultaneously. One way to do this is through normal CABAC reinitialization, but this results in the loss of all adopted data. This may increase the bitrate.

[0079] Secondly, each slice generates its own sub-bitstream, which can be sequentially fed into the mainstream. However, special information needs to be passed to the decoder so that these slices and their positions in the mainstream can be correctly identified. Two scenarios for signaling are supported: Position information can be stored in the image header (slice length information) or in each slice header (pointing as the start code). The byte array and position information at the end of each entropy slice increase the loss.

[0080] To mitigate the penalty introduced by signaling for entropy slices, it is crucial to use good encoding techniques for signaling. A noticeable penalty to entropy slice signaling is introduced in the frame when a start code is used for each slice, i.e., when too many additional bytes (e.g., a minimum of 4 bytes per slice) are added to the bitstream. Indeed, inserting entropy slice 90 with a start code has benefits in low-latency scenarios where the encoder can output the entropy slice immediately. In such cases, there is no prior signaling of possible entry points.

[0081] However, in strict scenarios with less latency, the opportunity to store the slice length (offset) seems more appropriate. One well-known method for encoding such information is variable-length coding (VLC) or exponential Golomb coding. The main feature of VLC is that it adds empty information (zeros) before the true information. With the help of these zeros, it is possible to determine the code that stores information about the offset length. We propose another technique that achieves this, the scheme of which is shown in Figure 15, where X = EntropySliceSize is the amount of bytes contained in the entropy slice. Each next X (offset) is defined as the difference in size between the offset of the previous, already encoded and signaled entropy slice and the current one. The main feature of this concept is that it forms a difference of size X, thus reducing the amount of data, which can be added up to 3 bits, and allows the decoder to extract appropriate information about the size of each entropy slice. Compared to VLC, this results in a number of bits saved in the entropy slice header.

[0082] In other words, for example, according to the embodiment of Figure 15, a concept is provided for entropy coding an information signal, enabling a higher compression rate compared to previously available concepts despite the parallel processing capability. According to this embodiment, section 20 of the entropy-coded data stream into which the sample array is coded comprises an entropy slice, such as 90 in Figure 8, into which different parts 12 of the sample array are entropy-coded, and a header, illustrated by a dashed line 300 in Figure 9, which contains information representing the starting position 302 of the entropy slice 90 measured in the entropy-decoding domain within the entropy-coded data stream, wherein the information includes a difference value representing the difference between the starting position of the previous entropy slice n-1 and the starting position of the predetermined entropy slice n for a predetermined entropy slice, and the difference is included in the data stream as a VLC bit sequence.

[0083] In particular, VLC bit sequences have a difference x of interval [0,2 a -1], [2 a ,2 b +2 a -1], [2 b +2 a ,2 c +2 b +2 a -1], can have a variable-length prefix y indicating the y-th position of a sequence of other numbers z, and a PCM-encoded suffix of the y-th position of a sequence of lengths a, b, c, ... If a, b, c, ... are chosen to be powers of 2, the corresponding y can be added, i.e., the result is a+1, b+2, c+3, and so on, all powers of 2, then a byte array can be stored. The number z is not limited to the three examples selected in Figure 15.

[0084] An encoder like the one in Figure 1 converts the difference between consecutive starting positions into a VLC bit sequence; that is, it first determines the prefix, that is, the difference exists within that sub-interval (the yth one), and then adds the suffix so that it is equal to the difference between the starting positions, 0, 2 a , 2 b +2 a The y-th bit is then properly designed by setting it to the negative value of the other y-th bit. A decoder like the one in Figure 6 derives the starting position of the current entropy slice n from the VLC bit sequence, i.e., first checks the prefix to get y, and then the difference is the value of the suffix plus 0, 2 a , 2 b +2 a The y-th and subsequent entree slices are then appropriately designed by setting the y-th and adding the difference to the starting point of the previous entree slice n-1.

[0085] Further benefits can be achieved through the segmentation of entropy slices, in other words, through lower latency transmission and faster decoding.

[0086] In video streaming, achieving higher resolutions (Full HD, Quad HD, etc.) leads to higher data transmission volumes. For time-sensitive scenarios, so-called low-latency use cases (<145ms), transfer time becomes a critical factor. Consider an ADSL upload link to a video conferencing application. Here, so-called random access points in the stream (usually called I-frames) are candidates for creating bottlenecks during transmission.

[0087] To solve this problem and minimize delays in transmission and decoding time, i.e., delays between terminals, new techniques can be applied to interleaved entropy slice schemes for parallel transmission and processing.

[0088] HEVC allows so-called wavefront processing on the decoder side. This is made possible by the use of entropy slices. In the normal case, data from all slices is delivered at once. The decoding process starts as soon as the encoded data reaches the wavefront-decoder-engine. To reduce the time it takes for the decoder to start and finish a frame, according to embodiments of the present invention, segmentation of the entropy slice into smaller chunks using an interleaved approach is employed. Thus, the encoder can deliver data corresponding to a particular entropy slice to the transport layer faster than in the normal case. This results in faster transmission and earlier initiation of the parallel decoding process at the client.

[0089] The generation of slice chunks can be further achieved by subdividing the entropy slice in further slices (dependent slices) that retain all dependencies. When done in each Maximum Encoding Unit (LCU) / Encoding Tree Unit (CTU), these chunks can be interleaved by additionally employing system layer techniques that enable the chunks to be transmitted in an interleaved format and restored by additional signaling, or at least provide knowledge of the original decoding order of the chunks. Such signaling can be a decoding order number (DON), as defined in the IETF RTP payload format for H.264 / AVC (RFC3984). Another system method is to assign different PIDs to each of them, further multiplex them, and thereby interleave them on the transport channel, thereby applying chunks of wavefront substreams to different transport streams, such as in the MEPG-2 system.

[0090] This approach can also be applied across frame boundaries if the following frame slice or entropy slice can already be decoded, for example in the wavefront method, based on the knowledge that the information required to decode the following frame entropy slice is based on the availability of interframe references. The data of these already decodeable frames that follow in the decoding order can be derived from additional information in the stream indicating the maximum possible / signaled motion vector length or the dependency of the data part on the previous frame, or from a fixed reference scheme indicating the signaled position at a fixed position in the sequence, such as a parameter set. This is further outlined below.

[0091] The image can be encoded by the largest encoding unit (LCU) – one entropy slice per row – as depicted in Figures 2 and 9. This is advantageous for the use of wavefront techniques on the decoder side. During the encoding process, the bitstream of each slice can be divided into segments of a fixed size. The resulting segments can then be interleaved and delivered for transmission. The fixed-size segments, due to their variable length, can pose problems at the end of the bitstream.

[0092] Here, there are two possible general solutions. The first solution involves generating a 1-byte segment (typically, the bitstream representation of a slice is a byte array) and controlling the consumption of bytes by each decoder engine, i.e., the decoder discovers the completion of the entropy slice.

[0093] A second solution is to use a finalization code at the end of the slice. In the last case, variable-length segments are possible, but this may also lead to a larger amount of data. Another method is signaling the entropy slice length. One such variation is described below.

[0094] The segment size and interleaving mode can be signaled in either a single SEI message or an SPS.

[0095] The transmission scheme is shown in Figure 16.

[0096] Thus, for example, according to the embodiment of Figure 16, a concept of entropy coding of a sample array is provided that enables lower latency compared to previously available concepts. According to this embodiment, the coded data stream 20 on which the information signal is coded comprises slices such as an entropy slice 90 or slices (shown on the left side) that are just completely independently decodeable, with different parts 12 of the information signal being coded (predictively and / or entropically), and the slice 90 is subdivided into chunks (shown on the right side) (shaded box 310) arranged in an interleaved manner within the coded data stream 20, the interleaving being represented by brackets 312.

[0097] As shown above and described in relation to other embodiments, the slice can be an entropy slice 90, which can then be a suitable subset of the slices of the frame, and thus the encoded data stream can be an entropy encoded data stream 20.

[0098] Interleaving slices 90 312 allows for lower latency because a decoder responsible for decoding any slice 90 does not need to wait for the duration consumed by the previous slice (by slice order 16) of other decoders. Rather, all available decoders can begin decoding their associated slices as soon as their first chunk 310 is available, and potential inter-slice dependencies are eliminated (see wavefront approach).

[0099] The different parts 12 can be entropically encoded into the entropy slice 90 using independently set probability estimates within the entropy slice, or, for example, by employing the aforementioned probability estimates between entropy slices such that the different parts 12 are encoded into the entropy slice 90 along their respective entropy coding paths 14, and the lower entropy slices have their corresponding parts 12 encoded therein, as previously described, in particular in spatially adjacent parts of the corresponding parts, using probability estimates derived from the probability estimates used in higher-rank entropy slices.

[0100] The entropy-encoded data stream 20 may optionally include a header, as shown in Figure 16. The header 300 may be associated with the frames (sample arrays) of the sequence 30 and may contain information representing the length of the entropy slice. Information regarding the length of the entropy slice 90 can be encoded within the header 300 using VLC code. Using knowledge of the length of the entropy slice, the decoding side can identify the last chunk and its length associated with each of the entropy slices 90. However, start codes or other representation schemes can also be used. The start position of a slice can also be identified by the decoding process, which knows exactly where the slice ends. Thus, it is possible to rely solely on the decoder's representation, which in some cases requires signaling between decoders when a "previous" entropy slice in the stream ends later than a "subsequent" slice. This may require "adaptive" signaling in the stream, which can be based on start codes, for specific cases.

[0101] For example, the entropy slice 90 can be placed after the header 300, as shown in Figure 16.

[0102] Chunk 310 can be of equal length, at least with respect to the initial adjacent portion of the sequence of chunks 310, starting from the first chunk in the order in which the chunks are placed within the entropy-encoded data stream 20. Subsequent chunks can vary in length. Subsequent chunks may be equal in length to the initial adjacent portion of the sequence, or they may be less. The length of a subsequent chunk can be derived from the aforementioned information in the header 300, which represents the length or starting position of the entropy slice 90. Chunks 310 can be placed within the entropy-encoded data stream 20 in a periodic manner, according to the order defined within the entropy slice. Chunks that are entirely within the previous cycle of the entropy slice can be skipped in the current and subsequent cycles.

[0103] Other information signals, as well as sample array sequences such as video signals, can similarly be transmitted by the data stream 20. Therefore, the different parts 12 do not have to be predetermined parts of a sample array, such as an image / frame.

[0104] As described above, the entropy slice 90 may have different portions 12 of the sample array 10 encoded therein by predictive coding using entropy coding of the prediction residuals of the inter-entropy slice predictions and / or inter-frame predictions. That is, as described above, the different portions may be spatially distinct portions of frame 10 or multiple frames 30. The latter case is applied based on knowledge derived from an inter-frame reference where the information necessary to decode the entropy slice of the following frame is available, for example, in a wavefront method, when the slice or entropy slice of the following frame can already be decoded. These already decodeable data of subsequent frames in the decoding order can be derived from the maximum allowed / signaled motion vector length or additional information in the stream that indicates the dependency of the data part of the previous frame, and the inter-entropy slice prediction may include intra-predictions, while the inter-frame prediction may include motion-compensated predictions. Examples are outlined below.

[0105] The independence of the probability estimates set within the entropy slices described above can relate to both probabilistic adaptation and contextual modeling. That is, a context selected within an entropy slice can be selected independently of other entropy slices, and the probability estimates of a context can similarly be initialized and adapted independently of any other entropy slice.

[0106] The corresponding entropy decoder can be configured as follows:

[0107] An entropy decoder configured to decode an entropy-encoded data stream 20 having an entropy slice 90 in which different parts of a frame are entropy-encoded, wherein the entropy slice is subdivided into chunks 310 arranged in an interleaved manner within the entropy-encoded data stream 20, can be configured as shown in Figure 6 and may further include a deinterleaver, represented by 314 in Figure 16, which is configured to deinterleave the chunks 310.

[0108] In particular, as illustrated in Figure 5, the entropy decoder may comprise multiple entropy decoders 130, such as threads running on different processing kernels, and the deinterleaver may be configured to transfer each entropy slice's chunk 310 to the entropy decoder 44 associated with that entropy slice.

[0109] In other words, an entropy slice can be subdivided into chunks, which can then be interleaved, and the decoder may include a deinterleaver to deinterleave the chunks, and operations on the entropy slice can be started in parallel along path 16, even before any entropy slice is received as a whole. It should be recalled that the length of a chunk is preferably measured in the entropy coding domain rather than the syntax domain, to correspond to many specific spatial parts / blocks in, for example, an image (even though the latter option is equally available).

[0110] Next, the use of temporal dependency is described. It can be used in addition to, or instead of, the extended forms of probability estimation described so far.

[0111] The wavefront processing pattern can be extended to entropy coding by a new probability calculation that also uses the inter-frame temporal dependency for each LCU, as described here.

[0112] As is well known, probabilities are reinitialized at the beginning of each frame (the first LCU). This causes probabilities already obtained in the previous frame to be lost. To reduce the loss of encoding efficiency, the probabilities of the end state of the image (320) or, in the case of using an entropy slice, the end state of the slice (322) can be passed from the reference frame 324 to the first LCU of the current frame 10 or the entropy slice 12, respectively (Figure 17). Since parallel wavefront processing can also cross frame boundaries, i.e., the encoding process of the slice of the previous frame may not have already finished while the slice of the frame is being encoded, such data for the corresponding slice in the reference frame can be derived not only from the end position but also from the previous position in the referenced slice. Therefore, signaling can be used to indicate the reference position, or it can be indicated by a scheme.

[0113] JPEG0007869289000003.jpg119168

[0114] The analysis process for the end state of the reference frame was tested using a probabilistic adaptation method, and the results are illustrated in Figures 10-19 (temporal graphs).

[0115] Another opportunity to use dates from other frames is to swap probabilities obtained between arrayed LCUs. The main idea is based on the assertion that the properties of the reference frame do not differ significantly from those of the current frame. To speed up the learning of probabilities along the LCUs in the frame, we can try passing the end state of each LCU in the current frame to the appropriate LCU. This proposal is illustrated in Figure 18.

[0116] Under a reference frame, different opportunities can be understood. For example, the last encoded frame can be used as a reference frame. Alternatively, only the last encoded frame from the same temporal layer can be used as a reference.

[0117] Furthermore, this approach can be combined with the methods already proposed above, as it involves using the last (slice) information from the reference frame, probabilistic adaptation, and the use of a second LCU for the upper line.

[0118] The spatial adaptation process described above can be modified as follows:

[0119] JPEG0007869289000004.jpg12170

[0120] Here, N can be 1 or 2 or more, and {i,j}1… N Each of these is selected from (located within) any of the previous entropy slices 90 (in the entropy slice order 16) in the current sample array 10 and its associated portion 12, where M can be 1 or 2 or more, and {i,j}'1… M It is located within the previous sample array 350. CU50{i,j}'1… M At least one of these can be positioned in the same location relative to p(n). CU50{i,j}1… N Refer to the above description for possible choices. The "mean" function can be a weighted sum, median function, or one of the others.

[0121] The spatial adaptation process described above can be replaced by the following equation.

[0122] JPEG0007869289000005.jpg10170

[0123] Here, M can be 1 or 2 or more, and {i,j}'1… MIt is located within the previous sample array. {i,j}'1… M At least one of these can be positioned in the same location relative to p(n). {i,j}1… N The above description is relevant to the possible choices. The function "mean" can be a weighted sum, median function, or one of the following: {i,j}'1… M At least one of them can be positioned in the same location relative to p(n).

[0124] As a specific extension of the use of arrayed information, one can apply an approach that utilizes dates obtained from other blocks from one or more reference frames.

[0125] The techniques mentioned earlier use only information obtained from immediate neighbors in the current frame or reference frame. However, this does not mean that the probabilities obtained in this case are the best. Adjacent LCUs do not necessarily have the best probabilistic model due to image segmentation (residuals). It is assumed that the best results can be achieved with the help of a block from which predictions are made. And therefore, this appropriate block can be used as a reference for the current LCU.

[0126] Thus, in the above adaptation example, {i,j}1… N and / or {i,j}'1… M This can be selected by a CU that serves as a predictor provider for p(n).

[0127] The proposed temporal probabilistic adaptation / initialization scheme can also be used without entropy slices or with a single entropy slice per frame.

[0128] According to the latter embodiment, the benefit in the rate of probabilistic adaptation is achieved by combining the probabilistic adaptations of temporally adjacent / related frames. Described therein is a decoder as shown in Figure 6, which is configured to reconstruct a sequence of sample arrays from an entropy-encoded data stream, and to entropy-decode the current frame of the entropy-encoded data stream in order to reconstruct the current sample array of the sequence of sample arrays, to perform entropy-decode along the entropy-encoded path using a probabilistic estimate, and to adapt the probabilistic estimate along the entropy-encoded path using parts decoded before the current frame, the entropy-decoded stage is configured to initialize or determine a probabilistic estimate for the current frame based on the probabilistic estimate used in decoding the frames decoded before the entropy-encoded data stream.

[0129] That is, for example, the probability estimate for the current frame is initialized based on the resulting probability estimate after the decoding of the frame decoded before the entropy-encoded data stream is completed. The buffer requirement is therefore low, since the end state of the probability estimate must simply be buffered up to the beginning of the decoding of the current frame. Of course, this embodiment can be combined with the embodiments in Figures 1-9, in that for the first part of each part 12, not only the probability estimate used for spatially adjacent parts in the previous entropy slice (if available) is used, but also, for example, the end state of the probability estimate of the corresponding (e.g., spatially) entropy slice in the previous frame in a weighted manner. Since parallel wavefront processing can also cross frame boundaries, i.e., the encoding process of the slice in the previous frame may not have already finished while the encoding of the slice in the frame is being done, such data of the corresponding slice in the reference frame can be derived in the referenced slice not only from the end position but also from the previous position. Therefore, signaling can be used to indicate the reference position, or it can be indicated by a scheme.

[0130] Furthermore, for example, the probability estimates used for coding parts / blocks of previously decoded frames are all buffered, not just for the final state, and the decoder performs entropy decoding of the current part (X) of a predetermined entropy slice, by optionally using the probability estimates used for adjacent parts (such as p4) of spatially adjacent previous entropy slices in the entropy decoding of spatially adjacent previous entropy slices (e.g., the slice containing X) in the entropy decoding of a spatially adjacent entropy slice (see above for the probability derivation of spatial connections), based on the probability estimates of each predetermined entropy slice applied using parts (e.g., including p1) that were decoded before the predetermined entropy slice, and the probability estimates used in the entropy decoding of the spatially corresponding parts of the entropy slices of previously decoded frames, in the entropy decoding of spatially adjacent previous entropy slices (e.g., the slice containing X). Furthermore, as mentioned above, the spatial correspondence within a part and the identification of which previously decoded frames are suitable for probabilistic adoption for the current frame can be determined with the help of motion information such as the motion index, motion vector, and other information of the current part / block.

[0131] Previously, the wavefronts extending during wavefront processing were described as extending diagonally primarily by one sample array 10 through coding / decoding performed sequentially on the sample arrays. However, this is not mandatory. Here, Figure 19 is referenced. Figure 19 shows portions from a sequence of sample arrays, the sample arrays of that sequence are represented so as to be arranged in a sample array coding order 380 which is determined among them and may or may not coincide with the presented time order. Figure 19 exemplifies the subdivision of sample array 10 into four entropy slices, each of which has been coded / decoded. Entropy slices that have already been coded / decoded are shown hatched. Four coding / decoding threads (coding / decoding stages) 382 are currently operating with respect to four entropy slices 12 of the sample array having index n. However, Figure 19 shows thread number 5 on the left, and this further thread 382 having number 5 in Figure 19 can operate to encode / decode the next sample array, i.e., n+1, in the frame n currently being encoded / decoded, where each reference part is already available, i.e., guaranteed to have already been processed by one of threads 1-4. These parts are referenced, for example, in the prediction shown as 64 in Figure 1.

[0132] Figure 19 illustrates, by dashed line 384, the line extending by sample array n+1, which is located at the same location as the boundary between, on the one hand, the already processed, i.e., already encoded / decoded portion of sample array n, i.e., the hatched portion in the sample array, and on the other hand, the unprocessed portion of sample array n, i.e., the unhatched portion of the sample array. Figure 19 also illustrates, by double arrows, the maximum possible length of the motion vector measured in the column and row directions, i.e., y, respectively. max and x max This is shown. Therefore, Figure 19 also shows a displaced version of line 384, in other words, the distance in the column direction y, by dashed line 386.max , in the row direction x max Line 386 is positioned at the maximum possible distance from line 384 so as not to fall below the following. As shown in the figure, the coding units 50 in sample array n+1, where any reference portion in sample array n is guaranteed to be found, are positioned so that they are entirely contained within the already processed portion of sample array n, in other words, they are set to half of sample array n+1 and are upstream of line 386. Thus, thread 5 can already operate to decode / encode these coding units, as shown in Figure 19. As shown in the figure, even a sixth thread can operate with respect to the second entropy slice in the entropy slice order 16 of sample array n+1. This causes the wavefront to extend not only spatially but also temporally, by the spatial-temporal space across the sequence 30 of the sample array.

[0133] It should be noted that the wavefront configuration mentioned above also works when combined with the probability estimation combination outlined above, across the entropy slice boundary.

[0134] Furthermore, with respect to the chunk configuration outlined above, it should be noted that the subdivision of an entropy slice into smaller pieces, i.e., chunks, is not limited to being performed in the entropy coding domain, i.e., the entropy compression domain. Consider the above argument: the entropy slices described above have the benefit of reducing the loss of coding efficiency, despite the possibility of wavefront processing by deriving probabilistic estimates from previously coded / decoded entropy slices or previously coded / decoded frames, i.e., by initializing and / or adapting probabilistic estimates based on such previous entropy slice probabilistic estimates. Each of these entropy slices is assumed to be entropy coded / decoded by a single thread in the case of wavefront processing. That is, when subdividing an entropy slice, it is not necessary to render the chunks to be codecable / decoded in parallel. Rather, the encoder simply provides an opportunity to output subparts of the bitstream of that entropy slice before finalizing the entropy coding, and the decoder provides an opportunity to operate with these subparts, i.e., chunks, before receiving the remaining chunks of the same entropy slice. Furthermore, interleaving becomes possible on the receiving end. However, it is not necessary to perform subdivision in the entropy coding domain in order to enable subsequent deinterleaving. In particular, by simply intermittently resetting the internal state of the probabilistic interval, i.e., the probabilistic interval width value and offset value of the entropy coding / decoding cores, respectively, it is possible to perform the subdivision of the proposed entropy slice into smaller chunks without significant loss of coding efficiency. However, the probabilistic estimates are not reset. Rather, they are continuously updated / adapted from the beginning to the end of the entropy slice, respectively. This policy makes it possible to subdivide the entropy slice into individual chunks by subdivision performed in the syntax element domain rather than the compressed bitstream domain.The subdivision can follow the spatial subdivision outlined below to facilitate signaling of the chunk interface to the decoder. Each chunk may be provided by its own chunk header representing its starting position in the sample array, measured with respect to the coding order 14, for example, with an index for its entropy slice, relative to the starting position of each entropy slice, or to a protruding position in the sample array 10, such as the upper left corner.

[0135] To more clearly illustrate the subdivision of the entropy slice into chunks according to the latter embodiment, Figure 20 is referenced. Figure 20 shows, for illustrative purposes only, a sample array 10 subdivided into four entropy slices. The currently encoded portion of the sample array 10 is indicated by hatching. Three threads are currently operating with respect to the entropy coding of the sample array 10 and output chunks of the entropy slice on a makeshift basis: for example, consider the first entropy slice in entropy slice order 16, corresponding to portion 12 of the sample array 10. After coding subpart 12a of portion 12, the encoder performs some termination steps to finalize the arithmetic bitstream created so far from subpart 12a, i.e., in the case where the entropy coding core 80 arithmetically codes from there to form chunk 390. The coding procedure then resumes with respect to the subsequent subpart 12b of the entropy slice 12 in coding order 14, while starting a new entropy bitstream. This means that internal states, such as the probability interval width value and probability interval offset value of the entropy coding core 80, are reset. However, the probability estimates are not reset; they remain unchanged. This is illustrated by arrow 392 in Figure 20. In Figure 20, it is illustrated that the entropy slice or portion 12 is subdivided into two or more sub-parts, and therefore even the second chunk 1b, according to some entropy, ends before reaching the end of portion 12 along the coding order 14, where the next chunk in the line is started by the others.

[0136] Simultaneously, the other threads operate on the second entropy slice or sub-slice 12 in entropy slice order 16. Once the first sub-slice of this second entropy slice / sub-slice 12 is finished, chunk 2a is output, where the remaining entropy coding of the second entropy slice begins, but the probability estimate remains valid at the end of chunk 2a.

[0137] Figure 20 attempts to illustrate, by time axis 394, that chunks 390 are output as soon as they are finalized. This leads to interleaving similar to that shown in Figure 16. Each chunk is packetized into packets and can be transported to the decoding side by several transport layers in any order. The transport layers are illustrated using arrows 396.

[0138] The decoder must reassign the chunks to their sub-parts 12a, 12b, and so on. For this purpose, each chunk 390 may have a header section 398 that indicates the starting position of its associated sub-part 12a or 12b, i.e., the sub-part whose syntax elements describing it are entropy-encoded within each chunk. Using this information, the decoder can associate each chunk 390 with its entropy slice and its sub-part within the portion 12 of that entropy slice.

[0139] For the sake of explanation, Figure 20 also illustrates the possibility that the connection between consecutive sub-parts 12a and 12b of the entropy slice 12 does not necessarily coincide with the boundary between consecutive coding units 50. Rather, such a connection can be defined at a deeper level of the exemplary multi-tree subdivision of the coding units described above. The positional information contained in the header 398 can indicate the beginning of the sub-part related to the current chunk 390 with sufficient precision to identify the position within the sequence of syntax elements described for each sub-block of each coding unit.

[0140] As the above discussion has shown, the loss of coding efficiency was minimal from the subdivision of entropy slices into chunks. While the entropy termination process and packetization may involve some loss of coding efficiency, the benefits of low latency are enormous.

[0141] Furthermore, it should be noted that the spatial subdivision chunking configuration mentioned above also works spatially and temporally across entropy slice boundaries, in combination with the probability estimation combination outlined above.

[0142] A decoder like the one in Figure 6 can cancel chunk transmission as follows. In particular, the decoder can check which entropy slice the current chunk belongs to. This check can be done based on the location information described above. Next, it can check whether the current chunk corresponds to the first sub-part of the corresponding entropy slice along the entropy coding path 14. In that case, the decoder can entropically decode the current chunk under the adaptation of each probability estimate, and when it entropically decodes other chunks corresponding to the second sub-part of a predetermined entropy slice along the entropy coding path, it can consider the state of each probability estimate that appears at the end of the entropy decoding of the current chunk. "Consider" can include setting the probability estimate at the beginning of chunk 1b to be equal to the probability estimate that appears at the end of sub-part 12a of chunk 1a by a probability adaptation starting from the state of the probability estimate at the beginning of chunk 1a, or equal to a combination of that and the entropy probability estimates from the other entropy slices described above. The above discussion is relevant, as it also involves the probabilistic initialization at the beginning of the first chunk 12a, which also forms the beginning of the corresponding entropy slice. In other words, if the current slice is the second or subsequent chunk in order 14, the decoder can entropy decode the current chunk along the entropy coding path 14 using a probabilistic estimate determined by the probabilistic estimate that appears at the end of the entropy decoding of the chunk corresponding to the subpart of the previous predetermined entropy slice, along the entropy coding path 14.

[0143] The above description reveals different methods that are useful for parallel encoding and decoding, and that can help optimize processes already present in the emerging HEVC video encoding standard. A brief overview of entropy slices has been presented. It has been shown how they can be formed, what benefits can be achieved by slicing, and what penalties can arise from these techniques. Many methods have been proposed that are assumed to improve the learning process of probabilities along the LCU (Largest Encoding Unit) in a frame by better utilizing local dependencies between LCUs, and even temporal dependencies between LCUs in different frames. Different combinations are claimed to provide improvements to both parallel and non-parallel concepts of encoding and decoding.

[0144] For example, the performance improvements in high efficiency achieved by the best mix of the proposed approaches are -0.4% intra-, -0.78% low-latency, and -0.63% random access compared to HM3.0 without entropy slicing, or -0.7% intra-, -1.95% low-latency, and -1.5% random access compared to the entropy slicing approach with normal reinitialization.

[0145] In particular, the following technologies have been presented as described above. • Use not only the local dependency of the LCU but also the temporal dependency to optimize the adaptation of the CABAC probability before encoding each LCU (see Figures 1-9, 17, and 18). • This achieves greater flexibility in decoding and allows the use of entropy slices so that specific regions within a frame are independent of each other. • Parallelization, such as wavefront processing, enables minimal signaling of the starting position of slice / entropy slices (see Figure 15). • Interleaved transfer of entropy slices / slices enables low-latency transfer in a parallel encoder-transmitter-receiver-decoder environment (see Figure 16).

[0146] All the methods described above have been combined and tested in HM3.0. The results obtained with HM3.0 as the reference point without any entropy slicing are presented in Tables 1 and 2 (where 2LCU is the use of the second LCU of the upper line, 2LCU+Prob.Adap is 2LCU merged with the probabilistic adaptation method, and Temporal is the use of temporal dependency (final state of the reference frame) by probabilistic adaptation for each LCU).

[0147] JPEG0007869289000006.jpg28114

[0148] JPEG0007869289000007.jpg28114

[0149] However, it is interesting to see how the proposed approach affects wavefront processing by reinitializing probabilities at the beginning of each line of the LCU. These results are illustrated in Tables 3 and 4 (where orig_neiInit is a comparison between HM3.0 without entropy slicing and with entropy slicing with initialization).

[0150] JPEG0007869289000008.jpg29130

[0151] JPEG0007869289000009.jpg28130

[0152] The results above indicate that fairly extensive use of intra-frame and inter-frame dependencies, along with a reasonable application of already acquired information, prevents mean loss.

[0153] The wavefront processing approach to HEVC video encoding and decoding combines the possibility of using dependencies between adjacent LCUs and temporal frame dependencies through the concept of wavefront parallel processing. In this way, losses can be reduced and performance improvements can be achieved.

[0154] The benefit in the speed of probabilistic adaptation has been achieved by performing probabilistic adaptation operations on spatially adjacent entropy slices.

[0155] As already stated above, all of the above embodiments can be combined with one another, and therefore, any reference to the possibility of a particular embodiment applies, of course, to the other embodiments as well.

[0156] While several embodiments have been described in the context of the apparatus, it is clear that these embodiments also represent descriptions of the corresponding methods, where blocks or devices correspond to method steps or functions of method steps. Similarly, embodiments described in the context of method steps also represent descriptions of the corresponding blocks, items, or functions of the corresponding apparatus. Some or all method steps can be performed by (or using) hardware devices such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, some one or more of the most important method steps can be performed by such devices.

[0157] The encoded signals of the present invention described above can be stored in a digital storage medium or transmitted through a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0158] Depending on the specific implementation requirements, embodiments of the present invention may be implemented in hardware or software. Implementation may be carried out using a digital storage medium, such as a floppy disk, DVD, CD, ROM, PROM, EPROM, EEPROM, or flash memory, which has electronically readable control signals stored thereon and cooperates (or can cooperate) with a programmable computer system so that each method is performed. Therefore, the digital storage medium may be computer-readable.

[0159] Some embodiments of the present invention include a data carrier having electronically readable control signals that can cooperate with a programmable computer system so that one of the methods described in this specification is performed.

[0160] Generally, embodiments of the present invention can be implemented as a computer program product comprising program code that is operable to perform one of the methods of the present invention when the computer program product runs on a computer. The program code can be stored, for example, on a machine-readable carrier.

[0161] Other embodiments include a computer program stored on a machine-readable carrier that performs one of the methods described in this specification.

[0162] In other words, one embodiment of the method of the present invention is a computer program having program code that, when the computer program runs on a computer, performs one of the methods described in this specification.

[0163] A further embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) having a computer program recorded thereon that performs one of the methods described in this specification. The data carrier, digital storage medium or recording medium is typically tangible and / or non-transient.

[0164] A further embodiment of the method of the present invention is a data stream or sequence of signals representing a computer program that performs one of the methods described in this specification. The data stream or sequence of signals may be configured to be transmitted, for example, by a data communication connection, such as the Internet.

[0165] Further embodiments include processing means, such as a computer or a programmable logic device, configured or adapted to perform one of the methods described in this specification.

[0166] Further embodiments include a computer on which a computer program is installed that performs one of the methods described in this specification.

[0167] Further embodiments of the present invention include an apparatus or system configured to transfer (e.g., electronically or optically) a computer program that performs one of the methods described in this specification to a recipient. The recipient may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server that transfers the computer program to the recipient.

[0168] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) can be used to perform some or all of the functions of the methods described herein. In some embodiments, a field-programmable gate array can cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0169] The embodiments described above are merely illustrative of the principles of the present invention. Modifications and changes to the configurations and details described in this specification will be obvious to those skilled in the art. Therefore, the present invention is intended to be limited only to the scope of the immediate claims and not to any specific details expressed in the manner of the embodiments described in this specification.

Claims

1. A decoder for reconstructing a sample array (10) from an entropy-encoded data stream, wherein the decoder is Multiple entropy slices in the entropy-encoded data stream are entropy-decoded to reconstruct various parts (12) of the sample array associated with the entropy slices, For each entropy slice, the entropy decoding is performed along each entropy coding path (14) using the respective probability estimation. Using the previously decoded portion of each entropy slice, the respective probability estimates are fitted along the respective entropy coding paths. The entropy decoding of the plurality of entropy slices is sequentially started using the entropy slice order (16). When entropy decoding a default entropy slice, the entropy decoding of the current portion of the default entropy slice is performed based on the respective probability estimates of the default entropy slice fitted using the previously decoded portion of the default entropy slice, and the probability estimates used when entropy decoding a preceding spatially adjacent entropy slice in the entropy slice order using the adjacent portion of the spatially adjacent entropy slice. It is configured in such a way, Here, each entropy slice has data for the corresponding portion of the sample array entropy encoded, the various portions form block rows of the sample array, the blocks are arranged in an orderly manner in rows and columns, and the portion corresponding to the entropy slice consists of the same number of blocks, and the direction of the entropy encoding path is parallel to the rows of the blocks. The decoder is configured to store the probability estimates that appear after entropy decoding the second block along the respective entropy coding path of the portion corresponding to the previous entropy slice in the entropy slice order, and to use the stored probability estimates for initialization before decoding the first block along the respective entropy coding path of the portion corresponding to each entropy slice. The entropy slice is subdivided into chunks, each chunk corresponding to a sub-part of the various parts, and is provided with a chunk header that clarifies the starting position of each chunk in the sample array relative to the upper left corner of the sample array. The decoder mentioned above is Using the aforementioned chunk header, each chunk is reassigned to the corresponding sub-part. Check if the current chunk corresponds to the first lower portion of the portion of the default entropy slice along the entropy coding path. If applicable, the current chunk is entropically decoded by fitting the respective probability estimates, and when entropically decoded another chunk corresponding to the second sub-part along the entropy coding path of the portion of the default entropy slice, the state of each probability estimate that appears at the end of the entropy decoding of the current chunk is taken into consideration. If not applicable, the current chunk is entropically decoded using a probability estimate that appears at the end of the entropy decoding of the chunk corresponding to the lower part of the default entropy slice, along the entropy coding path, of the lower part corresponding to the current chunk. It is configured in such a way. decoder.

2. The decoder according to claim 1, wherein the entropy slice order is selected such that the various parts follow each other along the entropy slice order in an oblique direction (16) with respect to the entropy coding path (14) of the entropy slices which extend parallel to each other.

3. The decoder according to claim 1 or 2, wherein for each entropy slice (90), the decoder is configured to initialize the probability estimation (94) for each entropy slice before decoding the first block of the portion (12) corresponding to each entropy slice along the entropy coding path (14), wherein the probability estimation appears along the respective entropy coding path after entropy decoding the second block (50) of the portion corresponding to the previous entropy slice in the entropy slice order (16).

4. The decoder according to claim 1 or 2, wherein for each entropy slice, after entropy decoding the current portion of the default entropy slice based on the respective probability estimates (94) of the default entropy slice, the decoder performs the entropy decoding along the respective entropy coding path and the fitting of the respective probability estimates along the respective entropy coding path such that the respective probability estimates (94) of the default entropy slice are fitted depending on the probability estimates that appear when entropy decoding the current portion of the default entropy slice and the adjacent portions of the spatially adjacent entropy slices.

5. The decoder according to claim 4, wherein the decoder is configured such that, after the current portion of the default entropy slice has been entropy-decoded based on the respective probability estimates of the default entropy slice, the fitting of the respective probability estimates of the default entropy slice is performed by averaging the results of the first fitting, which depends on the current portion of the default entropy slice and the probability estimates used in the entropy decoding of the adjacent portions of the spatially adjacent entropy slices.

6. The decoder according to claim 1 or 2, wherein the decoder is configured to guide the entropy decoding of directly consecutive entropy slices in entropy slice order so that the distance measured in block units along the entropy coding path of the currently decoded block corresponding to the directly consecutive entropy slice does not become less than two blocks.

7. The decoder according to claim 1 or 2, wherein the decoder is configured to guide the entropy decoding of directly consecutive entropy slices in entropy slice order such that the distance measured in block units along the entropy coding path of the currently decoded block corresponding to the directly consecutive entropy slice remains at 2 blocks.

8. The decoder according to claim 1 or 2, wherein the entropy slice is subdivided into chunks, and the decoder comprises a deinterleaver for deinterleaving the chunks, the deinterleaver configured to initiate parallel entropy decoding of the entropy slice along the entropy coding path even before any of the entropy slices have been fully received.

9. The decoder according to claim 1 or 2, wherein the sample array (10) is the current sample array in a sequence of sample arrays, and the decoder is configured to entropically decode the current portion of the default entropy slice based on the probability estimates of each of the default entropy slices fitted using the previously decoded portions of the default entropy slices, the probability estimates used when entropically decoding a preceding spatially adjacent entropy slice in the entropy slice order using adjacent portions of the spatially adjacent entropy slices, and the probability estimates used when decoding a previously decoded frame of the entropy-encoded data stream relating to another sample array other than the current sample array.

10. The decoder according to claim 1, wherein the entropy-encoded data stream is a layered data stream.

11. An encoder for encoding a sample array (10) into an entropy-encoded data stream, wherein the encoder is Multiple entropy slices, each associated with a different portion (12) of the sample array, are entropy encoded into the entropy-encoded data stream, For each entropy slice, the entropy coding is performed along each entropy coding path (14) using the respective probability estimation. Using the previously encoded portion of each entropy slice, the respective probability estimates are fitted along the respective entropy encoding path. The entropy coding of the plurality of entropy slices is sequentially started using the entropy slice order (16), When entropy coding a default entropy slice, the current portion of the default entropy slice is entropy coded based on the respective probability estimates of the default entropy slice fitted using the previously coded portion of the default entropy slice, and the probability estimates used when entropy coding a preceding spatially adjacent entropy slice in the entropy slice order using the adjacent portion of the spatially adjacent entropy slice. It is configured in such a way, Here, the encoder is configured such that each entropy slice has data for the corresponding portion of the sample array entropy encoded, the various portions form block rows of the sample array, the blocks are arranged in an orderly manner in rows and columns so that the portion corresponding to the entropy slice is composed of the same number of blocks, and the orientation of the entropy encoding path is configured to be parallel to the rows of the blocks. The encoder is configured to store the probability estimates that appear after entropy encoding the second block along the respective entropy encoding path of the portion corresponding to the previous entropy slice in the entropy slice order, and to use the stored probability estimates for initialization before encoding the first block along the respective entropy encoding path of the portion corresponding to each entropy slice. The entropy slice is subdivided into chunks, each chunk corresponding to a sub-part of the various parts, and is provided with a chunk header that clarifies the starting position of each chunk in the sample array relative to the upper left corner of the sample array, and each chunk can be reassigned to the corresponding sub-part using the chunk header. The encoder described above is Check if the current chunk corresponds to the first lower portion of the portion of the default entropy slice along the entropy coding path. If applicable, the current chunk is entropy coded by fitting the respective probability estimates, and when entropy coding another chunk corresponding to the second sub-part along the entropy coding path of the portion of the default entropy slice, the state of each probability estimate that appears at the end of the entropy coding of the current chunk is taken into consideration. If not applicable, the current chunk is entropy coded using a probability estimate that depends on the probability estimate that appears at the end of the entropy coding of the chunk corresponding to the lower part of the default entropy slice, along the entropy coding path of the lower part of the current chunk. It is configured in such a way. Encoder.

12. The encoder according to claim 11, wherein the entropy-encoded data stream is a layered data stream.

13. A method for reconstructing a sample array (10) from an entropy-encoded data stream, wherein the method is: Multiple entropy slices in the entropy-encoded data stream are entropy-decoded to reconstruct various parts (12) of the sample array associated with each entropy slice, For each entropy slice, the entropy decoding is performed along each entropy coding path (14) using the respective probability estimation. Using the previously decoded portion of each entropy slice, the respective probability estimates are fitted along the respective entropy coding paths. The entropy decoding of the plurality of entropy slices is sequentially started using the entropy decoding sequence (16). The entropy decoding of a default entropy slice includes the step of performing entropy decoding of the current portion of the default entropy slice based on the respective probability estimates of the default entropy slice fitted using the previously decoded portion of the default entropy slice, and the probability estimates used when entropy decoding a preceding spatially adjacent entropy slice in the entropy slice order using the adjacent portion of the spatially adjacent entropy slice, Here, each entropy slice has data for the corresponding portion of the sample array entropy encoded, the various portions form block rows of the sample array, the blocks are arranged in an orderly manner in rows and columns, and the portion corresponding to the entropy slice consists of the same number of blocks, and the direction of the entropy encoding path is parallel to the rows of the blocks. The probability estimates that appear after entropy decoding the second block along each of the entropy coding paths of the portion corresponding to the previous entropy slice in the order of the entropy slices are stored, and the stored probability estimates are used for initialization before decoding the first block along each of the entropy coding paths of the portion corresponding to each of the entropy slices. The entropy slice is subdivided into chunks, each chunk corresponding to a sub-part of the various parts, and is provided with a chunk header that clarifies the starting position of each chunk in the sample array relative to the upper left corner of the sample array. The aforementioned method, The steps include: reallocating each chunk to the corresponding sub-part using the chunk header; A step of checking whether the current chunk corresponds to the first sub-part of the portion of the default entropy slice along the entropy coding path, If applicable, the steps include: entropy decoding the current chunk by fitting the respective probability estimates, and entropy decoding another chunk corresponding to the second sub-chunk along the entropy coding path of the portion of the default entropy slice, taking into account the state of the respective probability estimates that appear at the end of the entropy decoding of the current chunk; If not applicable, the current chunk is entropy-decoded using a probability estimate that depends on the probability estimate that appears at the end of the entropy decoding of the chunk corresponding to the previous sub-part along the entropy coding path of the sub-part of the default entropy slice corresponding to the current chunk. Methods that further include this.

14. A method for encoding a sample array (10) into an entropy-encoded data stream, Each entropy slice is associated with a different part (12) of the sample array, and each of the multiple entropy slices is entropy encoded into the entropy encoded data stream, For each entropy slice, entropy coding is performed along the respective entropy coding path (14) using the respective probability estimation. Using the previously encoded portion of each entropy slice, the respective probability estimates are fitted along the respective entropy coding paths. Using the entropy slice order, the entropy coding of the plurality of entropy slices is sequentially started, When entropy coding a default entropy slice, the current portion of the default entropy slice is entropy coded based on the respective probability estimates of the default entropy slice fitted using the previously coded portion of the default entropy slice, and the probability estimates used when entropy coding a preceding spatially adjacent entropy slice in the entropy slice order using the neighbor portion of the spatially adjacent entropy slice. Includes steps, Here, each entropy slice has data for the corresponding portion of the sample array entropy encoded, the various portions form block rows of the sample array, the blocks are arranged in an orderly manner in rows and columns, and the portion corresponding to the entropy slice consists of the same number of blocks, and the direction of the entropy encoding path is parallel to the rows of the blocks. The probability estimates that appear after entropy coding the second block along each of the entropy coding paths for the portion corresponding to the previous entropy slice in the order of the entropy slices are stored, and the stored probability estimates are used for initialization before coding the first block along each of the entropy coding paths for the portion corresponding to each of the entropy slices. The entropy slice is subdivided into chunks, each chunk corresponding to a sub-part of the various parts, and is provided with a chunk header that clarifies the starting position of each chunk in the sample array relative to the upper left corner of the sample array. The aforementioned method, The steps include: reallocating each chunk to the corresponding sub-part using the chunk header; A step of checking whether the current chunk corresponds to the first lower portion of the portion of the default entropy slice along the entropy coding path, If applicable, the current chunk is entropy coded by fitting the respective probability estimates, and another chunk corresponding to the second sub-part of the portion of the default entropy slice along the entropy coding path, A step of considering the state of each of the probability estimates that appears at the end of the entropy coding of the current chunk, If not applicable, the probability estimate depends on the probability estimate that appears at the end of the entropy coding of the chunk corresponding to the previous sub-part along the entropy coding path of the sub-part of the default entropy slice corresponding to the current chunk. The steps include: Entropy encoding the current chunk using the above method Methods that further include this.

15. A computer-readable digital storage medium storing a computer program configured to perform the method described in claim 13 or 14 when executed on a computer.

16. A method for decoding a data stream, the method comprising the steps of receiving and decoding a data stream, the data stream encoding a sequence of a sample array, which, when subjected to decoding by a decoder, will decode the sequence of the sample array from the data stream by performing the method of claim 13.

Images