Parallel Entropy Coding

Abstract

A method and apparatus for encoding data into and decoding data from a bitstream are described. The method can efficiently perform parallel encoding and decoding, avoiding padding of substreams and thus reducing the amount of bits in the bitstream. Portions of input data channels are multiplexed and encoded into substreams. During multiplexing, a shuffling method is applied to obtain substreams of more uniform length. The amount of bits in the substreams can be further reduced by including only relevant significant bits in the trailing bits of the encoding process.

Description

[Technical field]

[0001] This disclosure relates to entropy encoding and decoding, and in particular to parallel entropy coding, and in particular to constructing coded substreams, including them in a bitstream, and parsing them from the bitstream. [Background technology]

[0002] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as broadcast digital TV, video transmission over the Internet and mobile networks, real-time conversation applications like video chat, video conferencing, DVDs and Blu-ray Discs, video content acquisition and editing systems, mobile device video recording, and camcorders in security applications.

[0003] Since the development of the block-based hybrid video coding approach in the H.261 standard in 1990, new video coding techniques and tools have been developed and have formed the basis for new video coding standards. One of the goals of most video coding standards has been to achieve bitrate reduction comparable to its predecessor without sacrificing picture quality. Additional video coding standards include MPEG-1 Video, MPEG-2 Video, VP8, VP9, ​​AV1, ITU-T H.262 / MPEG-2, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265, High Efficiency Video Coding (HEVC), ITU-T H.266, Versatile Video Coding (VVC), as well as extensions of these standards such as scalability and / or three-dimensional (3D) extensions.

[0004] The amount of video data required to depict even a relatively short video can be substantial, which can pose difficulties when the data is streamed or otherwise communicated over communication networks with limited bandwidth capacity. Thus, video data is typically compressed before being communicated over today's telecommunications networks. The size of the video can also be an issue when the video is stored on a storage device, since memory resources can be limited. Video compression devices sometimes use software and / or hardware at the source to code the video data prior to transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. With limited network resources and an ever-increasing demand for higher video quality, improved compression and decompression techniques that improve compression ratios without sacrificing much picture quality are desired.

[0005] The video encoding and decoding may be performed by standard video encoders and decoders, e.g., compatible with H.264 / AVC, HEVC (H.265), VVC (H.266), or other video coding technologies. Additionally, the video coding, or parts thereof, may be performed by neural networks. Entropy coding is widely used whether for coding or decoding still pictures or images or other source signals, such as feature channels of neural networks. In particular, arithmetic coding is gaining importance in newer coding approaches. Therefore, it would be desirable to improve the efficiency of entropy coding. Summary of the Invention

[0006] Embodiments of the present disclosure provide an apparatus and method for arithmetic encoding into and arithmetic decoding from a bitstream of data, the bitstream including coded bits and leading trailing bits.

[0007] Embodiments of the invention are defined by the features of the independent claims, further advantageous implementations of the embodiments are defined by the features of the dependent claims.

[0008] According to an embodiment, an encoding method is provided for entropy encoding data of multiple channels of the same size into a bitstream, comprising: multiplexing a portion of a first size from each of the multiple channels and thereafter encoding the multiplexed portion of the first size into a first sub-stream; multiplexing a portion of a second size from each of the multiple channels and thereafter encoding the multiplexed portion of the second size into a second sub-stream, the encoding being entropy encoding performed independently on the first sub-stream and the second sub-stream.

[0009] Multiplexing and encoding portions from different channels, instead of encoding each channel separately, offers the possibility to generate the completed sub-stream faster and / or to control the length of the individual sub-streams, for example to obtain sub-streams of more uniform length, which opens the possibility to perform entropy encoding and / or entropy decoding on multiple sub-streams in parallel.

[0010] In an exemplary implementation, the encoding method further includes generating multiple channels of the same size including pre-processed data of the multiple channels of different sizes to obtain multiple channels of the same size.

[0011] The option to obtain channels of the same size from channels of any size provides applicability of the method to different kinds of input data.

[0012] For example, the encoding method further includes multiplexing the first substream and the second substream into a bitstream together with a first substream length indicator indicating a length of the first substream and a second substream length indicator indicating a length of the second substream.

[0013] Indicating the length of the sub-streams in the bitstream allows for the provision of sub-streams with different sizes, and thus may lead to more flexible bitstream construction.

[0014] In an exemplary implementation, the first substream length indicator precedes the first substream in the bitstream, and the second substream length indicator precedes the second substream in the bitstream.

[0015] This feature avoids the need to buffer the entire bitstream in order to extract the individual substreams.

[0016] For example, the second substream length indicator precedes the first substream in the bitstream.

[0017] This bitstream structure, including a concatenated length indicator before the first substream, may provide for more efficient extraction of the substreams from the bitstream.

[0018] In an exemplary implementation, the entropy coding is arithmetic coding.

[0019] An efficient entropy coding mechanism may be provided, based on which rate reduction may be possible using arithmetic coding.

[0020] In an exemplary implementation, the encoding method further includes multiplexing the first substream and the second substream into a bitstream together with a first trailing bit length indicator indicating a length of the first trailing bits of the first substream and a second trailing bit length indicator indicating a length of the second trailing bits of the second substream.

[0021] The status of the encoder after encoding the last part, the trailing bits, may be signaled separately from the substream, allowing additional processing of the trailing bits.

[0022] For example, a first trailing bit length indicator precedes a first substream in the bitstream, and a second trailing bit length indicator precedes a second substream in the bitstream.

[0023] An advantage of this bitstream structure may be the possibility of immediate encoding of the substreams, without the need to buffer multiple substreams and their respective indices.

[0024] For example, a second trailing bit length indicator precedes the first substream in the bitstream.

[0025] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0026] In an exemplary implementation, the encoding method further includes the steps of: adding first trailing bits to the bitstream after the first substream, and adding second trailing bits to the bitstream after the second substream.

[0027] This bitstream structure makes it possible to decode the first substream and the corresponding trailing bits without extracting another substream.

[0028] For example, the first trailing bit is after the second substream in the bitstream.

[0029] Such a bitstream structure makes it possible to start decoding of the individual substreams before extracting the trailing bits from the bitstream.

[0030] In an exemplary implementation, the encoding method further includes padding the bitstream including the first substream length index, the second substream length index, the first trailing bit length index, the second trailing bit length index, the first substream, the second substream, the first trailing bits, and the second trailing bits with bits having a predetermined value to adjust the bitstream length to match an integer multiple of the predetermined amount of bytes.

[0031] This implementation may provide a bitstream that is appropriately conditioned for further processing, such as encapsulation into Network Adaptation Layer units or other packets.

[0032] For example, the first size is equal to the second size.

[0033] Using portions of the same size may result in more efficient performance. For example, it may be possible to use memory units that are appropriate for the hardware and software implementation.

[0034] In an exemplary implementation, every portion contained in the first substream and every portion contained in the second substream are an integer number K of symbols of data for the multiple channels, where K is greater than one.

[0035] Such an approach may provide efficient implementation in software and / or hardware.

[0036] For example, a symbol is a bit.

[0037] In an exemplary implementation, the encoding method further includes a step of selecting and subsequently applying a shuffling method for multiplexing the first size portions and the second size portions, the shuffling method being: selected from a set of predetermined shuffling methods and specifying an order of the first size portions and the second size portions.

[0038] The portions may be shuffled to achieve a more even distribution of substream sizes.

[0039] For example, the shuffling method performs a cyclic permutation of portions of a second size with portions of a first size.

[0040] Such shuffling methods may result in more uniform lengths of the sub-streams, as well as preferably simple implementation of such uniform lengths of the sub-streams.

[0041] For example, the encoding method is performed iteratively, in which case the shuffling method is selected according to: the difference between the length of the current first sub-stream and a statistical value based on the length of the past first sub-stream, and / or the difference between the length of the current second sub-stream and a statistical value based on the length of the past second sub-stream.

[0042] Shuffling may be performed iteratively across the portion to be coded to reduce the length difference between the sub-streams over time.

[0043] For example, the statistics may be based on at least one of an estimated mean, median, minimum, maximum, or growth rate.

[0044] These statistics may provide a suitable means to control the shuffling and thus improve the control of the substream size, which may allow a more uniform load within the streaming process.

[0045] In an exemplary implementation, the entropy coding is arithmetic coding, and the shuffling method is selected according to the current state of the range intervals in the arithmetic coding.

[0046] This can take into account whether the current range of values ​​of the interval for the arithmetic encoder is close to a predefined minimum range for encoding, thereby avoiding renormalization during encoding.

[0047] In an example implementation, the entropy encoding includes generating a first sub-stream with a first entropy encoder and generating a second sub-stream with a second entropy encoder, and the entropy encoding by the first entropy encoder and the second entropy encoder are performed at least partially in parallel.

[0048] Parallel encoding of the sub-streams may result in faster encoding of the complete bitstream.

[0049] For example, the channels may be output channels or latent representation channels of a neural network.

[0050] Neural networks typically provide channels of the same size or at least a fixed size, which makes the above embodiments and examples particularly suitable and readily applicable to these channels.

[0051] In an exemplary implementation, the entropy coding is arithmetic coding, and the method relates to encoding the multiplexed portion into a first substream or a second substream, including: arithmetically coding the multiplexed portion into coded bits and encoder status bits, the coded bits forming the substreams; determining minimum and maximum values ​​of an interval of the arithmetically coded input data; determining a quantity of leading trailing bits, which are consecutive encoder status bits, having the same value in a first most significant bit (MSB) representing the determined maximum value as in a second MSB representing the determined minimum value, the trailing bits being leading encoder status bits; and specifying the determined quantity of leading encoder status bits in a trailing bit length indicator.

[0052] Including leading trailing bits in the bitstream instead of full trailing bits reduces the amount of bits in the bitstream and therefore reduces the rate, e.g., reducing the amount of bits for a rate that is signaled with the same quality.

[0053] For example, the amount of leading encoder status bits, NumTrailingBits, is: NumTrailingBits = CLZ((LOW+RANGE‐1) XOR LOW) where CLZ() is the count of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval.

[0054] The amount of leading trailing bits may be determined precisely instead of rounding to the nearest byte boundary, for example, which may further reduce the amount of bits in the bitstream.

[0055] In an exemplary implementation, a bit with a value of 1 is added after the leading encoder status bits before being included in the bitstream.

[0056] Such an approach may follow any normal practice used to leave bits alternatively valued at 1.

[0057] For example, during arithmetic coding, the current minimum and current maximum values ​​of the interval are stored in a memory of a preset size; including the coded bits in the bitstream involves moving a predetermined amount of bits from fixed bits from the memory to the bitstream; and the fixed bits are consecutive bits having the same value at the MSB of the binary representation of the current minimum and current maximum values.

[0058] Such independent application of the method to two separate substreams provides a prerequisite for parallelization.

[0059] In an exemplary implementation, during arithmetic coding, if the difference between the amount of leading encoder status and a predetermined amount of bits from the fixed bits is below a predetermined threshold; trailing coded bits are generated from the leading encoder status bits by appending a bit whose value is one followed by zeros up to the predetermined amount of bits from the fixed bits; the trailing coded bits are included in the coded bits prior to including the coded bits in the bitstream; and an indicator of the zero leading encoder status bits is included in the bitstream.

[0060] Thus, signaling many leading trailing bits along with an indication of the amount of trailing bits corresponds to and can be avoided as a high processing effort, whereas having more coded bits and signaling zero leading trailing bits can result in less processing effort.

[0061] In an exemplary implementation, the arithmetic coding is a range coding.

[0062] It is thereby possible to improve hardware and software architectures that have limited registers or generally fast memory sizes to take advantage of arithmetic coding.

[0063] A decoding method is provided for entropy decoding a bitstream into multiple channels of equal-sized data, the method including the steps of: decoding a first substream independently of a second substream, and demultiplexing first-sized portions and second-sized portions from the first substream and the second substream into multiple channels of equal size.

[0064] Multiplexing and encoding portions from different channels, instead of encoding each channel separately, offers the possibility to decode substreams of more uniform length, which opens up the possibility of performing entropy decoding on multiple substreams in parallel.

[0065] For example, the decoding method further comprises the step of post-processing the multiple channels of the same size to obtain multiple channels of data of different sizes.

[0066] The option of acquiring data for channels of any size from channels of the same size provides improved applicability of the method to different kinds of data.

[0067] In an exemplary implementation, the decoding method further includes extracting the first substream and the second substream from the bitstream, along with a first substream length indicator indicating a length of the first substream and a second substream length indicator indicating a length of the second substream.

[0068] Indicating the length of the sub-streams in the bitstream allows for the provision of sub-streams with different sizes, and thus may lead to more flexible bitstream construction.

[0069] In an exemplary implementation, the first substream length indicator precedes the first substream in the bitstream, and the second substream length indicator precedes the second substream in the bitstream.

[0070] An advantage of this bitstream structure may be the possibility of immediate encoding or decoding of the substreams, without the need to buffer multiple substreams and their respective indices.

[0071] For example, the second substream length indicator precedes the first substream in the bitstream.

[0072] Providing a concatenated length indicator in front of the substreams can allow faster extraction of individual portions of the bitstream.

[0073] In an exemplary implementation, the entropy decoding is an arithmetic decoding.

[0074] Arithmetic coding is an efficient entropy coding technique that can contribute to rate reduction.

[0075] In an exemplary implementation, the decoding method further includes extracting the first substream and the second substream from the bitstream along with a first trailing bit length indicator indicating a length of the first trailing bits of the first substream and a second trailing bit length indicator indicating a length of the second trailing bits of the second substream.

[0076] Indicating the length of the sub-streams in the bitstream allows for the provision of sub-streams with different sizes, and thus may lead to more flexible bitstream construction.

[0077] For example, a first trailing bit length indicator precedes a first substream in the bitstream, and a second trailing bit length indicator precedes the second substream in the bitstream.

[0078] An advantage of this bitstream structure may be the possibility of immediate decoding of the substreams, without the need to buffer multiple substreams and their respective indices.

[0079] For example, a second trailing bit length indicator precedes the first substream in the bitstream.

[0080] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0081] In an exemplary implementation, the first trailing bits are in the bitstream after the first substream, and the second trailing bits are in the bitstream after the second substream.

[0082] This bitstream structure makes it possible to decode the first substream and the corresponding trailing bits without extracting another substream.

[0083] For example, the first trailing bit is after the second substream in the bitstream.

[0084] Such a bitstream structure makes it possible to start decoding of the individual substreams before extracting the trailing bits from the bitstream.

[0085] In an exemplary implementation, the first size is equal to the second size.

[0086] Using portions of the same size may result in more efficient performance, for example it may be possible to use memory units that are appropriate for the hardware and software implementation.

[0087] For example, every portion contained in the first substream and every portion contained in the second substream are an integer number K of symbols of data of the multiple channels, where K is greater than one.

[0088] Such an approach may provide efficient implementation in software and / or hardware.

[0089] For example, a symbol is a bit.

[0090] In an exemplary implementation, the encoding method further includes discarding remaining bits of the bitstream after extracting the first substream length index, the second substream length index, the first trailing bit length index, the second trailing bit length index, the first substream, the second substream, the first trailing bits, and the second trailing bits.

[0091] Such an approach can provide a bitstream that is appropriately conditioned for further processing, such as encapsulation into network adaptation layer units or other packets.

[0092] In an exemplary implementation, the decoding method further includes determining and applying a shuffling method for demultiplexing the first size portions and the second size portions, the shuffling method being one among a set of predetermined shuffling methods and specifying an order of the first size portions and the second size portions.

[0093] The portions may be shuffled to achieve a more even distribution of substream sizes.

[0094] For example, determining the shuffling method is based on control information contained in the bitstream.

[0095] In order to correctly shuffle the parts at the decoder side, the shuffling method used at the encoder side may be signaled in the bitstream.

[0096] In an exemplary implementation, the entropy decoding includes decoding the first substream with a first entropy decoder and decoding the second substream with a second entropy decoder, and the entropy decoding by the first entropy decoder and the second entropy decoder are performed at least partially in parallel.

[0097] Parallel decoding of the sub-streams may result in faster decoding of the complete bitstream.

[0098] For example, the channels may be output channels or latent representation channels of a neural network.

[0099] Neural networks typically provide channels of the same size or at least a fixed size, which makes the above embodiments and examples particularly suitable and readily applicable to these channels.

[0100] In an exemplary implementation, the entropy decoding is arithmetic decoding, and the method includes steps of decoding the multiplexed portion from the first sub-stream or the second sub-stream, including: extracting an amount of leading encoder status bits from a trailing bit length indicator, where the sub-stream includes coded bits and the trailing bits are leading encoder status bits; determining encoder status bits including appending zeros to the extracted leading encoder status bits up to a predetermined maximum length of the encoder status bits; and arithmetic decoding of the demultiplexed portion from the bits including the coded bits and the determined encoder status bits.

[0101] The reconstruction of the trailing bits from the leading encoder status bits provides for decoding from the coded bits and the trailing bits by using a smaller amount of bits in the bitstream.

[0102] For example, an encoder status bit determination may be constructed by appending to the extracted leading encoder status bit a bit whose value is one, followed by zeros up to a predefined maximum length of the encoder status bits.

[0103] This approach makes it possible to reconstruct the complete output of the arithmetic encoder and therefore provide the appropriate input to the arithmetic decoder.

[0104] In an exemplary implementation, the arithmetic decoding is a range decoding.

[0105] Range encoding may be particularly well suited to hardware and software architectures that have limited register or generally fast memory sizes.

[0106] In an exemplary implementation, a computer program stored on a non-transitory medium includes code instructions that, when executed on one or more processors, cause the one or more processors to perform the steps of any of the methods described above.

[0107] According to an embodiment, an apparatus for entropy encoding multiple channels of data of the same size into a bitstream includes a processing circuit configured to: multiplex portions of a first size from each of the multiple channels and thereafter encode the multiplexed portions of the first size into a first substream; multiplex portions of a second size from each of the multiple channels and thereafter encode the multiplexed portions of the second size into a second substream; the encoding being entropy encoding performed independently on the first substream and the second substream.

[0108] According to an embodiment, an apparatus for entropy decoding a bitstream into multiple channels of same-sized data includes: a processing circuit configured to decode a first substream independently of a second substream, and to demultiplex first-sized portions and second-sized portions from the first substream and the second substream into multiple channels of the same size.

[0109] The apparatus provides the advantages of the method described above.

[0110] The present invention can be implemented in hardware (HW) and / or software (SW) or any combination thereof. Furthermore, HW-based implementations may be combined with SW-based implementations.

[0111] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0112] In the following, embodiments of the invention are explained in more detail with reference to the accompanying drawings and drawings. [Figure 1] FIG. 1 is a schematic diagram illustrating the encoding of a message using an alphabet of two symbols by arithmetic coding. [Diagram 2] FIG. 2 is a schematic diagram of the first step of arithmetic (range) coding of a message using an alphabet of eight symbols. [Diagram 3] FIG. 3 is a schematic diagram of the second step of arithmetic (range) coding of a message using an alphabet of eight symbols. [Figure 4] FIG. 4 is a schematic diagram of the third step of arithmetic (range) coding of a message using an alphabet of eight symbols. [Diagram 5] FIG. 5 is a schematic diagram of the fourth step of arithmetic (range) coding of a message using an alphabet of eight symbols. [Figure 6] FIG. 6 is a schematic diagram of the final step of arithmetic (range) coding of a message using an alphabet of eight symbols. [Figure 7A] FIG. 7A is a schematic diagram of the minimum and maximum values ​​of the ranges during the coding step, expressed in binary numbers. [Figure 7B] FIG. 7B is a schematic diagram of the minimum and maximum values ​​of the ranges during the coding step, expressed in binary numbers. [Figure 7C] FIG. 7C is a schematic diagram of the minimum and maximum values ​​of the ranges during the coding step, expressed in binary numbers. [Figure 7D] FIG. 7D is a schematic diagram of the minimum and maximum values ​​of the ranges during the coding step, expressed in binary numbers. [Figure 8] FIG. 8 is a schematic diagram illustrating an exemplary arithmetic decoding. [Figure 9] FIG. 9 is a flow chart illustrating an exemplary encoding method. [Figure 10] FIG. 10 is a flow chart illustrating an exemplary decoding method. [Figure 11] FIG. 11 is a block diagram illustrating the use of arithmetic coding of some embodiments in picture coding. [Figure 12] FIG. 12 is a block diagram illustrating the use of arithmetic coding of some embodiments in picture decoding. [Figure 13] FIG. 13 is a schematic diagram of a single encoder that encodes channels of input data into a single stream. [Figure 14] FIG. 14 is a schematic diagram of a multi-core encoder that encodes channels of input data into sub-streams and concatenates the sub-streams into a bitstream. [Figure 15] FIG. 15 is a schematic diagram of a multi-core encoder that interleaves and encodes channels of input data into sub-streams and concatenates the sub-streams into a bitstream. [Figure 16] FIG. 16 is a schematic diagram showing the determination of leading and trailing identical bits. [Figure 17] FIG. 17 is a schematic diagram of a bitstream including a length indicator, coded bits, and leading trailing bits. [Figure 18] FIG. 18 is a schematic diagram of the pre-processing to obtain equal sized channels from the input data and the shuffling process to form the interleaved encoded sub-streams. [Figure 19] FIG. 19 is a block diagram illustrating an example of a video coding system configured to implement embodiments of the present invention. [Figure 20] FIG. 20 is a block diagram illustrating another example of a video coding system configured to implement embodiments of the present invention. [Figure 21] FIG. 21 is a block diagram showing an example of an encoding device or a decoding device. [Figure 22] FIG. 22 is a block diagram showing another example of the encoding device or the decoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0113] In the following description, reference is made to the accompanying drawings which form a part of this disclosure and which show by way of illustration certain aspects of embodiments of the invention or in which embodiments of the invention may be used. It is understood that embodiments of the invention may be used in other ways and may include structural or logical changes not shown in the drawings. Therefore, the following detailed description is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0114] For example, it is understood that disclosure related to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, if one or more particular method steps are described, the corresponding device may include one or more units, such as functional units, for performing the described one or more method steps (e.g., one unit performing one or more steps, or multiple units each performing one or more of the multiple steps), even if such one or more units are not explicitly shown or described in the drawings. On the other hand, if a particular apparatus is described based on one or more units, such as functional units, the corresponding method may include one step for performing the function of the one or more units (e.g., one step for performing the function of the one or more units, or multiple steps each performing one or more functions of the multiple units), even if such one or more steps are not explicitly shown or described in the drawings. Furthermore, it is understood that features of various exemplary embodiments and / or aspects described herein may be combined with each other, unless specifically stated otherwise.

[0115] Video coding typically refers to the processing of a video or a series of pictures forming a video sequence. In the field of video coding, instead of the term picture, the terms frame or image may be used synonymously. Video coding includes two parts: video encoding or video decoding. Video encoding is performed at the source side and typically involves processing an original video picture (e.g., by compression) to reduce the amount of data required to represent the video picture (for more efficient storage and / or transmission). Video decoding is performed at the destination side and typically involves the reverse processing compared to the encoder to reconstruct the video picture. The embodiments to the "coding" of a video picture (or pictures in general, as will be explained later) shall be understood to relate to both the "encoding" and the "decoding" of a video picture. The combination of the encoding and decoding parts is also referred to as CODEC (COding and DECoding).

[0116] In the case of lossless video coding, it is possible to reconstruct the original video picture, i.e. the reconstructed video picture has the same quality as the original video picture (assuming there are no transmission errors or other data losses during storage or transmission). In the case of non-lossless video coding, further compression is performed, e.g. by quantization, to reduce the amount of data representing the video picture, and the video picture cannot be perfectly reconstructed at the decoder side, i.e. the quality of the reconstructed video picture is lower or worse than the quality of the original video picture.

[0117] arithmetic encoding Entropy coding is typically used as lossless coding. Arithmetic coding is a type of entropy coding that encodes a message as a binary real number within an interval (range) that represents the message. Here, the term message denotes a sequence of symbols. The symbols are selected from a predefined alphabet of symbols. For example, the alphabet may consist of the two values ​​0 and 1. A message using such an alphabet is then a sequence of bits. The symbols (0 and 1) may occur in a message with different frequencies from each other. In other words, the symbol probabilities may be non-uniform. In fact, the less uniform the distribution, the higher the compression achievable by entropy codes in general and arithmetic codes in particular. Arithmetic coding utilizes a priori known probability models that specify the symbol probabilities for each symbol of the alphabet.

[0118] The alphabet need not be binary. Rather, the alphabet may be composed of, for example, eight values ​​0 through 7. In general, any alphabet having any size may be used. Typically, the alphabet is given by the range of values ​​of the data to be coded.

[0119] The interval representing the message is obtained by dividing the initial range according to the probability of the alphabet symbols on which the message is coded.

[0120] For example, let the current interval start out as the initial interval [0,1). For each symbol in the message, the following two steps are performed:

[0121] 1) Subdivide the current interval into subintervals, one for each possible alphabet symbol. The size of a symbol subinterval is proportional to the estimated probability that that symbol will be the next symbol in the message, according to a probability model (of the symbol source).

[0122] 2) Select the subinterval that corresponds to the next actual occurring symbol in the message, and make the selected subinterval the new current interval.

[0123] As a third step, enough bits are output to distinguish the current interval from all other possible intervals. This step can be performed already during the encoding in steps 1 and 2, or it can be performed after the encoding of the entire message. The length of the interval obtained after repeating steps 1) and 2) for all symbols of the message is obviously equal to the product of the probabilities of the individual symbols, which is also the probability of a particular sequence of symbols in the message.

[0124] In theory, an arithmetic coder would recursively divide the interval from 0 to 1 to code a message of any length, resulting in smaller and smaller intervals. In practice, the system is limited by a finite bit depth, and only discrete values ​​can be represented. Thus, the smaller the interval, the more precise the arithmetic is required. Furthermore, no output can be produced until the entire message has been read. A solution to both of these problems might be to output some bits as soon as they are known, and then double the length of the current interval for each output bit, so that it reflects only the (still) unknown part of the interval. In practice, the arithmetic can be done by storing the current interval in floating point or a sufficiently long integer that is not strictly rational.

[0125] A variant of arithmetic coders that has been improved for practical use is called a range coder, which does not use the interval [0,1) but a finite range of integers, for example from 0 to 255. This range is divided according to the probabilities of the alphabet symbols. If the remaining range is too small, the range can be renormalized in order to describe all alphabet symbols according to their probabilities.

[0126] In terms of the terminology used in this application, the current interval is given by its minimum value (denoted as LOW) and its maximum value (denoted as HIGH). The length of the interval is denoted as RANGE. In general, HIGH=LOW+RANGE, where RANGE is expressed as the number of (distinguishable) subintervals of the smallest size. The minimum range for encoding the symbols is BOTTOM. This minimum range ensures that the least probable symbols still have a valid range of at least 1. In other words, BOTTOM is a design parameter that can be determined based on the alphabet symbols and their probabilities to ensure that the range can still be divided into distinguishable intervals corresponding to all alphabet symbols.

[0127] The HIGH position indicates how many bits are needed to cover the initial maximum range. The BOTTOM position indicates how many bits are needed to cover the minimum range for encoding another symbol. The amount of bits between the HIGH position and a given TOP position corresponds to the smallest portion of bits that can be streamed (inserted) into the bitstream and become a coded bit.

[0128] Fig. 1 shows a schematic diagram of an exemplary procedure of arithmetic coding. The message to be coded is provided in an alphabet with two possible symbols A and B. The symbols of the alphabet {A,B} have probabilities P(A) = 2 / 3 and P(B) = 1 / 3. The message to be coded reads AABA.

[0129] Step 0 initially indicates an initial interval with length Range0110, which corresponds to HIGH, since LOW=0 (low0=0 at step 0). In the lower half of Range0, the leading bit of the binary representation of a number in Range0 is 1 111, and in the upper half of Range0, the leading bit is 0 112. In other words, a current interval in any step that falls in the upper half of the initial range will have its first leading bit 0, and a current interval that falls in the lower half of the initial range will have its first leading bit 1. Now, for example, if we assume that any number (code value) in the initial range can be represented in 8 bits, the range is 0 to 255, and HIGH=255.

[0130] In step 1, Range0 is split according to probability to code the first symbol A0. In this example, the initial interval is split into two intervals with sizes 1 / 3 122 (corresponding to symbol B0) and 2 / 3 121 (corresponding to symbol A0) of the total range size. Splitting according to probability means that the initial interval is split into a number of subintervals equal to the number of symbols in the alphabet (here, two symbols A and B), with the size of the intervals proportional to the respective probability of the symbol represented by the interval. Then, the upper subinterval Range1=P(A0)*Range0, corresponding to the message symbol A0, is selected as the current interval for the next step.

[0131] In step 2, the range of symbol A0, Range1 120, is split according to probability. The next message symbol is A. The remaining Range2=P(A1)*Range1, which describes message A0A1 131, is entirely within the upper half of Range0, which is encoded with bit 0, so adding bit 0 to the encoded bitstream is performed. In this particular exemplary implementation, bits are added to the bitstream as soon as possible, and a renormalization 140 is performed to double the resolution. The current maximum range is now the upper half of the initial range of step 0. The upper half of this current maximum range is assigned to bit 0, and the lower half of the current maximum range is assigned to bit 1.

[0132] The message A0A1B2141 in step 3 cannot be coded unambiguously because Range3 overlaps with bit 1 (the corresponding lower half of the current maximum range) and bit 0 (the corresponding upper half of the current maximum range), so a carry 150 is performed. Nothing is streamed (no bits are included in the bitstream). Range3 is divided in step 4 according to the probabilities of the symbols in the alphabet.

[0133] Here, the message to be encoded is A0A1B2A3151. Since Range4 still overlaps with both possible bits and there are no further symbols to be encoded, final processing 160 is performed in steps 5 and 6. This involves some renormalization being performed to create an unambiguous code 0011 for the message AABA.

[0134] 2 to 7 show an example of a range encoder, where the 8 symbols of the alphabet 220 {0,1,2,3,4,5,6,7} have probabilities according to the probability density function PDF 230 of the standard normal distribution. The symbols are mapped to ranges by a cumulative distribution function CDF 240. The message to be encoded reads 4420. The maximum starting range HIGH for encoding is represented by 8 bits, which in this example corresponds to a code value of 255. The minimum range for encoding the symbols is BOTTOM=16 in this example. This minimum range ensures that even the least probable symbols "0" and "7" still have a valid range of at least 1 when the cumulative distribution function is applied. For example, the normal distribution leads to a minimum interval of the following numbers for the 8 symbols: 1,1,2,3,4,2,2,1.

[0135] Fig. 2 shows that an initial range 210 Range0=HIGH=255=11111111.b (where .b is the binary representation of a number) is mapped to the symbols of the alphabet 220 according to a Gaussian probability density function PDF 230. The partitioning of Range0 is obtained from the cumulative distribution function CDF 240. This means that the first symbol of the message to be encoded, i.e. "4", is represented by any of the code values ​​within the interval with lower endpoint 250 Low1=71=01000111.b and excluded upper endpoint 260 Low1+Range1=199=11000111.b, which corresponds to Range1=128. total=const=16 270 means an operating precision of 1 / 16. This indicates that the subrange assigned to the least likely symbol is 1 / total=1 / 16 of the current range. total 270 is determined by the cumulative distribution function.

[0136] The binary representation of this interval is shown in FIG. 7A with the indications of HIGH, TOP, and BOTTOM positions. The HIGH position 740 indicates how many bits are required to cover the initial maximum range (in this case 8). The BOTTOM position 760 indicates how many bits are required to cover the minimum range for encoding another symbol (in this case 4). The amount of bits between the HIGH position 740 and a given TOP position 750 corresponds to the smallest portion of bits that can be streamed (inserted) into the bitstream and then become the coded bits. In the case of the arithmetic coding described above in relation to FIG. 1, there is only one bit between the HIGH position 740 and the TOP position 750. In the case of the range coding of this example, there are two bits between the HIGH position 740 and the TOP position 750. In FIG. 7A (as well as in FIGS. 7B-7D), the minimum value of the intervals 720a-d and the maximum value of the intervals 710a-d are represented in binary. This can actually correspond to two registers used for the purpose of storing the current interval and therefore the current result of the encoding.

[0137] In Figure 3, a cumulative distribution function CDF 340 is applied to Range1 = 128 starting from Low1 = 71 to map the symbols of the alphabet 220 into ranges. The next symbol in the message is "4". This results in a new current interval 350-360 representing the message 44. This interval, shown in Figure 7B, has a new lower limit (current minimum) 720b Low2 = 106 = 01101010.b and a new upper limit (current maximum) 710b Low2 + Range2 = 170 = 10101010.b. The new Range2 320 is equal to 64.

[0138] Figures 4 and 7C show the next step in the encoding procedure. As Range2 320 is still greater than BOTTOM, this range is again divided according to the cumulative distribution function 440. This results in a low value 720c of Low3=111=01101111.b and Range3=4 410 for symbol 2. Message 442 is therefore represented by the range from Low3 to Low3+Range3=115=01110011.b 710c.

[0139] Since the two bits 730 between the HIGH and TOP positions are equal, they can be output as coded bits in the stream 731. See Figure 7D.

[0140] Figure 5 shows that a proper mapping onto the range of symbols is not possible since Range3=4 410 is smaller than BOTTOM=16 and therefore a renormalization procedure is necessary. Figure 7D shows that all bit representations are shifted left until the new Range4 is greater than or equal to BOTTOM. Thus we arrive at new Range4=(4<<2)=16 610. The new upper limit 710e is obtained after two left shifts from the current one 710d. The same shift is applied to Low3, except for the two coded bits, which are already part of the stream Low4=188=10111100.b 720d.

[0141] The message 442 is now encoded by any value between 444=0110111100.b and 460=0111001100.b.

[0142] Figure 6 shows the coding of the last symbol “0”. The probability distribution results in the lower value of the interval, Low5=188=10111100.b 650 and Range5=1 620. Since there are no further symbols to be coded, Low5 and Range5 describe the range interval of the trailing bits that represent (together with the bits being coded) the coded message. In general, any value from the interval can be used (in the bit stream) to represent the interval and therefore the coded message (sequence of symbols). Thus, the trailing bits can be chosen arbitrarily from this final range interval. In this example, Range5=1 620 results in a single value for the trailing bits, namely Low5=188=10111100.b 650. Thus, message 4420 is coded by appending trailing bits to the bits to be coded, resulting in coded value 444=0110111100.b.

[0143] Arithmetic Decoding Figure 8 shows an exemplary decoding process. The decoder receives the coded values ​​(bits) sequentially. The received coded values ​​820 are within the complete Range0=HIGH=255 810. The probability distribution function is known at the decoder side, so the mapping of symbols 220 to the range is done by a cumulative distribution function 830. Since the decoder does not know the inverse of this mapping, the decision of the symbol requires a search process. The decoder makes a first guess 840 on the coded symbol by selecting the most likely symbol "4", calculates the low value 841 of the range corresponding to this symbol, and checks whether the received coded value 820 is higher than this low value 841.

[0144] Because the received coded value 820 is less than the low value 841 of the first guess 840, the next guess 850 is a symbol that maps to a lower value in the range, i.e., one of the symbols "0", "1", "2" or "3". Selecting a low value that is approximately in the middle of the remaining interval of symbols leads to a faster decoding process since fewer steps are required to obtain the correct symbol. In this example, the next low value 851 selected for comparison corresponds to the symbol "2".

[0145] The test for the symbol "2" results in the received coded value being higher than the low value 851 of the range which encodes the symbol "2". Thus, the received coded value can represent the symbols "2" or "3". A final check 860 reveals that the low value 861 of the range which corresponds to "3" is higher than the coded value 820. Thus, the received coded value 820 is decoded as the symbol "2".

[0146] Multi-Channel Entropy Coding FIG. 13 illustrates a scheme for a single encoder 1320. Here, the term "single encoder" indicates that the encoder operates serially, i.e., encodes the input sequentially rather than in parallel. The input data may consist of or include multiple channels 1310. Thus, in this exemplary encoding process, portions of each channel are encoded sequentially. A first portion-sized portion from each of the channels 1330 is encoded (resulting in a multiplexing of portions of the same first size from different channels), followed by a second portion-sized portion from each of the channels 1340, resulting in a multiplexing of portions of the same second size from different channels). Trailing bits 1350 remaining in the encoder after encoding the last portion are appended back to the main single stream.

[0147] In a single (or single-core) entropy encoder, there is only one finalization step at the end of the coding, where the stream is padded with 0 bits to align the bytes. There is no problem in signaling a few extra bits. However, it is difficult to parallelize such encoding and accordingly the decoding.

[0148] 14 shows an exemplary scheme of a parallel (e.g., multi-core) encoder 1420. Each of the input data channels 1410 can be encoded into an individual sub-stream that includes coded bits 1430-1433 and trailing bits 1440-1443. The length of the sub-stream 1450 is signaled.

[0149] In a parallel processing implementation, the bitstream consists of several substreams that are concatenated in a final step. Each of the substreams needs to be finalized. This is due to the fact that the substreams are encoded independently of each other, so that the encoding (and therefore the decoding) of one substream does not require the previous encoding (or decoding) of one or more other substreams.

[0150] Entropy coding, especially arithmetic coding finalization, may involve encoding one or more trailing bits into the bitstream and / or padding to the nearest byte boundary or a predetermined amount of bits boundary. However, when multiple substreams are coded in parallel, the padding may result in a huge amount of meaningless padding bits. This problem may be solved if the number of trailing bits in each thread is not rounded to the nearest byte boundary, but rather a significant amount of leading bits are determined among the trailing bits and their amount is specified in the bitstream.

[0151] This is exemplarily shown in Figure 15, where trailing bits 1540-1543 are added to the bitstream 1570 immediately following the coded data 1530-1533. The length 1560 of the trailing bits is also included in the bitstream 1570. The multi-core encoder in this exemplary embodiment multiplexes and possibly shuffles portions of several input data channels 1510 before encoding them into multiple sub-streams.

[0152] Note that the complete partial multiplex to generate the substreams does not have to be formed before encoding the substreams. Instead, the entropy encoder can receive the parts directly - part by part - from the different channels and process them into substreams. The term shuffling refers to the sequence of parts within a multiplex (and thus within a substream after encoding the multiplex).

[0153] An input data channel may refer to a channel obtained by processing some data with a neural network. For example, the input data may be a feature channel, such as an output channel or a latent representation channel, of a neural network. In an exemplary implementation, the neural network may be a deep neural network and / or a convolutional neural network. The neural network may be trained to process pictures (still or video). The processing may be for picture encoding and reconstruction, or for computer vision such as object recognition, classification, segmentation, etc. In general, the present disclosure is not limited to any particular type of task or neural network. The present disclosure is readily applicable to encoding and decoding channels of neural networks, but is not limited to such applications. Rather, the present disclosure is applicable to encoding any type of data coming from multiple channels, which should be generally understood as any source of data. Furthermore, the channels may be provided by pre-processing of the source data.

[0154] FIG. 18 exemplarily illustrates pre-processing of input data channels 1810, denoted by Ch0, Ch1, Ch2, and Ch3, with different sizes, along with their corresponding probability distributions 1820, denoted as ProbCh0, ProbCh1, ProbCh2, and ProbCh3. The term size in this case indicates the number (amount) of bits, symbols, or elements of the channel. In general, channels such as neural network channels may have more dimensions, such as vertical and horizontal, but these typically do not play a role in entropy coding, which is performed serially on the channel elements. Depending on the entropy coding and the channel type, the coding may be based on channel bits, channel symbols, or channel elements in general (e.g., a probability model may be provided). However, the present disclosure may also be applicable to split a channel into new channels of the same size in more than one dimension.

[0155] These input channels 1810 of different sizes are pre-processed to obtain channels of the same size 1830. Input channels that are larger than the required size may therefore be split (1812). As can be seen in Fig. 18, for example, at least one of the channels (here two channels Ch1 and Ch2) is split into two channels (for example ch0a and ch0b, and ch2a and ch2b). The splitting may be done in any conceivable way, for example, the channel is split into n pieces (n being an integer larger than the number of consecutive pieces). Alternatively, the pieces are not consecutive, but are formed by repeatedly assigning each symbol or each k symbols from the channel to one or a new channel (interleaving the channel pieces into the new channel).

[0156] The corresponding probability distributions of the split channels Ch0 and Ch2 are adapted to the new channels ch0a, ch0b, ch2a, and ch2b of the same size. In other words, the new channels (e.g., ch0a and ch0b) may have distributions that are different from the distribution of the original channel (e.g., Ch0) from which they were derived.

[0157] If a channel (e.g., Ch3) is smaller than said same size, after splitting it may be padded with zeros to result in a new channel of the same size 1813. Alternatively, padding may be performed after splitting: for example, if the last part of a split channel is smaller than the same size, it may be padded with zeros as shown in Figure 18. However, padding does not have to be done only in the last new channel, but can be inserted in more new channels, e.g. distributed among the new channels.

[0158] For example, a channel (such as Ch0 and Ch2) may be split into n new channels, but may have a size that is not divisible by n (e.g., not divisible by 2 in the case of FIG. 18). In such a case, it may not be possible to form n channels of the same size with only the data from the channel (Ch0 or Ch2). To overcome this problem, the channel or one or more of the n new channels may be padded. Note that in some example implementations, there may be additional reasons for padding. For example, data from a channel (Ch0 or Ch2) may be split into new channels (ch0a, ch0b, ch2a, and ch2b) not on a bit basis, but on a symbol basis, such as a byte or symbol or other size. Thus, the channel size in symbols, not in bits, needs to be divisible by n.

[0159] Padding with zeros is just one exemplary option. Padding may be performed with bits or symbols of any value. It may also be padded with repetitions of channel bits or symbols, etc.

[0160] At the decoder side, new channels are to be decoded. To form channels of different sizes (e.g. Ch0 and Ch2), the padding shall be removed at the decoder side. To achieve this, the decoder needs information about these pre-processing steps. For example, the decoder is configured to remove the padding based on knowledge of the sizes of the different sized channels. For example, the channel sizes may be defined by a standard or may be set by side information etc. The information about the pre-processing steps may also include the size of the new channels.

[0161] Following pre-processing, each of the channels is divided into portions of the same size. In a first step, portions of a first size 1840 are taken from each channel and these portions (possibly with corresponding distributions) are multiplexed. This multiplexing is entropy coded into a first sub-stream 1530. Furthermore, portions of a second size 1841 from each channel are multiplexed and then coded into a second sub-stream 1531. The first size and the second size may be the same or different. Entropy coding is performed separately to generate the first sub-stream 1530 and to generate the second sub-stream 1531. As mentioned above, the channel portions may be multiplexed with respective probability model side information. In an exemplary embodiment, such side information may correspond to a hyper-prior obtained by a variational auto-encoder with a hyper-prior sub-network. However, there may be embodiments and implementations in which the probability model does not need to be provided as side information multiplexed with the channel portions. For example, the probability model may be updated in a context-adaptive manner, based on previously encoded and / or decoded data, etc.

[0162] Multiplexing and encoding portions from different channels, instead of encoding each channel separately, offers the possibility to generate the completed sub-stream faster and / or to control the length of the individual sub-streams, for example to obtain sub-streams of more uniform length, which opens the possibility to perform entropy encoding and / or entropy decoding on multiple sub-streams in parallel.

[0163] The present disclosure is not limited to providing only portions of the same size or portions of a first size and a second size different from the first size. Furthermore, portions of other sizes, such as a third size 1842 and / or a fourth size 1843, may be included in the first sub-stream or the second sub-stream. This third size may be equal to the first size or the second size. This is illustrated in FIG. 18, where the i-th channel ch i is split into 4 portions: portion.i0, portion.i1, portion.i2, and portion.i3. It is possible to control the substream size by splitting all channels into equal sized portions and then forming substreams containing portions from different channels. This may be called dynamic partitioning.

[0164] For example, an individual substream can have its length extended by including more parts and still be adjusted to the desired length. Even if the multiplexing of parts to form the first substream and the second substream can have the same size, after entropy coding the first substream and the second substream can have different sizes. It may therefore be desirable to adapt the substream sizes to reduce their variance, for example by setting the size and / or number of parts included in the multiplex. For example, if there are fewer individual substreams that are longer, a shorter length indicator needs to be signaled or less padding is required.

[0165] The entropy coding may be arithmetic coding or range coding. In these cases, the coding results in coded bits 1530-1533 and trailing bits 1540-1543 (if present). These trailing bits are the state of the encoder after coding the last part. Note that the present disclosure is not limited to embedding the coded bits and the trailing bits separately in the bitstream. There may be encoders that output all bits as coded bits in the bitstream. The present substream based on independent coding and decoding is equally applicable to such encoders and decoders.

[0166] The substreams are multiplexed into a bitstream 1570 with substream length indicators 1550 indicating the length of each substream.

[0167] In an embodiment (as shown in FIG. 15), a substream length indicator 1550 precedes each substream in the bitstream 1570. Additionally, a second substream length indicator may precede the first substream 1530, i.e., multiple substream length indicators are concatenated and included in the bitstream 1570 before the multiple substreams. The multiple substreams may relate to some or all of the substreams generated from a channel, for example, one picture or picture portion or another container of a predefined number of pictures or channel data. The number of substreams included in the multiple may be set or fixed to a predefined number that may, but need not, correspond to the number of channels of the same size.

[0168] Furthermore, the trailing bits may be signaled with the length indicator. There are trailing bits for each substream, i.e., a first trailing bit and a second trailing bit. Thus, a first trailing bit length indicator and a second trailing bit length indicator may be included in the bitstream 1570. The trailing bit length indicator 1560 may precede their respective substreams in the bitstream 1570. The second trailing bit length indicator may also precede the first substream 1530. Thus, the concatenated trailing bit length indicator 1560 precedes the first substream 1530. The order of the substream length indicators and the trailing bit length indicators should be predetermined so that both the encoder and the decoder can form and analyze the bitstream amenably.

[0169] Including all these length indicators in the bitstream avoids padding of individual substreams and / or trailing bits. Notably, in some implementations, padding may be performed after multiple substreams and corresponding indicators have been included in the bitstream. However, this disclosure does not require padding because there may be bitstream structures or protocols that do not require such alignment to a particular raster of bits or symbols.

[0170] Instead of concentrating the indicators for the multiple substreams, in one embodiment, the trailing bits 1540 of the first substream may be included in the bitstream after the first substream 1530, and the trailing bits 1541 of the second substream may be after the second substream 1531 in the bitstream 1570. An advantage of this bitstream structure may be the possibility of immediate encoding or decoding of the substreams without the need to buffer the multiple substreams and their respective indicators. Furthermore, the first trailing bits 1540 may be after the second substream 1531.

[0171] A bitstream 1570 constructed as in any of the above example embodiments may be padded to adjust the length of the bitstream to match an integer multiple of a predetermined amount of bits, such as a byte, word, or doubleword, etc. Such an approach can provide a bitstream that is appropriately sized for further processing, such as encapsulation into a network adaptation layer unit or other packet.

[0172] The multiple channel data 1510 consists of symbols, e.g. bits. Every portion in the first substream 1530 and every portion in the second substream 1531 may be an integer K times these symbols, where K is greater than 1.

[0173] To achieve a more uniform distribution of substream sizes, the portions may be shuffled. Shuffling has already been mentioned in connection with FIG. 18. This corresponds to an interleaving of portions performed synchronously at the encoding and decoding sides, i.e. using the same rules at the encoding and decoding sides to ensure compatibility between encoding and decoding. A shuffling method is selected from a set of predefined methods and then applied to specify the order of the portions of the first size and the portions of the second size. The portions 1860 (e.g., portion 0a0, portion 0b0, portion 1a0, portion 2a0, portion 2b0 and portion 3a0) are shuffled synchronously. The exemplary scheme of FIG. 18 shows a cyclic permutation 1850 of the portions. Furthermore, the corresponding probability distributions 1861 associated with each portion (e.g., Prob prt.0a0, Prob prt.0b0, Prob prt.1a0, Prob prt.2a0, Prob prt.2b0, and Prob prt.3a0) are subjected to entropy coding (e.g., arithmetic coding). Thus, the portions and associated probability portions (models) can be understood as being synchronously shuffled.

[0174] Shuffling (and possibly shuffling method selection) may be performed iteratively over the portions to be coded to reduce the length difference between the coded first substream and the coded second substream, typically between substreams over time. Thus, the shuffling method may take into account the difference between the length of the current (e.g., first or second) substream and a statistic based on the lengths of past substreams and / or the difference between the length of the current substream and a statistic based on the lengths of past substreams. Such a shuffling method may result in any shuffling 1852 of the portions, i.e. any possible permutation.

[0175] This may involve collecting statistical measures of the past substreams to obtain a distribution of the lengths of the past substreams, which can be obtained and used simultaneously at the encoder and decoder side, since both sides are pre-processing the same substreams.

[0176] The statistics may be based, for example, on an estimated mean, median, minimum or maximum of past and / or current sub-stream lengths, or the rate of growth of the sub-stream lengths, or the rate of decrease of the sub-stream lengths, or a combination thereof, or another statistical measure (estimator).

[0177] If the entropy coding is arithmetic coding, the shuffling method may also take into account the current state of the range intervals in the arithmetic coding process.

[0178] The current state of the range interval can also provide detailed information of the substream length and can be taken into account for the estimation of the growth rate of the substream based on a particular channel. In particular, if the current state of the range interval is small, it indicates that the substream length is larger and vice versa. Based on such growth estimation, it is possible to select an appropriate shuffling method.

[0179] As mentioned above, proper shuffling can provide several advantages, as shown in the following example. Now, assume that the shuffling may be a circular rotation of the channels. In this example, assume that we have three channels Ch1, Ch2, and Ch3, each with a different growth rate of substream size (also called length in this case): the substreams grow by 10 bytes for the part from Ch1, by 2 bytes for the part from Ch2, and by 20 bytes for the part from Ch3. Thus, after encoding the first three parts in parallel into substreams S1, S2, and S3, without shuffling, the following stream lengths are achieved:

[0180] - The first part from each channel is encoded without shuffling, in the order Ch1, Ch2, and Ch3. This results in three parallel stream sizes of 10, 2, and 20 bytes for S1, S2, and S3, respectively.

[0181] - If the second parts are encoded without shuffling in the same order of Ch1, Ch2, and Ch3, parallel stream sizes of 20, 4, and 40 bytes are achieved after including the second parts from each of the three channels.

[0182] - If the third part is encoded without shuffling in the same order of Ch1, Ch2, and Ch3, we obtain parallel stream sizes of 30, 6, and 60 after including the third part from each of the three channels.

[0183] As can be seen in this example, the parallel substreams are of significantly different lengths. This may be undesirable for some applications. To remedy this situation, shuffling may be performed. In particular, the order in which the portions are taken from each channel may be changed.

[0184] It may be assumed that the same channels Ch1, Ch2, and Ch3 mentioned above have the same growth rate. If a shuffling by circular shift of the channel order Ch1, Ch2, Ch3 is performed, the following results are obtained:

[0185] - Here the first part is taken from the three channels in the same order as in the example above, i.e. the order is Ch1, Ch2, and Ch3, which results in the same parallel stream sizes of 10, 2, and 20 bytes.

[0186] - The second part is taken in a shuffled order, specifically in a circularly shifted order (shifting right by one place): Ch3, Ch1, and Ch2. This results in stream sizes of 30, 12, and 22 bytes, respectively. These stream sizes are obtained by adding the sizes 20, 10, and 2, corresponding to the shuffled channels, to the streams of lengths 10, 2, and 20 bytes from the previous step. As can be seen, after the second part, the sizes 30, 12, and 22 after shuffling show a lower variance than the sizes 20, 4, and 40 bytes from the previous example without shuffling.

[0187] - The third part is retrieved, again in shuffled order, by a repeated cyclic shift to the right in this example, resulting in the order of channels Ch2, Ch3, and Ch1. The resulting stream sizes of the three parallel streams are 32, 32, and 32. This size comes from adding 2, 20, and 10 bytes, respectively, to the stream sizes of 30, 12, and 22 bytes from the previous step (adding the second part). As can be seen, after the second shuffling, the lengths of the parallel streams (corresponding to the sub-streams above) are equalized.

[0188] In real applications, the growth may not be so easily and accurately estimable. In particular, it is not necessarily static as in the above example. Nevertheless, shuffling may improve the equalization of the substream lengths (sizes). To do so, an estimation of the growth rate may contribute to the performance. As mentioned above, the growth of the (encoded) substream may be estimated based on previously coded (decoded) parts or substreams. However, the current state of the range interval may provide a closer indication. A large range interval indicates a shorter substream length and a smaller contribution to the growth rate. A small range interval indicates a larger substream length, which corresponds to a larger contribution to the growth rate. In other words, the length of the range interval is inversely proportional to the growth rate of the stream. The proportionality is not necessarily linear.

[0189] Thus, during encoding and decoding, a shuffling mechanism may be applied. The shuffling may be similar to that described above, for example, the kth portion from each channel is encoded (or decoded) in the kth order of the channels, followed by the (k+1)th portion from each channel being encoded (or decoded) in the (k-1)th order of the channels. In an exemplary implementation, the (k+1)th order is obtained from the kth order by circularly shifting the kth order. The circular shift may be right or left. It may be advantageous if the shift is by one channel. However, the disclosure is not limited thereto, and the shift step may be different from 1. As already described above, the shuffling order may also be specifically selected and signaled.

[0190] In another exemplary embodiment, the portions of a channel that are encoded into a substream with a longer length and a faster growth rate may be exchanged, i.e. shuffled, with the portions of a channel that are encoded into a substream with a shorter length and a slower growth rate. In particular, the method may determine the length of the substream and the growth rate of the substream's length. Based on the result of this determination, the method shuffles the portions. For the above example, this corresponds to an exchange of channels Ch2 and Ch3. The exchange encodes the portion from Ch3 that contributes to a larger growth rate (20 bytes per portion) into a substream S2 of shorter length. The portion from Ch2 that contributes to a smaller growth rate (5 bytes per portion) into a substream S3 of larger length. This results in an increased growth of S2 and a decreased growth of S3. Thus, the above-mentioned length difference between the encoded substreams may be reduced. In a real application, the growth may not necessarily be static as in the above example.

[0191] The entropy encoding into the first sub-stream 1530 and the second sub-stream 1531 may be performed in parallel, for example by one of the multi-core encoders 1420, 1520 of Figures 14 and 15. In some implementations, only a portion of the entropy encoding may be performed in parallel.

[0192] The decoding method includes entropy decoding the first substream 1530 and the second substream 1531 into multiplexed parts. The entropy decoding is performed separately for the first and second substreams. The multiple channels 1810 are obtained from demultiplexing the first size part 1840 and the second size part 1841. The first size and the second size may be the same. The same size channels 1620 may be post-processed to obtain multiple channels 1810 of different sizes. This post-processing includes adding (concatenating) the split channels. The channels that are padded with zeros are clipped to obtain the input data, i.e. the padding is removed. Information about these steps is included in the same size channels. The channels of any size may be output channels or latent representation channels of a neural network.

[0193] The entropy decoding may be, for example, arithmetic decoding or range decoding, as described in the arithmetic coding section above. However, the present disclosure is not limited to the application of arithmetic encoders and decoders with respect to the formation of the substreams. Rather, any entropy coding and / or run-length coding, etc. may be applied to the channel data. The entropy coding may be context-adaptive. These embodiments correspond to the coding examples described above.

[0194] The substreams are extracted from the bitstream 1570 with a first substream length indicator indicating the length of the first substream and a second substream length indicator indicating the length of the second substream. For example, the substreams can be extracted where the length indicators of the substreams are known prior to extraction.

[0195] Thus, the sub-stream length indicators 1550 may precede their respective sub-streams in the bitstream 1570. Additionally, the second sub-stream length indicator may also precede the first sub-stream 1530.

[0196] By signaling the concatenated length indicators before the substreams, the decoder can extract the substreams simultaneously.

[0197] A first trailing bit length indicator indicating the length of the first trailing bits and a second trailing bit length indicator indicating the length of the second trailing bits may be extracted from the bitstream 1570. The trailing bit length indicator 1560 may precede their respective substreams in the bitstream 1570. The second trailing bit length indicator may also precede the first substream 1530.

[0198] The trailing bits of the first substream 1540 may be included in the bitstream after the first substream 1530, and the trailing bits of the second substream 1541 may follow the second substream 1531 in the bitstream 1570. Additionally, the first trailing bits 1540 may follow the second substream 1531.

[0199] The data in the multiple channels 1510 consists of symbols, which may be bits. Every portion decoded and demultiplexed from the first substream 1530 and every portion decoded and demultiplexed from the second substream 1531 may be an integer number K times these symbols, where K is greater than 1.

[0200] There may be remaining bits after extraction of the first substream length indicator, the second substream length indicator, the first trailing bit length indicator, the second trailing bit length indicator, the first substream 1530, the second substream 1531, the first trailing bits 1541, and the second trailing bits 1541. The bits remaining in the bitstream 1570 after extraction of the last trailing bit are the result of padding the entire bitstream 1570 to match an integer multiple of a predetermined number of bits to provide a bitstream that is properly aligned for further processing, such as encapsulation into a network adaptation layer unit or other packet. The remaining bits may be discarded.

[0201] During the multiplexing of parts in the encoding process, a shuffling method may have been applied. For example, this method is signaled in the control information included in the bitstream. Thus, the decoder can parse the shuffling method indicator from the bitstream. The indicator may for example be an index to a list of shuffling methods that may be defined by a standard or configurable according to a standard, and possibly signaled in the bitstream less frequently than the shuffling method indicator itself.

[0202] In this manner, a shuffling method can be determined from a set of predefined shuffling methods and applied to demultiplex the first size portions 1840 and the second size portions 1841. The shuffling method specifies the order of the portions within the substream. Furthermore, the shuffling method can also define the portion lengths of the substreams in example implementations where the portion lengths may vary. In some embodiments, the portion lengths may be the same across channels and / or across substreams.

[0203] The entropy decoding may be performed in parallel, for example by a multi-core decoder, or further, only a portion of the entropy decoding may be performed in parallel.

[0204] Handling Trailing Bits 9 is a flow chart illustrating an exemplary method for arithmetically encoding input data into a bitstream 1750. An example of the encoded bitstream 1750 is provided in FIG.

[0205] The method may initialize an initial range S910 used by the encoder. For example, such an initial range may correspond to the initial range 210 of FIG. 2 as described above. The encoding starts with the first symbol of the message to be encoded and proceeds through all symbols of the message to obtain coded bits.

[0206] In step S920, the current symbol from the message is coded using an arithmetic code, for example as described in connection with FIGS.

[0207] After the coding loop S920-S950, the coded bits are included in the bitstream 1570. However, there are still trailing bits remaining in the register, which indicate the state of the encoder. The interval describing the trailing bits is the current range remaining after encoding the last symbol and streaming the coded bits 1730-1731. The trailing bits, which together with the coded bits 1730-1731 form the arithmetically coded data, can generally be chosen arbitrarily from this interval.

[0208] However, the trailing bits may be selected to maximize the amount of trailing zeros in the leading trailing bits, which may not be included in the bitstream 1750 and therefore may be clipped.

[0209] The determined minimum and maximum values ​​S960 of the interval may include the amount of most significant bits MSB that are identical. These identical leading bits are consecutive trailing bits, i.e., they form a set of consecutive trailing bits. The amount of these leading trailing bits 1740-1741 may be determined S970.

[0210] An indication of the determined amount of these identical leading bits and leading trailing bits may be included in the bit stream S980.

[0211] NumTrailingBits is the amount of leading trailing bits. NumTrailingBits=CLZ((LOW+RANGE-1) XOR LOW) where CLZ() is the count of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval. XOR denotes the exclusive logical OR operation.

[0212] An example for determining the identical leading bits 1740-1741 and their quantity is shown in Figure 16. After encoding the last symbol and including the coded bits 1730-1731 in the bitstream 1750, there are 16 trailing bits remaining in the encoder. The trailing bits are represented by the current minimum 1610 Low=1123 and the range of the current interval 1620, Range=67.

[0213] In Figure 16, these values ​​are given in binary representation: Current Maximum 1630 High-1 = (Low + Range-1) = (1123 + 67-1) is determined from Current Minimum 1610 and Current Range 1620. (HIGH-1) XOR Low 1640 results in a zero bit where the bits in LOW 1610 and HIGH-1 1630 are identical, and a one bit elsewhere.

[0214] The leading zeros in this value 1640 represent the same leading bit in the trailing bits. Therefore, the count of leading zeros, CLZ(), is CLZ((low+range-1) XOR low) = 8.

[0215] In this example, there are 8 identical leading bits. The trailing bits in the current interval, 1650, are selected as the value between Low and Hight-1 by zeroing out all bits after the first bit that differs in Low and High-1, thus arriving at 0b.0000.0100.1000.0000 1650.

[0216] As mentioned above, the trailing zeros 1680 can be clipped along with bit 1 1670, which is the first bit that differs between Low and High-1, since this bit is always 1 and does not need to be signaled. Thus, the leading trailing bits 1660 in this example are formed by the 8 bits 00000100.

[0217] The determined number of leading zeros is included in an indicator of the amount of leading bits, which precedes 1720 the coded bits 1730-1731 and the leading trailing bits 1740-1741 in the bitstream 1750.

[0218] However, leading trailing bits 1740-1741 may also be included in bitstream 1570 with an additional bit 1670 whose value is 1 immediately following leading trailing bits 1660. This additional bit is the first bit that differs between low and high-1. The indication of the amount of leading trailing bits 1720 includes the additional bit 1 in this case.

[0219] The method for arithmetic coding may be performed separately for the multiple substreams. An exemplary embodiment is given in Fig. 14. The method described above here is exemplarily applied to the first and second substreams, resulting in the first coded bits 1430a, the second coded bits 1431, the first leading trailing bits 1440a and the second leading trailing bits 1441. A bitstream 1750 is formed from these parts by inserting the first leading trailing bits 1740 and the second leading trailing bits 1741 following the first coded bits 1730 and the second coded bits 1731 into the bitstream 1750 as shown diagrammatically in Fig. 17. The first leading trailing bits 1740 immediately follow the second coded 1731 bits, and no padding is required.

[0220] The bitstream 1750 formed from the first and second substreams may also include length indicators for the first coded bits 1710 and the second coded bits 1711. These first and second length indicators 1710-1711 may precede their respective coded bits 1730-1731 in the bitstream 1750.

[0221] Additionally, a second length indicator 1711 may precede the first coded bit 1730. This is exemplarily shown in Figure 17. The length indicators 1710-1711 are concatenated for each of the coded bits 1730-1731 and included in the bitstream before the coded bits 1730-1731.

[0222] An indicator of the amount of second leading trailing bits 1721 may precede the first coded bit 1730 in the bitstream 1750. The example of Figure 17 includes concatenated indicators 1720-1721 immediately followed by the first coded bit 1730-1731.

[0223] The bitstreams 1750 that have been combined together as in any of the above example embodiments may be padded to align the bitstream length to match an integer multiple of a predefined amount of bytes, which may be a word or doubleword, for example.

[0224] The arithmetic coding of the first and second sub-streams may be performed in parallel, for example by one of the multi-core encoders 1420, 1520 in Figures 14 and 15. In some implementations, only a portion of the arithmetic coding may be performed in parallel.

[0225] As mentioned above, the method of arithmetic coding may be implemented as range coding, which may have a predefined total range; the predefined size of the memory is equal to or greater than the number of bits to represent the total range (log_2 of the total range).

[0226] The memory can hold the minimum and maximum values ​​of the current range interval. For example, such a minimum value in the finite register can correspond to binary representation 720c in Fig. 7A-D, and such a maximum value in the finite register can have binary representation 710c. After one or more iterations of the encoding loop, there can be fixed bits that are consecutive bits having the same value in the MSB of the binary representation of the current minimum value and the current maximum value. A predetermined amount of these fixed bits are moved from the memory to the bit stream. This corresponds, for example, to the 2 bits between the HIGH and TOP positions in Fig. 7A-D. In an exemplary implementation, there can be 16 bits between the HIGH and TOP positions. The invention is not limited to any of these examples.

[0227] It may happen that the difference between the amount of leading trailing bits and a predefined amount of bits from the fixed bits between the HIGH and TOP positions is less than a predefined threshold, for example there may be 16 bits between the HIGH and TOP positions and 15 leading trailing bits. In this case it is more efficient to include the leading trailing bits in the coded bits. Thus the trailing coded bits are generated from the leading trailing bits. Bit 1 1670 has to be added to the leading trailing bits 1660 and the bits may be padded with zeros to reach the predefined number of fixed bits. Thus the expensive signaling of many leading trailing bits together with an indication of the amount of trailing bits is avoided. For example at least 4 bits are needed to signal an amount of 15 leading trailing bits. Instead it may be cheaper to have more coded bits and signal zero leading trailing bits. They can be coded efficiently, for example if they occur frequently.

[0228] The above-mentioned predetermined threshold may be determined empirically, for example taking into account the amount of bits between the HIGH and TOP positions and the amount of signaling used to indicate the length of the leading trailing bits.

[0229] FIG. 10 is a flow chart illustrating an exemplary method for arithmetic decoding of data from a bitstream 1750. A decoder receives a bitstream S1010 and may extract from the bitstream an indication of the length of the coded bits S1020. Coded bits 1730-1731 are extracted from the bitstream and decoded consecutively S1040-S1060. Once all coded bits have been decoded, leading trailing bits 1740-1741 are extracted S1070 according to their quantitative indications 1720-1721 that may be extracted from the bitstream S1030. From the leading trailing bits 1660, the complete trailing bits 1650 need to be recovered S1080 in order to be decoded. The leading trailing bits 1660 are padded with zeros up to a predetermined maximum length of the trailing bits. The recovered trailing bits can then be decoded S1090.

[0230] Another exemplary implementation is one in which, if there is sufficient memory available, a complete substream can be formed before decoding. The coded bits and leading trailing bits are extracted from the bitstream. The trailing bits can be restored as described above. The coded bits and determined trailing bits pertain to the substream. The complete substream is then decoded.

[0231] However, the invention is not limited to any of these example implementations.

[0232] If bit 1 1670 following the same leading bit 1660 in the trailing bits 1650 was not signaled, it must be included in the trailing bit determination process. Bit 1 1670 is appended to the leading trailing bits 1660 before the padding 1680.

[0233] Indicators of the amount of leading trailing bits 1720-1721 may precede the coded bits 1730-1731 as well as the leading trailing bits 1740-1741 in the bitstream 1750.

[0234] The method for arithmetic decoding may also be performed separately for the multiple substreams. To recover the individual substreams for decoding, the first coded bits 1730 and the second coded bits 1731, followed by the first leading trailing bits 1740 and the second leading trailing bits 1741 are extracted. The first coded bits 1730 and the first leading trailing bits 1740 form a first substream, and the second coded bits 1731 and the second leading trailing bits 1741 form a second substream. For each substream, the trailing bits are determined as described above for a single substream. Each substream is decoded individually.

[0235] The first coded bits 1730 and the second coded bits 1731 may be extracted along with an index for their respective lengths, referred to as the first length index 1710 and the second length index 1720.

[0236] The first length indicator 1710 of the coded bits may precede the first coded bit 1730 in the bitstream 1750, and the second length indicator 1711 precedes the second coded bit 1731. Furthermore, the second length indicator 1711 may also precede the first coded bit 1730.

[0237] An indicator of the amount of second trailing bits 1721 may also precede the first coded bit 1730 .

[0238] After extraction of the first coded bits 1730, the second coded bits 1731, the first leading trailing bits 1740, and the second leading trailing bits 1741, there may be remaining bits that can be discarded. The bits remaining in the bitstream 1750 after extraction of the last trailing bit are the result of padding the entire bitstream 1750 to match an integer multiple of a predetermined amount of bits.

[0239] The arithmetic decoding may be performed in parallel, for example by a multi-core decoder, or further, only a portion of the arithmetic decoding may be performed in parallel.

[0240] The method of arithmetic decoding may be implemented as range coding.

[0241] The arithmetic coding of the present disclosure can be easily applied to the coding of neural network feature maps, or classical picture (still or moving) coding and decoding. Neural networks may be used for any purpose, in particular for coding and decoding pictures (still or moving), or for coding and decoding picture-related data, such as motion flow or motion vectors or other parameters. Neural networks may also be used for computer vision applications, such as image classification, depth detection, segmentation map determination, discriminative object recognition, etc.

[0242] The method of entropy coding of multiple channels described in the above section may be combined with the processing of trailing bits described in this section. The first and second substreams are formed by the first and second coded bits, respectively, including the multiplexed and coded portions. The trailing bits referred to in "entropy coding of multiple channels" may correspond to leading trailing bits, which are leading encoder status bits. The determined amount of leading encoder status bits is indicated in the trailing bits length indicator.

[0243] Implementation in picture coding One possible development can be seen in FIGS.

[0244] Fig. 11 is a schematic block diagram of an exemplary video encoder 20 configured to implement the techniques of the present application. In the example of Fig. 11, the video encoder 20 includes an input 102 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy coding unit 270, and an output 272 (or an output interface 272). The entropy coding unit 270 may implement the arithmetic coding method or device described above.

[0245] The mode selection unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a partitioning unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 as shown in FIG. 11 may also be referred to as a hybrid video encoder or a video encoder with a hybrid video codec.

[0246] The encoder 20 may be arranged to receive, for example via an input 201, a picture 17 (or picture data 17), for example a picture of a sequence of pictures forming a video or a video sequence. The received picture or picture data may also be referred to as a preprocessed picture 19 (or preprocessed picture data 19). For simplicity, the following description refers to picture 17. Picture 17 may also be referred to as a current picture or a picture to be coded (particularly in video coding to distinguish the current picture from other pictures, for example from previously coded and / or decoded pictures of the same video sequence, i.e. a video sequence which also includes the current picture).

[0247] A (digital) picture is, or can be thought of as, a two-dimensional array or matrix of samples with intensity values. The samples in the array may also be referred to as pixels (short for picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or picture defines the size and / or resolution of the picture. For color representation, typically three color components are used, i.e. a picture may be represented with or may contain three sample arrays. In an RBG format or color space, a picture contains corresponding red, green, and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance / chrominance format or color space, e.g. YCbCr, which contains a luminance component denoted Y (sometimes L is used instead) and two chrominance components denoted Cb and Cr. The luminance (or abbreviated luma) component Y represents the brightness or gray level intensity (as in a gray scale picture, for example), and the two chrominance (or abbreviated chroma) components Cb and Cr represent the chrominance or color information components. Thus, a picture in YCbCr format contains a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). A picture in RGB format can be converted or transformed into YCbCr format and vice versa, a process also known as color transformation or conversion. If a picture is monochrome, the picture may contain only a luminance sample array. Thus, a picture may be, for example, an array of luma samples in monochrome format, or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0248] An embodiment of the video encoder 20 may include a picture partitioning unit (not shown in FIG. 11 ) configured to partition a picture 17 into multiple (typically non-overlapping) picture blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC) or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265 / HEVC and VVC). The picture partitioning unit may be configured to use the same block size for all pictures of a video sequence and a corresponding grid that defines the block size, or to vary the block size between pictures, subsets, or groups of pictures and partition each picture into the corresponding blocks. The abbreviation AVC stands for Advanced Video Coding.

[0249] In further embodiments, the video encoder may be configured to directly receive blocks 203 of picture 17, such as one, some or all of the blocks forming picture 17. Picture blocks 203 may also be referred to as current picture blocks or picture blocks to be coded.

[0250] Similar to the picture 17, the block 203 is or can be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values), again with smaller dimensions than the picture 17. In other words, the block 203 may for example contain one sample array (e.g. a luma array in the case of a monochrome picture 17, or a luma or chroma array in the case of a color picture) or three sample arrays (e.g. a luma and two chroma arrays in the case of a color picture 17), or any other quantity and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of the block 203 defines the size of the block 203. Thus, the block may for example be an MxN (M columns and N rows) array of samples, or an MxN array of transform coefficients.

[0251] As shown in FIG. 11, an embodiment of the encoder 20 may be configured to encode the picture 17 block by block, eg, encoding and prediction is performed for each block 203 .

[0252] The embodiment of video encoder 20 shown in FIG. 11 may be further configured to partition and / or encode a picture using slices (also referred to as video slices), where a picture may be partitioned into or encoded using one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTUs).

[0253] Embodiments of video encoder 20 as shown in FIG. 11 may be further configured to partition and / or encode a picture using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where a picture may be partitioned into or encoded using one or more tile groups (typically non-overlapping), each tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., complete or fractional blocks.

[0254] 12 illustrates an example of a video decoder 30 configured to implement the techniques of the present application. The video decoder 30 is configured to receive coded picture data 21 (e.g., coded bitstream 21), e.g., as coded by encoder 20, to obtain a decoded picture 131. The coded picture data or bitstream includes information for decoding the coded picture data, e.g., data representing picture blocks (and / or tile groups or tiles) of a coded video slice and associated syntax elements.

[0255] The entropy decoding unit 304 is configured to analyze the bitstream 21 (or in general, the coded picture data 21) and, e.g., perform entropy decoding on the coded picture data 21 to obtain, e.g., quantization coefficients 309 and / or decoded coding parameters (not shown in FIG. 12 ), e.g., any or all of inter prediction parameters (e.g., reference picture indexes and motion vectors), intra prediction parameters (e.g., intra prediction modes or indices), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to an encoding scheme as described with respect to the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter prediction parameters, intra prediction parameters and / or other syntax elements to the mode application unit 360 and to provide other parameters to other units of the decoder 30. Video decoder 30 may receive syntax elements at a video slice level and / or a video block level. In addition to or as an alternative to slices and their respective syntax elements, tile groups and / or tiles and their respective syntax elements may be received and / or used. The entropy decoding may implement any of the arithmetic decoding methods or apparatus described above.

[0256] A reconstruction unit 314 (e.g. an adder or summation unit 314) may be configured to add the reconstructed residual block 313 to the prediction block 365 to obtain a reconstructed block 315 in the sample domain, e.g. by adding sample values ​​of the reconstructed residual block 313 and sample values ​​of the prediction block 365.

[0257] The embodiment of video decoder 30 shown in FIG. 12 may be configured to partition and / or decode pictures using slices (also referred to as video slices), where a picture may be partitioned or decoded into one or more slices (typically non-overlapping), and each slice may include one or more blocks (e.g., CTUs).

[0258] The embodiment of video decoder 30 shown in FIG. 12 may be configured to partition and / or decode a picture using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where a picture may be partitioned or decoded into one or more tile groups (typically non-overlapping), each of which may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., complete or fractional blocks.

[0259] Other variations of the video decoder 30 may be used to decode the encoded picture data 21. For example, the decoder 30 may generate an output video stream without a loop filtering unit 320. For example, a non-transform-based decoder 30 may inverse quantize the residual signal directly for a particular block or frame, without the inverse transform processing unit 312. In another implementation, the video decoder 30 may combine the inverse quantization unit 310 and the inverse transform processing unit 312 into a single unit.

[0260] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current step may be further processed and then output to the next step. For example, after the interpolation filtering, the motion vector derivation, or the loop filtering, further processing such as clipping or shifting may be performed on the processing result of the interpolation filtering, the motion vector derivation, or the loop filtering.

[0261] Hardware and software implementation Several further implementations in hardware and software are described below.

[0262] Any of the encoding devices described above in connection with Figures 19-22 may provide means for performing a multiplexing of portions of a first size from each of the channels and a multiplexing of portions of a second size from each of the channels. Processing circuitry within any of these example devices is then configured to encode the multiplexed portions into substreams and perform this entropy encoding independently on the first substream and the second substream.

[0263] The decoding device in any of Figures 19-22 may include processing circuitry adapted to perform a decoding method, such as the steps of entropy decoding the first substream independently from the second substream, and demultiplexing first size portions and second size portions from the first substream and the second substream into multiple channels of data.

[0264] In summary, a method and apparatus for encoding data into and decoding data from a bitstream is described. The method can efficiently perform parallel encoding and decoding, avoiding padding of substreams and thus reducing the amount of bits in the bitstream. Portions of input data channels are multiplexed and encoded into substreams. During multiplexing, a shuffling method is applied to obtain substreams of more uniform length. The amount of bits in the substreams can be further reduced by including only relevant significant bits in the trailing bits of the encoding process.

[0265] According to an embodiment, a method is provided for arithmetically coding input data into a bitstream, comprising the steps of: arithmetically coding the input data into coded bits and trailing bits; including the coded bits into the bitstream; determining minimum and maximum values ​​of an interval of the arithmetically coded input data; determining a quantity of leading trailing bits, which are consecutive trailing bits, having the same value in a first most significant bit (MSB) representing a determined maximum value as in a second MSB representing a determined minimum value; and including an indication of the determined quantity of leading trailing bits and the leading trailing bits into the bitstream.

[0266] Including leading trailing bits in the bitstream instead of the complete trailing bits reduces the amount of bits in the bitstream and thus can reduce the rate for the same quality. In other words, the determined leading trailing bits alone (without the remaining trailing bits) are sufficient to define a range interval, and therefore the remaining trailing bits do not need to be included in the bitstream but can be reconstructed at the decoder based on the specified amount of leading trailing bits. This allows saving the rate without degrading the quality.

[0267] According to an exemplary implementation, the method calculates the amount of leading trailing bits, NumTrailingBits, as follows: NumTrailingBits = CLZ((LOW+RANGE‐1) XOR LOW) where CLZ() is the count of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval.

[0268] The amount of leading trailing bits may be determined precisely, for example, instead of rounding to the nearest byte boundary, potentially further reducing the amount of bits in the bitstream.

[0269] For example, an indication of the amount of leading trailing bits precedes the coded bits and the leading trailing bits in the bitstream.

[0270] An indication of the amount of leading bits that precede the coded bits and leading trailing bits in the bitstream can allow efficient extraction from the bitstream without buffering the entire bitstream.

[0271] In an exemplary implementation, a bit with a value of 1 is added after the leading trailing bits before being included in the bitstream.

[0272] Such an approach may alternatively be used to leave bits with a value of 1. As mentioned in the detailed description, in some known codecs, such as the Current-Video codec, bits with a value of 1 are specified in the bitstream.

[0273] In particular, the steps of arithmetically coding, determining minimum and maximum intervals, and determining the amount of leading trailing bits are performed separately for the first sub-stream and the second sub-stream, resulting in first coded bits, second coded bits, first leading trailing bits, and second leading trailing bits; and the method includes inserting the first coded bits and the second coded bits, the subsequent first leading trailing bits, and the second leading trailing bits into the bitstream.

[0274] Such independent application of the method to two separate substreams provides a prerequisite for parallelization.

[0275] For example, the method further includes multiplexing the first coded bits and the second coded bits into a bitstream together with a first length indicator indicating a length of the first coded bits and a second length indicator indicating a length of the second coded bits.

[0276] Indicating the length of the sub-streams in the bitstream allows for the provision of sub-streams with different sizes, and thus may lead to more flexible bitstream construction.

[0277] In an example implementation, the first length indicator precedes the first coded bit in the bitstream and the second length indicator precedes the second coded bit in the bitstream.

[0278] An advantage of this bitstream structure may be the possibility of immediate encoding or decoding of the substreams, without the need to buffer multiple substreams and their respective indices.

[0279] For example, the second length indicator precedes the first coded bit in the bitstream.

[0280] Providing a length indicator concatenated in front of the coded bits can allow for faster extraction of individual portions of the bitstream.

[0281] For example, an indication of the amount of second leading trailing bits precedes the first coded bit in the bitstream.

[0282] Such a bitstream structure allows for faster extraction of individual parts of the bitstream, making the bitstream more suitable for parallel decoding.

[0283] In an exemplary implementation, the method further includes padding the bitstream including the first coded bits, the second coded bits, the first leading trailing bits, and the second leading trailing bits with bits having a predetermined value to adjust the bitstream length to match an integer multiple of the predetermined amount of bytes.

[0284] This approach can provide a bitstream that is appropriately conditioned for further processing, such as encapsulation into network adaptation layer units or other packets.

[0285] Further, the arithmetic coding includes encoding the first sub-stream with a first arithmetic encoder and encoding the second sub-stream with a second arithmetic encoder, and the arithmetic coding by the first arithmetic encoder and the second arithmetic encoder are performed at least partially in parallel.

[0286] The above mentioned possibility of performing the coding method separately for at least two sub-streams allows for parallel coding, which may help to facilitate improved coding efficiency.

[0287] In an exemplary implementation, the arithmetic coding is a range coding.

[0288] Range encoding may be particularly well suited to hardware and software architectures that have limited register or generally fast memory sizes.

[0289] For example, during arithmetic coding, the current minimum and current maximum values ​​of the interval are stored in a memory of a preset size; including the coded bits in the bitstream involves moving a predetermined amount of bits from the fixed bits from the memory to the bitstream; and the fixed bits are consecutive bits having the same value that are the MSBs of the binary representation of the current minimum and the current maximum values.

[0290] Outputting the coded bits in portions larger than one bit may provide for more efficient software and / or implementation.

[0291] In an exemplary implementation, during arithmetic coding, the amount of leading trailing bits approaches a predetermined amount of bits from the fixed bits (730); trailing coded bits are generated from the leading trailing bits (1160) by appending a bit (1670) whose value is one followed by zeros up to the predetermined amount of bits from the fixed bits (730); the trailing coded bits are included in the coded bits prior to including the coded bits in the bitstream; and an indicator of the zero leading trailing bits is included in the bitstream.

[0292] Thus, the increased effort of signaling many leading trailing bits along with an indication of the amount of trailing bits can be avoided, rather the signaling effort is reduced to having more coded bits and signaling zero leading trailing bits.

[0293] According to an embodiment, a method is provided for arithmetically decoding data from a bitstream, comprising: extracting from the bitstream an indication of an amount of leading trailing bits; extracting a plurality of coded bits from the bitstream; extracting from the bitstream leading trailing bits specified by the extracted indication of the amount of leading trailing bits; determining trailing bits including appending zeros to the extracted leading trailing bits up to a predetermined maximum length of the trailing bits; arithmetically decoding a coded value represented by the coded bits and bits including the determined trailing bits, thereby obtaining said data.

[0294] The reconstruction of the trailing bits from the leading trailing bits provides for decoding from the coded bits and the trailing bits using a smaller amount of bits in the bitstream.

[0295] For example, a trailing bit decision may be constructed by appending to the extracted leading trailing bit a bit whose value is one followed by zeros up to a predefined maximum length of the trailing bits.

[0296] This approach makes it possible to reconstruct the complete output of the arithmetic encoder and therefore provides a particularly suitable input for the arithmetic decoder.

[0297] In an exemplary implementation, an indication of the amount of leading trailing bits precedes the coded bits and the leading trailing bits in the bitstream.

[0298] This bitstream structure provides faster extraction from the bitstream.

[0299] In an exemplary implementation, the method includes the steps of extracting first coded bits and second coded bits, and subsequent first leading trailing bits and second leading trailing bits from a bitstream, where the first coded bits and the first leading trailing bits form a first substream and the second coded bits and the second leading trailing bits form a second substream; determining first trailing bits of the first substream; determining second trailing bits of the second substream; arithmetically decoding a first coded value represented by the first coded bits and the first bits including the determined first trailing bits; and arithmetically decoding a second coded value represented by the second coded bits and the second bits including the determined second trailing bits.

[0300] Such independent application of the method to the two separate sub-streams may result in a more efficient decoding process and provides a prerequisite for parallelization.

[0301] For example, the method further includes extracting the first coded bits and the second coded bits from the bitstream along with a first length indicator indicating a length of the first coded bits and a second length indicator indicating a length of the second coded bits.

[0302] Indicating the length of the sub-streams in the bitstream allows for the provision of sub-streams with different sizes, and thus may lead to more flexible bitstream construction.

[0303] In an example implementation, the first length indicator precedes the first coded bit in the bitstream and the second length indicator precedes the second coded bit in the bitstream.

[0304] An advantage of this bitstream structure may be the possibility of immediate decoding of the substreams without the need to buffer multiple substreams and their respective indices, which can be achieved to expedite the processing flow.

[0305] For example, the second length indicator precedes the first coded bit in the bitstream.

[0306] Providing a length indicator concatenated in front of the coded bits can allow for faster extraction of individual portions of the bitstream.

[0307] For example, an indication of the amount of second leading trailing bits precedes the first coded bit in the bitstream.

[0308] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0309] In an example implementation, the method further includes discarding remaining bits of the bitstream after extracting the first coded bits, the second coded bits, the first leading trailing bits, and the second leading trailing bits.

[0310] Such an approach can provide a bitstream that is appropriately conditioned for further processing, such as encapsulation into network adaptation layer units or other packets.

[0311] For example, the arithmetic decoding includes decoding the first sub-stream with a first arithmetic decoder and decoding the second sub-stream with a second arithmetic decoder, and the arithmetic decoding by the first arithmetic decoder and the second arithmetic decoder are performed at least partially in parallel.

[0312] Parallel decoding of the sub-streams may result in faster decoding of the complete bitstream.

[0313] In an exemplary implementation, the arithmetic decoding is a range decoding.

[0314] Range encoding may be particularly well suited to hardware and software architectures that have limited register or generally fast memory sizes.

[0315] In an exemplary implementation, a computer program stored on a non-transitory medium includes code instructions that, when executed on one or more processors, cause the one or more processors to perform the steps of any of the methods described above.

[0316] According to an embodiment, an apparatus is provided for arithmetically coding input data into a bitstream, the apparatus comprising processing circuitry configured to: arithmetically code the input data into coded bits and trailing bits; include the coded bits in the bitstream; determine minimum and maximum values ​​of an interval of the arithmetically coded input data; determine a quantity of leading trailing bits, the leading trailing bits being consecutive trailing bits, the leading trailing bits having the same value in a first most significant bit (MSB) representing the determined maximum value as in a second MSB representing the determined minimum value; and include an indication of the determined quantity of leading trailing bits and the leading trailing bits in the bitstream.

[0317] According to an embodiment, an apparatus is provided for arithmetically decoding data from a bitstream, the apparatus comprising processing circuitry configured to: extract from the bitstream an indication of an amount of leading trailing bits; extract from the bitstream a plurality of coded bits; extract from the bitstream leading trailing bits specified by the extracted indication of the amount of leading trailing bits; determine trailing bits including appending zeros to the extracted leading trailing bits up to a predetermined maximum length of the trailing bits; and arithmetically decode a coded value represented by the coded bits and bits including the determined trailing bits, thereby obtaining said data.

[0318] The apparatus provides the advantages of the method described above.

[0319] Any of the encoding devices described above in relation to Figures 19-22 may provide means for performing arithmetic coding of input data into coded bits and preceding trailing bits. Processing circuitry within any of these exemplary devices is configured to encode the input data and determine the leading trailing bits of the encoder status after encoding the last bit of the coded bits according to the method described above.

[0320] The decoding device of any of Figures 19-22 may include processing circuitry adapted to perform a decoding method, such as that described above, which includes extracting the coded bits and leading trailing bits along with an indication of their respective quantities. The trailing bits may be reconstructed from the leading trailing bits and decoded together with the coded bits to obtain the data.

[0321] In summary, a method and apparatus for encoding data into and decoding data from a bitstream is described. The method is capable of reducing the length of the bitstream by including only relevant significant bits in the trailing bits of the encoding process. The amount of these leading trailing bits is determined and it is possible to configure the trailing bits having a minimum amount. An indication of the amount of leading trailing bits is included in the bitstream. Thus, no padding is required and fewer bits need to be signaled.

[0322] In the following embodiments of the video coding system 10, the video encoder 20 and the video decoder 30 are described based on FIGS. 19 and 20 in conjunction with the above-mentioned FIGS. 11 and 12 or other encoders and decoders, such as neural network-based encoders and decoders.

[0323] 19 is a schematic block diagram illustrating an example coding system 10, e.g., video coding system 10 (or coding system 10 for short), capable of using techniques of the present application. Video encoder 20 (or encoder 20 for short) and video decoder 30 (or decoder 30 for short) of video coding system 10 represent examples of devices that may be configured to perform techniques according to various examples described herein.

[0324] As shown in FIG. 19, coding system 10 includes a source device 12 configured to provide encoded picture data 21 to a destination device 14, which may, for example, decode encoded picture data 13.

[0325] The source device 12 includes an encoder 20 and may additionally, i.e. optionally, include a picture source 16, a pre-processing processor (or pre-processing unit) 18, such as a picture pre-processing unit 18, and a communication interface or unit 22.

[0326] Picture source 16 may include or be any kind of picture capture device, such as a camera capturing real-world pictures, and / or any kind of picture generation device, such as a computer graphics processor generating computer-animated pictures, or any kind of other device acquiring and / or providing real-world pictures, computer-generated pictures (e.g. screen content or virtual reality (VR) pictures), and / or any combination thereof (e.g. augmented reality (AR) pictures). Picture source could be any kind of memory or storage storing any of the above mentioned pictures.

[0327] To distinguish between the pre-processing unit 18 and the processing performed by the pre-processing unit 18 , the pictures and the picture data 17 may also be referred to as unprocessed pictures or unprocessed picture data 17 .

[0328] The pre-processing unit 18 is configured to receive the (unprocessed) picture data 17 and perform pre-processing on the picture data 17 to obtain a pre-processed picture 19 or pre-processed picture data 19. The pre-processing performed by the pre-processing unit 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction, or noise reduction. It can be understood that the pre-processing unit 18 may be an optional component.

[0329] A video encoder 20 is arranged to receive pre-processed picture data 19 and to provide encoded picture data 21 (further details are described above with reference to, for example, FIG. 11).

[0330] The communications interface 22 of the source device 12 can be configured to receive the encoded picture data 21 and to transmit the encoded picture data 21 (or any further processed version of it) via the communications channel 13 to another device, such as the destination device 14 or any other device, for storage or direct reconstruction.

[0331] The destination device 14 includes a decoder 30 (e.g., a video decoder 30) and may additionally, i.e. optionally, include a communications interface or unit 28, a post-processing processor 32 (or post-processing unit 32), and a display device 34.

[0332] The communications interface 28 of the destination device 14 is configured to receive the encoded picture data 21 (or any further processed version thereof), for example directly from the source device 12 or from any other source, for example from a storage device, for example from a storage device for encoded picture data, and to provide the encoded picture data 21 to a decoder 30.

[0333] The communication interface 22 and the communication interface 28 may be configured to transmit or receive the encoded picture data 21 or the encoded data 13 via a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any kind of network, such as a wired or wireless network or any combination thereof, or any kind of private and public network, or any combination thereof.

[0334] The communications interface 22 may, for example, be configured to package the encoded picture data 21 into a suitable format, such as packets, and / or process the encoded picture data using any type of transmission coding or processing for transmission over a communications link or network.

[0335] The communications interface 28, forming a counterpart of the communications interface 22, can for example be configured to receive transmitted data and to process the transmitted data using any kind of corresponding transmission decoding or processing and / or unpackaging to obtain the encoded picture data 21.

[0336] Both communication interface 22 and communication interface 28 may be configured as a unidirectional communication interface, as indicated by the arrow in FIG. 19 for communication channel 13 from source device 12 to destination device 14, or as a bidirectional communication interface, and may be configured, for example, to send and receive messages, for example, to set up connections, to verify and exchange communication links and / or any other information related to data transmission, for example encoded picture data transmission.

[0337] The decoder 30 is arranged to receive the encoded picture data 21 and to provide decoded picture data 31 or decoded pictures 31 (further details are explained, for example, on the basis of FIG. 12).

[0338] The post-processing processor 32 of the destination device 14 is configured to post-process the decoded picture data 31 (also called reconstructed picture data), e.g., a decoded picture 31, to obtain post-processed picture data 33, e.g., a post-processed picture 33. The post-processing performed by the post-processing unit 32 may include, e.g., color format conversion (e.g., from YCbCr to RGB), color correction, cropping, resampling, or any other processing, e.g., to prepare the decoded picture data 31 for display by a display device 34.

[0339] A display device 34 of destination device 14 is configured to receive the post-processed picture data 33 and display the picture, for example to a user or viewer. The display device 34 may be or include any type of display that presents the reconstructed picture, for example an integrated or external display or monitor. The display may include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any other type of display.

[0340] 19 depicts source device 12 and destination device 14 as separate devices, an embodiment of the devices may also include both or both functionality, source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software, or by separate hardware and / or software, or any combination thereof.

[0341] As will be clear to a person skilled in the art based on the specification, the presence and (exact) division of functions or functions of various units in the source device 12 and / or destination device 14 shown in FIG. 19 may vary depending on the actual device and application.

[0342] Encoder 20 (e.g., video encoder 20) or decoder 30 (e.g., video decoder 30) or both encoder 20 and decoder 30 may be implemented by processing circuitry as shown in FIG. 20, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, dedicated to video coding, or any combination thereof. Encoder 20 may be implemented by processing circuitry 46 to implement various modules described with respect to encoder 20 of FIG. 11 and / or any other encoder system or subsystem described herein. Decoder 30 may be implemented by processing circuitry 46 to implement various modules described with respect to decoder 30 of FIG. 12 and / or any other decoder system or subsystem described herein. The processing circuitry may be configured to perform various operations as described below. Where the techniques are implemented in part in software, as shown in Figure 22, a device may store the software instructions on a suitable non-transitory computer-readable storage medium and execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Either video encoder 20 or video decoder 30 may be integrated as part of a combined encoder / decoder (CODEC) within a single device, as shown, for example, in Figure 20.

[0343] Source device 12 and destination device 14 may include any of a wide range of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video gaming console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiving device, a broadcast transmitting device, and the like, and may or may not use any type of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

[0344] In some cases, the video coding system 10 shown in FIG. 19 is merely an example, and the techniques of the present application may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between the encoding device and the decoding device. In other examples, data may be retrieved from local memory, streamed over a network, etc. A video encoding device may encode data and store it in memory, and / or a video decoding device may retrieve data from memory and decode it. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data to memory and / or retrieve data from memory and decode it.

[0345] For ease of explanation, embodiments of the present invention are described herein with reference to reference software, such as High Efficiency Video Coding (HEVC) or Generic Video Coding (VVC), the next generation video coding standard developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Moving Picture Experts Group (MPEG) Joint Research Team on Video Coding (JCT-VC). Those skilled in the art will appreciate that embodiments of the present invention are not limited to HEVC or VVC.

[0346] 21 is a schematic diagram of a video coding device 400 according to an embodiment of the present disclosure. The video coding device 400 is suitable for implementing the disclosed embodiments as described herein. In an embodiment, the video coding device 400 may be a decoder, such as the video decoder 30 of FIG. 19, or an encoder, such as the video encoder 20 of FIG. 19.

[0347] Video coding device 400 includes an ingress port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an egress port 450 (or output port 450) for transmitting data, and a memory 460 for storing data. Video coding device 400 may also include optical-to-electrical (OE) and electrical-to-optical (EO) components coupled to ingress port 410, receiver unit 420, transmitter unit 440, and egress port 450 for entering and leaving optical or electrical signals.

[0348] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., multi-core processors), FPGAs, ASICs, and DSPs. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the disclosed embodiments described above. For example, the coding module 470 implements, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module 470 provides a significant improvement to the functionality of the video coding device 400 and affects the transformation of the video coding device 400 into different states. Alternatively, the coding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0349] Memory 460 may include one or more disks, tape drives, and solid state drives, may be used as overflow data storage devices, may store programs when such programs are selected for execution, and may store instructions and data loaded during the execution of programs. Memory 460 may be volatile and / or non-volatile, and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0350] FIG. 22 is a simplified block diagram of an apparatus 500 that can be used as either or both of the source device 120 and the destination device 14 of FIG. 19, according to an example embodiment.

[0351] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices, now existing or later developed, capable of manipulating or processing information. Although the disclosed implementations may be implemented using a single processor, such as processor 502, as shown, advantages in speed and efficiency may be obtained using more than one processor.

[0352] The memory 504 in the device 500 may be a read-only memory (ROM) device or a random access memory (RAM) device in implementation. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 that is accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510, which include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include applications 1 through N, which further include a video coding application that performs the methods described herein, including encoding and decoding using arithmetic coding as described above.

[0353] The apparatus 500 may also include one or more output devices, such as a display 518. The display 518, in one example, may be a touch-sensitive display that combines a display with touch-sensitive elements operable to sense touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0354] Although depicted herein as a single bus, bus 512 of device 500 may be comprised of multiple buses. Additionally, secondary storage 514 may be directly coupled to other components of device 500 or may be accessed over a network, and may include a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, device 500 may be implemented in a wide variety of configurations.

[0355] Although embodiments of the present invention are described primarily in the context of video coding, it should be noted that embodiments of coding system 10, encoder 20 and decoder 30 (and corresponding system 10), as well as other embodiments described herein, may be configured for still picture processing or coding, i.e., processing or coding of individual pictures independent of any preceding or subsequent pictures, as in video coding. In general, in cases where picture processing coding is limited to a single picture 17, only inter prediction units 244 (encoder) and 344 (decoder) may not be available. All other functions (also called tools or techniques) of the video encoder 20 and the video decoder 30 can be equally used for still picture processing, e.g., residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, partitioning 262 / 362, intra prediction 254 / 354, and / or loop filtering 220, 320, as well as entropy coding 270 and entropy decoding 304.

[0356] For example, embodiments of the encoder 20 and the decoder 30, and the functions described herein with respect to the encoder 20 and the decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium or transmitted over a communication medium as one or more instructions or codes and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium, which corresponds to a tangible medium, such as a data storage medium, or a communication medium, which includes any medium that facilitates the transfer of a computer program from one place to another, for example according to a communication protocol. Thus, a computer-readable medium may generally correspond to (1) a tangible computer-readable storage medium that is non-transitory, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to retrieve instructions, code, and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0357] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly referred to as a computer-readable medium. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but rather relate to non-transitory tangible storage media. Disk and disk, as used herein, include compact disks (CDs), laser disks, optical disks, digital versatile disks (DVDs), floppy disks, and Blu-ray disks, where disks typically reproduce data magnetically and disks reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer readable media.

[0358] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Thus, the term "processor" as used herein may refer to any of the aforementioned structures, or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided in dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. The techniques may also be fully implemented in one or more circuits or logic elements.

[0359] The techniques of the present disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC), or a set of ICs (e.g., a chip set). To emphasize the functional aspects of devices configured to implement the disclosed techniques, various components, modules, or units are described in the present disclosure, but do not necessarily require realization by different hardware units. Rather, as described above, the various units may be combined in a codec hardware unit in association with appropriate software and / or firmware, or may be provided by a collection of interoperating hardware units including one or more processors as described above.

Claims

1. 1. A coding method for entropy coding multiple channels of same-sized data into a bitstream, comprising: multiplexing portions of a first size from each of the channels of the plurality of channels and thereafter encoding the multiplexed portions of the first size into a first substream; multiplexing portions of a second size from each of the channels of the plurality of channels and thereafter encoding the multiplexed portions of the second size into a second substream; wherein the encoding is entropy encoding performed independently on the first sub-stream and the second sub-stream.

2. 2. The encoding method of claim 1, further comprising the step of generating the same-sized channels including pre-processed data of the different-sized channels to obtain the same-sized channels.

3. 3. The method of claim 1, further comprising multiplexing the first substream and the second substream into the bitstream together with a first substream length indicator indicating a length of the first substream and a second substream length indicator indicating a length of the second substream.

4. The encoding method according to claim 3, the first substream length indicator precedes the first substream in the bitstream; and A method of encoding, wherein the second substream length indicator precedes the second substream in the bitstream.

5. 5. A method of encoding as claimed in claim 4, wherein the second sub-stream length indicator precedes the first sub-stream in the bitstream.

6. Coding method according to any one of claims 1 to 5, wherein the entropy coding is an arithmetic coding.

7. 7. The encoding method of claim 6, further comprising multiplexing the first substream and the second substream into the bitstream along with a first trailing bit length indicator indicating a length of a first trailing bit of the first substream and a second trailing bit length indicator indicating a length of a second trailing bit of the second substream.

8. The encoding method according to claim 7, the first trailing bit length indicator precedes the first substream in the bitstream; and The method of encoding, wherein the second trailing bit length indicator precedes the second substream in the bitstream.

9. 9. The method of claim 8, wherein the second trailing bit length indicator precedes the first substream in the bitstream.

10. A method for encoding according to any one of claims 7 to 9, comprising: appending the first trailing bits to the bitstream after the first substream; and appending said second trailing bits to said bitstream after said second substream. The encoding method further comprises:

11. 11. The method of claim 10, wherein the first trailing bits follow the second substream in the bitstream.

12. In the encoding method of any one of claims 7 to 11, when claim 6 cites claim 3, the encoding method further comprises a step of padding the bitstream including the first substream length indicator, the second substream length indicator, the first trailing bit length indicator, the second trailing bit length indicator, the first substream, the second substream, the first trailing bits, and the second trailing bits with bits having a predetermined value to adjust the length of the bitstream to match an integer multiple of a predetermined byte amount.

13. 13. A method according to any one of the preceding claims, wherein the first size is equal to the second size.

14. 14. The encoding method according to claim 1, wherein all parts included in the first substream and all parts included in the second substream are an integer K times the number of symbols of the data of the multiple channels, K being greater than 1.

15. 15. The method of claim 14, wherein the symbols are bits.

16. 16. The method of encoding according to claim 1, further comprising the step of selecting and subsequently applying a shuffling method for the multiplexing of the portions of the first size and the portions of the second size, the shuffling method being: selected from a set of pre-defined shuffling methods; and specifying an order of the portions of the first size and the portions of the second size.

17. 17. A method of encoding as claimed in claim 16, wherein the shuffling method performs a cyclic permutation of the portions of the second size with respect to the portions of the first size.

18. 17. The encoding method of claim 16, wherein the shuffling method, when performed iteratively, comprises: the difference between the length of the current first sub-stream and a statistic based on the length of past first sub-streams; and / or The difference between the current length of the second substream and a statistic based on the past lengths of the second substream. An encoding method selected according to:

19. 20. The method of claim 18, wherein the statistics are based on at least one of an estimated mean, median, minimum, maximum, or growth rate.

20. A method for encoding according to any one of claims 16 to 19, the entropy coding is arithmetic coding; and A method of encoding, wherein the shuffling method is selected according to a current state of a range interval in the arithmetic coding.

21. A method for encoding according to any one of claims 1 to 20, the entropy encoding includes generating the first substream with a first entropy encoder and generating the second substream with a second entropy encoder; and A method of encoding, wherein the entropy encoding by the first entropy encoder and the second entropy encoder is performed at least partially in parallel.

22. 22. The method according to any one of claims 1 to 21, wherein the channels are output channels or latent representation channels of a neural network.

23. A method for encoding according to any one of claims 7 to 22, the entropy coding is arithmetic coding; and The method relates to encoding a multiplexed portion into the first sub-stream or the second sub-stream, the method comprising: arithmetically encoding the multiplexed portions into coded bits and encoder status bits, the coded bits forming the substreams; determining minimum and maximum values ​​of an interval of the arithmetically coded input data; determining a quantity of leading trailing bits, which are consecutive encoder status bits, in a first most significant bit (MSB) representing the determined maximum value having the same value as in a second most significant bit (MSB) representing the determined minimum value, said trailing bits being leading encoder status bits; and specifying the determined amount of the leading encoder status bits in the trailing bit length indicator; 13. An encoding method comprising:

24. 24. The encoding method of claim 23, wherein the amount of leading encoder status bits, NumTrailingBits, is: NumTrailingBits = CLZ((LOW+RANGE‐1) XOR LOW) where CLZ() is a count of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval.

25. A method according to any one of claims 23 to 24, wherein a bit having a value of 1 is added after the leading encoder status bits before being included in the bitstream.

26. A method for encoding according to any one of claims 24 to 25, During said arithmetic coding, the current minimum value and the current maximum value of said interval are stored in a memory of a preset size; Including the coded bits in the bitstream includes moving a predetermined amount of bits from a fixed bit stream from the memory to the bitstream; and The coding method, wherein the fixed bits are consecutive bits having the same value in the MSBs of the binary representations of the current minimum value and the current maximum value.

27. 27. The encoding method according to claim 26, if, during the arithmetic coding, a difference between an amount of the leading encoder status bits and a predetermined amount of bits from the fixed bits falls below a predetermined threshold; generating trailing coded bits from the leading encoder status bits by appending a bit having a value of one followed by zeros up to a predetermined amount of bits from the fixed bits; the trailing coded bits are included with the coded bits prior to including the coded bits in the bitstream; and A method wherein an indication of a zero leading encoder status bit is included in the bitstream.

28. 28. A method according to any one of claims 7 to 27, wherein the arithmetic coding is a range coding.

29. 1. A decoding method for entropy decoding a bitstream into equal sized multiple channel data, comprising: entropy decoding the first substream independently of the second substream; demultiplexing first size portions and second size portions from the first sub-stream and the second sub-stream into the same sized multiple channels; A decoding method comprising:

30. 30. The method of decoding of claim 29, further comprising the step of post-processing the multiple channels of the same size to obtain multiple channels of data of different sizes.

31. 31. A method for decoding according to any one of claims 29 to 30, further comprising the step of extracting the first sub-stream and the second sub-stream from the bitstream together with a first sub-stream length indicator indicating a length of the first sub-stream and a second sub-stream length indicator indicating a length of the second sub-stream.

32. 32. The method of claim 31, further comprising: the first substream length indicator precedes the first substream in the bitstream; and The method of decoding, wherein the second substream length indicator precedes the second substream in the bitstream.

33. 33. A method of decoding as recited in claim 32, wherein the second substream length indicator precedes the first substream in the bitstream.

34. 34. A method according to any one of claims 29 to 33, wherein said entropy decoding is an arithmetic decoding.

35. 35. The method of decoding as recited in claim 34, further comprising extracting the first substream and the second substream from the bitstream along with a first trailing bit length indicator indicating a length of a first trailing bit of the first substream and a second trailing bit length indicator indicating a length of a second trailing bit of the second substream.

36. 36. The method of claim 35, the first trailing bit length indicator precedes the first substream in the bitstream; and The method of decoding, wherein the second trailing bit length indicator precedes the second substream in the bitstream.

37. 37. A method of decoding as recited in claim 36, wherein the second trailing bit length indicator precedes the first substream in the bitstream.

38. A method of decoding according to any one of claims 35 to 37, comprising: the first trailing bits are in the bitstream after the first substream; and The method of decoding, wherein the second trailing bits follow the second substream in the bitstream.

39. 40. A method of decoding as recited in claim 38, wherein the first trailing bits follow the second substream in the bitstream.

40. 40. A method for decoding according to any one of claims 29 to 39, wherein the first size is equal to the second size.

41. 41. A method for decoding as claimed in any one of claims 29 to 40, wherein all parts contained in the first substream and all parts contained in the second substream are an integer K times the symbols of the data of the multiple channels, K being greater than 1.

42. 42. A method of decoding as claimed in claim 41, wherein the symbols are bits.

43. A decoding method as described in claim 42, where claim 34 refers to claim 31 and claim 41 refers to claim 35, further comprising a step of discarding remaining bits of the bitstream after extracting the first substream length index, the second substream length index, the first trailing bit length index, the second trailing bit length index, the first substream, the second substream, the first trailing bit, and the second trailing bit.

44. A method of decoding according to any one of claims 29 to 43, comprising: determining and applying a shuffling method for the demultiplexing of the portions of the first size and the portions of the second size, the shuffling method comprising: one from a set of predefined shuffling methods, and specifying an order of the portions of the first size and the portions of the second size.

45. 45. A method of decoding as recited in claim 44, wherein determining the shuffling method is based on control information included in the bitstream.

46. A method for decoding according to any one of claims 29 to 45, comprising the steps of: the entropy decoding includes decoding the first substream with a first entropy decoder and decoding the second substream with a second entropy decoder; and A method of decoding, wherein the entropy decoding by the first entropy decoder and the second entropy decoder is performed at least partially in parallel.

47. 47. A method according to any one of claims 29 to 46, wherein said channels are output channels of a neural network or latent representation channels.

48. A method for decoding according to any one of claims 35 to 47, comprising the steps of: the entropy decoding is arithmetic coding, and The method relates to decoding a multiplexed portion from the first sub-stream or the second sub-stream, the method comprising: extracting an amount of leading encoder status bits from the trailing bit length indicator, the substream including coded bits and the trailing bits being leading encoder status bits; determining the encoder status bits including appending zeros to the extracted leading encoder status bits up to a predetermined maximum length of the encoder status bits; and performing arithmetic decoding of the demultiplexed portion from the coded bits and bits including the determined encoder status bit; A decoding method comprising:

49. 49. A method of decoding as recited in claim 48, wherein the step of determining the encoder status bits comprises appending to the extracted leading encoder status bits a bit having a value of one followed by zeros up to a predetermined maximum length of the encoder status bits.

50. 50. A method of decoding according to any one of claims 34 to 39, 48 or 49, wherein the arithmetic decoding is range decoding.

51. A computer program stored on a non-transitory medium, comprising code instructions which, when executed on one or more processors, cause said one or more processors to perform the steps of the method according to any one of claims 1 to 50.

52. 1. An apparatus for entropy encoding multiple channels of same-sized data into a bitstream, comprising: multiplexing portions of a first size from each of the channels of the plurality of channels and thereafter encoding the multiplexed portions of the first size into a first substream; a processing circuit configured to multiplex a second size portion from each of the channels of the plurality of channels and thereafter encode the multiplexed second size portion into a second substream; The apparatus, wherein the encoding is entropy encoding performed independently on the first sub-stream and the second sub-stream.

53. 1. An apparatus for entropy decoding a bitstream into equal sized multiple channel data, comprising: entropy decoding the first substream independently of the second substream; an apparatus comprising: a processing circuit configured to demultiplex first size portions and second size portions from the first substream and the second substream into the multiple channels of the same size.

Images