Manipulation of tail bits in arithmetic encoding and decoding

Arithmetic encoding and decoding techniques that utilize leading trailing bits in the bitstream enhance video coding efficiency by reducing bitstream size and enabling parallel processing, addressing the need for high compression ratios with minimal quality loss.

JP7582576B2Active Publication Date: 2024-11-13HUAWEI TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
JP2023548750
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-04-26
Publication Date
2024-11-13
Estimated Expiration
2041-04-26

AI Technical Summary

Technical Problem

Existing video coding techniques face challenges in achieving high compression ratios with minimal image quality loss, particularly in scenarios where video data needs to be streamed over limited bandwidth networks or stored in memory-constrained devices.

Method used

The method involves arithmetic encoding and decoding that includes leading trailing bits in the bitstream, determining the minimum and maximum values of the interval, and indicating the amount of leading trailing bits, allowing for efficient extraction and parallel processing of substreams.

Benefits of technology

This approach reduces the bitstream size without compromising quality by including only necessary leading trailing bits, enabling faster encoding and decoding, particularly suitable for hardware and software architectures with limited resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007582576000001
    Figure 0007582576000001
  • Figure 0007582576000002
    Figure 0007582576000002
  • Figure 0007582576000003
    Figure 0007582576000003
Patent Text Reader

Abstract

A method and apparatus for encoding data into and decoding data from a bitstream are described. The method is capable of reducing the length of the bitstream by including only relevant significant bits in the tail bits of the encoding process. The amount of these leading tail bits can be determined and the tail bits with the lowest amount can be constructed. An indication of the amount of leading tail bits is included in the bitstream. Thus, no padding is required and, as a result, fewer bits need to be signaled.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] This disclosure relates to arithmetic encoding and decoding, and in particular to the manipulation of tail bits, particularly their inclusion in and parsing from bitstreams. [Background technology]

[0002] Video coding (video encoding and decoding) is used in a wide range of digital video applications, such as digital television broadcasting, video transmission over the Internet and mobile networks, real-time conversation applications such as video chat, video conferencing, DVD, and Blu-ray Disc, video content acquisition and editing systems, mobile device video recording, and camcorders for security applications.

[0003] Since the development of the block-based hybrid video coding approach in the H.261 standard in 1990, new video coding techniques and tools have been developed and formed the basis of new video coding standards. One of the goals of most video coding standards has been to achieve a reduction in bit rate compared to the previous one without sacrificing image quality. Further video coding standards include MPEG-1 Video, MPEG-2 Video, VP8, VP9, ​​AV1, ITU-T H.262 / MPEG-2, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265, High Efficiency Video Coding (HEVC), ITU-T H.266, Versatile Video Coding (VVC), and extensions of these standards, such as scalability and / or three-dimensional (3D) extensions.

[0004] The amount of video data required to render even a relatively short video can be significant, which can result in difficulties when the data is to be streamed or otherwise communicated over a communication network with limited bandwidth capacity. Thus, video data is typically compressed before being communicated over modern telecommunication networks. The size of the video can also be an issue when the video is stored on a storage device, since memory resources may be limited. Video compression devices often use software and / or hardware at the source to code the video data before transmission or storage, thereby reducing the amount of data required to represent a digital video image. The compressed data is then received at the destination by a video decompression device, which decodes the video data. In view of limited network resources and an ever-increasing demand for high video quality, improved compression and decompression techniques are desired that improve compression ratios with little or no sacrifice in image quality.

[0005] The video encoding and decoding may be performed by standard video encoders and decoders conforming to, for example, H.264 / AVC, HEVC (H.265), VVC (H.266), or other video coding techniques, or the video coding, or parts thereof, may be performed by neural networks.

[0006] Entropy coding has been widely used in any encoding or decoding of still pictures or images, or other source signals, such as feature channels of neural networks. In particular, arithmetic coding has gained importance in newer coding techniques. Therefore, it may be desirable to improve the efficiency of entropy coding. Summary of the Invention

[0007] SUMMARY OF THE DISCLOSURE Embodiments of the present disclosure provide an apparatus and method for arithmetically encoding and decoding data into and from a bitstream, where the bitstream includes coded bits and leading tail bits.

[0008] Embodiments of the invention are defined by the features of the independent claims, further advantageous implementations of the embodiments are defined by the features of the dependent claims.

[0009] According to an embodiment, there is provided a method for arithmetically encoding input data into a bitstream, the method comprising the steps of: arithmetically encoding the input data into coded bits and trailing bits; including the coded bits in the bitstream; determining a minimum and a maximum value of an interval of the arithmetically encoded input data; determining a quantity of leading trailing bits that are consecutive trailing bits and have the same value in a first most significant bit (MSB) representing the determined maximum value as in a second MSB representing the determined minimum value; and including in the bitstream an indication of the determined quantity of leading trailing bits, and the leading trailing bits.

[0010] Including the leading tail bits in the bitstream rather than the full tail bits can reduce the amount of bits in the bitstream, and thus, for example, reduce the rate with the same quality. In other words, the determined leading tail bits (without the remaining tail bits) are enough to define the range interval, and therefore the remaining tail bits do not need to be included in the bitstream and can be reconstructed at the decoder based on the indicated amount of leading tail bits. This can save rate without reducing quality.

[0011] According to an implementation example, the method includes steps in which the amount of leading trailing bits, NumTrailingBits, is determined by NumTrailingBits=CLZ((LOW+RANGE-1)XOR LOW), where CLZ() is the number of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval.

[0012] The amount of leading trailing bits may be determined precisely, for example, rather than rounded to the nearest byte boundary, which may further reduce the amount of bits in the bitstream.

[0013] For example, an indication of the amount of leading tail bits precedes the coded bits and the leading tail bits in the bitstream.

[0014] An indication of the amount of preceding bits that precede the coded bits and preceding tail bits in the bitstream may allow efficient extraction from the bitstream without buffering the entire bitstream.

[0015] In an example implementation, a bit having a value of 1 is appended to each leading trailing bit before being included in the bitstream.

[0016] Such an approach may be used instead of removing the 1-valued bits. As mentioned in the detailed description, currently in some known codecs, such as video codecs, 1-valued bits are represented in the bitstream.

[0017] In particular, the steps of arithmetic encoding, determining minimum and maximum values ​​of the interval, and determining the amount of leading tail bits are performed separately for the first sub-stream and the second sub-stream, resulting in first coded bits, second coded bits, first leading tail bits, and second leading tail bits; the method comprises the step of inserting the first coded bits and second coded bits followed by the first leading tail bits and second leading tail bits into the bitstream.

[0018] Such independent application of the method to two separate substreams provides a prerequisite for parallelization.

[0019] For example, the method further comprises multiplexing the first coded bits and the second coded bits into a bitstream together with a first length indication indicating a length of the first coded bits and a second length indication indicating a length of the second coded bits.

[0020] By indicating the length of the sub-streams in the bitstream, it becomes possible to provide sub-streams with different sizes, and thus may help to achieve a more flexible bitstream configuration.

[0021] In an example implementation, the first length indication precedes the first coded bits in the bitstream, and the second length indication precedes the second coded bits in the bitstream.

[0022] An advantage of this bitstream structure may be that it allows for immediate encoding or decoding of the substreams without the need to buffer multiple substreams and their respective indications.

[0023] For example, the second length indication precedes the first coded bit in the bitstream.

[0024] Providing a length indication concatenated in front of the coded bits may allow faster extraction of individual portions of the bitstream.

[0025] For example, an indication of the amount of second leading tail bits precedes the first coded bit in the bitstream.

[0026] Such a bitstream structure allows for much faster extraction of individual portions of the bitstream, making the bitstream more suitable for parallel decoding.

[0027] In an example implementation, the method further includes padding the bitstream including the first coded bit, the second coded bit, the first leading tail bit, and the second leading tail bit with bits having predetermined values ​​to align the bitstream length to an integer multiple of the predetermined amount of bytes.

[0028] Such an approach may provide a properly aligned bitstream for further processing, such as, for example, encapsulation into a network adaptation layer unit or other packets.

[0029] Further, the arithmetic encoding includes encoding the first sub-stream using a first arithmetic encoder and encoding the second sub-stream using a second arithmetic encoder, wherein the arithmetic encoding using the first arithmetic encoder and the second arithmetic encoder are performed at least partially in parallel.

[0030] As mentioned above, being able to perform the encoding method separately on at least two sub-streams allows for parallel encoding, which may help facilitate improved encoding efficiency.

[0031] In the implementation, the arithmetic encoding is a range encoding.

[0032] Range encoding may be particularly suitable for hardware and software architectures that have limited registers or generally fast memory sizes.

[0033] For example, during arithmetic encoding, the current minimum and current maximum values ​​of the interval are stored in a memory of a preconfigured size; including the coded bits in the bitstream involves moving a predefined amount of bits from the memory into the bitstream of stable bits; stable bits are consecutive bits that have the same value in the MSBs of the binary representation of the current minimum and current maximum values.

[0034] Partially outputting more than one coded bit may provide for more efficient software and / or implementation.

[0035] In an example implementation, during arithmetic encoding, the amount of leading tail bits is close to a predefined amount of stable bits; tail coded bits are generated from the leading tail bits by appending a 1 bit having a value of 1 followed by 0 up to the predefined amount of stable bits; tail coded bits are included in the coded bits before including the coded bits in the bitstream; and an indication of leading tail bits of 0 is included in the bitstream.

[0036] Thus, the increased effort of signaling many leading tail bits together with an indication of the amount of tail bits may be avoided. Instead, the signaling effort to have more coded bits and signal 0 leading tail bits may be reduced.

[0037] According to an embodiment, there is provided a method for arithmetically decoding data from a bitstream, the method comprising the steps of: extracting from the bitstream an indication of an amount of leading tail bits; extracting a plurality of coded bits from the bitstream; extracting from the bitstream leading tail bits specified by the indication of the amount of extracted leading tail bits; determining tail bits including appending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; and obtaining the data by arithmetically decoding a coded value represented by bits including the coded bits and the determined tail bits.

[0038] Reconstructing the tail bits from the preceding tail bits provides decoding from the coded bits and the tail bits by using a smaller amount of bits in the bitstream.

[0039] For example, the step of determining the tail bits may comprise appending to the preceding extracted tail bits a bit having a value of 1 followed by zeros up to a predetermined maximum length of the tail bits.

[0040] This approach makes it possible to reconstruct the complete output of the arithmetic encoder, thereby providing a particularly suitable input for the arithmetic decoder.

[0041] In an implementation, an indication of the amount of leading tail bits precedes the coded bits and the leading tail bits in the bitstream.

[0042] This bitstream structure may provide for more efficient extraction from the bitstream.

[0043] In an example implementation, the method includes extracting a first coded bit and a second coded bit followed by a first leading tail bit and a second leading tail bit from a bitstream, the first coded bit and the first leading tail bit forming a first substream, the second coded bit and the second leading tail bit forming a second substream; determining a first tail bit for the first substream, determining a second tail bit for the second substream, arithmetically decoding a first coded value represented by the first coded bit and the first bits including the determined first tail bit; and arithmetically decoding a second coded value represented by the second coded bit and the second bits including the determined second tail bit.

[0044] Such independent application of the method to the two separate sub-streams may result in a more efficient decoding process, providing a prerequisite for parallelization.

[0045] For example, the method further comprises extracting the first coded bits and the second coded bits from the bitstream together with a first length indication indicating a length of the first coded bits and a second length indication indicating a length of the second coded bits.

[0046] By indicating the length of the sub-streams in the bitstream, it is possible to provide sub-streams with different sizes, thus resulting in more flexible bitstream configurations.

[0047] In an example implementation, the first length indication precedes the first coded bits in the bitstream, and the second length indication precedes the second coded bits in the bitstream.

[0048] An advantage of this bitstream structure may be that it allows for immediate decoding of the substreams without requiring buffering of multiple substreams and their respective indications, which may be achieved to expedite processing flow.

[0049] For example, the second length indication precedes the first coded bit in the bitstream.

[0050] Providing a length indication concatenated in front of the coded bits may allow faster extraction of individual portions of the bitstream.

[0051] For example, an indication of the amount of second leading tail bits precedes the first coded bit in the bitstream.

[0052] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0053] In an example implementation, the method further comprises discarding remaining bits of the bitstream after extracting the first coded bits, the second coded bits, the first leading tail bits, and the second leading tail bits.

[0054] Such an approach may provide a properly aligned bitstream for further processing, such as, for example, encapsulation into a network adaptation layer unit or other packets.

[0055] For example, the arithmetic decoding may include decoding a first sub-stream using a first arithmetic decoder and decoding a second sub-stream using a second arithmetic decoder, where the arithmetic decoding using the first arithmetic decoder and the second arithmetic decoder are performed at least partially in parallel.

[0056] Parallel decoding of the sub-streams may result in faster decoding of the full bitstream.

[0057] In an implementation, the arithmetic decoding is range decoding.

[0058] Range encoding may be particularly suitable for hardware and software architectures that have limited registers or generally fast memory sizes.

[0059] In an implementation, a computer program is stored on a non-transitory medium and includes code instructions that, when executed on one or more processors, cause the one or more processors to perform any of the steps of the methods described above.

[0060] According to an embodiment, an apparatus is provided for arithmetically encoding input data into a bitstream, the apparatus comprising processing circuitry configured to: arithmetically encode the input data into coded bits and tail bits; include the coded bits into the bitstream; determine a minimum and a maximum value of an interval of the arithmetically encoded input data; determine a quantity of leading tail bits that are consecutive tail bits and have the same value in a first MSB representing the determined maximum value as in a second MSB representing the determined minimum value; and indicate the determined quantity of leading tail bits and include the leading tail bits in the bitstream.

[0061] According to an embodiment, an apparatus is provided for arithmetically decoding data from a bitstream, the apparatus comprising processing circuitry configured to: extract from the bitstream an indication of an amount of leading tail bits; extract a plurality of coded bits from the bitstream; extract from the bitstream leading tail bits specified by the indication of the amount of extracted leading tail bits; determine tail bits including appending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; and obtain the data by arithmetically decoding a coded value represented by bits including the coded bits and the determined tail bits.

[0062] The apparatus provides the advantages of the method described above.

[0063] The invention may be implemented in hardware (HW) and / or software (SW), or in any combination thereof. Also, HW-based implementations may be combined with SW-based implementations.

[0064] The details of one or more embodiments are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will become apparent from the description, drawings, and claims. [Brief description of the drawings]

[0065] In the following, embodiments of the invention are explained in more detail with reference to the accompanying figures and drawings.

[0066] [Figure 1] FIG. 2 is a schematic diagram illustrating the encoding of a message using an alphabet having two symbols using arithmetic coding.

[0067] [Diagram 2] FIG. 1 is a schematic diagram of the first stage of arithmetic (range) coding of a message using an alphabet with eight symbols.

[0068] [Diagram 3] FIG. 2 is a schematic diagram of the second stage of arithmetic (range) coding of a message using an alphabet with eight symbols.

[0069] [Figure 4] FIG. 13 is a schematic diagram of the third stage of arithmetic (range) coding of a message using an alphabet with eight symbols.

[0070] [Diagram 5] FIG. 13 is a schematic diagram of a fourth stage of arithmetic (range) coding of a message using an alphabet with eight symbols.

[0071] [Figure 6] FIG. 1 is a schematic diagram of the final stage of arithmetic (range) coding of a message using an alphabet with eight symbols.

[0072] [Figure 7A] FIG. 2 is a schematic diagram illustrating the minimum and maximum values ​​of a range during the coding stage, expressed as binary numbers. [Figure 7B] FIG. 2 is a schematic diagram illustrating the minimum and maximum values ​​of a range during the coding stage, expressed as binary numbers. [Figure 7C] FIG. 2 is a schematic diagram illustrating the minimum and maximum values ​​of a range during the coding stage, expressed as binary numbers. [Figure 7D] FIG. 2 is a schematic diagram illustrating the minimum and maximum values ​​of a range during the coding stage, expressed as binary numbers.

[0073] [Figure 8] FIG. 2 is a schematic diagram illustrating an exemplary arithmetic decoding.

[0074] [Figure 9] 1 is a flow diagram illustrating an exemplary encoding method.

[0075] [Figure 10] 1 is a flow diagram illustrating an exemplary decoding method.

[0076] [Figure 11] FIG. 2 is a block diagram illustrating the employment of arithmetic encoding of some embodiments in image encoding.

[0077] [Figure 12] FIG. 2 is a block diagram illustrating the employment of arithmetic encoding of some embodiments in image decoding.

[0078] [Figure 13] FIG. 2 is a schematic diagram of a single encoder for encoding channels of input data into a single stream.

[0079] [Figure 14] 1 is a schematic diagram of a multi-core encoder for encoding channels of input data into sub-streams and concatenating the sub-streams into a bitstream.

[0080] [Figure 15] 1 is a schematic diagram of a multi-core encoder for interleaving and encoding channels of input data into sub-streams and concatenating the sub-streams into a bitstream.

[0081] [Figure 16] FIG. 13 is a schematic diagram illustrating the determination of identical leading tail bits.

[0082] [Figure 17] A schematic diagram of a bitstream including a length indication, coded bits, and leading tail bits.

[0083] [Figure 18]FIG. 1 is a schematic diagram of the pre-processing to obtain equal-sized channels from the input data, and the shuffling process to form the interleaved and encoded sub-streams.

[0084] [Figure 19] 1 is a block diagram illustrating an example of a video coding system configured to implement embodiments of the present invention.

[0085] [Figure 20] 2 is a block diagram illustrating another example of a video coding system configured to implement embodiments of the present invention.

[0086] [Figure 21] 1 is a block diagram illustrating an example of an encoding device or a decoding device.

[0087] [Figure 22] 1 is a block diagram illustrating another example of an encoding device or a decoding device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0088] In the following description, reference is made to the accompanying drawings which form a part of this disclosure and which show by way of illustration certain aspects of embodiments of the invention or in which embodiments of the invention may be used. It is understood that embodiments of the invention may be used in other ways and may include structural or logical changes not shown in the drawings. Therefore, the following detailed description should not be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.

[0089] For example, it is understood that disclosure related to a described method may also apply to a corresponding device or system configured to perform the method, and vice versa. For example, when one or more particular method steps are described, the corresponding device may include one or more units for performing the described method steps (e.g., one unit performing one or more steps, or multiple units performing one or more of the multiple steps, respectively), e.g., functional units, even if such one or more units are not explicitly described or shown in the figures. On the other hand, for example, when a particular apparatus is described based on one or more units, e.g., functional units, the corresponding method may include one step performing the function of one or more units (e.g., one step performing the function of one or more units, or multiple steps performing one or more of the functions of multiple units, respectively), even if such one or more steps are not explicitly described or shown in the figures. Furthermore, it is understood that the features of various exemplary embodiments and / or aspects described herein can be combined with each other, unless otherwise specified.

[0090] Video coding typically refers to the processing of a series of images forming a video or a video sequence. Instead of the term image, the terms frame or image may be used as synonyms in the field of video coding. Video coding includes two parts: video encoding and video decoding. Video encoding is performed on the source side and typically involves processing the original video image (e.g., by compression) to reduce the amount of data required to represent the video image (for more efficient storage and / or transmission). Video decoding is performed on the destination side and typically involves the reverse processing compared to the encoder to reconstruct the video image. An embodiment referring to "coding" a video image (or a general image, as described later) should be understood to relate to both "encoding" and "decoding" of the video image. The combination of the encoding and decoding parts is also referred to as a codec (coding and decoding).

[0091] In the case of lossless video coding, the original video image can be reconstructed, i.e. the reconstructed video image has the same quality as the original video image (assuming there are no transmission errors or other data losses during storage or transmission). In the case of lossy video coding, further compression, e.g. by quantization, is performed to reduce the amount of data representing the video image. The video image cannot be completely reconstructed at the decoder, i.e. the quality of the reconstructed video image is low or poor compared to the quality of the original video image.

[0092] Arithmetic Encoding Entropy coding is typically employed as lossless coding. Arithmetic coding is a type of entropy coding that encodes a message as a binary real number within an interval (range) that represents the message. In this specification, the term message refers to a sequence of symbols. The symbols are selected from a predefined alphabet of symbols. For example, the alphabet can consist of two values ​​0 and 1. A message using such an alphabet is therefore a sequence of bits. The symbols (0 and 1) can occur in a message with mutually different frequencies. In other words, the symbol probabilities can be non-uniform. Indeed, the more non-uniform the distribution, the higher the compression achievable by entropy coding in general, and arithmetic coding in particular. Arithmetic coding uses an a priori probability model that specifies the symbol probabilities for each symbol of the alphabet.

[0093] The alphabet need not be binary. Rather, the alphabet may consist of eight values, for example, from 0 to 7. In general, any alphabet having any size may be used. Typically, the alphabet is given by the value range of the coded data.

[0094] The interval representing the message is obtained by separating the initial range according to the probability of the alphabet symbols on which the message is coded.

[0095] For example, let the current interval be the initial interval [0,1). For each symbol of the message, the following two steps are performed: 1) Subdivide the current interval into subintervals, one for each possible alphabet symbol. The size of a symbol subinterval is proportional to the estimated probability that the symbol is the next symbol in the message according to a probability model (of the symbol source). 2) Select the subinterval that corresponds to the next actually occurring symbol in the message, and make the selected subinterval the new current interval.

[0096] As a third step, enough bits are output to distinguish the current interval from all other possible intervals. This step can be performed already during the encoding in steps 1 and 2, or after the encoding of the entire message. The length of the interval obtained after repeating steps 1) and 2) for all symbols of the message is obviously equal to the product of the probabilities of the individual symbols, which is also the probability of a certain sequence of symbols in the message.

[0097] In theory, an arithmetic coder would encode a message of any length by recursively separating the interval from 0 to 1, resulting in smaller and smaller intervals. In practice, the system is limited by a finite bit depth, where only discrete values ​​can be represented. Smaller intervals would therefore require more precise arithmetic. Also, no output would be produced until the entire message has been read. A solution to both of these problems could be to output some bits as soon as they are known, doubling the length of the current interval for each output bit, so that it reflects only the (yet) unknown part of the interval. In practice, the arithmetic could be done by storing the current interval in a sufficiently long integer, rather than in floating point or strictly rational numbers.

[0098] A variant of an arithmetic coder, improved for practical use, which uses a finite range of integers, e.g. 0 to 255, rather than the interval [0,1), is called a range coder. The range is separated according to the probabilities of the alphabet symbols. In order to describe all the alphabet symbols according to their probabilities, the ranges can be renormalized if the remaining ranges become too small.

[0099] In terms of the terminology employed herein, the current interval is given by its minimum value (denoted as LOW) and its maximum value (denoted as HIGH). The length of the interval is denoted as RANGE. In general, HIGH=LOW+RANGE, where RANGE is expressed as the number of (distinguishable) subintervals of minimum size. The minimum range for encoding the symbols is BOTTOM. This minimum range ensures that even the least likely symbols have at least one valid range. In other words, BOTTOM is a design parameter that can be determined based on the alphabet symbols and their probabilities to ensure that the range can still be divided into distinguishable intervals corresponding to all alphabet symbols.

[0100] The HIGH position indicates how many bits are needed to cover the initial maximum range. The BOTTOM position indicates how many bits are needed to cover the minimum range for encoding another symbol. The amount of bits between the HIGH position and the predefined TOP position corresponds to the smallest portion of bits that can be streamed (inserted) into the bitstream and become coded bits.

[0101] Figure 1 illustrates in schematic form an exemplary procedure for arithmetic coding. The message to be coded is provided in an alphabet with two possible symbols A and B, where the symbols {A,B} of the alphabet have probabilities P(A) = 2 / 3 and P(B) = 1 / 3. The message to be encoded can be read as AABA.

[0102] Since LOW=0 at the beginning (at stage 0, 0 =0), where stage 0 is the length Range 0 The initial interval has a value of 110. 0 In the lower half of Range 0 The leading bits of the binary representation of the numbers in are 1 111, while the leading bits of the numbers in 0In the upper half of the initial range, the leading bits are 0 112. In other words, a current interval in any of the stages that is in the upper half of the initial range has a first leading bit of 0, while a current interval that is in the lower half of the initial range has a first leading bit of 1. Now, for example, assuming that any number (code value) in the initial range can be represented by 8 bits, the range is 0 to 255, so that HIGH=255.

[0103] In stage 1, Range 0 According to the probability, the first symbol A 0 In this example, the initial interval is divided into 1 / 3 122 (symbol B 0 ) and 2 / 3 121 (corresponding to symbol A 0 The division according to probability means that the initial interval is divided into a number of subintervals equal to the number of symbols in the alphabet (here, two symbols A and B), with the size of the intervals proportional to the respective probabilities of the symbols represented by the interval. 0 The upper subinterval Range corresponding to 1 =P(A 0 )*Range 0 is then selected as the current interval for the next step.

[0104] In stage 2, symbol A 0 Range 1 120 is divided according to probability. The next message symbol is A. Message A 0 A 1 The remaining range describes 131. 2 =P(A 1 )*Range 1 is a Range encoded with bit 0. 0Since the current maximum range is entirely within the upper half of stage 0, adding bit 0 to the encoded bitstream is performed. In this particular implementation, bits are added to the bitstream as soon as possible and a renormalization 140 is performed to double the resolution, where the current maximum range is the upper half of the initial range of stage 0. The upper half of this current maximum range is assigned to bit 0 and the lower half of the current maximum range is assigned to bit 1.

[0105] Message A in Phase 3 0 A 1 B 2 141 cannot yet be encoded clearly because Range 3 This is because the range overlaps bit 1 (the corresponding lower half of the current maximum range) and bit 0 (the corresponding upper half of the current maximum range), and therefore a carry 150 is performed. Nothing is streamed (no bits are included in the bitstream). 3 is divided in step 4 according to the probabilities of the symbols in the alphabet.

[0106] Here, the message to be encoded is A 0 A 1 B 2 A 3 It can be read as 151. Range 4 still overlaps both possible bits and there are no further symbols to be encoded, so finalization 160 is performed in steps 5 and 6. This involves several renormalizations being performed to create the unambiguous code 0011 for the message AABA.

[0107] 2 to 7 show an example for a range encoder in which the 8 symbols 220 of the alphabet {0,1,2,3,4,5,6,7} have probabilities according to the probability density function PDF 230 for a standard normal distribution. The symbols are mapped onto the range by the cumulative distribution function CDF 240. The encoded message reads 4420. The maximum starting range for the encoding HIGH is represented in this example by 8 bits corresponding to a code value of 255. The minimum range for encoding the symbols is BOTTOM=16 in this example. This minimum range ensures that the least likely symbols "0" and "7" still have a valid range of at least 1 when the cumulative distribution function is applied. For example, a normal distribution results in the following number of minimum intervals per 8 symbols: 1,1,2,3,4,2,2,1.

[0108] Figure 2 shows the initial range 210 0 2. Range 255 = HIGH = 255 = 1111 1111.b, where b is the binary representation of the number, which is mapped onto the symbols of the alphabet 220 according to a Gaussian probability density function PDF 230. 0 The partitioning of is obtained from the cumulative distribution function CDF 240. This is because the first symbol of the message to be encoded, i.e. "4", falls at the lower end 250Low 1 =71=0100 0111.b and Range 1 =128 corresponds to the upper endpoint to be excluded 260 = Low 1 +Range 1 =199=1100 0111.b. Sum=Constant=16 270 implies an operating precision of 1 / 16. This indicates that the subrange assigned to the least likely symbol is 1 / sum=1 / 16 of the current range. The sum 270 is determined by the cumulative distribution function.

[0109] The binary representation of this interval is shown in FIG. 7A with the indications of HIGH, TOP and BOTTOM positions. The HIGH position 740 indicates how many bits are needed to cover the initial maximum range (in this case 8). The BOTTOM position 760 indicates how many bits are needed to cover the minimum range (in this case 4) for encoding another symbol. The amount of bits between the HIGH position 740 and the predefined TOP position 750 corresponds to the smallest part of bits that can be streamed (inserted) into the bitstream and then become coded bits. For the arithmetic coding described above with reference to FIG. 1, there is only one bit between the HIGH position 740 and the TOP position 750. For the range coding of this example, there are two bits between the HIGH position 740 and the TOP position 750. In FIG. 7A, (and also in FIGS. 7B-7D), the minimum value of the interval 720a-d and the maximum value of the interval 710a-d are represented in binary. This may actually correspond to two registers used to store the current interval and therefore the current result of the encoding.

[0110] In FIG. 3, the cumulative distribution function CDF340 is Low 1 Range starting from =71 1 = 128 to map the symbols of alphabet 220 onto the range. The next symbol in the message is "4". This results in a new current interval 350-360, which represents message 44. This interval, shown in FIG. 7B, is now bounded by a new lower limit (current minimum) 720b Low 2 =106=0110 1010.b and new upper limit (current maximum value) 710b Low 2 +Range 2 =170=1010 1010.b. New Range 2 320 is equal to 64.

[0111] 4 and 7C show the next steps in the encoding procedure. 2Since 320 is still greater than BOTTOM, the range is again divided according to the cumulative distribution function 440. This results in a low 3 =111 =0110 1111.b Low value 720c and Range 3 =4 410 occurs. Therefore, the message 442 is 3 ~Low 3 +Range 3 =115=0111 0011.b is represented by the range 710c.

[0112] Since the two bits 730 between the HIGH and TOP positions are equal, they can be output as coded bits in the stream 731. See Figure 7D.

[0113] Figure 5 shows the Range 3 7D illustrates that proper mapping of symbols onto the range is not possible because BOTTOM=4 410 is less than BOTTOM=16, and therefore a renormalization procedure is necessary. 4 indicates that the entire bit representation is shifted left until is greater than or equal to BOTTOM. 4 =(4<<2)=16 610 is reached. A new upper limit 710e is obtained from the current one 710d after two left shifts. 4 =188=1011 1100.b The same shift, except for the two coded bits, is Low, which is part of 720d. 3 applies to.

[0114] Here, message 442 is encoded with any value between 444=01 1011 1100.b and 460=01 1100 1100.b.

[0115] Figure 6 shows the encoding of the last symbol "0". The probability distribution is 5 =188=1011 1100.b 650 and Range 5= 1 620. Since there are no more symbols to be encoded, 5 and Range 5 describes the range interval of the tail bits which (together with the coded bits) represent the encoding message. In general, any value from the interval can be used (included in the bitstream) to represent the interval and thus also the coded message (sequence of symbols). Thus, the tail bits can be arbitrarily selected from this final range interval. In this example, Range 5 =1 620 is a single value for the last bit, i.e., Low 5 =188=1011 1100.b 650. Thus, message 4420 is encoded by appending the tail bits to the coded bits, resulting in coded value 444=01 1011 1100.b.

[0116] Arithmetic Decoding 8 shows an exemplary decoding process. The decoder receives the coded values ​​(bits) sequentially. The received coded values ​​820 are 0 =HIGH=255 810. The probability distribution function is known at the decoder side, and therefore the mapping of the symbols 220 onto the ranges according to the cumulative distribution function 830 is known. The decoder does not know the inverse of this mapping, so a search process is required to determine the symbol. The decoder makes a first guess 840 about the encoded symbol by selecting the most likely symbol "4", calculates the low value 841 of the range corresponding to this symbol and checks whether the received coded value 820 is higher than this low value 841.

[0117] Because the received coded value 820 is less than the low value 841 of the first guess 840, the next guess 850 is one of the symbols that maps to a lower value in the range, i.e., symbols "0", "1", "2", or "3". Selecting a low value approximately in the middle of the remaining interval of symbols results in a faster decoding process because fewer steps are required to obtain the correct symbol. In this example, the next low value 851 selected for comparison corresponds to the symbol "2".

[0118] The test for the symbol "2" results in the received coded value being higher than the low value 851 of the range, which encodes the symbol "2". Thus, the received coded value may represent the symbols "2" or "3". A final check 860 reveals that the low value 861 of the range, which corresponds to "3", is higher than the coded value 820. Therefore, the received coded value 820 is decoded as the symbol "2".

[0119] Multi-Channel Entropy Coding FIG. 13 shows a scheme of a single encoder 1320. Here, the term "single encoder" refers to the fact that the encoder operates serially, i.e., it encodes the input sequentially, not in parallel. The input data may consist of or include multiple channels 1310. Thus, in this exemplary encoding process, portions of each of the channels are encoded sequentially. A first portion size portion from each of the channels 1330 is encoded (resulting in a multiplexing of identical first size portions from different channels), followed by a second portion size portion from each of the channels 1340 (resulting in a multiplexing of identical second size portions from different channels). Tail bits 1350 remaining in the encoder after encoding of the last portion are appended to the end of the main single stream.

[0120] In a single (or single-core) entropy encoder, there is only one finalization stage at the end of the coding, where the stream is padded with 0 bits to make the bytes aligned. There is no problem in signaling a few extra bits. However, it is difficult to parallelize such encoding, and correspondingly, to parallelize the decoding.

[0121] 14 shows an example scheme of a parallel (e.g. multi-core) encoder 1420. Each of the input data channels 1410 may be encoded into an individual sub-stream comprising coded bits 1430-1433 and tail bits 1440-1443. The length of the sub-stream 1450 is signaled.

[0122] In a parallel processing implementation, the bitstream consists of multiple substreams that are concatenated in a final stage. Each of the substreams needs to be finalized. The reason for this is that the substreams are encoded independently of each other, such that the encoding (and therefore the decoding) of one substream does not require the prior encoding (or decoding) of another substream or substreams.

[0123] Finalization of entropy encoding, particularly arithmetic encoding, may involve encoding one or more of the tail bits into the bitstream and / or padding to the nearest byte boundary or boundary of a predefined amount of bits. However, when multiple substreams are encoded in parallel, the padding may result in the inclusion of a large amount of meaningless padding bits. This problem may be solved if the number of tail bits in each thread is not rounded to the nearest byte boundary, but the most significant leading bits are determined among the tail bits and their amount is specified in the bitstream.

[0124] This is exemplarily shown in Figure 15, where tail bits 1540-1543 are added to the bitstream 1570 directly after the coded data 1530-1533. The length of tail bits 1560 is also included in the bitstream 1570. The multi-core encoder in this example embodiment multiplexes and optionally shuffles portions of multiple input data channels 1510 before encoding the portions into multiple sub-streams.

[0125] It should be noted that a complete partial multiplex for generating a substream does not need to be formed prior to encoding the substream. Conversely, the entropy encoder may directly receive the parts, piece by piece, from the different channels and process them into a substream. The term shuffle refers to the sequence of parts within a multiplex (and, consequently, within a substream after encoding the multiplex).

[0126] An input data channel may refer to a channel obtained by processing some data by a neural network. For example, the input data may be a feature channel, such as an output channel or a latent representation channel of a neural network. In an implementation, the neural network may be a deep neural network and / or a convolutional neural network, or the like. The neural network may be trained to process (still or video) images. The processing may be for image encoding and reconstruction, or computer vision, such as object recognition, classification, segmentation, or the like. In general, the present disclosure is not limited to any particular type of task or neural network. The present disclosure is readily applicable to encoding and decoding channels of a neural network, but it is not limited to such application. Rather, the present disclosure is applicable to encoding any kind of data coming from multiple channels, generally understood as any source of data. Also, the channels may be provided by pre-processing of the source data.

[0127] FIG. 18 exemplarily illustrates pre-processing of input data channels 1810, denoted by Ch0, Ch1, Ch2, and Ch3, with different sizes, along with their corresponding probability distributions 1820 (denoted as ProbCh0, ProbCh1, ProbCh2, and ProbCh3). The term size here refers to the number (quantity) of bits, symbols, or elements of the channel. In general, channels such as neural network channels may have more dimensions, such as vertical and horizontal, but these typically do not play a role in entropy coding, which is performed serially (sequentially) on channel elements. Depending on the entropy coding and channel type, encoding may be based on channel bits, channel symbols, or generally on channel elements (e.g., a probability model may be provided). However, the present disclosure may also be applicable to separating channels into new channels of the same size with more than one dimension.

[0128] These input channels 1810 of different sizes are pre-processed to obtain channels of the same size 1830. Thus, input channels larger than the required size can be separated 1812. As can be seen from Fig. 18, for example, at least one of the channels (here, two channels Ch1 and Ch2) is separated into two channels (e.g., ch0a and ch0b, and ch2a and ch2b). The separation can occur in any conceivable manner. For example, the channel is divided into n pieces (n is an integer larger than the contiguous parts). Alternatively, the pieces are not contiguous, but rather are formed by repeatedly assigning each symbol or each k symbols from the channel to one or a new channel (interleaving the channel parts into the new channel).

[0129] The corresponding probability distributions of the separated channels Ch0 and Ch2 are fitted to new channels ch0a, ch0b, ch2a, and ch2b of the same size. In other words, the new channels (e.g., ch0a and ch0b) may have distributions that are different from the distribution of the original channel (e.g., Ch0) from which they were derived.

[0130] If a channel (e.g., Ch3) is smaller than the same size, it may be padded with 0's 1813, resulting in a new channel of the same size after separation. Alternatively, padding may be performed after separation. For example, if the last part of a separated channel is smaller than the same size, it may be padded with 0's as shown in Figure 18. However, padding does not have to be done only on the last new channel, but may be inserted in more new channels, e.g., distributed among the new channels.

[0131] For example, a channel (such as Ch0 and Ch2) may be separated into n new channels, but may have a size that is not divisible by n (e.g., not divisible by 2 in the case of FIG. 18). In such a case, it is not possible to form n channels of the same size using only data from a channel (Ch0 or Ch2). To overcome this problem, the channel or one or more of the n new channels may be padded. Note that in some implementations, there may be additional reasons for padding. For example, data from a channel (Ch0 or Ch2) may be separated into new channels (ch0a, ch0b, ch2a, and ch2b) based on symbols or other sizes, such as bytes or symbols, rather than on a bit basis. Thus, the channel size in units of symbols, rather than in units of bits, needs to be divisible by n.

[0132] Padding with zeros is just one exemplary option. Padding can be performed with bits or symbols of any value. It can be padded with repetition of channel bits or symbols, or the like.

[0133] At the decoder side, the new channel is decoded. The padding shall be removed at the decoder side to form channels of different sizes (e.g. Ch0 and Ch2). To achieve this, the decoder needs information about these pre-processing steps. For example, the decoder is configured to remove the padding based on knowledge of the sizes of the different sized channels. For example, the channel sizes may be defined by a standard or may be configured by side information or the like. The information about the pre-processing steps may also include the size of the new channel.

[0134] Following pre-processing, each of the channels of the same size is divided into parts. In a first stage, parts 1840 of a first size are obtained from each of the channels, and these parts (possibly with corresponding distributions) are multiplexed. This multiplexing is entropy encoding into a first sub-stream 1530. Furthermore, parts 1841 of a second size from each channel are multiplexed and then encoded into a second sub-stream 1531. The first size and the second size can be the same or different. The entropy encoding is performed separately to generate the first sub-stream 1530 and to generate the second sub-stream 1531. As mentioned above, the channel parts can be multiplexed with respective probability model side information. In an exemplary embodiment, such side information can correspond to a hyper-prior obtained by a modified auto-encoder with a hyper-prior sub-network. However, there can be embodiments and implementations in which a probability model does not need to be provided as side information multiplexed with the channel parts. For example, the probability model may be updated in a context-adaptive manner based on previously encoded and / or decoded data or the like.

[0135] Multiplexing and encoding portions from different channels, rather than encoding each channel separately, offers the possibility to produce a complete sub-stream faster and / or to control the length of the individual sub-streams, e.g., to obtain more uniform lengths of the sub-streams, which opens up the possibility to perform entropy encoding and / or entropy decoding on multiple sub-streams in parallel.

[0136] The present disclosure is not limited to providing only portions of the same size, or portions of a first size and a second size different from the first size. In addition, portions of other sizes, such as a third size 1842 and / or a fourth size 1843, may be included in the first or second sub-stream. This third size may be equal to the first or second size. This is illustrated in FIG. 18, where the i-th channel ch i is split into four parts: part.i0, part.i1, part.i2, and part.i3. The substream size can be controlled by splitting all channels in parts of the same size and then forming substreams that contain parts from different channels. This can be called dynamic partitioning.

[0137] For example, the individual substreams can extend their length by including more portions and still be adjusted to the desired length. The multiplexing of portions to form the first substream and the second substream may have the same size, but after entropy encoding, the first substream and the second substream may have different sizes. It may therefore be desirable to adapt the substream sizes to reduce their discrepancy, for example by configuring the size and / or number of portions to include in the multiplex. For example, if there are fewer individual substreams that are longer, fewer length indications need to be signaled or less padding is required.

[0138] The entropy encoding can be arithmetic encoding or range encoding. In these cases, the encoding results in coded bits 1530-1533 and tail bits 1540-1543 (if present). These tail bits are the status of the encoder after encoding the last part. Note that the present disclosure is not limited to incorporating coded bits and tail bits separately in the bitstream. There may be encoders that output all bits as coded bits in the bitstream. The present substream based on independent encoding and decoding is also applicable to such encoders and decoders.

[0139] The substreams are multiplexed into a bitstream 1570 with substream length indications 1550 indicating the length of each substream.

[0140] In an embodiment (as shown in FIG. 15), a substream length indication 1550 precedes each substream in the bitstream 1570. In addition, a second substream length indication may also precede the first substream 1530, i.e., the multiple substream length indications are concatenated and included in the bitstream 1570 before the multiple substreams. The multiple substreams may be some or all of the substreams generated from a channel, for example, associated with an image or image portion, or a predefined number of images, or another container of channel data. How many substreams are included in the plurality may also be configurable or fixed to a predefined number that may, but need not, correspond to the number of channels of the same size.

[0141] In addition, the tail bits may be signaled with the length indication. There are tail bits for each substream, namely a first tail bit and a second tail bit. Thus, a first tail bit length indication and a second tail bit length indication may be included in the bitstream 1570. A tail bit length indication 1560 may precede their respective substreams in the bitstream 1570. A second tail bit length indication may also precede the first substream 1530. Thus, the concatenated tail bit length indication 1560 precedes the first substream 1530. The order of the substream length indications and tail bit length indications is predefined, so that both the encoder and the decoder are able to form and parse the bitstream adaptively.

[0142] Including all of these length indications in the bitstream avoids padding of individual substreams and / or trailing bits. In particular, in some implementations padding may be performed after multiple substreams and corresponding indications may be included in the bitstream. However, this disclosure does not require padding, as there may be bitstream structures or protocols that do not require such alignment of bits or symbols to a particular raster.

[0143] Instead of concentrating the instructions for the multiple substreams, in an embodiment, the tail bits of the first substream 1540 may be included in the bitstream following the first substream 1530, and the tail bits of the second substream 1541 may follow the second substream 1531 in the bitstream 1570. An advantage of this bitstream structure may be that it allows for immediate encoding or decoding of the substreams without requiring buffering of multiple substreams and their respective instructions. Additionally, the first tail bits 1540 may follow the second substream 1531.

[0144] A bitstream 1570 constructed as in any of the above example embodiments may be padded to align the bitstream length to coincide with an integer multiple of a predetermined amount of bits, such as a byte, word, or doubleword, or the like. Such an approach may provide a properly aligned bitstream for further processing, such as, for example, encapsulation into a network adaptation layer unit or other packet.

[0145] The data in the multiple channels 1510 consists of symbols, e.g., bits. The total portion included in the first substream 1530 and the total portion included in the second substream 1531 may be an integer number K times these symbols, where K is greater than 1.

[0146] To achieve a more uniform distribution of substream sizes, the parts can be shuffled. Shuffling has already been mentioned with reference to Fig. 18. It corresponds to interleaving of parts performed synchronously on the encoding and decoding sides, i.e. using the same rules on the encoding and decoding sides to ensure compatibility between encoding and decoding. A shuffling method is selected from a set of predefined methods and then applied to specify the order of the parts of the first size and the parts of the second size. The parts 1860 (e.g. part 0a0, part 0b0, part 1a0, part 2a0, part 2b0, and part 3a0) are shuffled synchronously. The exemplary scheme in Fig. 18 shows a cyclic permutation 1850 of the parts. Also, the corresponding probability distributions 1861 (e.g., Prob prt.0a0, Prob prt.0b0, Prob prt.1a0, Prob prt.2a0, Prob prt.2b0, and Prob prt.3a0) associated with each portion are applied for entropy coding (e.g., arithmetic coding). Thus, the portions and associated probability portions (models) can be considered as being synchronously shuffled.

[0147] The shuffling (and possibly the shuffling method selection) may be performed iteratively on the encoded portions, between the first encoded substream and the second encoded substream, generally to reduce the difference in length between the substreams over time. Thus, the shuffling method may take into account the difference between the length of the current (e.g., first or second) substream and a statistic based on the length of the past substreams, and / or the difference between the length of the current substream and a statistic based on the length of the past substreams. Such a shuffling method may result in any arbitrary shuffling 1852 of the portions, i.e., any possible reordering.

[0148] This may involve collecting statistical measurements of past substreams to obtain a distribution of the lengths of the past substreams, which can be obtained and used simultaneously on the encoder and decoder side, since both sides have previously processed the same substreams.

[0149] The statistics may be based, for example, on an estimated mean, median, minimum, or maximum of past and / or current sub-stream lengths, or the rate of increase in the sub-stream length, or the rate of decrease in the sub-stream length, or a combination thereof, or another statistical measure (estimator).

[0150] If the entropy encoding is arithmetic encoding, the shuffle method may also take into account the current state of the range interval in the arithmetic encoding process.

[0151] The current state of the range interval may also provide detailed information of the substream length and allow for an estimation of the rate of growth of the substream based on a particular channel. In particular, when the current state of the range interval is small, it is an indication that the substream length is larger, and vice versa. Based on such an estimation of growth, an appropriate shuffle method may be selected.

[0152] As mentioned above, a proper shuffle may provide several advantages, as illustrated in the following example. Now, assume that the shuffle may be a circular rotation (cyclic permutation) of the channels. In this example, there are three channels Ch1, Ch2, Ch3, each with a different rate of growth of the substream size (also referred to herein as length). The substreams grow by 10 bytes per portion from Ch1, 2 bytes per portion from Ch2, and 20 bytes per portion from Ch3. Thus, after first encoding the three portions in parallel into substreams S1, S2, and S3 without shuffling, the following stream lengths are achieved: The first portion from each channel is encoded without shuffling, in the order Ch1, Ch2, and Ch3. This results in three parallel stream sizes of 10, 2, and 20 bytes for S1, S2, and S3, respectively. If the second portions are encoded in the same order of Ch1, Ch2, and Ch3, with no shuffling, then parallel stream sizes of 20, 4, and 40 bytes are achieved after including the second portions from each of the three channels. If the third part is encoded in the same order of Ch1, Ch2, and Ch3, but without shuffling, then parallel stream sizes of 30, 6, and 60 are obtained after including the third part from each of the three channels.

[0153] As can be seen in this example, the parallel substreams are of significantly different lengths, which may be undesirable for some applications. To remedy this situation, a shuffle may be performed. In particular, the order in which the portions are taken from each channel may be changed.

[0154] Let us assume the same channels Ch1, Ch2 and Ch3 as mentioned above, with the same growth rate. If a shuffle by circular shift of the channel order Ch1, Ch2, Ch3 is performed, the following results are obtained. The first portion is now taken from the three channels in the same order as in the example above, i.e. Ch1, Ch2, Ch3, resulting in identical parallel stream sizes of 10, 2 and 20 bytes. The second part is obtained in shuffled order, specifically in circularly shifted (one shift to the right) order, Ch3, Ch1, and Ch2. This results in stream sizes of 30, 12, and 22 bytes, respectively. These stream sizes are obtained by adding the sizes of 20, 10, and 2, corresponding to the shuffled channels, to the streams of length 10, 2, and 20 bytes from the previous stage. As can be seen, the sizes after shuffling 30, 12, and 22 after the second part show lower variation than the sizes from the previous example where 20, 4, and 40 bytes are not shuffled. The third portion is obtained again in a shuffled order, in this example by repeated cyclic shifts to the right, resulting in the order of channels Ch2, Ch3 and Ch1. The resulting stream sizes of the three parallel streams are 32, 32 and 32, which is the result of adding 2, 20 and 10 more bytes to the stream sizes of 30, 12 and 22 bytes of the previous stage (adding the second portion), respectively. As can be seen, after the second shuffle the lengths of the parallel streams (corresponding to the sub-streams described above) are equalized.

[0155] In practical applications, the growth may not be so easy and may not be accurately estimable. In particular, it is not necessarily stationary as in the above example. Nevertheless, shuffling may improve the equalization of the substream lengths (sizes). To do so, an estimation of the growth rate may contribute to performance. As explained above, the growth of the (encoded) substream may be estimated based on the previously coded (decoded) parts or substreams. However, the current state of the range interval may provide a closer indication. If the range interval is large, the substream length is smaller, indicating a lower contribution to the growth rate. If the range interval is small, a larger length of the substream is indicated, which corresponds to a larger contribution to the growth rate. In other words, the length of the range interval is inversely proportional to the growth rate of the stream. The proportionality is not necessarily linear.

[0156] Therefore, during encoding and decoding, a shuffling mechanism may be applied. The shuffling may be similar to that described above. For example, after encoding (or decoding) the kth portion from each channel in the kth channel, the (k+1)th portion from each channel is encoded in the (k-1)th channel. In an implementation example, the (k+1)th is obtained from the kth by cyclically shifting the kth. The cyclic shift may be right or left. It may be advantageous if the shift is one channel. However, the present disclosure is not limited thereto, and the step of the shift may be different from 1. As already mentioned above, the shuffling order may also be specifically selected and signaled.

[0157] In another exemplary embodiment, the portions of a channel that are encoded into a substream with a larger length and a higher growth rate can be exchanged, i.e. shuffled, with the portions of the channel that are encoded into a substream with a smaller length and a lower growth rate. In particular, the method can determine the length of the substream and the growth rate of the substream. Based on the result of this determination, the method shuffles the portions. For the above example, this corresponds to the exchange of channel Ch2 and channel Ch3. This exchange encodes the portion from Ch3 that contributes to a higher growth rate (20 bytes per portion) into a substream S2 of smaller length. The portion from Ch2 that contributes to a lower growth rate (5 bytes per portion) is encoded into a substream S3 of larger length. This results in an increase in the growth of S2 and a decrease in the growth of S3. Thus, the above mentioned difference in length between the encoded substreams can be reduced. In practical applications, the growth may not necessarily be constant as in the above example.

[0158] The entropy encoding into the first sub-stream 1530 and the second sub-stream 1531 may be performed in parallel, for example by one of the multi-core encoders 1420, 1520 in Figures 14 and 15. In some implementations, only a portion of the entropy encoding may be performed in parallel.

[0159] The decoding method involves entropy decoding the first substream 1530 and the second substream 1531 into multiplexed parts. Entropy decoding may be performed separately for the first and second substreams. The multiple channels 1810 may be obtained from demultiplexing the first size portion 1840 and the second size portion 1841. The first size and the second size may be identical. The channels of the same size 1620 may be post-processed to obtain multiple channels of different sizes 1810. This post-processing involves adding (concatenating) the separated channels. The zero padded channels are clipped, i.e. the padding is removed, to obtain the input data. Information about these stages may be included in the same size channels. The channels of any size may be output channels or latent representation channels of a neural network.

[0160] The entropy decoding may be, for example, arithmetic decoding or range decoding, as described in the arithmetic encoding section above. However, this disclosure regarding the formation of sub-streams is not limited to the application of arithmetic encoders and decoders. Rather, any entropy coding and / or run-length coding, or the like, may be applied to the channel data. The entropy coding may be context-adaptive. These embodiments correspond to the encoding examples described above.

[0161] The substreams are extracted from the bitstream 1570 with a first substream length indication indicating the length of the first substream and a second substream length indication indicating the length of the second substream. For example, the substreams may be extracted when the length indications of the substreams are known prior to extraction.

[0162] Thus, the sub-stream length indications 1550 may precede their respective sub-streams in the bitstream 1570. In addition, the second sub-stream length indication may also precede the first sub-stream 1530.

[0163] By signaling the concatenated length indications before the sub-streams, the decoder can extract the sub-streams simultaneously.

[0164] A first tail bit length indication indicating the length of the first tail bit and a second tail bit length indication indicating the length of the second tail bit may be extracted from the bitstream 1570. The tail bit length indication 1560 may precede their respective substreams in the bitstream 1570. The second tail bit length indication may also precede the first substream 1530.

[0165] The tail bits of the first substream 1540 may be included in the bitstream following the first substream 1530, and the tail bits of the second substream 1541 may follow the second substream 1531 in the bitstream 1570. Additionally, the first tail bits 1540 may follow the second substream 1531.

[0166] The data of the multiple channels 1510 consists of symbols, which may be bits. The total portion decoded and demultiplexed from the first substream 1530 and the total portion decoded and demultiplexed from the second substream 1531 may be an integer number K times these symbols, where K is greater than 1.

[0167] After extraction of the first substream length indication, the second substream length indication, the first tail bit length indication, the second tail bit length indication, the first substream 1530, the second substream 1531, the first tail bit 1541, and the second tail bit 1541, there may be remaining bits. The bits remaining in the bitstream 1570 after extraction of the last tail bit are the result of padding the entire bitstream 1570 to match a predetermined integer number of bits to provide a properly aligned bitstream for further processing, such as for encapsulation into a network adaptation layer unit or other packet. The remaining bits may be discarded.

[0168] During the multiplexing of parts in the encoding process, there may be a shuffle method applied. For example, this method is signaled in the control information included in the bitstream. Thus, the decoder may parse the shuffle method indication from the bitstream. The indication may be, for example, an index to a list of shuffle methods that may be defined by the standard or that may be configurable according to the standard and signaled in the bitstream, possibly less frequently than the shuffle method indication itself.

[0169] Thus, a shuffle method may be determined from a predefined shuffle method setting and applied to demultiplex the first size portions 1840 and the second size portions 1841. The shuffle method specifies the order of the portions within the substreams. In addition, the shuffle method may also define the portion lengths of the substreams in implementations where the portion lengths may vary. In some embodiments, the portion lengths may be the same across channels and / or across substreams.

[0170] The entropy decoding may be performed in parallel, for example by a multi-core decoder. In addition, only a portion of the entropy decoding may be performed in parallel.

[0171] Manipulating trailing bits 9 is a flow diagram illustrating an exemplary method for arithmetically coding input data into a bitstream 1750. An example of the encoded bitstream 1750 is provided in FIG.

[0172] The method may initialize an initial range S910 used by the encoder. For example, such an initial range may correspond to the initial range 210 in FIG. 2, as described above. The encoding starts with the first symbol of the message to be encoded and proceeds through all symbols of the message to obtain coded bits.

[0173] In step S920, the current symbol from the message is coded using arithmetic coding, for example as described with reference to Figures 2 to 7.

[0174] After the coding loop S920-S950, the coded bits are included in the bitstream 1570. However, the tail bits still remain in the registers indicating the status of the encoder. The interval describing the tail bits is the current range remaining after encoding the last symbol and streaming the coded bits 1730-1731. The tail bits which together with the coded bits 1730-1731 form the arithmetic coded data can generally be arbitrarily selected from this interval.

[0175] However, the tail bits may be selected to maximize the amount of trailing zeros in the leading tail bits. These trailing zeros may not be included in the bitstream 1750 and therefore may be clipped.

[0176] The determined minimum and maximum values ​​S960 of the interval may include the amount of most significant bits MSB that are identical. These identical leading bits are consecutive trailing bits, i.e., they form a set of consecutive trailing bits. The amount of these leading trailing bits 1740-1741 may be determined S970.

[0177] An indication of the determined amount of these identical leading and trailing bits may be included in the bitstream S980.

[0178] The amount of leading trailing bits, NumTrailingBits, is determined as follows: NumTrailingBits=CLZ((LOW+RANGE-1)XOR LOW), where CLZ() is the number of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval. XOR denotes an exclusive logical OR operation.

[0179] An example of determining identical leading bits 1740-1741 and their quantity is illustrated in Fig. 16. After encoding the last symbol and including the coded bits 1730-1731 in the bitstream 1750, 16 trailing bits remain in the encoder. The trailing bits are represented by the current minimum 1610Low=1123 and the range of the current interval 1620Range=67. In Fig. 16, these values ​​are given in binary representation. The current maximum 1630High-1=(Low+Range-1)=(1123+67-1) is determined from the current minimum 1610 and the current range 1620. (High-1) XOR Low 1640 results in a 0 bit in positions where the bits in LOW 1610 and HIGH-1 1630 are identical, and a 1 bit otherwise.

[0180] A leading zero in this value 1640 indicates the same leading bit in the tail bit. Thus, the number of leading zeros, CLZ(), yields CLZ((low + range-1) XOR low) = 8.

[0181] In this example, there are 8 identical leading bits. The last bit in the current interval 1650 is selected as a value between Low and High-1 by making all bits after the first bit that differs in Low and High-1 0, thus yielding 0b.0000.0100.1000.0000 1650.

[0182] As mentioned above, the trailing 0 1680 may be clipped along with bit 1 1670, which is the first bit that differs in Low and High-1, because this bit is always 1 and does not need to be signaled. Thus, the leading trailing bits 1660 in this example are formed by the 8 bits 0000 0100.

[0183] The determined number of leading zeros is included in an indication of the amount of leading bits, which precedes 1720 in the bitstream 1750 the coded bits 1730-1731 and the leading tail bits 1740-1741.

[0184] However, leading tail bits 1740-1741 may be included in bitstream 1570 with bit 1670 of value 1 added immediately after leading tail bit 1660. This added bit is the first bit that differs in Low and High-1. The indication of the amount of leading tail bits 1720 in this case includes the additional bit 1.

[0185] The method for arithmetic coding may be performed separately on the multiple substreams. An exemplary embodiment is given in Fig. 14. The method described above is exemplary applied to the first and second substreams, resulting in the first coded bits 1430a, the second coded bits 1431, the first leading tail bits 1440a and the second leading tail bits 1441. A bitstream 1750 is formed from these pieces as illustrated diagrammatically in Fig. 17 by inserting the first coded bits 1730 and the second coded bits 1731 followed by the first leading tail bits 1740 and the second leading tail bits 1741 into the bitstream 1750. The first leading tail bits 1740 immediately follow the second coded bits 1731, and no padding is required.

[0186] The bitstream 1750 formed from the first and second substreams may also include length indications of the first and second coded bits 1710 and 1711. These first and second length indications 1710-1711 may precede the coded bits 1730-1731, respectively, in the bitstream 1750.

[0187] In addition, a second length indication 1711 may also precede the first coded bits 1730. This is exemplarily shown in Figure 17. The length indications 1710-1711 are concatenated for each of the coded bits 1730-1731 and included in the bitstream before the coded bits 1730-1731.

[0188] An indication of the amount of second leading tail bits 1721 may precede the first coded bit 1730 in the bitstream 1750. The example in Figure 17 includes concatenated indications 1720-1721 immediately followed by the first coded bits 1730-1731.

[0189] Bitstreams 1750 concatenated together as in any of the above exemplary embodiments may be padded to align the bitstream length to match an integer multiple of a predetermined amount of bytes, which may be, for example, a word or doubleword.

[0190] The arithmetic encoding of the first and second sub-streams may be performed in parallel, for example by one of the multi-core encoders 1420, 1520 in Figures 14 and 15. In some implementations, only a portion of the arithmetic encoding may be performed in parallel.

[0191] As mentioned above, the method of arithmetic encoding can be implemented as range coding, which can have a predefined total range; the preconfigured size of the memory is equal to or greater than the number of bits representing the total range (log_2 of the total range).

[0192] The memory may hold the minimum and maximum values ​​of the current range interval. For example, such a minimum value in a finite register may correspond to binary representation 720c in Figs. 7A-7D, and such a maximum value in a finite register may have binary representation 710c. After one or more iterations of the encoding loop, there may be stable bits, which are consecutive bits that have the same value in the MSB of the binary representation of the current minimum value and the current maximum value. A predetermined amount of these stable bits are moved from the memory to the bit stream. This corresponds, for example, to the two bits between the HIGH and TOP positions in Figs. 7A-7D. In an example implementation, there may be 16 bits between the HIGH and TOP positions. The invention is not limited to any of these examples.

[0193] It may happen that the difference between the amount of leading tail bits and a predefined amount of bits of stable bits between the HIGH and TOP positions is below a predefined threshold, for example there may be 16 bits between the HIGH and TOP positions and 15 leading tail bits. In this case it is more efficient to include the leading tail bits in the coded bits. Thus tail coded bits are generated from the leading tail bits. Bit 1 1670 needs to be added to the leading tail bits 1660, and the bits can be padded with 0 to reach a predefined number of stable bits. Thus, costly signaling of many leading tail bits with an indication of the amount of tail bits is avoided. For example, at least 4 bits are needed to signal an amount of 15 leading tail bits. Instead it may be less costly to have more coded bits and signal 0 leading tail bits. For example, they can be efficiently encoded if they occur frequently.

[0194] The predefined threshold mentioned above may be empirically determined, for example, taking into account the amount of bits between the HIGH and TOP positions and the amount of signaling used to indicate the length of the leading tail bits.

[0195] 10 is a flow diagram illustrating an exemplary method for arithmetic decoding of data from a bitstream 1750. A decoder may receive a bitstream S1010, from which an indication of the length of the coded bits may be extracted S1020. Coded bits 1730-1731 are extracted from the bitstream and decoded successively S1040-S1060. When all the coded bits have been decoded, leading tail bits 1740-1741 may also be extracted S1030 from the bitstream, extracted according to an indication of their quantity 1720-1721 S1070. From the leading tail bits 1660, the complete tail bits 1650 need to be recovered to be decoded S1080. The leading tail bits 1660 are padded with zeros up to a predefined maximum length of tail bits. The recovered tail bits may be decoded S1090.

[0196] In another implementation, if there is enough memory available, the entire substream may be formed before decoding. The coded bits and leading tail bits are extracted from the bitstream. The tail bits may be restored as described above. The coded bits and determined tail bits for the substream. The entire substream is then decoded.

[0197] However, the invention is not limited to any of these implementations.

[0198] If bit 1 1670 following the same leading bit 1660 in the trailing bits 1650 was not signaled, it needs to be included in the trailing bit determination process. Bit 1 1670 is appended to the leading trailing bit 1660 before the padding 1680.

[0199] The indications 1720-1721 of the amount of leading tail bits may precede in the bitstream 1750 the leading tail bits 1740-1741 as well as the coded bits 1730-1731.

[0200] The method for arithmetic decoding may also be performed separately on multiple substreams. To recover the individual substreams for decoding, the first coded bits 1730 and the second coded bits 1731 followed by the first leading tail bits 1740 and the second leading tail bits 1741 are extracted. The first coded bits 1730 and the first leading tail bits 1740 form the first substream, and the second coded bits 1731 and the second leading tail bits 1741 form the second substream. For each substream, the tail bits are determined as described above for a single substream. Each substream is decoded separately.

[0201] The first coded bits 1730 and the second coded bits 1731 may be extracted along with an indication of their respective lengths, referred to as a first length indication 1710 and a second length indication 1720 .

[0202] The first length indication of the coded bits 1710 may precede the first coded bits 1730 in the bitstream 1750, and the second length indication 1711 precedes the second coded bits 1731. In addition, the second length indication 1711 may also precede the first coded bits 1730.

[0203] An indication of the amount of second tail bits 1721 may also precede the first coded bits 1730 .

[0204] After extraction of the first coded bits 1730, the second coded bits 1731, the first leading tail bits 1740, and the second leading tail bits 1741, there may be remaining bits that can be discarded. The bits remaining in the bitstream 1750 after extraction of the last tail bit are the result of padding the entire bitstream 1750 to match a predetermined integer number of bits.

[0205] The arithmetic decoding may be performed in parallel, for example by a multi-core decoder. In addition, only a portion of the arithmetic decoding may be performed in parallel.

[0206] The method of arithmetic decoding may be implemented as range coding.

[0207] The arithmetic coding of the present disclosure can be easily applied to the encoding of neural network feature maps or to conventional image (still or video) encoding and decoding. Neural networks can be used for any purpose, in particular for encoding and decoding images (still or video) or image related data such as motion flow or motion vectors or other parameters. Neural networks can also be used for computer vision applications such as image classification, depth detection, segmentation map determination, object recognition on identification cards, or the like.

[0208] The method of entropy coding of multiple channels described in the above section may be combined with the manipulation of tail bits described in this section. The first and second sub-streams are formed by the first and second coded bits, respectively, including the multiplexed and encoded parts. The tail bits referred to in "entropy coding of multiple channels" may correspond to the leading tail bits, which are leading encoder status bits. The determined amount of leading encoder status bits is indicated in the tail bit length indication.

[0209] Implementation in image coding One possible deployment can be seen in FIGS.

[0210] Figure 11 shows a schematic block diagram of an example video encoder 20 configured to implement the techniques of the present application. In the example of Figure 11, the video encoder 20 includes an input 201 (or an input interface 201), a residual calculation unit 204, a transform processing unit 206, a quantization unit 208, an inverse quantization unit 210, an inverse transform processing unit 212, a reconstruction unit 214, a loop filter unit 220, a decoded picture buffer (DPB) 230, a mode selection unit 260, an entropy encoding unit 270, and an output 272 (or an output interface 272). The entropy coding 270 may implement an arithmetic coding method or device as described above.

[0211] The mode selection unit 260 may include an inter prediction unit 244, an intra prediction unit 254, and a partitioning unit 262. The inter prediction unit 244 may include a motion estimation unit and a motion compensation unit (not shown). The video encoder 20 as shown in FIG. 11 may also be referred to as a hybrid video encoder or a video encoder according to a hybrid video codec.

[0212] The encoder 20 may be configured to receive, for example, via an input 201, an image 17 (or image data 17), for example an image of a sequence of images forming a video or a video sequence. The received image or image data may be a preprocessed image 19 (or preprocessed image data 19). For the sake of brevity, reference is made in the following description to the image 17. The image 17 may also be referred to as the current image or the image to be coded (particularly in video coding, to distinguish the current image from other images, for example previously encoded and / or decoded images of the same video sequence, i.e. the video sequence which also includes the current image).

[0213] A (digital) image is or can be considered as a two-dimensional array or matrix of samples with intensity values. The samples in the array can also be called pixels (short for picture element) or pels. The number of samples in the horizontal and vertical directions (or axes) of the array or image determines the size and / or resolution of the image. For color representation, typically three color components are used. That is, an image can be represented as or contain three sample arrays. In an RGB format or color space, an image contains corresponding red, green and blue sample arrays. However, in video coding, each pixel is typically represented in a luminance and chrominance format or color space, e.g. YCbCr, which contains a luminance component denoted by Y (sometimes L is also used instead) and two chrominance components denoted by Cb and Cr. The luminance (or luma for short) component Y represents the brightness or intensity of the gray level (e.g. as in a grayscale image), while the two chrominance (or chroma for short) components Cb and Cr represent the chromaticity or color information components. Thus, an image in YCbCr format includes a luminance sample array of luminance sample values ​​(Y) and two chrominance sample arrays of chrominance values ​​(Cb and Cr). An image in RGB format may be converted or transformed to YCbCr format or vice versa, a process also known as color conversion or color transformation. If an image is monochrome, it may only include a luminance sample array. Thus, an image may be, for example, an array of luma samples in a monochrome format, or an array of luma samples and two corresponding arrays of chroma samples in 4:2:0, 4:2:2, and 4:4:4 color formats.

[0214] An embodiment of the video encoder 20 may comprise an image partitioning unit (not shown in FIG. 11 ) configured to partition an image 17 into multiple (typically non-overlapping) image blocks 203. These blocks may also be referred to as root blocks, macroblocks (H.264 / AVC) or coding tree blocks (CTBs) or coding tree units (CTUs) (H.265 / HEVC and VVC). The image partitioning unit may be configured to use the same block size for all images of a video sequence and a corresponding grid defining the block size, or to vary the block size between images or between subsets or groups of images, and to partition each image into corresponding blocks. The abbreviation AVC stands for Advanced Video Coding.

[0215] In a further embodiment, the video encoder may be configured to directly receive blocks 203 of image 17, e.g. one, some or all of the blocks forming image 17. Image block 203 may also be referred to as a current image block or an image block to be coded.

[0216] Here as well as in the image 17, the image block 203 is or can be considered as a two-dimensional array or matrix of samples with intensity values ​​(sample values), although with smaller dimensions than the image 17. In other words, the block 203 may contain, for example, one sample array (e.g. a luma array in case of a monochrome image 17, or a luma array or a chroma array in case of a color image), or three sample arrays (e.g. one luma array and two chroma arrays in case of a color image 17), or any other number and / or type of arrays depending on the color format applied. The number of samples in the horizontal and vertical directions (or axes) of the block 203 determines the size of the block 203. Thus, the block may be, for example, an M×N (M columns×N rows) array of samples, or an M×N array of transform coefficients.

[0217] An embodiment of a video encoder 20 as shown in FIG. 11 may be configured to encode an image 17 block-by-block, eg, encoding and prediction is performed for each block 203.

[0218] An embodiment of video encoder 20 as shown in FIG. 11 may further be configured to partition and / or encode an image using slices (also referred to as video slices), where an image may be partitioned or encoded using one or more slices (typically without overlap), and each slice may include one or more blocks (e.g., CTUs).

[0219] An embodiment of video encoder 20 as shown in FIG. 11 may further be configured to partition and / or encode an image using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where an image may be partitioned or encoded using one or more tile groups (typically non-overlapping), and each tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may, for example, be rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., full or partial blocks.

[0220] 12 illustrates an example of a video decoder 30 configured to implement the techniques of the present application. The video decoder 30 is configured to receive encoded image data 21 (e.g., encoded bitstream 21), e.g., encoded by encoder 20, to obtain a decoded image 331. The encoded image data or bitstream includes information for decoding the encoded image data, e.g., data representing image blocks and associated syntax elements of encoded video slices (and / or tile groups or tiles).

[0221] The entropy decoding unit 304 is configured to parse the bitstream 21 (or the encoded image data 21 in general) and, for example, perform entropy decoding on the encoded image data 21 to obtain, for example, quantization coefficients 309 and / or decoded coding parameters (not shown in FIG. 12 ), for example, any or all of inter prediction parameters (e.g., reference image index and motion vectors), intra prediction parameters (e.g., intra prediction mode or index), transform parameters, quantization parameters, loop filter parameters, and / or other syntax elements. The entropy decoding unit 304 may be configured to apply a decoding algorithm or scheme corresponding to the encoding schemes described with respect to the entropy encoding unit 270 of the encoder 20. The entropy decoding unit 304 may be further configured to provide the inter prediction parameters, intra prediction parameters, and / or other syntax elements to the mode application unit 360 and other parameters to other units of the decoder 30. The video decoder 30 may receive the syntax elements at a video slice level and / or a video block level. Additionally or alternatively to slices and their respective syntax elements, tile groups and / or tiles and their respective syntax elements may be received and / or used. The entropy decoding may implement any of the arithmetic decoding methods or apparatuses mentioned above.

[0222] The reconstruction unit 314 (e.g., an adder or summator 314) may be configured to add the reconstructed residual block 313 to the prediction block 365, e.g., by adding sample values ​​of the reconstructed residual block 313 and sample values ​​of the prediction block 365, to obtain a reconstructed block 315 in the sample domain.

[0223] An embodiment of the video decoder 30 as shown in FIG. 12 may be configured to partition and / or decode an image using slices (also referred to as video slices), where an image may be partitioned or decoded using one or more slices (typically without overlap), and each slice may include one or more blocks (e.g., CTUs).

[0224] An embodiment of video decoder 30 as shown in FIG. 12 may be configured to partition and / or decode an image using tile groups (also referred to as video tile groups) and / or tiles (also referred to as video tiles), where an image may be partitioned or decoded using one or more tile groups (typically without overlap), and each tile group may include, for example, one or more blocks (e.g., CTUs) or one or more tiles, where each tile may be, for example, rectangular in shape and may include one or more blocks (e.g., CTUs), e.g., full or partial blocks.

[0225] Other variations of the video decoder 30 may be used to decode the encoded image data 21. For example, the decoder 30 may generate an output video stream without a loop filtering unit 320. For example, a non-transform-based decoder 30 may directly inverse quantize the residual signal without an inverse transform processing unit 312 for a particular block or frame. In another implementation, the video decoder 30 may have the inverse quantization unit 310 and the inverse transform processing unit 312 combined into a single unit.

[0226] It should be understood that in the encoder 20 and the decoder 30, the processing result of the current stage may be further processed and then output to the next stage. For example, after the interpolation filtering, the motion vector derivation or the loop filtering, further operations such as Clip or Shift may be performed on the processing result of the interpolation filtering, the motion vector derivation or the loop filtering.

[0227] Hardware and software implementation Several further implementations in hardware and software are described below.

[0228] Any of the encoding devices described above with reference to Figures 19-22 may provide means for performing arithmetic encoding of the input data into the coded bits and the leading tail bits. The processing circuitry in any of these example devices is configured to encode the input data and to determine the leading tail bits of the encoder status after encoding the last bit of the coded bits according to the methods described above.

[0229] The decoding device in any of Figures 19-22 may include processing circuitry adapted to perform a decoding method. The method described above includes extracting the coded bits and the preceding tail bits together with an indication of their respective quantities. The tail bits are reconstructed from the preceding tail bits, which may be decoded together with the coded bits to obtain the data.

[0230] In summary, a method and apparatus is described for encoding data into a bitstream and decoding data from the bitstream. The method is capable of reducing the length of the bitstream by including only relevant significant bits in the tail bits of the encoding process. The amount of these leading tail bits can be determined and the tail bits with the lowest amount can be constructed. An indication of the amount of leading tail bits is included in the bitstream. Thus, no padding is required and, as a result, fewer bits need to be signaled.

[0231] According to an embodiment, an encoding method is provided for entropy encoding data of multiple channels of the same size into a bitstream, the method comprising: multiplexing a first size portion from each of the multiple channels and then encoding the first size multiplexed portion into a first substream; multiplexing a second size portion from each of the multiple channels and then encoding the second size multiplexed portion into a second substream; wherein the encoding is entropy encoding into the first substream and the second substream, which are performed independently.

[0232] Multiplexing and encoding portions from different channels, rather than encoding each channel separately, offers the possibility to produce a complete sub-stream faster and / or to control the length of the individual sub-streams, e.g., to obtain more uniform lengths of the sub-streams, which opens up the possibility to perform entropy encoding and / or entropy decoding on multiple sub-streams in parallel.

[0233] In an implementation, the encoding method further comprises generating multiple channels of equal size, including pre-processing data of the multiple channels of different sizes to obtain the multiple channels of equal size.

[0234] The option to obtain equal-sized channels from any-sized channels provides applicability of the method to different kinds of input data.

[0235] For example, the encoding method further comprises multiplexing the first substream and the second substream into a bitstream together with a first substream length indication indicating a length of the first substream and a second substream length indication indicating a length of the second substream.

[0236] By indicating the length of the sub-streams in the bitstream, it is possible to provide sub-streams with different sizes, thus resulting in more flexible bitstream configurations.

[0237] In an exemplary embodiment, the first substream length indication precedes the first substream in the bitstream, and the second substream length indication precedes the second substream in the bitstream.

[0238] This feature avoids the need to buffer the entire bitstream in order to extract the individual substreams.

[0239] For example, the second sub-stream length indication precedes the first sub-stream in the bitstream.

[0240] This bitstream structure, including a concatenated length indication before the first substream, may provide for more efficient extraction of the substreams from the bitstream.

[0241] In an implementation, the entropy encoding is an arithmetic encoding.

[0242] Arithmetic encoding is an efficient entropy coding that can contribute to rate reduction.

[0243] In an implementation example, the encoding method further comprises multiplexing the first sub-stream and the second sub-stream into a bitstream together with a first tail bit length indication indicating a length of the first tail bit of the first sub-stream and a second tail bit length indication indicating a length of the second tail bit of the second sub-stream.

[0244] The status of the encoder after encoding the last part, the tail bits, can be signaled separately from the substream, which offers the possibility of additional processing of the tail bits.

[0245] For example, a first tail bit length indication precedes a first substream in the bitstream, and a second tail bit length indication precedes a second substream in the bitstream.

[0246] An advantage of this bitstream structure may be the possibility of immediate encoding of the substreams without the need for multiple substreams and buffering their respective indications.

[0247] For example, a second tail bit length indication precedes the first substream in the bitstream.

[0248] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0249] In an exemplary implementation, the encoding method further comprises appending first tail bits to a bitstream following the first substream, and appending second tail bits to a bitstream following the second substream.

[0250] This bitstream structure allows decoding of the first substream and the corresponding tail bits without the extraction of another substream.

[0251] For example, the first tail bits follow in the bitstream the second substream.

[0252] Such a bitstream structure makes it possible to start decoding the individual substreams before extracting the tail bits from the bitstream.

[0253] In an example implementation, the encoding method further includes padding the bitstream including the first sub-stream length indication, the second sub-stream length indication, the first tail bit length indication, the second tail bit length indication, the first sub-stream, the second sub-stream, the first tail bit, and the second tail bit with bits having a predetermined value to align the bitstream length to match an integer multiple of the predetermined amount of bytes.

[0254] This implementation may provide a properly aligned bitstream for further processing, for example, by a network adaptation layer unit or encapsulation into other packets.

[0255] For example, the first size is equal to the second size.

[0256] Using portions of the same size may result in more efficient performance because, for example, suitable memory units may be used for hardware and software implementations.

[0257] In an example implementation, the total portion included in the first substream and the total portion included in the second substream are an integer number K times the symbols of the data of the multiple channels, where K is greater than one.

[0258] Such an approach may provide efficient implementation in software and / or hardware.

[0259] For example, a symbol is a bit.

[0260] In an implementation example, the encoding method further comprises selecting and subsequently applying a shuffling method for multiplexing the first size portions and the second size portions, where the shuffling method is selected from a set of predefined shuffling methods and specifies an order of the first size portions and the second size portions.

[0261] To achieve a more even distribution of substream sizes, the portions may be shuffled.

[0262] For example, the shuffle method performs a cyclic permutation of the portions of a second size with respect to the portions of a first size.

[0263] With such a shuffling method, substreams of more uniform length can be easily implemented.

[0264] For example, the encoding method is performed iteratively, where a shuffling method is selected according to a difference between a statistical value based on the length of the current first substream and a past length of the first substream, and / or a difference between a statistical value based on the length of the current second substream and a past length of the second substream.

[0265] Shuffling may be performed repeatedly on the encoded portions, reducing the length difference between the sub-streams over time.

[0266] For example, the statistics may be based on at least one of an estimated mean, median, minimum, maximum, or growth rate.

[0267] These statistics may provide a convenient means for controlling the shuffle and therefore the sub-stream size. In addition, this may allow for a more uniform load during the streaming process.

[0268] In an implementation, the entropy encoding is an arithmetic encoding, and the shuffle method is selected according to the current state of the range intervals in the arithmetic encoding.

[0269] The method may take into account whether the current range of values ​​in the interval of the arithmetic encoder is close to a predetermined minimum range for encoding, thereby avoiding renormalization during encoding.

[0270] In an example implementation, the entropy encoding includes generating a first sub-stream using a first entropy encoder and generating a second sub-stream using a second entropy encoder, and the entropy encoding using the first entropy encoder and the second entropy encoder are performed at least partially in parallel.

[0271] Parallel encoding of the sub-streams may result in faster encoding of the entire bitstream.

[0272] For example, the channel may be an output channel or a latent representation channel of a neural network.

[0273] Neural networks typically provide channels of the same size, or at least a fixed size, making the above embodiments and examples particularly suitable and more easily applicable to these channels.

[0274] In an implementation example, the entropy encoding is arithmetic encoding, and the method for encoding the multiplexed portion into a first substream or a second substream comprises the steps of: arithmetically encoding the multiplexed portion into coded bits and encoder status bits, where the coded bits form the substream; determining a minimum and a maximum value of the arithmetically encoded input data interval; determining an amount of preceding tail bits having the same value in a first most significant bit MSB representing the determined maximum value as in a second MSB representing the determined minimum value, which are successive encoder status bits, where the tail bits are preceding encoder status bits; and indicating the determined amount of preceding encoder status bits in a tail bit length indication.

[0275] Including leading tail bits in the bitstream rather than full tail bits can reduce the amount of bits in the bitstream, and therefore reduce the rate, e.g., reduce the amount of bits signaled, rate, for the same quality.

[0276] For example, the amount of leading encoder status bits NumTrailingBits is determined by NumTrailingBits=CLZ((LOW+RANGE-1)XOR LOW), where CLZ() is the number of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval.

[0277] The amount of leading trailing bits may be determined precisely, for example, rather than rounded to the nearest byte boundary, which may further reduce the amount of bits in the bitstream.

[0278] In an example implementation, the preceding encoder status bits are suffixed with a bit having a value of 1 before being included in the bitstream.

[0279] Such an approach may follow normal practices that may alternatively be used to remove one-valued bits.

[0280] For example, during arithmetic encoding, the current minimum and current maximum values ​​of the interval are stored in a memory of a preconfigured size; including the coded bits in the bitstream involves moving a predefined amount of bits from the memory into the bitstream of stable bits; stable bits are consecutive bits that have the same value in the MSBs of the binary representation of the current minimum and current maximum values.

[0281] Such independent application of the method to two separate substreams provides a prerequisite for parallelization.

[0282] In an example implementation, during arithmetic encoding, if the difference between the amount of leading encoder status and a predefined amount of bits of stable bits is below a predefined threshold; tail coded bits are generated from the leading encoder status bits by appending a 1 bit having a value of 1 followed by 0's up to the predefined amount of bits of stable bits; the tail coded bits are included in the coded bits before including the coded bits in the bitstream; and an indication of a leading encoder status bit of 0 is included in the bitstream.

[0283] Thus, costly signaling of many leading tail bits along with an indication of the amount of tail bits can be avoided. Instead, it may be less costly to have more coded bits and signal leading tail bits of 0.

[0284] In the implementation, the arithmetic encoding is a range encoding.

[0285] Range encoding may be particularly suitable for hardware and software architectures that have limited registers or generally fast memory sizes.

[0286] According to an embodiment, a decoding method is provided for entropy decoding a bitstream into multiple channels of same-sized data, the method comprising: entropy decoding a first substream independently of a second substream; and demultiplexing first-sized portions and second-sized portions from the first and second substreams into multiple channels of same size.

[0287] Rather than encoding each channel separately, multiplexing and encoding portions from different channels offers the possibility of decoding substreams of more uniform length, which opens up the possibility of performing entropy decoding on multiple substreams in parallel.

[0288] For example, the decoding method may further include post-processing the multiple channels of the same size to obtain data for multiple channels of different sizes.

[0289] The option of obtaining data for channels of any size from channels of the same size provides the applicability of the method to different kinds of data.

[0290] In an implementation, the decoding method further includes extracting the first substream and the second substream from the bitstream, together with a first substream length indication indicating a length of the first substream and a second substream length indication indicating a length of the second substream.

[0291] By indicating the length of the sub-streams in the bitstream, it is possible to provide sub-streams with different sizes, thus resulting in more flexible bitstream configurations.

[0292] In an example implementation, the first substream length indication precedes the first substream in the bitstream, and the second substream length indication precedes the second substream in the bitstream.

[0293] An advantage of this bitstream structure may be the possibility of immediate encoding or decoding of the substreams without the need for buffering multiple substreams and their respective indications.

[0294] For example, the second sub-stream length indication precedes the first sub-stream in the bitstream.

[0295] Providing a length indication concatenated in front of the substreams may allow for faster extraction of individual portions of the bitstream.

[0296] In an implementation, the entropy decoding is arithmetic decoding.

[0297] Arithmetic encoding is an efficient entropy coding that can contribute to rate reduction.

[0298] In an implementation example, the decoding method further comprises extracting the first sub-stream and the second sub-stream from the bitstream along with a first tail bit length indication indicating a length of the first tail bits of the first sub-stream and a second tail bit length indication indicating a length of the second tail bits of the second sub-stream.

[0299] By indicating the length of the sub-streams in the bitstream, it is possible to provide sub-streams with different sizes, thus resulting in more flexible bitstream configurations.

[0300] For example, a first tail bit length indication precedes a first substream in the bitstream, and a second tail bit length indication precedes a second substream in the bitstream.

[0301] An advantage of this bitstream structure may be the possibility of immediate decoding of the substreams without the need for buffering multiple substreams and their respective indications.

[0302] For example, a second tail bit length indication precedes the first substream in the bitstream.

[0303] Such a bitstream structure offers further possibilities for faster extraction of individual parts of the bitstream.

[0304] In an example implementation, a first tail bit follows the first sub-stream in the bitstream, and a second tail bit follows the second sub-stream in the bitstream.

[0305] This bitstream structure allows decoding of the first substream and the corresponding tail bits without the extraction of another substream.

[0306] For example, the first tail bits follow in the bitstream the second substream.

[0307] Such a bitstream structure makes it possible to start decoding the individual substreams before extracting the tail bits from the bitstream.

[0308] In an implementation, the first size is equal to the second size.

[0309] Using portions of the same size may result in more efficient performance because, for example, suitable memory units may be used for hardware and software implementations.

[0310] For example, the total portion included in the first substream and the total portion included in the second substream are an integer K times the symbols of the data of the multiple channels, where K is greater than one.

[0311] Such an approach may provide efficient implementation in software and / or hardware.

[0312] For example, a symbol is a bit.

[0313] In an implementation example, the encoding method further includes a step of discarding remaining bits of the bitstream after extracting the first sub-stream length indication, the second sub-stream length indication, the first tail bit length indication, the second tail bit length indication, the first sub-stream, the second sub-stream, the first tail bit, and the second tail bit.

[0314] Such an approach may provide a properly aligned bitstream for further processing, such as, for example, encapsulation into a network adaptation layer unit or other packets.

[0315] In an implementation example, the decoding method further comprises determining and applying a shuffling method for demultiplexing the first size portions and the second size portions, where the shuffling method is one of a set of predefined shuffling methods and specifies an order of the first size portions and the second size portions.

[0316] To achieve a more even distribution of substream sizes, the portions can be shuffled.

[0317] For example, the decision on the shuffling method is based on control information included in the bitstream.

[0318] In order to correctly shuffle the parts at the decoder side, the shuffling method used at the encoder side can be signaled in the bitstream.

[0319] In an example implementation, the entropy decoding includes decoding the first sub-stream using a first entropy decoder and decoding the second sub-stream using a second entropy decoder, where the entropy decoding using the first entropy decoder and the second entropy decoder are performed at least partially in parallel.

[0320] Parallel decoding of the sub-streams may result in faster decoding of the full bitstream.

[0321] For example, the channel may be an output channel or a latent representation channel of a neural network.

[0322] Neural networks typically provide channels of the same size, or at least a fixed size, making the above embodiments and examples particularly suitable and more easily applicable to these channels.

[0323] In an implementation example, the entropy decoding is arithmetic decoding, and the method for decoding the multiplexed portion from the first sub-stream or the second sub-stream includes the steps of: extracting an amount of a leading encoder status bit from a tail bit length indication, where the sub-stream includes coded bits and the tail bits are leading encoder status bits; determining an encoder status bit including appending zeros to the extracted leading encoder status bits up to a predetermined maximum length of the encoder status bits; and arithmetically decoding the multiplexed portion from the bits including the coded bits and the determined encoder status bits.

[0324] Reconstructing the tail bits from the preceding encoder status bits provides decoding from the coded bits and the tail bits by using a smaller amount of bits in the bitstream.

[0325] For example, determining an encoder status bit may consist of appending one bit of a value of 1 to the extracted preceding encoder status bit, followed by zeros up to a predetermined maximum length of the encoder status bit.

[0326] This approach makes it possible to reconstruct the complete output of the arithmetic encoder and therefore provide the appropriate input to the arithmetic decoder.

[0327] In an implementation, the arithmetic decoding is range decoding.

[0328] Range encoding may be particularly suitable for hardware and software architectures that have limited registers or generally fast memory sizes.

[0329] In an implementation, a computer program is stored on a non-transitory medium and includes code instructions that, when executed on one or more processors, cause the one or more processors to perform any of the steps of the methods described above.

[0330] According to an embodiment, an apparatus for entropy encoding multiple channels of data of the same size into a bitstream comprises a processing circuit configured to: multiplex a first size portion from each of the multiple channels and then encode the first size multiplexed portion into a first substream; multiplex a second size portion from each of the multiple channels and then encode the second size multiplexed portion into a second substream, where the encoding is entropy encoding into the first substream and the second substream performed independently.

[0331] According to an embodiment, an apparatus for entropy decoding a bitstream into multiple channels of same-sized data comprises a processing circuit configured to entropy decode a first substream independently from a second substream and to demultiplex first-sized and second-sized portions from the first and second substreams into multiple channels of same size, the apparatus providing the advantages of the method described above.

[0332] Any of the encoding devices described above with reference to Figures 19-22 may provide means for performing the multiplexing of the first size portions from each of the channels and the multiplexing of the second size portions from each of the channels, with the processing circuitry in any of these example devices configured to then encode the multiplexed portions into substreams and perform this entropy coding independently on the first and second substreams.

[0333] The decoding device in any of Figures 19-22 may include a processing circuit adapted to perform a decoding method as described above, comprising entropy decoding the first sub-stream independently from the second sub-stream, and demultiplexing first-sized portions and second-sized portions from the first and second sub-streams into data for multiple channels.

[0334] In summary, a method and apparatus is described for encoding data into a bitstream and decoding data from the bitstream. The method is capable of efficiently performing parallel encoding and decoding, avoiding padding of substreams and thus reducing the amount of bits in the bitstream. Portions of input data channels are multiplexed and encoded into substreams. During multiplexing, a shuffling method is applied to obtain substreams of more uniform length. The amount of bits in the substreams can be further reduced by including only the relevant most significant bits in the tail bits of the encoding process.

[0335] In the following embodiments of video coding system 10, video encoder 20 and video decoder 30 are described based on FIGS. 19 and 20 with reference to FIGS. 11 and 12 mentioned above, or other encoders and decoders, such as neural network-based encoders and decoders.

[0336] 19 is a schematic block diagram illustrating an example of a coding system 10, e.g., a video coding system 10 (or coding system 10 for short), that may utilize the techniques of the present application. A video encoder 20 (or encoder 20 for short) and a video decoder 30 (or decoder 30 for short) of video coding system 10 represent examples of devices that may be configured to perform techniques in accordance with various examples described herein.

[0337] As shown in FIG. 19, coding system 10 includes, for example, a source device 12 configured to provide encoded image data 21 to a destination device 14 for decoding the encoded image data 13.

[0338] The source device 12 comprises an encoder 20 and may additionally, that is, optionally, comprise an image source 16 , a pre-processor (or pre-processing unit) 18 , for example an image pre-processor 18 , and a communication interface or unit 22 .

[0339] Image source 16 may include or be any type of image capture device, such as a camera that captures real-world images, and / or any type of image generation device, such as a computer graphics processor that generates computer-animated images, or any type of other device that obtains and / or provides real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images), and / or any combination thereof (e.g., augmented reality (AR) images). Image source may be any type of memory or storage that stores any of the above mentioned images.

[0340] To distinguish from the pre-processor 18 and the processing performed by the pre-processing unit 18 , the image or image data 17 may also be referred to as raw image or raw image data 17 .

[0341] The pre-processor 18 is configured to receive the (raw) image data 17 and perform pre-processing on the image data 17 to obtain a pre-processed image 19 or pre-processed image data 19. The pre-processing performed by the pre-processor 18 may include, for example, cropping, color format conversion (e.g., from RGB to YCbCr), color correction or noise removal. It can be understood that the pre-processing unit 18 may be an optional component.

[0342] Video encoder 20 is configured to receive pre-processed image data 19 and provide encoded image data 21 (further details are described above with reference to, for example, FIG. 11).

[0343] The communications interface 22 of the source device 12 may be configured to receive the encoded image data 21 via the communications channel 13 and to transmit the encoded image data 21 (or any further processed version thereof) to another device, such as the destination device 14 or any other device, for storage or direct reconstruction.

[0344] The destination device 14 comprises a decoder 30 (eg, a video decoder 30), and may additionally, i.e. optionally, comprise a communications interface or unit 28, a post-processor 32 (or post-processing unit 32), and a display device 34.

[0345] The communications interface 28 of the destination device 14 is configured to receive the encoded image data 21 (or any further processed version thereof), for example directly from the source device 12 or from any other source, for example a storage device, for example an encoded image data storage device, and to provide the encoded image data 21 to a decoder 30.

[0346] The communications interface 22 and the communications interface 28 may be configured to transmit or receive (13) the encoded image data 21 or encoded data between the source device 12 and the destination device 14 via a direct communications link, e.g., a direct wired or wireless connection, or via any type of network, e.g., a wired or wireless network or any combination thereof, or any type of private and public network or any combination thereof.

[0347] The communications interface 22 may be configured, for example, to package the encoded image data 21 into an appropriate format, for example into packets, and / or to process the encoded image data using any type of transmission encoding or processing for transmission over a communications link or network.

[0348] Communications interface 28, which is the counterpart of communications interface 22, may be configured, for example, to receive transmitted data and process the transmitted data using any type of corresponding transmission decoding or processing and / or depackaging to obtain encoded image data 21.

[0349] Both communication interface 22 and communication interface 28 may be configured as unidirectional communication interfaces, as indicated by the arrow of communication channel 13 in FIG. 19 pointing from source device 12 to destination device 14, or as bidirectional communication interfaces, for example to send and receive messages, for example to set up connections, and to confirm and exchange any other information related to the communication link and / or data transmission, for example encoded image data transmission.

[0350] The decoder 30 is arranged to receive the encoded image data 21 and to provide decoded image data 31 or a decoded image 31 (further details are explained above, for example with reference to FIG. 12).

[0351] The post-processor 32 of the destination device 14 is configured to post-process the decoded image data 31 (also called reconstructed image data), e.g. the decoded image 31, to obtain post-processed image data 33, e.g. the post-processed image 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g. from YCbCr to RGB), color correction, cropping or resampling or any other processing, for example with the purpose of preparing the decoded image data 31 for display by, for example, a display device 34.

[0352] A display device 34 of the destination device 14 is configured to receive the post-processed image data 33 for displaying the image, for example to a user or viewer. The display device 34 may be or comprise any type of display for presenting the reconstructed image, for example an integrated or external display or monitor. The display may include, for example, a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro LED display, a liquid crystal on silicon (LCoS), a digital light processor (DLP), or any type of other display.

[0353] 19 illustrates source device 12 and destination device 14 as separate devices, an embodiment of the devices may include both or both functionality, source device 12 or corresponding functionality and destination device 14 or corresponding functionality. In such an embodiment, source device 12 or corresponding functionality and destination device 14 or corresponding functionality may be implemented using the same hardware and / or software or by separate hardware and / or software or any combination thereof.

[0354] As will be clear to a person skilled in the art based on the description, the functions of different units or the presence and (strict) separation of functions within the source device 12 and / or destination device 14 shown in FIG. 19 may vary depending on the actual device and application.

[0355] The encoder 20 (e.g., video encoder 20) or the decoder 30 (e.g., video decoder 30) or both the encoder 20 and the decoder 30 may be implemented via processing circuitry as shown in FIG. 20, such as one or more microprocessors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, hardware, dedicated video coding, or any combination thereof. The encoder 20 may be implemented via processing circuitry 46 to embody various modules described in connection with the encoder 20 of FIG. 11 and / or any other encoder system or subsystem described herein. The decoder 30 may be implemented via processing circuitry 46 to embody various modules described in connection with the decoder 30 of FIG. 12 and / or any other decoder system or decoder subsystem described herein. The processing circuitry may be configured to perform various operations as described below. 22, where these techniques are implemented partially in software, a device may store instructions for the software on a suitable non-transitory computer-readable storage medium and may execute the instructions in hardware using one or more processors to perform the techniques of this disclosure. Either video encoder 20 and video decoder 30 may be integrated as part of a combined encoder / decoder (codec) in a single device, for example, as shown in FIG.

[0356] Source device 12 and destination device 14 may comprise any of a wide range of devices, including any type of handheld or stationary device, such as a notebook or laptop computer, a mobile phone, a smart phone, a tablet or tablet computer, a camera, a desktop computer, a set-top box, a television, a display device, a digital media player, a video gaming console, a video streaming device (such as a content service server or a content delivery server), a broadcast receiver device, a broadcast transmitter device, etc., and may use no operating system or any type of operating system. In some cases, source device 12 and destination device 14 may be equipped for wireless communication. Thus, source device 12 and destination device 14 may be wireless communication devices.

[0357] In some cases, the video coding system 10 illustrated in FIG. 19 is merely an example, and techniques herein may be applied to video coding settings (e.g., video encoding or video decoding) that do not necessarily include any data communication between encoding and decoding devices. In other examples, data may be retrieved from local memory, streamed over a network, etc. A video encoding device may encode and store data in memory, and / or a video decoding device may retrieve and decode data from memory. In some examples, encoding and decoding are performed by devices that do not communicate with each other, but simply encode data to memory and / or retrieve and decode data from memory.

[0358] For ease of explanation, embodiments of the present invention are described herein by reference to, for example, reference software for High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), the next generation video coding standard developed by the ITU-T Video Coding Experts Group (VCEG) and the ISO / IEC Motion Picture Experts Group (MPEG) Joint Collaboration Team on Video Coding (JCT-VC). Those skilled in the art will appreciate that embodiments of the present invention are not limited to HEVC or VVC.

[0359] 21 is a schematic diagram of a video coding device 400 according to an embodiment of the present disclosure. The video coding device 400 is suitable for implementing the embodiments of the present disclosure described herein. In an embodiment, the video coding device 400 may be a decoder, such as the video decoder 30 of FIG. 19, or an encoder, such as the video encoder 20 of FIG. 19.

[0360] Video coding device 400 comprises an ingress port 410 (or input port 410) and a receiver unit (Rx) 420 for receiving data, a processor, logic unit, or central processing unit (CPU) 430 for processing data, a transmitter unit (Tx) 440 and an egress port 450 (or output port 450) for transmitting data, and a memory 460 for storing data. Video coding device 400 may also comprise optical / electrical (OE) and electrical / optical (EO) components coupled to the ingress port 410, the receiver unit 420, the transmitter unit 440, and the egress port 450 for egress or ingress of optical or electrical signals.

[0361] The processor 430 is implemented by hardware and software. The processor 430 may be implemented as one or more CPU chips, cores (e.g., as a multi-core processor), FPGA, ASIC, and DSP. The processor 430 communicates with the ingress port 410, the receiver unit 420, the transmitter unit 440, the egress port 450, and the memory 460. The processor 430 includes a coding module 470. The coding module 470 implements the above-disclosed embodiments. For example, the coding module 470 implements, processes, prepares, or provides various coding operations. Thus, the inclusion of the coding module 470 provides substantial improvements to the functionality of the video coding device 400, resulting in the transformation of the video coding device 400 into different states. Alternatively, the coding module 470 is implemented as instructions stored in the memory 460 and executed by the processor 430.

[0362] Memory 460 may comprise one or more disks, tape drives, and solid state drives, and may be used as an overflow data storage device for storing programs when the programs are selected for execution, as well as for storing instructions and data read during execution of the programs. Memory 460 may be, for example, volatile and / or non-volatile, and may be read only memory (ROM), random access memory (RAM), ternary content addressable memory (TCAM), and / or static random access memory (SRAM).

[0363] FIG. 22 is a simplified block diagram of an apparatus 500 that may be used as either or both of the source device 12 and the destination device 14 from FIG. 19, according to an example embodiment.

[0364] The processor 502 in the device 500 may be a central processing unit. Alternatively, the processor 502 may be any other type of device or devices, now existing or later developed, that can manipulate or process information. Although the disclosed implementations may be practiced with a single processor as shown, such as the processor 502, speed and efficiency advantages may be realized with more than one processor.

[0365] The memory 504 in the apparatus 500 may be a read-only memory (ROM) device or a random access memory (RAM) device in implementation. Any other suitable type of storage device may be used as the memory 504. The memory 504 may include code and data 506 that is accessed by the processor 502 using a bus 512. The memory 504 may further include an operating system 508 and application programs 510, which include at least one program that enables the processor 502 to perform the methods described herein. For example, the application programs 510 may include applications 1-N, which further include a video coding application that performs the methods described herein, including encoding and decoding using arithmetic coding as described above.

[0366] The apparatus 500 may also include one or more output devices, such as a display 518. The display 518, in one example, may be a touch-sensitive display that combines a display with a touch-sensitive element operable to detect touch input. The display 518 may be coupled to the processor 502 via the bus 512.

[0367] Although shown herein as a single bus, bus 512 of device 500 may be comprised of multiple buses. Additionally, secondary storage 514 may be directly coupled to other components of device 500 or may be accessed over a network, and may comprise a single integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, device 500 may be implemented in a wide variety of configurations.

[0368] Although the embodiments of the present invention have been described mainly based on video coding, it should be noted that the embodiments of the coding system 10, the encoder 20, and the decoder 30 (and correspondingly the system 10), as well as other embodiments described herein, may also be configured for still image processing or coding, i.e., processing or coding of an individual image independent of any preceding or subsequent images, as in video coding. In general, when image processing coding is limited to a single image 17, only the inter prediction units 244 (encoder) and 344 (decoder) may not be available. All other functions (also referred to as tools or techniques) of the video encoder 20 and the video decoder 30 may be equally used for still image processing, e.g., residual calculation 204 / 304, transform 206, quantization 208, inverse quantization 210 / 310, (inverse) transform 212 / 312, partitioning 262 / 362, intra prediction 254 / 354, and / or loop filtering 220, 320, as well as entropy coding 270 and entropy decoding 304.

[0369] The embodiments of, e.g., the encoder 20 and the decoder 30, and the functions described herein with reference to, e.g., the encoder 20 and the decoder 30, may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored on a computer-readable medium and transmitted as one or more instructions or code over a communication medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium, such as a data storage medium, or a communication medium including any medium that facilitates transfer of a computer program from one place to another, e.g., according to a communication protocol. Thus, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium, or (2) a communication medium, such as a signal or carrier wave. A data storage medium may be any available medium that can be accessed by one or more computers or one or more processors to obtain instructions, code and / or data structures for implementation of the techniques described in this disclosure. A computer program product may include a computer-readable medium.

[0370] By way of example, and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is referred to as a computer-readable medium, as appropriate. For example, if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio waves, and microwaves, the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio waves, and microwaves are included within the definition of media. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transitory media, but instead cover non-transitory, tangible storage media. Disk and disc as used herein include compact discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks and Blu-ray discs, where disks typically reproduce data magnetically, while discs reproduce data optically with a laser. Combinations of the above should also be included within the scope of computer readable media.

[0371] The instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Thus, the term "processor" as used herein may refer to any of the foregoing structures, or any other structure suitable for implementing the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules configured for encoding and decoding, or may be incorporated into a combined codec. Also, the techniques may be implemented entirely in one or more circuit or logic elements.

[0372] The techniques of this disclosure may be implemented in a wide variety of devices or apparatuses, including a wireless handset, an integrated circuit (IC) or a set of ICs (e.g., a chipset). Various components, modules or units are described in this disclosure to highlight functional aspects of devices configured to perform the disclosed techniques, but do not necessarily require realization by different hardware units. Rather, the various units may be combined into a codec hardware unit in conjunction with suitable software and / or firmware, as described above, or may be provided by a collection of interoperable hardware units including one or more processors, as described above. (Other possible items) (Item 1) A method for arithmetically encoding input data into a bitstream (1750), comprising: arithmetically encoding the input data into coded bits (1730 to 1731) and tail bits; Including the coded bits (1730 to 1731) in the bitstream (1750) (S940); determining a minimum value (1610) and a maximum value (1630) of a range of the arithmetically encoded input data (S960); are consecutive trailing bits, The first most significant bit (MSB) representing the determined maximum value (1630) has the same value as the second most significant bit (MSB) representing the determined minimum value (1610). determining the amount of leading tail bits (S970); and an indication of the determined amount of leading tail bits (1720-1721); and The preceding and trailing bits (1740 to 1741) into the bitstream (1750) (S980). A method for providing the above. (Item 2) The amount of leading trailing bits (1740-1741), NumTrailingBits, NumTrailingBits=CLZ((LOW+RANGE-1)XOR LOW) is determined by CLZ() is the number of leading zeros, LOW is the minimum value of the interval (1610), and RANGE is the range of the interval (1620). The method according to item 1. (Item 3) 3. The method according to claim 1 or 2, wherein the indication of the amount of leading tail bits (1720-1721) precedes in the bitstream the coded bits (1730-1731) and the leading tail bits (1740-1741). (Item 4) 4. The method of any one of items 1 to 3, wherein a bit (1670) having a value of 1 is post-pended to the leading tail bit (1660) prior to the step of including it in the bitstream. (Item 5) the step of arithmetically encoding, the step of determining minimum and maximum values ​​of an interval (S960), and the step of determining the amount of leading tail bits (S970) are performed separately for the first sub-stream and the second sub-stream, resulting in first coded bits (1730), second coded bits (1731), first leading tail bits (1740), and second leading tail bits (1741); The method comprises inserting the first coded bits (1730) and the second coded bits (1731) followed by the first leading tail bits (1740) and the second leading tail bits (1741) into the bitstream. 5. The method according to any one of items 1 to 4. (Item 6) 6. The method of claim 5, further comprising a step of multiplexing the first coded bits (1730) and the second coded bits (1731) into the bitstream (1750) together with a first length indication (1710) indicating a length of the first coded bits (1730) and a second length indication (1711) indicating a length of the second coded bits (131). (Item 7) the first length indication (1710) precedes the first coded bits (1730) in the bitstream (1750); the second length indication (1711) precedes the second coded bits (1731) in the bitstream (1750); The method according to item 6. (Item 8) 8. The method of claim 7, wherein the second length indication (1711) precedes the first coded bit (1730) in the bitstream (1750). (Item 9) 9. The method of claim 8, wherein the indication (1721) of the amount of the second leading tail bits precedes the first coded bit (1730) in the bitstream (1750). (Item 10) 10. The method of any one of items 5 to 9, further comprising padding the bitstream (1750) including the first coded bit (1730), the second coded bit (1731), the first leading tail bit (1740), and the second leading tail bit (1741) with bits having a predetermined value to align a bitstream length to match an integer multiple of a predetermined amount of bytes. (Item 11) the arithmetic encoding includes encoding the first sub-stream with a first arithmetic encoder and encoding the second sub-stream with a second arithmetic encoder; the arithmetic encoding using the first arithmetic encoder and the second arithmetic encoder is performed at least partially in parallel. 11. The method according to any one of items 5 to 10. (Item 12) 12. The method according to any one of claims 1 to 11, wherein the arithmetic encoding is a range encoding. (Item 13) During the arithmetic encoding, the current minimum value (720a-d) and the current maximum value (710a-d) of the interval are stored in a memory of a preconfigured size; The step of including the coded bits in the bitstream includes moving a predefined amount of stable bits (730) from the memory to the bitstream; The stable bits (730) are consecutive bits having the same value in the MSB of the binary representation of the current minimum values ​​(720a-d) and the current maximum values ​​(710a-d). Item 12. The method according to item 11. (Item 14) If, during the arithmetic encoding, a difference between the amount of leading trailing bits and a predefined amount of bits of the stable bits (730) is below a predefined threshold; Tail coded bits are generated from the preceding tail bits (1660) by postpending a bit (1670) having a value of 1 followed by zeros up to a predefined amount of bits of the stable bits (730); the tail coded bits are included in the coded bits prior to the step of including the coded bits in the bitstream; An indication of leading trailing bits of zero is included in the bitstream. Item 14. The method according to item 13. (Item 15) A method for arithmetically decoding data from a bitstream (1750), comprising: extracting an indication of the amount of leading and trailing bits (1720-1721) from the bit stream (1750) (S1030); extracting a plurality of coded bits (1730-1731) from the bitstream (1750) (S1040); extracting the leading tail bits (1740-1741) designated by the indication (1720-1721) of the amount of the extracted leading tail bits from the bit stream (1750) (S1070); determining tail bits (S1080), including postpending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; and obtaining the data by arithmetically decoding a coded value represented by the coded bits and bits including the determined tail bit. A method for providing the above. (Item 16) 16. The method of claim 15, wherein the step of determining the tail bits (1650) comprises postpending to the extracted preceding tail bits a bit (1670) having a value of 1 followed by zeros (1650) up to the predetermined maximum length of the tail bits. (Item 17) 17. The method according to claim 15 or 16, wherein the indication (1720-1721) of the amount of leading tail bits precedes the coded bits (1730-1731) and the leading tail bits (1740-1741) in the bitstream (1750). (Item 18) The method comprises extracting from the bitstream (1750) a first coded bit (1730) and a second coded bit (1731) followed by a first leading tail bit (1740) and a second leading tail bit (1741), the first coded bits (1730) and the first leading tail bits (1740) form a first substream; the second coded bits (1731) and the second leading tail bits (1741) form a second substream; determining a first tail bit for the first substream; determining second tail bits for the second substream; arithmetically decoding a first coded value represented by the first coded bits and the first bits including the determined first tail bits; arithmetically decoding a second coded value represented by the second coded bits and the second bits including the determined second tail bits. 18. The method according to any one of items 15 to 17. (Item 19) Item 19. The method of item 18, further comprising a step of extracting the first coded bits (1730) and the second coded bits (1731) from the bitstream (1750) together with a first length indication (1710) indicating a length of the first coded bits and a second length indication (1711) indicating a length of the second coded bits. (Item 20) the first length indication (1710) precedes the first coded bits (1730) in the bitstream; the second length indication (1711) precedes the second coded bits (1731) in the bitstream. Item 19. The method according to item 19. (Item 21) 21. The method of claim 20, wherein the second length indication (1711) precedes the first coded bits (1730) in the bitstream (1750). (Item 22) 22. The method of claim 21, wherein the indication of the amount of the second leading tail bits (1712) precedes the first coded bits (1730) in the bitstream (1750). (Item 23) 23. The method of any one of claims 18 to 22, further comprising a step of discarding remaining bits of the bitstream (1750) after extracting the first coded bits (1730), the second coded bits (1731), the first leading tail bits (1740), and the second leading tail bits (1741). (Item 24) the arithmetic decoding includes decoding the first sub-stream with a first arithmetic decoder and decoding the second sub-stream with a second arithmetic decoder; the arithmetic decoding using the first arithmetic decoder and the second arithmetic decoder is performed at least partially in parallel. 24. The method according to any one of items 18 to 23. (Item 25) 25. The method according to any one of items 15 to 24, wherein the arithmetic decoding is range decoding. (Item 26) 26. A computer program stored on a non-transitory medium, comprising code instructions which, when executed on one or more processors, cause said one or more processors to perform the steps of the method according to any one of items 1 to 25. (Item 27) 1. An apparatus for arithmetically encoding input data into a bitstream, comprising: arithmetically encoding the input data into coded bits and tail bits; Including the coded bits (1730-1731) in the bitstream (1750); determining a minimum value (1610) and a maximum value (1630) of an interval of the arithmetically encoded input data; are consecutive trailing bits, The first MSB representing the determined maximum value (1630) has the same value as the second MSB representing the determined minimum value (1610). Determining the amount of leading tail bits (1740-1741); and An indication of the determined amount of leading tail bits (1720-1721); and The preceding and trailing bits (1730 to 1731) in the bitstream (1750). 16. An apparatus comprising: a processing circuit configured to: (Item 28) An apparatus for arithmetically decoding data from a bitstream (1750), comprising: extracting an indication (1720-1721) of the amount of leading tail bits from said bitstream (1750); extracting a plurality of coded bits (1730-1731) from the bitstream (1750); extracting from the bitstream (1750) the leading tail bits (1740-1741) specified by the indication (1720-1721) of the amount of the leading tail bits extracted; determining tail bits including appending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; obtaining the data by arithmetically decoding a coded value represented by the coded bits and bits including the determined tail bit. 16. An apparatus comprising: a processing circuit configured to:

Claims

1. 1. A method for arithmetically encoding input data into a bitstream, comprising: arithmetically encoding the input data into coded bits and tail bits; including the coded bits in the bitstream; determining minimum and maximum values ​​of an interval of the arithmetically encoded input data; are consecutive trailing bits, In the first most significant bit MSB representing the determined maximum value, the same value is present as in the second most significant bit MSB representing the determined minimum value. determining the amount of leading tail bits; and an indication of the determined amount of leading tail bits; and The preceding trailing bits in the bitstream. A method for providing the above.

2. The amount of leading trailing bits, NumTrailingBits, is NumTrailingBits=CLZ((LOW+RANGE-1)XOR LOW) is determined by CLZ() is the number of leading zeros, LOW is the minimum value of the interval, and RANGE is the range of the interval. The method of claim 1.

3. The method of claim 1 or 2, wherein the indication of the amount of leading tail bits precedes the coded bits and the leading tail bits in the bitstream.

4. 4. A method according to claim 1, wherein, prior to the step of including in the bitstream, a bit having a value of 1 is appended to the leading tail bits.

5. the steps of arithmetically encoding, determining minimum and maximum interval values, and determining an amount of leading tail bits are performed separately for a first sub-stream and a second sub-stream, resulting in first coded bits, second coded bits, first leading tail bits, and second leading tail bits; the method comprising inserting the first coded bits and the second coded bits followed by the first leading tail bits and the second leading tail bits into the bitstream.

5. The method according to any one of claims 1 to 4.

6. 6. The method of claim 5, further comprising multiplexing the first coded bits and the second coded bits into the bitstream along with a first length indication indicating a length of the first coded bits and a second length indication indicating a length of the second coded bits.

7. the first length indication precedes the first coded bits in the bitstream; the second length indication precedes the second coded bits in the bitstream. The method according to claim 6.

8. The method of claim 7 , wherein the second length indication precedes the first coded bits in the bitstream.

9. The method of claim 8 , wherein the indication of the amount of the second leading tail bits precedes the first coded bits in the bitstream.

10. 10. The method of claim 5, further comprising padding the bitstream, including the first coded bit, the second coded bit, the first leading tail bit, and the second leading tail bit, with bits having a predetermined value to align a bitstream length to an integer multiple of a predetermined amount of bytes.

11. the arithmetic encoding includes encoding the first sub-stream with a first arithmetic encoder and encoding the second sub-stream with a second arithmetic encoder; the arithmetic encoding using the first arithmetic encoder and the second arithmetic encoder is performed at least partially in parallel.

11. The method according to any one of claims 5 to 10.

12. The method of claim 1 , wherein the arithmetic encoding is a range encoding.

13. During the arithmetic encoding, the current minimum value and the current maximum value of the interval are stored in a memory of a preconfigured size; The step of including the coded bits in the bitstream includes moving a predefined amount of stable bits from the memory to the bitstream; the stable bits are consecutive bits having the same value in the MSBs of the binary representations of the current minimum value and the current maximum value; The method of claim 11.

14. if, during the arithmetic encoding, a difference between the amount of leading trailing bits and a predefined amount of the stable bits is below a predefined threshold; tail coded bits are generated from the preceding tail bits by appending a bit having a value of 1 followed by zeros up to a predefined amount of bits of the stable bits; the tail coded bits are included in the coded bits prior to the step of including the coded bits in the bitstream; An indication of leading trailing bits of zero is included in the bitstream. The method of claim 13.

15. 1. A method for arithmetically decoding data from a bitstream, comprising: extracting from said bitstream an indication of the amount of leading tail bits; extracting a plurality of coded bits from the bitstream; extracting from the bitstream the leading tail bits specified by the indication of the amount of the leading tail bits extracted; determining the tail bits, comprising appending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; and obtaining the data by arithmetically decoding a coded value represented by the coded bits and bits including the determined tail bit. A method for providing the above.

16. 16. The method of claim 15, wherein the step of determining the tail bits comprises appending to the extracted preceding tail bits a bit having a value of 1 followed by zeros up to the predetermined maximum length of the tail bits.

17. 17. A method according to claim 15 or 16, wherein the indication of the amount of leading tail bits precedes in the bitstream the coded bits and the leading tail bits.

18. The method comprises: extracting from the bitstream a first coded bit and a second coded bit followed by a first leading tail bit and a second leading tail bit; the first coded bits and the first leading tail bits forming a first sub-stream; the second coded bits and the second leading tail bits forming a second sub-stream; determining a first tail bit for the first substream; determining second tail bits for the second substream; arithmetically decoding a first coded value represented by the first coded bits and the first bits including the determined first tail bits; and arithmetically decoding a second coded value represented by the second coded bits and second bits including the determined second tail bits; 18. The method of any one of claims 15 to 17, comprising:

19. 20. The method of claim 18, further comprising extracting the first coded bits and the second coded bits from the bitstream along with a first length indication indicating a length of the first coded bits and a second length indication indicating a length of the second coded bits.

20. the first length indication precedes the first coded bits in the bitstream; the second length indication precedes the second coded bits in the bitstream.

20. The method of claim 19.

21. 21. The method of claim 20, wherein the second length indication precedes the first coded bits in the bitstream.

22. 22. The method of claim 21, wherein the indication of the amount of the second leading tail bits precedes the first coded bits in the bitstream.

23. 23. The method of claim 18, further comprising discarding remaining bits of the bitstream after extracting the first coded bits, the second coded bits, the first leading tail bits, and the second leading tail bits.

24. the arithmetic decoding includes decoding the first sub-stream with a first arithmetic decoder and decoding the second sub-stream with a second arithmetic decoder; the arithmetic decoding using the first arithmetic decoder and the second arithmetic decoder is performed at least partially in parallel.

24. The method according to any one of claims 18 to 23.

25. 25. The method of any one of claims 15 to 24, wherein the arithmetic decoding is range decoding.

26. 26. A computer program stored on a non-transitory medium comprising code instructions that, when executed on one or more processors, cause said one or more processors to perform the steps of the method according to any one of claims 1 to 25.

27. 1. An apparatus for arithmetically encoding input data into a bitstream, comprising: arithmetically encoding the input data into coded bits and tail bits; including the coded bits in the bitstream; determining minimum and maximum values ​​of an interval of the arithmetically encoded input data; are consecutive trailing bits, In the first MSB representing the determined maximum value, the same value is present as in the second MSB representing the determined minimum value. determining the amount of leading trailing bits; and an indication of the determined amount of leading tail bits; and The preceding trailing bits in the bitstream.

16. An apparatus comprising: a processing circuit configured to:

28. 1. An apparatus for arithmetically decoding data from a bitstream, comprising: extracting from the bitstream an indication of the amount of leading tail bits; extracting a plurality of coded bits from the bitstream; extracting from the bitstream the leading tail bits specified by the indication of the amount of the leading tail bits extracted; determining the tail bits, including appending zeros to the extracted leading tail bits up to a predetermined maximum length of the tail bits; and obtaining the data by arithmetically decoding a coded value represented by the coded bits and bits including the determined tail bit.

16. An apparatus comprising: a processing circuit configured to:

Citation Information

Patent Citations

  • jctvc-l0226: update vps and vps_extension

    JP2016508331A

  • Coding concept allowing parallel processing, transport demultiplexer, and video bit stream

    JP2018198447A

  • Electronic device and method

    JP2020017993A

  • Arithmetic coder byte stuffing signaling for video coding

    US20210092454A1