Improvements to trellis coded quantization for video compression

Abstract

Various improvements to trellis coded quantization (TCQ) are described. An improvement modifies the coefficient syntax to preserve parity information in the first coding pass, thereby allowing the decoder to determine the parity of each coefficient without requiring the second pass. An improvement proposes a hybrid state machine approach that allows for better coding gains than a lower-state trellis alone, while reducing the computational complexity of a higher-state trellis. An improvement focuses on enhancing the end-of-block (EOB) decision algorithm using a simplified rate-distortion (RD) cost computation, which can reduce the computational complexity of the trellis search. An improvement introduces a node pruning technique using a dynamic threshold based on the distance of the current position from the DC coefficient, which aims to reduce the number of RD cost calculations performed during the trellis search by pruning nodes that are unlikely to contribute to the optimal solution.

Description

Atty. Doc. No. GOGL-2264-A-WO PATENTIMPROVEMENTS TO TRELLIS CODED QUANTIZATION FOR VIDEO COMPRESSIONCROSS-REFERENCE TO RELATED APPLICATION

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 692,574, filed September 9, 2024, which is incorporated herein in its entirety by reference.BACKGROUND

[0002] Digital video streams may represent video using a sequence of frames or still images. Digital video can be used for various applications including, for example, video conferencing, high-definition video entertainment, video advertisements, or sharing of usergenerated videos. A digital video stream can contain a large amount of data and consume a significant amount of computing or communication resources of a computing device for processing, transmission, or storage of the video data. Various approaches have been proposed to reduce the amount of data in video streams, including lossy and lossless coding techniques.SUMMARY

[0003] This application relates to encoding and decoding of image data, video stream data, or both for transmission and / or storage. Disclosed herein are aspects of systems, methods, and apparatuses for encoding and decoding using trellis coded quantization (TCQ).

[0004] An aspect of the teachings herein is a method for coding a current block. The method can include encoding, in a first coding pass, a base level value and a mid level value for a transform coefficient of a set of transform coefficients in the current block using trellis coded quantization (TCQ), wherein a value of the transform coefficient is equal to the base level value plus the mid level value plus a high level value, determining that, for the transform coefficient, a sum of the base level value and the mid level value for the transform coefficient is equal to or greater than a threshold value below a maximum value for the sum, encoding, in a second coding pass, the high level value for the transform coefficient, wherein the value of the transform coefficient is equal to or greater than the maximum value, and wherein a parity of the transform coefficient is indicated by the sum of the base level valueand the mid level value, and encoding a next transform coefficient of the set of transform coefficients using the parity as input to the TCQ.

[0005] In a variation of this method, the high level value is an odd integer.

[0006] In a variation of this method, encoding the high level value comprises encoding the high level value using a bypass coding scheme.

[0007] In a variation of this method, encoding the base level value and the mid level value comprises encoding the base level value and the mid level value using a multi- symbol encoder.

[0008] In a variation of this method, the transform coefficient is a discrete cosine transform coefficient.

[0009] In a variation of this method, the method includes decoding the transform coefficient using TCQ, calculating the parity of the transform coefficient as a sum of the base level value, the mid level value, and the high level value multiplied by an even integer, and decoding a next transform coefficient using TCQ according to a state based on the parity. In some implementations, the even integer is 2, and the threshold value is the maximum value minus 1.

[0010] An aspect of the teachings herein is another method for coding a current block. The method can include decoding, from a first pass syntax, a base level value and a mid level value of a transform coefficient of a set of transform coefficients of the current block using trellis coded quantization (TCQ), decoding a high level value of the transform coefficient using TCQ, determining a parity of the transform coefficient based on a sum of the base level value, the mid level value, and the high level value multiplied by an even number, and decoding a next transform coefficient of the set of transform coefficients using the TCQ according to a state based on the parity.

[0011] In a variation of the method, the even number is two.

[0012] In a variation of the method, the set of transform coefficients comprises a set of discrete cosine transform coefficients.

[0013] As aspect of the teachings herein is another method for coding a current block. The method can include selecting a first trellis state machine having a first number of states for trellis coded quantization (TCQ), selecting a second trellis state machine having a second number of states for TCQ, the second number of states being greater than the first number of states, coding, in a scan order, a proper subset of transform coefficients of the current block using the first trellis state machine, and coding, in the scan order, one or more subsequent transform coefficients of the current block using the second trellis state machine.

[0014] In a variation of this method, the first trellis state machine has four states, and the second trellis state machine has eight states.

[0015] In a variation of this method, the proper subset of transform coefficients corresponds to a low frequency region of the block.

[0016] In a variation of this method, a transition from the first trellis state machine to the second trellis state machine is based on a defined position within the scan order.

[0017] In a variation of this method, a transition from the first trellis state machine to the second trellis state machine is based on a predetermined threshold value.

[0018] In a variation of this method, coding the current block comprises decoding the current block.

[0019] In a variation of this method, the method includes mapping a state of the first trellis state machine after coding a final transform coefficient of the proper subset of transform coefficients to a state of the second trellis state machine before coding the one or more subsequent transform coefficients.

[0020] An aspect of the teachings herein is another method for coding a current block. The method can include generating quantized transform coefficients for the current block using trellis coded quantization (TCQ), determining a first candidate end-of-block (EOB) position based on a scan order of the quantized transform coefficients, determining a second candidate EOB position based on the scan order of the quantized transform coefficients, wherein the second candidate EOB position has a lower frequency than the first candidate EOB position, and the second candidate EOB position is separated from the first candidate EOB position by at least one position, each position having a zero value for its quantized transform coefficient, calculating a first coding efficiency value associated with the first candidate EOB position, calculating a second coding efficiency value associated with the second candidate EOB position, selecting an EOB position for the current block as whichever of the first coding efficiency value or the second coding efficiency value is lower, and encoding the quantized transform coefficients into an encoded bitstream in the scan order starting at the EOB position.

[0021] In a variation of this method, the first coding efficiency value comprises a combination of a distortion resulting from using a first non-zero quantized transform coefficient value of the first candidate EOB position and a bit cost for sending the first nonzero quantized transform coefficient value and each zero value of the at least one position.

[0022] In a variation of this method, the second coding efficiency value comprises a combination of a distortion resulting from using a second non-zero quantized transformcoefficient value of the second candidate EOB position and a zero value for a first non-zero quantized coefficient and a bit cost for sending the second non-zero quantized transform coefficient value.

[0023] In a variation of this method, the first candidate EOB position is separated from the second candidate EOB position by multiple zero coefficients.

[0024] An aspect of the teachings herein is another method for coding a current block. The method can include calculating a set of efficiency cost measures, each efficiency cost measure corresponding to a respective state in a trellis state machine at a current transform coefficient position in the current block being encoded using trellis coded quantization (TCQ), determining a minimum efficiency cost value from among the set of efficiency cost measures, determining a dynamic threshold based on a distance of the current transform coefficient position from a DC coefficient position of the current block, and pruning states having an efficiency cost measure greater than a pruning threshold value based on the minimum efficiency cost measure and the dynamic threshold.

[0025] In a variation of this method, the pruning threshold value comprises a product of the minimum efficiency cost measure and the dynamic threshold.

[0026] In a variation of this method, the dynamic threshold is increased as the distance of the current transform coefficient position from the DC coefficient position decreases.

[0027] In a variation of this method, the dynamic threshold is decreased as the distance of the current transform coefficient position from the DC coefficient position increases.

[0028] In a variation of this method, the dynamic threshold is determined based on a quantization parameter (QP).

[0029] In a variation of this method, the dynamic threshold is determined based on a block size of the block of discrete cosine transform (DCT) coefficients.

[0030] An aspect of the teachings herein is an apparatus for coding a current block that includes a processor configured to execute any of the methods described herein.

[0031] An aspect of the teachings herein is an apparatus for coding a current block that includes a processor and a memory storing instructions that, when executed, cause the processor to perform operations according to any of the methods described herein.

[0032] As aspect of the teachings herein is a non-transitory, computer-readable medium storing instructions that, when executed, cause a processor to perform operations according to any of the methods described herein.

[0033] An aspect of the teachings herein is a non-transitory, computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a current block encoded according to any of the encoding methods described herein.

[0034] An aspect of the teachings herein is a non-transitory, computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a current block to be decoded according to any of the decoding methods described herein.

[0035] Aspects of the present disclosure are disclosed in the following detailed description of the embodiments, the appended claims, and the accompanying figures.BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The description herein refers to the accompanying drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0037] FIG. 1 is a schematic of a video encoding and decoding system.

[0038] FIG. 2 is a block diagram of an example of a computing device that can implement a transmitting station or a receiving station.

[0039] FIG. 3 is a diagram of a typical video stream to be encoded and subsequently decoded.

[0040] FIG. 4 is a block diagram of an encoder according to implementations of this disclosure.

[0041] FIG. 5 is a block diagram of a decoder according to implementations of this disclosure.

[0042] FIG. 6 is a trellis diagram depicting a four-state trellis state machine according to implementations of this disclosure.

[0043] FIG. 7 is a trellis diagram depicting an eight-state trellis state machine according to implementations of this disclosure.

[0044] FIG. 8 is a schematic diagram illustrating an example of an encoding flow using TCQ according to implementations of this disclosure.

[0045] FIG. 9A is a diagram depicting a coding syntax for multi-pass quantization.

[0046] FIG. 9B is a diagram depicting a modified coding syntax for multi-pass TCQ according to implementations of this disclosure.

[0047] FIG. 10 is a flow diagram of a method for encoding a current block using TCQ according to implementations of this disclosure.

[0048] FIG. 11 is a flow diagram of a method for decoding a current bitstream using TCQ according to implementations of this disclosure.

[0049] FIG. 12 is a flow diagram of a method for coding a current block using TCQ according to implementations of this disclosure.

[0050] FIG. 13 is a diagram of an example showing a coefficient scan for an 8x8 block according to implementations of this disclosure.

[0051] FIG. 14 is a schematic diagram illustrating another example of an encoding flow using TCQ according to implementations of this disclosure.

[0052] FIG. 15 is a flow diagram of a method for encoding a current block using TCQ according to implementations of this disclosure.

[0053] FIG. 16 is a flow diagram of a method for coding transform coefficients of a current block using TCQ according to implementations of this disclosure.DETAILED DESCRIPTION

[0054] Video compression schemes may include breaking respective images, or frames, of a video stream into smaller portions, such as coding tree blocks (CTBs) or coding tree units (CTUs) (sometimes referred to as superblocks), and generating an encoded bitstream using techniques to limit the information included for respective CTUs thereof. The bitstream can be decoded to re-create the source frames from the limited information. Encoding CTUs to or decoding CTUs from the bitstream can include predicting the values of pixels or CTUs based on similarities with other pixels or CTUs in the same frame or in one or more other frames that have already been coded.

[0055] Those similarities can be determined using intra prediction, which attempts to predict the pixel values of a coding unit (CU) (coding block, etc.) of a CTU using pixels peripheral to the CU (e.g., pixels that are in the same frame as the CU, but that are outside the CU). During encoding, the result of an intra-prediction mode performed against a CU is a prediction unit (PU) (prediction block, predictor block, etc.). A prediction residual can be determined based on a difference between the pixel values of the CU and the pixel values of the PU. The prediction residual and the intra-prediction mode used to ultimately obtain that prediction residual can then be encoded to a bitstream. During decoding, the prediction residual is reconstructed into a CU using a PU produced based on the intra-prediction mode and is thereafter included in an output video stream.

[0056] Similarly, inter prediction attempts to predict the pixel values of a CU of a CTU using pixels from one or more reference frames. During encoding, the result of an interprediction mode performed against a CU is also a PU. A prediction residual can be determined based on a difference between the pixel values of the CU and the pixel values ofthe PU. The prediction residual and the inter-prediction mode used to ultimately obtain that prediction residual can then be encoded to a bitstream. During decoding, the prediction residual is reconstructed into a CU using a PU produced based on the inter-prediction mode and is thereafter included in an output video stream.

[0057] A frame, and hence its CTUs and CUs may include a luminance, also referred to as luma, component and two chrominance, also referred to as chroma, components. These luma and chroma components may in some cases be referred to as luma blocks and chroma blocks. The luma component may, for example, be expressed within a Y plane and the chroma components may be expressed either within U and V planes or Cr and Cb planes. The luma component is understood to include some number of luma samples and each chroma component is understood to include some number of chroma samples. Generally, the luma samples provide measures of brightness throughout a frame and thus represent the structural qualities of the video content of the frame, whereas the chroma samples provide measures of color throughout the frame. The number of luma samples can indicate the spatial resolution (or simply the resolution) of the frame.

[0058] One useful technique for further reducing the amount of data sent from an encoder to a decoder to recreate these samples is a lossy coding referred to as quantization, which is a process used to map a large set of input values to a smaller set. Scalar quantization, which is one of the most commonly used methods, involves quantizing each individual sample of a signal independently. While scalar quantization is relatively simple and easy to implement, it often results in a significant loss of information, particularly in high-compression scenarios, as it does not consider the correlation between successive samples in a signal.

[0059] TCQ is an advanced quantization technique that overcomes several limitations of scalar quantization. TCQ is an extension of the principle of trellis-coded modulation (TCM) and utilizes a trellis structure to encode a sequence of quantized values. In TCQ, the input signal is divided into blocks, and each block is represented as a path through a trellis, with each state in the trellis corresponding to one or more different quantization levels. The encoder selects the path through the trellis that minimizes a coding efficiency metric, a distortion metric, or some combination thereof, thereby producing a sequence of quantized values that are highly correlated with the input signal.

[0060] The application of TCQ in video and image compression provides benefits over scalar quantization. One of the primary advantages is the ability of TCQ to exploit the inherent redundancy in the signal by considering the correlation between successive samples. This results in a more efficient representation of the signal, allowing for higher compressionratios, generally with little to no corresponding increase in distortion, and with a small increase in bits. Furthermore, the trellis structure in TCQ enables the use of sophisticated coding techniques, such as Viterbi decoding, which can further reduce the bit rate required for transmission or storage while maintaining high signal fidelity.

[0061] There are several challenges to implementing TCQ in practical applications. For example, many codecs code transform coefficients one than one pass. Where two passes are used, the first pass be used to signal base values (such as 0 or 1) and mid range values (such as 2 and 3), while the second pass may be used to signal high range values (greater than 3). This approach does not preserve coefficient parity in the first pass, which is necessary for TCQ decoding. Another issue is the high computational complexity associated with TCQ, which can impede its adoption in environments where processing resources are limited or where real-time processing is important. Additionally, existing TCQ systems may not optimally balance the trade-off between compression efficiency and computational demand, potentially leading to either excessive resource consumption or suboptimal video quality.

[0062] Implementations of this disclosure describe several improvements to TCQ for video compression that address these challenges, thereby facilitating adoption of TCQ in a codec. In some implementations, a modified coefficient syntax preserves parity information in the first coding pass, enabling more efficient TCQ. In some implementations, a hybrid state machine switches between 4-state and 8-state trellises within a coefficient block to optimize coding gain versus complexity. In some implementations, an enhanced end-of-block (EOB) decision process evaluates multiple candidate positions to reduce unnecessary trellis computations. In some implementations, dynamic node pruning based on accumulated path costs further reduces computational complexity of the trellis search. These techniques allow for more efficient implementation of TCQ in image codecs, video codecs, or both, providing better rate-distortion performance and / or reduced computational complexity compared to conventional approaches.

[0063] Implementations of this disclosure describe improvements to TCQ for video compression. Further details of techniques for image and / or video coding using improved TCQ are described herein with initial reference to a system in which the disclosure may be implemented.

[0064] FIG. 1 is a schematic of a video encoding and decoding system 100. A transmitting station 102 can be, for example, a computer having an internal configuration of hardware such as that described in FIG. 2. However, other implementations of thetransmitting station 102 are possible. For example, the processing of the transmitting station 102 can be distributed among multiple devices.

[0065] A network 104 can connect the transmitting station 102 and a receiving station 106 for encoding and decoding of the video stream. Specifically, the video stream can be encoded in the transmitting station 102, and the encoded video stream can be decoded in the receiving station 106. The network 104 can be, for example, the Internet. The network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other means of transferring the video stream from the transmitting station 102 to, in this example, the receiving station 106.

[0066] The receiving station 106, in one example, can be a computer having an internal configuration of hardware such as that described in FIG. 2. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed among multiple devices.

[0067] Other implementations of the video encoding and decoding system 100 are possible. For example, an implementation can omit the network 104. In another implementation, a video stream can be encoded and then stored for transmission at a later time to the receiving station 106 or any other device having memory. In one implementation, the receiving station 106 receives (e.g., via the network 104, a computer bus, and / or some communication pathway) the encoded video stream and stores the video stream for later decoding. In an example implementation, a real-time transport protocol (RTP) is used for transmission of the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, e.g., video streaming protocol based on the Hypertext Transfer Protocol (HTTP).

[0068] When used in a video conferencing system, for example, the transmitting station 102 and / or the receiving station 106 may include the ability to both encode and decode a video stream as described below. For example, the receiving station 106 could be a video conference participant who receives an encoded video bitstream from a video conference server (e.g., the transmitting station 102) to decode and view and further encodes and transmits his or her own video bitstream to the video conference server for decoding and viewing by other participants.

[0069] FIG. 2 is a block diagram of an example of a computing device 200 that can implement a transmitting station or a receiving station. For example, the computing device 200 can implement one or both of the transmitting station 102 and the receiving station 106 of FIG. 1. The computing device 200 can be in the form of a computing system includingmultiple computing devices, or in the form of one computing device, for example, a mobile phone, a tablet computer, a laptop computer, a notebook computer, a desktop computer, and the like. The computing device 200 and / or one or more components thereof may be, be similar to, include, or be included in, an apparatus for performing one or more techniques, processes, and / or methods described herein.

[0070] A processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device, or multiple devices, capable of manipulating or processing information now existing or hereafter developed. For example, although the disclosed implementations can be practiced with one processor as shown (e.g., the processor 202), advantages in speed and efficiency can be achieved by using more than one processor.

[0071] A memory 204 in computing device 200 can be a read only memory (ROM) device or a random access memory (RAM) device in an implementation. However, other suitable types of storage device can be used as the memory 204. The memory 204 can include code and data 206 that is accessed by the processor 202 using a bus 212. The memory 204 can further include an operating system 208 and application programs 210, the application programs 210 including at least one program that permits the processor 202 to perform the techniques described herein. For example, the application programs 210 can include applications 1 through N, which further include a video coding application that performs the techniques described herein. The video coding application may include computer-executable instructions that, when executed by the processor 202, are configured to cause the processor 202 and / or an apparatus (e.g., the computing device 200 and / or one or more components thereof) including the processor 202 to perform one or more aspects of one or more techniques, processes, and / or methods described herein. The computing device 200 can also include a secondary storage 214, which can, for example, be a memory card used with a mobile computing device. Because the video communication sessions may contain a significant amount of information, they can be stored in whole or in part in the secondary storage 214 and loaded into the memory 204 as needed for processing.

[0072] The computing device 200 can also include one or more output devices, such as a display 218. The display 218 may be, in one example, a touch sensitive display that combines a display with a touch sensitive element that is operable to sense touch inputs. The display 218 can be coupled to the processor 202 via the bus 212. Other output devices that permit a user to program or otherwise use the computing device 200 can be provided in addition to or as an alternative to the display 218. When the output device is or includes a display, thedisplay can be implemented in various ways, including by a liquid crystal display (LCD), a cathode-ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0073] The computing device 200 can also include or be in communication with an image-sensing device 220, for example, a camera, or any other image-sensing device 220 now existing or hereafter developed that can sense an image such as the image of a user operating the computing device 200. The image-sensing device 220 can be positioned such that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image-sensing device 220 can be configured such that the field of vision includes an area that is directly adjacent to the display 218 and from which the display 218 is visible.

[0074] The computing device 200 can also include or be in communication with a soundsensing device 222, for example, a microphone, or any other sound-sensing device now existing or hereafter developed that can sense sounds near the computing device 200. The sound-sensing device 222 can be positioned such that it is directed toward the user operating the computing device 200 and can be configured to receive sounds, for example, speech or other utterances, made by the user while the user operates the computing device 200.

[0075] Although FIG. 2 depicts the processor 202 and the memory 204 of the computing device 200 as being integrated into one unit, other configurations can be utilized. The operations of the processor 202 can be distributed across multiple machines (wherein individual machines can have one or more processors) that can be coupled directly or across a local area or other network. The memory 204 can be distributed across multiple machines such as a network-based memory or memory in multiple machines performing the operations of the computing device 200. Although depicted here as one bus, the bus 212 of the computing device 200 can be composed of multiple buses. Further, the secondary storage 214 can be directly coupled to the other components of the computing device 200 or can be accessed via a network and can comprise an integrated unit such as a memory card or multiple units such as multiple memory cards. The computing device 200 can thus be implemented in a wide variety of configurations.

[0076] FIG. 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes a number of adjacent frames 304. While three frames are depicted as the adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individualframes, for example, a frame 306. At the next level, the frame 306 can be divided into a series of planes or segments 308. The segments 308 can be subsets of frames that permit parallel processing, for example. The segments 308 can also be subsets of frames that can separate the video data into separate colors. For example, a frame 306 of color video data can include a luminance plane and two chrominance planes. The segments 308 may be sampled at different resolutions.

[0077] Whether or not the frame 306 is divided into segments 308, the frame 306 may be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also be of any other suitable size such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise noted, the terms block and macroblock are used interchangeably herein.

[0078] FIG. 4 is a block diagram of an encoder 400 according to implementations of this disclosure. The encoder 400 can be implemented, as described above, in the transmitting station 102, such as by providing a computer software program stored in memory, for example, the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the transmitting station 102 to encode video data in the manner described in FIG. 4. The encoder 400 can also be implemented as specialized hardware included in, for example, the transmitting station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0079] The encoder 400 has the following stages to perform the various functions in a forward path (shown by the solid connection lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy encoding stage 408. The encoder 400 may also include a reconstruction path (shown by the dotted connection lines) to reconstruct a frame for encoding of future blocks. In FIG. 4, the encoder 400 has the following stages to perform the various functions in the reconstruction path: a dequantization stage 410, an inverse transform stage 412, a reconstruction stage 414, and a loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode the video stream 300.

[0080] When the video stream 300 is presented for encoding, respective adjacent frames 304, such as the frame 306, can be processed in units of blocks. At the intra / inter prediction stage 402, respective blocks can be encoded using intra-frame prediction (also called intra prediction) or inter- frame prediction (also called inter prediction). In any case, a predictionblock can be formed. In the case of intra prediction, a prediction block may be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter prediction, a prediction block may be formed from samples in one or more previously constructed reference frames.

[0081] Next, the prediction block can be subtracted from the current block at the intra / inter prediction stage 402 to produce a residual block (also called a residual). The transform stage 404 transforms the residual into transform coefficients in, for example, the frequency domain using block-based transforms. The quantization stage 406 converts the transform coefficients into discrete quantum values, which are referred to as quantized transform coefficients, using a quantizer value or a quantization level. For example, the transform coefficients may be divided by the quantizer value and truncated.

[0082] The quantized transform coefficients are then entropy encoded by the entropy encoding stage 408. The entropy-encoded coefficients, together with other information used to decode the block (which may include, for example, syntax elements such as used to indicate the type of prediction used, transform type, motion vectors, a quantizer value, or the like), are then output to the compressed bitstream 420. The compressed bitstream 420 can be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 can also be referred to as an encoded video stream or encoded video bitstream, and the terms will be used interchangeably herein.

[0083] The reconstruction path (shown by the dotted connection lines) can be used to ensure that the encoder 400 and a decoder 500 (described below with respect to FIG. 5) use the same reference frames to decode the compressed bitstream 420. The reconstruction path performs functions that are like functions that take place during the decoding process (described below with respect to FIG. 5), including dequantizing the quantized transform coefficients at the dequantization stage 410 and inverse transforming the dequantized transform coefficients at the inverse transform stage 412 to produce a derivative residual block (also called a derivative residual). At the reconstruction stage 414, the prediction block that was predicted at the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. The loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0084] Other variations of the encoder 400 can be used to encode the compressed bitstream 420. In some implementations, a non-transform based encoder can quantize the residual signal directly without the transform stage 404 for certain blocks or frames. In someimplementations, an encoder can have the quantization stage 406 and the dequantization stage 410 combined in a common stage.

[0085] FIG. 5 is a block diagram of a decoder 500 according to implementations of this disclosure. The decoder 500 can be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program can include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to decode video data in the manner described in FIG. 5. The decoder 500 can also be implemented in hardware included in, for example, the transmitting station 102 or the receiving station 106.

[0086] The decoder 500, similar to the reconstruction path of the encoder 400 discussed above, includes in one example the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a post filtering stage 514. Other structural variations of the decoder 500 can be used to decode the compressed bitstream 420.

[0087] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by the quantizer value), and the inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce a derivative residual that can be identical to that created by the inverse transform stage 412 in the encoder 400. Using header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter prediction stage 508 to create the same prediction block as was created in the encoder 400 (e.g., at the intra / inter prediction stage 402).

[0088] At the reconstruction stage 510, the prediction block can be added to the derivative residual to create a reconstructed block. The loop filtering stage 512 can be applied to the reconstructed block to reduce blocking artifacts. Other filtering can be applied to the reconstructed block. In this example, the post filtering stage 514 is applied to the reconstructed block to reduce blocking distortion, and the result is output as the output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and the terms will be used interchangeably herein. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 canproduce the output video stream 516 without the post filtering stage 514 or otherwise omit the post filtering stage 514.

[0089] The above encoder and decoder implementations generally refers to scalar quantization. As described in further detail below, a quantization stage at an encoder (such as the quantization stage 406 shown in FIG. 4) and a dequantization stage at an encoder or decoder (such as the dequantization stage 410 shown in FIG. 4 or the dequantization stage 504 shown in FIG. 5) may use TCQ. TCQ may be employed to efficiently encode and decode transform coefficients such as, for example, Discrete Cosine Transform (DCT) coefficients. A trellis state machine may be used to guide the quantization of the transform coefficients.

[0090] FIG. 6 is a trellis diagram 600 depicting a four-state trellis state machine according to implementations of this disclosure. The trellis diagram 600 illustrates the structure and operation of a TCQ scheme that may be used in image and video coding systems.

[0091] The trellis diagram 600 consists of four states, labeled 0 (binary 00), 1 (binary 01), 2 (binary 10), and 3 (binary 11), represented by nodes in the diagram. The term "state" in the context of a trellis state machine refers to a specific configuration or condition of the state machine. Each state represents a different quantization option or path through the trellis. In a 4-state trellis state machine, for example, there are four possible states, each representing a respective quantizer or quantization level, at least one of which is different from another.

[0092] The states correspond to different quantization options or paths through the trellis. The trellis diagram 600 shows the possible transitions between states from one stage to the next. Each transition is represented by an arrow or branch connecting two states. In this four- state trellis, each state has two possible transitions to the next stage, creating a branching structure. The transitions between states are associated with specific quantization decisions. In some implementations, odd-numbered states may correspond to one set of quantizers, while even-numbered states may correspond to another set of quantizers. This alternating structure allows for more flexible and efficient quantization of transform coefficients.

[0093] For example, each state may have a different quantizer and entropy model. In an example, a quantizer, probability model for entropy coding, and state transitions can be assigned to each state as shown in Table 1, below.

[0094] Table 1

[0095] During the encoding process, the TCQ algorithm may traverse the trellis to find the optimal path that minimizes a cost function, such as rate-distortion (RD) cost. This path determines the sequence of quantization decisions for a set of transform coefficients. In the context of Table 1, the trellis diagram 600 shows how the encoder may transition between states based on the parity of the coefficient being encoded. For each state, there are two possible transitions to the next state, corresponding to whether the coefficient parity is 0 or 1 (e.g., is even or odd).

[0096] In some implementations, for example, the trellis diagram 600 may be implemented in the quantization stage 406 of the encoder 400. As transform coefficients are processed, the encoder may use the trellis structure to determine the optimal quantization path. The encoder may start at an initial state, typically state 0. For each transform coefficient, the encoder may determine the parity of the coefficient (0 or 1) and apply the appropriate quantizer (Q0 or QI) as specified in Table 1 for the current state before transitioning to the next transform coefficient. The encoder may calculate the rate-distortion cost for this quantization decision and store this information for path selection. After processing all coefficients, the encoder may select the path through the trellis that minimizes the overall rate-distortion cost. The encoder may then encode the quantized coefficients along with the selected path information. The transform coefficients may be processed in a coding order, such as a reverse zig-zag scan order.

[0097] The alternating structure of quantizers (Q0 and QI) in the trellis allows for more flexible quantization. States 0 and 1 use quantizer Q0, while states 2 and 3 use quantizer QI. This arrangement may provide better adaptation to local statistics of the transform coefficients. The trellis structure also introduces dependencies between adjacent coefficients, as the state transitions depend on the parity of the previous coefficient. This interdependence may allow the encoder to exploit correlations between coefficients, potentially leading to improved coding efficiency. In some implementations, the encoder may use multiple passes through the trellis to refine the quantization decisions. The encoder may also adapt the trellis structure or quantizers based on the local characteristics of the video content or the target bit rate.

[0098] The use of TCQ in the quantization stage 406 may provide a more sophisticated quantization scheme compared to scalar quantization, potentially leading to improved ratedistortion performance in the overall encoding process. The trellis structure allows the encoder to make quantization decisions that consider both the current coefficient and the context provided by previously encoded coefficients, which may result in more efficient compression of the video data.

[0099] The trellis diagram 600 in FIG. 6 may be used repeatedly for different coefficient groups or blocks within a frame. Quantizers and entropy models may differ depending on a variety of factors, such as color component being coded, block size, etc. The four- state trellis state machine represented by FIG. 6 represents a balance between computational complexity and coding efficiency. More states can improve coding efficiency with the downside of increasing computational complexity.

[0100] FIG. 7 is a trellis diagram 700 depicting an eight-state trellis state machine according to implementations of this disclosure. The trellis diagram 700 illustrates the structure and operation of a TCQ scheme that may be used in video coding systems. A quantizer and probability model can be assigned to each state. In an example, a quantizer and state transitions can be assigned to each state as shown in Table 2, below.

[0101] Table 2 s j Next State (coeff s Next State (coeff s _ iJ Trellis State j \ j Quantizer s j j parity = 0) i parity = 1) j §

[0102] FIG. 7 illustrates a trellis diagram 700 depicting an eight-state trellis state machine for TCQ. This expanded trellis structure offers a more complex and a potentially more efficient quantization scheme for transform coefficients in video coding systems than that shown in FIG. 6.

[0103] The trellis diagram 700 comprises eight states, labeled from 0 to 7 (binary 000, 001, 010, Oi l, 100, 101, 110, 111) as indicated by the state indexes 702. Each column in thediagram represents a state in the trellis. The arrows connecting the states represent possible transitions between states from one stage to the next.

[0104] Table 2 provides the state transition rules and quantizer assignments for this eightstate trellis. The table shows how the next state is determined based on the current state and the parity of the coefficient being processed. For example, if the current state is 0 and the coefficient parity is 0, the next state will be 0. However, if the coefficient parity is 1, the next state will be 4.

[0105] The quantizer assignments in Table 2 show that states 0, 1, 4, and 5 use quantizer Q0, while states 2, 3, 6, and 7 use quantizer QI. This alternating pattern of quantizers may allow for more adaptive quantization compared to the four-state trellis, potentially capturing more complex patterns in the transform coefficients.

[0106] In the context of TCQ, the trellis diagram 700 and Table 2 work together to guide the quantization process. For example, during encoding, the TCQ algorithm may traverse the trellis, evaluating different paths through the states for each transform coefficient. At each stage, the encoder may consider the parity of the current coefficient and the current state to determine the possible next states and corresponding quantizers. The encoder may calculate the rate-distortion cost for each possible transition, considering both the quantization error (e.g., in the reconstructed transform coefficients) and the bit cost of encoding the state transition. After processing all coefficients, the encoder may select the path through the trellis that minimizes the overall rate-distortion cost. The selected path determines the sequence of quantizers used for the transform coefficients, which may result in more efficient compression compared to scalar quantization or simpler TCQ schemes.

[0107] In some cases, the eight- state trellis may offer advantages over the four- state trellis. For example, the larger number of states may allow for more nuanced quantization decisions, potentially adapting better to local statistics of the transform coefficients. The eight-state trellis may capture longer-range dependencies between coefficients, as the current state can depend on the parity of multiple previous coefficients. The additional states and transitions may allow the encoder to find a more optimal balance between quantization error and bit rate. However, the increased complexity of the eight-state trellis may also require more computational resources for both encoding and decoding. The choice between a four- state and eight- state trellis (or even more complex structures) may depend on the specific requirements of the video coding application, such as the desired balance between compression efficiency and computational complexity.

[0108] In some implementations, the encoder may dynamically switch between different trellis structures or adapt the trellis parameters based on the characteristics of the video content or the target bit rate. This adaptive approach may further enhance the flexibility and efficiency of the TCQ scheme in video coding systems.

[0109] In the examples above, two different quantizers (Q0 and QI) are used, shown in Table 3, below. Q0 is identical to the current scalar quantizer, while QI has the non-zero points shifted * quantizer step towards zero. Each quantizer uses its own context-based probability model to account for the shifted positions.

[0110] Table 3

[0111] While the transform coefficients in Table 3 extend between coefficient values of - 2 and +2, Table 3 can be extended to cover more coefficient values, both positive and negative. Table 3 shows an example of symmetric quantizers, but asymmetric quantizers (such as those used in the JPEG 2K) may also be used with the teachings herein.

[0112] FIG. 8 is a schematic diagram illustrating an example 800 of an encoding flow using TCQ according to implementations of this disclosure. The example 800 uses the four- state trellis state machine described in FIG. 6 to explain the encoding flow.

[0113] The states 802 maintained in the encoding flow of FIG. 8 include four coded states in addition to an “all_zero” state 804. Encoding starts with the transform coefficient at the end-of-block (EOB) position (e.g., the last non-zero coefficient) and proceeds in the scan order towards the last (or final) coefficient in the scan order, which is the first transform coefficient (coeff[0]) in the reverse zig-zag scan order, e.g., the DC coefficient when the transform coefficients are DCT transform coefficients. This EOB position is an initial EOB position and may be one of several candidate EOB positions because the EOB position may be moved to other positions within the coding order. In this example, the first transform coefficient is referred to as an initial transform coefficient. A coding efficiency measure (such as an RD cost) for the all_zero state is determined at node 810 by assuming that the initial transform coefficient is zero. A transition is made to the node 812 for the next non-zero transform coefficient (at the next candidate EOB position) in the coding order, and the coding efficiency measure is updated with the error resulting from coding the second, non-zerotransform coefficient as a zero value. This all_zero state processing can continue as described for some or all non-zero transform coefficients.

[0114] At every node of (e.g., for every non-zero transform coefficient considered at) the all_zero state, the cost of transitioning from the all_zero state to a coded state is determined using the trellis structure, in this case, the trellis structure based on the example of the trellis diagram 600. Namely, a set of nodes 806 (i.e., the filled circles) is associated with quantizer Q0, and another set of nodes 808 (i.e., the hatched circles) is associated with quantizer QI. This division reflects the alternating quantizer structure described earlier, where different states use different quantizers, and the transitions between nodes depend on the parity as described previously.

[0115] At node 810, when the coefficient parity of the initial transform coefficient is even, transition is made to node 814 (state 0), which is in the set of nodes 806 with which the quantizer Q0 is used for quantizing the (e.g., non-zero) transform coefficient. Otherwise, when the coefficient parity is odd, transition is made to node 816 (state 2), which is in the set of nodes 808 with which the quantizer QI is used for quantizing the transform coefficient.

[0116] Similarly, at node 812, when the coefficient parity of the next (second) non-zero transform coefficient is even, transition is made to node 818 (state 0), which is in the set of nodes 806 with which the quantizer Q0 is used for quantizing the (e.g., non-zero) transform coefficient. Otherwise, when the coefficient parity is odd, transition is made to node 822 (state 2), which is in the set of nodes 808 with which the quantizer QI is used for quantizing the transform coefficient.

[0117] Processing at node 814 (state 0), when applicable, includes quantizing the initial transform coefficient using quantizer Q0 and determining a RD cost for state 0. Then, when the coefficient parity of the next (second) non-zero transform coefficient is even, transition is made to node 818 (state 0), which is in the set of nodes 806 with which the quantizer Q0 is used for quantizing the next non-zero transform coefficient. Otherwise, when the coefficient parity of the next non-zero transform coefficient is odd, transition is made to node 822 (state 2), which is in the set of nodes 808 with which the quantizer QI is used for quantizing the next non-zero transform coefficient. At node 818 (state 0) or node 822 (state 2), whichever is applicable, the (second) quantized transform coefficient is used to determine (update, calculate, etc.) the RD cost of state 0 or state 2, and the parity of the next (third) non-zero transform is considered for transition to the next node.

[0118] Processing at node 816 (state 2), when applicable, includes quantizing the initial transform coefficient using quantizer QI and determining a RD cost for state 2. Then, whenthe coefficient parity of the next (second) non-zero transform coefficient is even, transition is made to node 820 (state 1), which is in the set of nodes 806 with which the quantizer Q0 is used for quantizing the next non-zero transform coefficient. Otherwise, when the coefficient parity of the non-zero transform coefficient is odd, transition is made to node 824 (state 3), which is in the set of nodes 808 with which the quantizer QI is used for quantizing the next non-zero transform coefficient. At node 820 (state 1) or node 824 (state 3), whichever is applicable, the (second) quantized transform coefficient is used to determine (update, calculate, etc.) the RD cost of state 1 or state 3, and the parity of the next (third) non-zero transform is considered for transition to the next node.

[0119] Processing at either node 818 (state 0) or node 822 (state 2) after transition from the node 812 (all_zero state), whichever is applicable, is like the above. At the node, the second non-zero transform coefficient is quantized (by quantizer Q0 or quantizer QI) and used to update the RD cost for state 0 or stage 2 before transition to the next node based on the parity of the next (third) non-zero transform coefficient.

[0120] In summary, an encoder may evaluate multiple paths through this trellis structure to determine the optimal sequence of quantization decisions. The encoder may consider factors such as the distortion introduced by quantization, the number of bits required to encode the quantized values, and the cost in bits of encoding the initial state, if any. A balancing between the distortion and the bits (e.g., a RD cost) may be used to make decisions. With the 4-state trellis, at each of the states, the RD cost is calculated for each state transition, and for the coding states (states 0 - 3) keeping the best option (e.g., the lowest RD cost) out of the 2 (or 3) incoming candidates at each state. After the last coefficient in the scan order (e.g., the DC coefficient) is processed, the lowest RD cost out of the coding states (states 0 - 3) and the all_zero state is selected for coding the block.

[0121] The encoding flow described using the example 800 of FIG. 8 may be applied to individual blocks of transform coefficients or to larger units of video data, depending on the specific implementation of the video coding system. The TCQ process may be integrated with other encoding techniques, such as prediction, transform coding, and entropy coding, to achieve overall improvements in video compression efficiency.

[0122] As described initially, improvements to TCQ can be used to address limitations in practical implementation of TCQ in video codecs due to high computational complexity and other issues.

[0123] One issue that arises using TCQ is the use, in some codecs, of two-pass coding for coefficient magnitudes. The coefficient syntax may use three types of syntax elements for themagnitudes. In some implementations, so-called “base” and “mid” range values of coefficient magnitudes are coded in a first pass, while signs (e.g., positive or negative) and so-called “high” range values, when applicable, are coded in a second pass. Because a different probability model is used for each quantizer, the decoder needs to be able to determine coefficient parity with only the first pass (base + mid) syntax. An example may be used to illustrate the problem assuming the coefficient coding syntax below in Table 4.

[0124] Table 4

[0125] In this syntax, for example, if the (base + mid) value is 16, and the high value is 1, the total value is 17, which is an odd number (or odd integer). However, if the (base + mid) value is 16, and the high value is 2, the total value is 18, which is an even number (or even integer). In the latter case, the correct parity from the first pass is preserved for dequantization using TCQ. However, in the former case, the parity from the first pass is not preserved. This can result in a mismatch between the block reconstructed at the decoder as compared to the encoder, which in turn can reduce image quality (e.g., in the block itself, due to incorrect prediction using the block, etc.).

[0126] To preserve parity in the first pass, coding the high range value, where present, is modified. The high range value may be modified before the value is added to the (base + mid) value so that the parity of the (base + mid) value is unchanged. For example, the high range value may be multiplied by two before it is added to the (base + mid) value, thereby preserving the correct parity from the first pass.

[0127] The coefficient coding syntax for this improvement is shown below in Table 5.

[0128] Table 5

[0129] As can be seen from the second pass syntax of Table 5, to allow all possible magnitudes to be coded, the high range value is coded when the (base + mid) value is equal to the maximum value (here, 15 or 17) or is equal to the second- to-maximum value (here, 14 or 16). This change is diagrammatically represented in FIGS. 9A and 9B. FIG. 9A is a diagram depicting a coding syntax 900 for multi-pass quantization. The coding syntax 900 of FIG. 9A diagrams the coding syntax of Table 4. FIG. 9B is a diagram depicting a modified coding syntax 910 for multi-pass TCQ according to implementations of this disclosure. The coding syntax 910 diagrams the coding syntax of Table 5.

[0130] Comparing FIGS. 9 A and 9B, each syntax includes a base level value syntax 902 associated with a low coefficient value range and a mid level value syntax 904 associated with a mid coefficient value range. A high level value syntax 906 in FIG. 9A is associated with a high coefficient value range. The low range may be encoded using a multi-symbol encoder, while the mid range and high range may be encoded using separate syntax elements. The high range is encoded in a second pass, after the low range and mid range have been encoded.

[0131] The coding syntax 910 of FIG. 9B differs from the coding syntax 900 of FIG. 9B in that the high range is encoded so as not to affect the parity of earlier values, e.g., those encoded in a first pass, here (base + mid). Because parity is a binary factor, multiplying the high range value by an even number would not change the parity when added to the sum of the earlier values. Two is the smallest whole even number, so it is desirable to use a multiplier of two as shown in Table 5 and illustrated by a high level value syntax 912 in FIG. 9B. The switch from the level before the high range to the high range level is modified to ensure that all possible values (i.e., magnitudes) can be coded. The test for switching depends on the multiplier. In this example, the high level value syntax 912 in FIG. 9B is encoded when the sum of the base and mid level values is greater than or equal to the second-to- maximum value instead of greater than or equal to the maximum value as in FIG. 9A. This can also be seen in Table 5 as compared to Table 4.

[0132] The modified coefficient syntax preserves the parity information of the first coding pass, enabling TCQ decisions as determined by the encoder when decoding, such as in the dequantization stage 504 of the decoder 500. The modified or updated coefficient coding syntax may be used alone or with one or more of the other improvements to TCQ described herein. Although only a base level value is described as required for any particular transform coefficient, the mid level value may be present for all transform coefficients, even when equal to zero, and the high level value may be present for all transform coefficients, evenwhen equal to zero, such that low level value + mid level value + high level value = the magnitude of the transform coefficient value.

[0133] FIG. 10 is a flow diagram of a method 1000 for encoding a current block using TCQ according to implementations of this disclosure. The method 1000 may be performed by an encoder (such as the encoder 400 shown in FIG. 4) and / or one or more components of the computing device 200 depicted in FIG. 2. In general, an encoder uses, when TCQ is used for quantization, a starting value for the high level coding syntax based on what multiplier will be used to maintain parity of the sum of previously-coded values. The starting value is lower than the maximum value defined by the sum of the maximum values that can be coded by the previous levels. The sum of the maximum values is modified based on the multiplier so that the full range of values can be coded. FIG. 10 is one example of encoding, but others are possible given the teachings herein.

[0134] At step 1002, an encoder encodes, in a first coding pass, a base level value and, where applicable, a mid level value for each transform coefficient of a set of transform coefficients in the current block. For the purposes of this example, the method 1000 assumes that both a base level value and a mid level value are encoded for the current block. The current block may be a block of DCT coefficients, for example, which are generated by transforming a residual block of video data. The base level value and the mid level value may be encoded using a multi-symbol encoder, for example. The base level value represents a lower range of transform coefficient magnitudes, while the mid level value represents a middle range of transform coefficient magnitudes.

[0135] At step 1004, the encoder determines, for a transform coefficient of the set of transform coefficients, whether the sum of the base level value and the mid level value for the transform coefficient is equal to or greater than a defined threshold that is based on the multiplier that is used to maintain parity. The multiplier is used to reduce the threshold below the threshold based on the maximum value that can be coded using the previous levels (i.e., the first pass). In this example, the defined threshold is equal to the second- to-maximum value. If the sum of the base level value and the mid level value for the transform coefficient is equal to or greater than the defined threshold, the encoder proceeds to step 1006. Otherwise, the encoder proceeds to step 1008.

[0136] At step 1006, the encoder encodes a high level value for the transform coefficient, which is a difference between the already coded values and the original value of the transform coefficient. The high level value may be encoded using a bypass coding scheme,for example. The high level value is encoded after the first pass (e.g., in a second coding pass).

[0137] At step 1008, the encoder excludes the second pass encoding for the transform coefficient. That is, the encoder does not encode further values for the transform coefficient. The encoder continues to process the remaining transform coefficients in the current block, repeating steps 1002, 1004, 1006, and 1008 in the scan order as necessary. Following the encoding, the encoder may output the encoded bitstream. The encoded bitstream may include the encoded base level values, mid level values, and high level values for the transform coefficients in the current block. The encoded bitstream may also include other information, such as syntax elements indicating the type of prediction used, transform type, motion vectors, initial state for DCQ, etc.

[0138] FIG. 11 is a flow diagram of a method 1100 for decoding a current bitstream using TCQ according to implementations of this disclosure. The method 1100 may be performed by a decoder (such as the decoder 500 shown in FIG. 5) and / or one or more components of the computing device 200 depicted in FIG. 2. A decoder (such as the decoder 500 shown in FIG. 5) may receive a current bitstream. The current bitstream may include the encoded base level values, and, where applicable, encoded mid level values and encoded high level values for the transform coefficients of a current block. The current bitstream may also include other information, such as syntax elements indicating the type of prediction used, transform type, motion vectors, initial state for DCQ, etc. In this example, it is assumed that the current block being decoded has a quantized transform coefficient that has an encoded high level value and thus has both an encoded base level value and an encoded mid level value.

[0139] At step 1102, the decoder starts decoding a block by decoding the transform coefficients of the block in a decoding order, e.g., one that starts at the EOB position. Decoding a transform coefficient includes entropy decoding encoded level values that were encoded before the high level value. In this example, the base level and mid level values are entropy decoded using the probability model based on the initial trellis state for the block. Thereafter, at step 1104, the quantized base level and mid level values are dequantized using the quantizer based on the initial trellis state for the block.

[0140] At step 1106, decoding the transform coefficient continued by entropy decoding the encoded high level value. In this example, the high level value is entropy decoded using the probability model based on the initial trellis state for the block. The high level value may be decoded using a bypass decoding scheme, for example. Thereafter, at step 1108, thequantized high level value is dequantized using the quantizer based on the initial trellis state for the block.

[0141] At step 1110, the decoder determines a parity for the transform coefficient using the dequantized level values. In this example, the base level and mid level values are added together with the high level value multiplied by the even number (e.g., 2). The next transform coefficient can begin decoding by the method 1100, using the next trellis state based on transitioning from the initial trellis state using the parity determined at step 1110 as long as the next transform coefficient has a high level value. Where the transform coefficient does not have a high level value, the method 1100 can be modified by performing steps 1102 and 1104 before parity is determined at step 1110 (omitting steps 1106 and 1108). Parity is determined in this example using the base level value and, where applicable, the sum of the base level value and the mid level value.

[0142] The method 1100 (in its entirety or modified as described) continues for transform coefficients in the scan order until all transform coefficients are dequantized. Although not shown in FIG. 11, the dequantized transform coefficients are used to reconstruct respective blocks, such as described with respect to the reconstruction stage of the encoder 400 or with respect to the decoder 500. Briefly, after inverse transformation, the residual block is added to a prediction block to reconstruct the block for output or display.

[0143] The preservation of parity information in the first coding pass addresses a challenge in implementing TCQ that encodes transform coefficient using multi-pass coding. By ensuring that the parity is known from the first pass, the use of different probability models for odd and even coefficients facilitate effective TCQ. This can lead to improved coding efficiency compared to conventional scalar quantization methods.

[0144] Some implementations may include different thresholds for triggering the encoding of the high level value, or different multiplication factors for the high level value. The high range or level value could also be encoded using a different coding scheme, such as a run-length coding scheme. The specific implementation may be optimized based on the characteristics of the video content or the requirements of the codec.

[0145] As mentioned initially, multiple state trellis state machines exist. A state machine with fewer states reduces computational complexity. More states increase complexity and storage requirements but generally improve compression. The teachings herein describe a hybrid state machine that uses two or more state machines for coding transform coefficients of a block. In general, the hybrid state machine uses more than one trellis state machine with respectively higher number of states as the frequency of the transform coefficients increases.In practice, this means that there can be a switch from one trellis state machine to another trellis state machine at one or more positions in the coding sequence.

[0146] An example of using a hybrid state machine for TCQ herein is described with regards to FIGS. 12-14. The hybrid state machine balances coding gain and complexity. The hybrid state machine may be used alone or with one or more of the other improvements to TCQ described herein. The example of FIGS. 12-14 describe a combination of two trellis state machines, but the teachings herein can be extended to a hybrid state machine including a combination of three or more trellis state machines.

[0147] FIG. 12 is a flow diagram of a method 1200 for coding a current block using TCQ according to implementations of this disclosure. The method 1200 may be performed by a coder (such as the encoder 400 shown in FIG. 4 or the decoder 500 shown in FIG. 5) and / or one or more components of the computing device 200 depicted in FIG. 2. The method 1200 is described in the context of coding a block of transform coefficients, such as DCT coefficients, using TCQ.

[0148] At step 1202, the coder determines a first trellis state machine having a first number of states. In some implementations, the first trellis state machine may have four states. The first trellis state machine may be used to code a proper subset of transform coefficients in the block. The four-state machine may provide adequate coding efficiency for many coefficients having a relatively high frequency while keeping computational complexity relatively low.

[0149] At step 1204, the coder determines a second trellis state machine having a second number of states, the second number of states being greater than the first number of states. In some implementations, the second trellis state machine may have eight states. The second trellis state machine may be used to one or more remaining transform coefficients in the block. The eight-state machine may be used for lower frequency coefficients where the additional complexity is justified by improved coding efficiency.

[0150] Each of an encoder and a decoder may determine the first trellis state machine at step 1202 and the second trellis state machine at step 1204 based on an a priori decision known to each of the encoder and decoder. In some implementations, the encoder can determine the first trellis state machine at step 1202 and the second trellis state machine at step 1204 by selecting from multiple state machines (e.g., pairs of state machines) using, for example, a rate-distortion loop. Thereafter, the encoder can signal the selections, e.g., using one or more indices into a table. The decoder may then determine the first trellis state- l-machine at step 1202 and the second trellis state machine at step 1204 by decoding the identifiers of the state machines sent by the encoder.

[0151] At step 1206, the coder codes a proper subset of transform coefficients of the block using the first trellis state machine. In some implementations, the proper subset of transform coefficients may be determined based on a scan position of the transform coefficients. For example, the proper subset of transform coefficients may include all transform coefficients that are located within a predetermined scan position range starting at the EOB position (e.g., up to a defined scan position). The proper subset of transform coefficients corresponds to a high frequency region of the block, for example. The defined scan position for switching from one trellis state machine to the next.

[0152] Coding the proper subset of transform coefficients at an encoder at step 1206, such as the encoder 400, can include quantizing each transform coefficient using the quantizer indicated by the current state of the first trellis state machine and entropy encoding the quantized transform coefficient using the probability model indicated by the current state of the first trellis state machine. Similarly, coding the proper subset of transform coefficients at a decoder at step 1206, such as the decoder 500, can include entropy decoding each quantized transform coefficient using the probability model indicated by the current state of the first trellis state machine and dequantizing each transform coefficient using the quantizer indicated by the current state indicated by the first trellis state machine.

[0153] At step 1208, the coder codes a transform coefficient of the block that is subsequent to the proper subset using the second trellis state machine. In some implementations, the coder codes all remaining (subsequent) transform coefficients of the block after a defined scan position. The transform coefficient(s) may correspond to a high frequency region of the block, for example. This approach recognizes that low frequency coefficients often contain more significant information and may benefit from the more sophisticated coding provided by the larger state machine.

[0154] Coding the transform coefficient(s) at an encoder at step 1208, such as the encoder 400, can include quantizing each transform coefficient using the quantizer indicated by the current state of the second trellis state machine and entropy encoding the quantized transform coefficient using the probability model indicated by the current state of the second trellis state machine. Similarly, coding the transform coefficient(s) at a decoder at step 1208, such as the decoder 500, can include entropy decoding each quantized transform coefficient using the probability model indicated by the current state of the second trellis state machine anddequantizing each transform coefficient using the quantizer indicated by the current state indicated by the second trellis state machine.

[0155] As described previously, a decoder may further determine a residual block by inverse transforming the dequantized transform coefficients from steps 1206 and 1208, determine a prediction block, and add the prediction block to the residual block to reconstruct the current block.

[0156] As mentioned, FIG. 12 describes an example of a hybrid state machine including two trellis state machines. An example 1300 of a scan order described in steps 1206 and 1208 is shown in FIG. 13. FIG. 13 is a diagram of the example 1300 showing a scan order for an 8x8 block 1302 of transform coefficients according to implementations of this disclosure. The example 1300 illustrates how the hybrid state machine approach might be applied to a block of transform coefficients when the scan order is a diagonal scan. Other scan orders are possible.

[0157] The transform coefficients after the EOB position are all zero, so the first transform coefficient coded at step 1206 is the transform coefficient at the EOB position. The coefficients are scanned in the scan order, starting from the farthest (e.g., the highest frequency) AC coefficient (at the EOB position) and moving towards the DC (i.e., the lowest frequency) coefficient. In this example, the first trellis state machine is used for coding coefficients in the region labeled default (the proper subset of transform coefficients at step 1206), while the second trellis state machine is used for coding coefficients (the coefficients at step 1208) in the region labeled low frequency (LF). The position 1304 is the defined scan position referred to in steps 1206 and 1208, where the transition to the second trellis state machine occurs.

[0158] This hybrid state machine approach allows for flexible complexity-performance tradeoffs within a single block of transform coefficients. By using a simpler trellis state machine (i.e., with fewer states) for a subset of coefficients and a more complex trellis state machine (i.e., with more states) for the remaining coefficient(s), coding efficiency and computational complexity can be balanced. The approach uses the more complex trellis state machine for lower frequency coefficients, which may be more sensitive to quantization errors. The higher complexity state machine may be justified for the low frequency coefficients, as those may have a greater impact on the overall quality of the reconstructed video.

[0159] The transition (i.e., the defined scan position) from the first trellis state machine to the second trellis state machine can be based on various factors. In some implementations, thetransition may occur at a position corresponding to a defined frequency value (e.g., depending on the block size). The transition to the second trellis state machine may occur after a defined number of coefficients have been coded or when a defined number of coefficients remain to be coded (e.g., based on block size, EOB position, or both). In some implementations, the defined scan position may be based on magnitudes of the coefficients, an accumulated coding cost, other relevant metrics, or some combination thereof. By basing the defined scan position on such variables, the hybrid state machine can adapt to the characteristics of the specific block being coded.

[0160] To use a hybrid state machine, there needs to be a mapping between states when transitioning from one trellis state machine to another trellis state machine. More specifically, the state of the transform coefficient before the coefficient at the defined scan position determined using one trellis state machine needs to be used to transition to the next state for the next transform coefficient at the defined scan position, where the next state is a state in the other trellis state machine. In an example where the first trellis state machine is a 4-state trellis state machine, and the second trellis state machine is an 8-state trellis state machine, the state mapping is shown in Table 7.

[0161] Table 7

[0162] This mapping ensures that the state transitions are consistent when the coder switches between trellis state machines.

[0163] How the mapping is used can be seen with reference to FIG. 14. FIG. 14 FIG. 8 is a schematic diagram illustrating an example 1400 of an encoding flow using TCQ according to implementations of this disclosure. The example 1400 of FIG. 14 proceeds like the example 800 of FIG. 8 except that the example 1400 transitions between states 1402 of the 4- state trellis state machine to states 1404 of the 8-state trellis state machine within a single block of coefficients to affect a hybrid state machine instead of using a single trellis state machine. FIG. 14 is simplified by omitting the all_zero state. Moreover, only four stages are shown.

[0164] At stage n, the transitions to stage n+1 are based on the parity of the current transform coefficient according to the 4-state trellis state diagram. At stage n+1, the nexttransform coefficient, which is quantized according the 4- state trellis date diagram at the indicated state, is the transform coefficient just before the defined scan position where transition occurs to using the 8-state trellis state diagram. The transitions to stage n+2 from stage n+1 are based on the 8-state trellis state diagram. More particularly, each of the states 0 - 3 is transitioned from the mapped (new) state according to the next trellis state diagram, in this example the 8-state trellis state diagram. Accordingly, the current state 0 at stage n+1 transitions to stage n+2 according to new state 0 (Table 7) in the 8-state trellis state diagram — to state 0 or state 4 at stage n+2 depending on parity (Table 4). Similarly, the current state 1 at stage n+1 transitions to stage n+2 according to new state 2 in the 8-state trellis state diagram — to state 1 or state 5 at stage n+2 depending on parity, the current state 2 at stage n+1 transitions to stage n+2 according to new state 4 in the 8-state trellis state diagram — to state 2 or state 6 at stage n+2 depending on parity, and the current state 3 at stage n+1 transitions to stage n+2 according to new state 6 in the 8-state trellis state diagram — to state 7 or state 3 at stage n+2 depending on parity. The transitions from stage n+2 to stage n+3 are performed according to the 8-state trellis stage machine.

[0165] The example 1400 of FIG. 14 describes an encoding flow where a single transition point between two trellis state machines exists. As explained previously however, more than two trellis state machines may be used for coding transform coefficients of a block.Moreover, the transition point (e.g., the defined coding position) can vary, increasing the options at each stage.

[0166] This hybrid state machine approach provides a flexible solution to the challenge of balancing coding efficiency and computational complexity in TCQ. By allowing for more sophisticated coding of important coefficients while using simpler coding for less critical coefficients, the hybrid state machine approach enables improved rate-distortion performance without excessive increases in computational demands. This can lead to better video quality at the same bitrate, or reduced bitrate for the same quality, particularly in scenarios where computational resources are constrained.

[0167] As explained previously and can be seen from this description, TCQ can be computationally intensive. Another improvement that can be used either alone or with the improvements herein is to consider adjustment to the EOB position. As explained with regards to FIG. 8, the encoding flow starts with processing non-zero coefficients from the farthest AC non-zero coefficient (which is referred to above as the initial EOB candidate position) along the coefficient scan order. As the complexity of the trellis search is proportional to the scan position of the initial EOB candidate, it would be beneficial to movethe EOB position closer to the DC coefficient while preserving most of the coding gain. An example that achieves this goal is shown in FIG. 15.

[0168] FIG. 15 is a flow diagram of a method 1500 for encoding a current block using TCQ according to implementations of this disclosure. The method 1500 may be performed by an encoder (such as the encoder 400 shown in FIG. 4) and / or one or more components of the computing device 200 depicted in FIG. 2. The method 1500 is described in the context of encoding a block of transform coefficients, such as DCT coefficients, using TCQ.

[0169] At step 1502, the encoder generates quantized transform coefficients for a current block using TCQ. The current block may be a block of quantized DCT coefficients, for example, which are generated by transforming a block of video data and quantizing the transform coefficients using TCQ. The encoder may use a trellis state machine, such as the 4- state or 8-state trellis state machine described in FIG. 6 or FIG. 7, to guide the quantization of the transform coefficients.

[0170] At step 1504, the encoder determines a first candidate EOB position based on a scan order of the quantized transform coefficients. The scan order may be a diagonal scan order, for example, as illustrated in FIG. 13. The first candidate EOB position may be the farthest AC non-zero coefficient along the coefficient scanning line.

[0171] At step 1506, the encoder determines a second candidate EOB position based on the scan order of the quantized transform coefficients. The second candidate EOB position may have a lower frequency than the first candidate EOB position and may be separated from the first candidate EOB position by at least one position, each having a zero value for its quantized transform coefficient. In some implementations, the second candidate EOB position may be the next non-zero coefficient after the first candidate EOB position. In some implementations, the first candidate EOB position may be separated from the second candidate EOB position by multiple zero coefficients. This allows the method to consider more distant EOB positions that might provide better coding efficiency.

[0172] At step 1508, the encoder calculates a first coding efficiency value associated with the first candidate EOB position. The first coding efficiency value may comprise a combination of a distortion resulting from using a first non-zero quantized transform coefficient value of the first candidate EOB position and a bit cost for sending the first nonzero quantized transform coefficient value and each zero value of the at least one position. The distortion may be measured using a metric such as mean squared error (MSE) or sum of squared error (SSE). The bit cost may be determined based on the number of bits required to encode the non-zero quantized transform coefficient value and the zero values.

[0173] At step 1510, the encoder calculates a second coding efficiency value associated with the second candidate EOB position. The second coding efficiency value may comprise a combination of a distortion resulting from using a second non-zero quantized transform coefficient value of the second candidate EOB position and a zero value for the first non-zero quantized coefficient and a bit cost for sending the second non-zero quantized transform coefficient value. The distortion may be measured using a metric such as MSE or SSE. The bit cost may be determined based on the number of bits required to encode the non-zero quantized transform coefficient value.

[0174] At step 1512, the encoder selects an EOB position for the current block as whichever of the first coding efficiency value or the second coding efficiency value is lower. The EOB position with the lower coding efficiency value may result in a more efficient coding of the transform coefficients in the block. The EOB position with the lower coding efficiency value may be considered more efficient in terms of rate-distortion performance.

[0175] At step 1514, the encoder encodes the quantized transform coefficients into an encoded bitstream in the scan order starting at the selected EOB position. The encoder may use an entropy coding scheme, such as variable length coding (VLC) or arithmetic coding, to encode the quantized transform coefficients. The encoded bitstream may also include other information, such as syntax elements indicating the type of prediction used, transform type, motion vectors, quantizer value, or the like.

[0176] In some implementations, the method 1500 may include evaluating more than two candidate EOB positions, using different criteria for selecting candidate EOB positions, or incorporating additional factors into the coding efficiency calculations. The specific implementation can be optimized based on the characteristics of the video content and the requirements of the codec. In some implementations, the method 1500 can be implemented in conjunction with other TCQ improvements, such as the modified coefficient syntax, the hybrid state machine, and / or the dynamic node pruning technique described in this disclosure. This can lead to further improvements in compression efficiency and computational complexity.

[0177] This enhanced EOB decision process evaluates multiple candidate EOB positions to reduce unnecessary trellis computations. By considering both a first candidate EOB position (typically the last non-zero coefficient) and a second candidate EOB position (a lower frequency position separated by zero coefficients), the method can potentially identify a more efficient coding strategy. In some implementations, by potentially selecting an earlier EOB position, the method 1500 can avoid unnecessary computations on coefficients that arelikely do not contribute significantly to image quality. In some implementations, the method 1500 can adapt to different types of content, potentially choosing different EOB strategies for blocks with different characteristics. In some implementations, this approach can be integrated into existing TCQ frameworks without requiring major changes to the overall coding structure.

[0178] In an implementation, the following algorithm may be used assuming nonzero coefficients (coeffl and coeff2) located respectively in location 1 and location! with multiple zeros in between. In this algorithm location 1 > location!, where the DC coefficient is identified as position 0 and the position count increases by 1 in a scan order per AC transform coefficient. The coding efficiency value is a rate-distortion error (RD) using a constant lambda to balance the

[0179] 1. Compute related costs of setting location 1 as EOB position according to ratedistortion optimization 1 (RDO1) = distortionl + lambda*bit_costl, where distortionl assumes sending non-zero coeffl, and bit_costl includes sending coeffl plus all zeros between location 1 and location!.

[0180] 2. Compute relates costs of setting location! as EOB position according to ratedistortion optimization 2 (RDO2) = distortion! + lambda*bit_cost2, where distortion! assumes sending a zero in location 1 and the nonzero coeff2 in location!, and bit_cost2 includes sending coeff2.

[0181] 3. In most cases, it is expected that distortionl < distortion! and bit_costl > bit_cost2. Accordingly, RDO1 is compared to RDO2. If RDO1 <= RDO2, EOB is set to locationl. Otherwise, EOB is set to location!.

[0182] As referenced previously, trellis coded quantization can be computationally expensive. A 4-state trellis state machine, for example, needs to perform 4x3 (assuming additionally 0 is also evaluated from each node) rate-distortion (RD) cost calculations at each position. Accordingly, a technique to prune those nodes in the graph that are unlikely to “win”, that is, are unlikely to produce the lowest RD cost, is desirable. One example of a pruning technique that can be used either alone or in combination with any or all of the other techniques described herein is shown by FIG. 16.

[0183] FIG. 16 is a flow diagram of a method 1600 for encoding transform coefficients of a current block using TCQ according to implementations of this disclosure. The method 1600 may be performed by a coder (such as the encoder 400 shown in FIG. 4) and / or one or more components of the computing device 200 depicted in FIG. 2. The method 1600 is describedin the context of coding a current block of transform coefficients, such as DCT coefficients, using TCQ.

[0184] At step 1602, the encoder calculates a set of efficiency cost measures, each efficiency cost measure corresponding to a respective state in a trellis state machine at a current transform coefficient position in the current block. The term "efficiency cost measure" refers to a metric that quantifies the trade-off between compression efficiency and computational complexity. In the context of TCQ, the efficiency cost measure may be the RD cost, which is a measure of the trade-off between the distortion introduced by quantization and the number of bits required to encode the quantized values. The trellis state machine may be a 4-state or 8-state trellis state machine, for example, as described in FIG. 6 or FIG. 7. The current transform coefficient position may be a position in a scan order of the transform coefficients, such as a diagonal scan order as illustrated in FIG. 13.

[0185] At step 1604, the encoder determines a minimum efficiency cost value from among the set of efficiency cost measures. The minimum efficiency cost value may correspond to the state with the lowest RD cost, for example. At step 1606, the method 1600 determines a dynamic threshold based on a distance (e.g., a position count) of the current transform coefficient position from a DC coefficient position of the current block. The DC coefficient position is located in the upper-left comer of the block. The dynamic threshold may be determined based on the distance of the current transform coefficient position from the DC coefficient position, the quantization parameter (QP), or the block size of the block of DCT coefficients. In some implementations, the dynamic threshold may be increased as the distance of the current transform coefficient position from the DC coefficient position decreases. This is because nodes with higher accumulated RD cost have a slimmer chance of catching up with nodes with lower RD cost as the current transform coefficient position gets closer to the DC coefficient position. Conversely, the dynamic threshold may be decreased as the distance from the DC coefficient position increases. This strategy recognizes that coefficients closer to the DC coefficient are generally more important for image quality and should be evaluated more thoroughly.

[0186] At step 1608, the coder prunes states having an efficiency cost measure greater than a pruning threshold value based on the minimum efficiency cost measure and the dynamic threshold. The term “pruning threshold value” refers to a value that is used to determine which states are pruned from the trellis. The pruning threshold value is based on the minimum efficiency cost measure and the dynamic threshold. The pruning threshold value may be defined as a product of the minimum efficiency cost measure and the dynamicthreshold. For example, if the minimum efficiency cost measure is 10 and the dynamic threshold is 1.5, the pruning threshold value would be 15. The states with an efficiency cost measure greater than the pruning threshold value are considered unlikely to be part of the optimal quantization path and are therefore pruned from the trellis.

[0187] Some implementations may include different methods for calculating the dynamic threshold, incorporating additional factors into the pruning decision, and / or applying the pruning technique selectively based on content characteristics or encoding parameters. The specific implementation can be optimized based on the requirements of the codec and the target applications.

[0188] In summary, assume that the accumulated costs of 4 nodes at position i are Cost_0, Cost_l, Cost_2, and Cost_3. Cost_min denotes the minimum cost of those 4 nodes. Then, those nodes with a cost larger than Cost_min * Threshold_i may be pruned such that any further RD cost calculation from pruned nodes are skipped. The threshold at each position i (Threshold_i) is determined by its distance to the DC position. The closer the position is to the DC coefficient, nodes with a higher accumulated RD cost have reduced chances to catch up with those with lower RD cost.

[0189] The method 1600 provides a more efficient approach to encoding transform coefficients using TCQ. By pruning states with high efficiency cost measures, the coder can reduce the number of trellis paths that need to be evaluated, thereby improving computational efficiency. The dynamic threshold allows the encoder to adapt the pruning process to different coding scenarios, such as different QP values or block sizes. This can lead to further improvements in computational efficiency and compression efficiency. In some implementations, the dynamic threshold allows the pruning to adapt to different parts of the coefficient block, potentially preserving more paths for important coefficients while aggressively pruning less important ones. The method 1600 may be implemented in conjunction with other TCQ techniques, such as the modified coefficient syntax, the hybrid state machine, and / or the enhanced EOB decision process described in this disclosure. This can lead to further improvements in compression efficiency and computational complexity.

[0190] According to the disclosure herein, improvements to TCQ may be achieved by using one or more of the TCQ improvements described herein.

[0191] The word “example” or the like is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as “example” or the like is not necessarily to be construed as being preferred or advantageous over other aspects or designs. Rather, use of the word “example” or the like is intended to present concepts in a concretefashion. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise or clearly indicated otherwise by the context, the statement “X includes A or B” is intended to mean any of the natural inclusive permutations thereof. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied under any of the foregoing instances. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more,” unless specified otherwise or clearly indicated by the context to be directed to a singular form. Moreover, use of the term “an implementation” or the term “one implementation” throughout this disclosure is not intended to mean the same embodiment or implementation unless described as such. As used herein, the terms “determine” and “identify”, or any variations thereof, includes selecting, ascertaining, computing, looking up, receiving, determining, establishing, obtaining, or otherwise identifying or determining in any manner whatsoever using one or more of the devices described herein.

[0192] Implementations of the transmitting station 102 and / or the receiving station 106 (and the algorithms, methods, instructions, etc., stored thereon and / or executed thereby, including by the encoder 400 and the decoder 500) can be realized in hardware, software, or any combination thereof. The hardware can include, for example, computers, intellectual property (IP) cores, application- specific integrated circuits (ASICs), programmable logic arrays, optical processors, programmable logic controllers, microcode, microcontrollers, servers, microprocessors, digital signal processors, or any other suitable circuit. In the claims, the term “processor” should be understood as encompassing any of the foregoing hardware, either singly or in combination. The terms “signal” and “data” are used interchangeably. Further, portions of the transmitting station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0193] Further, in one aspect, for example, the transmitting station 102 or the receiving station 106 can be implemented using a general-purpose computer or general-purpose processor with a computer program that, when executed, carries out any of the respective methods, algorithms, and / or instructions described herein. In addition, or alternatively, for example, a special purpose computer / processor can be utilized that can contain other hardware for carrying out any of the methods, algorithms, or instructions described herein.

[0194] The transmitting station 102 and the receiving station 106 can, for example, be implemented on computers in a video conferencing system. Alternatively, the transmitting station 102 can be implemented on a server, and the receiving station 106 can beimplemented on a device separate from the server, such as a handheld communications device. In this instance, the transmitting station 102, using an encoder 400, can encode content into an encoded video signal and transmit the encoded video signal to the communications device. In turn, the communications device can then decode the encoded video signal using a decoder 500. Alternatively, the communications device can decode content stored locally on the communications device, for example, content that was not transmitted by the transmitting station 102. Other suitable transmitting and receiving implementation schemes are available. For example, the receiving station 106 can be a generally stationary personal computer rather than a portable communications device, and / or a device including an encoder 400 may also include a decoder 500.

[0195] Further, all or a portion of implementations of the present disclosure can take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium can be any device that can, for example, tangibly contain, store, communicate, or transport the program for use by or in connection with any processor. The (e.g., non-transitory) medium can be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable mediums are also available.

[0196] The above-described embodiments, implementations, and aspects have been described to facilitate easy understanding of this disclosure and do not limit this disclosure. On the contrary, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which scope is to be accorded the broadest interpretation as is permitted under the law to encompass all such modifications and equivalent arrangements.

Claims

What is claimed is:

1. A method for coding a current block, the method comprising: decoding, from a first pass syntax, a base level value and a mid level value of a transform coefficient of a set of transform coefficients of the current block using trellis coded quantization (TCQ); decoding a high level value of the transform coefficient using TCQ; determining a parity of the transform coefficient based on a sum of the base level value, the mid level value, and the high level value multiplied by an even number; and decoding a next transform coefficient of the set of transform coefficients using the TCQ according to a state based on the parity.

2. The method of claim 1, wherein the even number is two.

3. The method of one of claim 1 or claim 2, wherein the set of transform coefficients comprises a set of discrete cosine transform coefficients.

4. The method of any one of claims 1 to 3, wherein decoding the high level value comprises decoding the high level value using a bypass coding scheme.

5. The method of any one of claims 1 to 3, wherein decoding the base level value and the mid level value comprises decoding the base level value and the mid level value using a multi-symbol encoder.

6. A method for coding a current block, the method comprising: encoding, in a first coding pass, a base level value and a mid level value for a transform coefficient of a set of transform coefficients in the current block using trellis coded quantization (TCQ), wherein a value of the transform coefficient is equal to the base level value plus the mid level value plus a high level value; determining that, for the transform coefficient, a sum of the base level value and the mid level value for the transform coefficient is equal to or greater than a threshold value below a maximum value for the sum; encoding, in a second coding pass, the high level value for the transform coefficient, wherein the value of the transform coefficient is equal to or greater than the maximum value,and wherein a parity of the transform coefficient is indicated by the sum of the base level value and the mid level value; and encoding a next transform coefficient of the set of transform coefficients using the parity as input to the TCQ.

7. The method of claim 6, wherein the high level value is an odd integer.

8. The method of one of claim 6 or claim 7, wherein the transform coefficient is a discrete cosine transform coefficient.

9. The method of any one of claims 6 to 8, comprising: decoding the transform coefficient using TCQ; calculating the parity of the transform coefficient as a sum of the base level value, the mid level value, and the high level value multiplied by an even integer; and decoding a next transform coefficient using TCQ according to a state based on the parity.

10. The method of claim 9, wherein the even integer is 2, and the threshold value is the maximum value minus 1.

11. A method for coding a current block, the method comprising: selecting a first trellis state machine having a first number of states for trellis coded quantization (TCQ); selecting a second trellis state machine having a second number of states for TCQ, the second number of states being greater than the first number of states; coding, in a scan order, a proper subset of transform coefficients of the current block using the first trellis state machine; and coding, in the scan order, one or more subsequent transform coefficients of the current block using the second trellis state machine.

12. The method of claim 11, wherein the first trellis state machine has four states, and the second trellis state machine has eight states.

13. The method of one of claims 11 or 12, wherein the proper subset of transform coefficients corresponds to a low frequency region of the block.

14. The method of any one of claims 11 to 13, wherein a transition from the first trellis state machine to the second trellis state machine is based on a defined position within the scan order.

15. The method of any one of claims 11 to 14, wherein a transition from the first trellis state machine to the second trellis state machine is based on a predetermined threshold value.

16. The method of any one of claims 11 to 15, wherein coding the current block comprises decoding the current block.

17. The method of any of claims 11 to 16, comprising: mapping a state of the first trellis state machine after coding a final transform coefficient of the proper subset of transform coefficients to a state of the second trellis state machine before coding the one or more subsequent transform coefficients.

18. A method for coding a current block, the method comprising: generating quantized transform coefficients for the current block using trellis coded quantization (TCQ); determining a first candidate end-of-block (EOB) position based on a scan order of the quantized transform coefficients; determining a second candidate EOB position based on the scan order of the quantized transform coefficients, wherein the second candidate EOB position has a lower frequency than the first candidate EOB position, and the second candidate EOB position is separated from the first candidate EOB position by at least one position, each position having a zero value for its quantized transform coefficient; calculating a first coding efficiency value associated with the first candidate EOB position; calculating a second coding efficiency value associated with the second candidate EOB position; selecting an EOB position for the current block as whichever of the first coding efficiency value or the second coding efficiency value is lower; and encoding the quantized transform coefficients into an encoded bitstream in the scan order starting at the EOB position.

19. The method of claim 18, wherein the first coding efficiency value comprises a combination of a distortion resulting from using a first non-zero quantized transform coefficient value of the first candidate EOB position and a bit cost for sending the first nonzero quantized transform coefficient value and each zero value of the at least one position.

20. The method of one of claim 18 or claim 19, wherein the second coding efficiency value comprises a combination of a distortion resulting from using a second non-zero quantized transform coefficient value of the second candidate EOB position and a zero value for a first non-zero quantized coefficient and a bit cost for sending the second non-zero quantized transform coefficient value.

21. The method of any one of claims 18 to 20, wherein the first candidate EOB position is separated from the second candidate EOB position by multiple zero coefficients.

22. A method for coding a current block, the method comprising: calculating a set of efficiency cost measures, each efficiency cost measure corresponding to a respective state in a trellis state machine at a current transform coefficient position in the current block being encoded using trellis coded quantization (TCQ); determining a minimum efficiency cost value from among the set of efficiency cost measures; determining a dynamic threshold based on a distance of the current transform coefficient position from a DC coefficient position of the current block; and pruning states having an efficiency cost measure greater than a pruning threshold value based on the minimum efficiency cost measure and the dynamic threshold.

23. The method of claim 22, wherein the pruning threshold value comprises a product of the minimum efficiency cost measure and the dynamic threshold.

24. The method of one of claim 22 or claim 23, wherein the dynamic threshold is increased as the distance of the current transform coefficient position from the DC coefficient position decreases.

25. The method of any one of claims 22 to 24, wherein the dynamic threshold is decreased as the distance of the current transform coefficient position from the DC coefficient position increases.

26. The method of any one of claims 22 to 25, wherein the dynamic threshold is determined based on a quantization parameter (QP).

27. The method of any one of claims 22 to 25, wherein the dynamic threshold is determined based on a block size of the block of discrete cosine transform (DCT) coefficients.

28. An apparatus for coding a current block, the apparatus comprising: a processor configured to execute the method of any one of claims 1 to 27.

29. An apparatus for coding a current block, the apparatus comprising: a processor; and a memory storing instructions that, when executed, cause the processor to perform operations according to the method of any one of claims 1 to 27.

30. A non-transitory, computer-readable medium storing instructions that, when executed, cause a processor to perform operations according to the method of any one of claims 1 to 27.

31. A non-transitory, computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a current block encoded according to the method of any one of claims 6 to 27.

32. A non-transitory, computer-readable medium storing an encoded bitstream, the encoded bitstream comprising a current block to be decoded according to the method of any one of claims 1 to 5 and 11 to 17.

Images