Vector quantization for prediction residual coding

Abstract

A device for decoding a current block includes a memory and a processor. The processor is configured to execute instructions stored in the memory to: decode from a compressed bitstream a flag indicating whether a residual block of the current block is encoded using vector quantization (VQ); in response to the flag indicating that the residual block is encoded using VQ, execute instructions to: decode a parameter indicating an entry in a codebook; and decode the residual block using the entry; in response to the flag indicating that the residual block is not encoded using VQ, execute instructions to: decode the residual block based on a skip flag indicating whether the current block is encoded using transform skipping; and reconstruct the current block using the residual block.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Patent Application Serial No. 62 / 940,271, filed on November 26, 2019, the entire disclosure of which is incorporated herein by reference. Background Art

[0003] A digital video stream can represent a video using a series of frames or still images. Digital video can be used for a variety of applications, including, for example, video conferencing, high-definition video entertainment, video advertising, or sharing of user-generated videos. Digital video streams can contain large amounts of data and consume significant computing or communication resources of computing devices used to process, transmit, or store video data. Various methods have been proposed to reduce the amount of data in video streams, including encoding and decoding techniques. Summary of the Invention

[0004] A first aspect is an apparatus for decoding a current block of a video. The apparatus includes a memory and a processor. The processor is configured to execute instructions stored in the memory to: decode, from a compressed bitstream, a flag indicating whether a residual block of the current block is encoded using vector quantization (VQ); in response to the flag indicating that the residual block is encoded using VQ, execute instructions to: decode a parameter indicating an entry in a codebook; and decode the residual block using the entry; in response to the flag indicating that the residual block is not encoded using VQ, execute instructions to: decode the residual block based on a skip flag indicating whether the current block is encoded using transform skipping; and reconstruct the current block using the residual block.

[0005] A second aspect is an apparatus for encoding a current block using vector quantization (VQ). The apparatus includes a memory and a processor. The processor is configured to execute instructions stored in the memory to: determine whether the current block is encoded without using a transform skip mode; and in response to determining that the current block is encoded without using a transform skip mode, execute instructions to: obtain a residual block for the current block; select a VQ gain index and a VQ shape index using the residual block; encode the VQ gain index in a compressed bitstream; and encode the VQ shape index in the compressed bitstream.

[0006] A third aspect is a method for encoding a current block using vector quantization (VQ). The method includes: determining that the current block is not encoded using a transform skip mode; obtaining a residual block of the current block; selecting a VQ gain index and a VQ shape index using the residual block; encoding the VQ gain index in a compressed bitstream; and encoding the VQ shape index in the compressed bitstream.

[0007] These and other aspects of the present disclosure are disclosed in the following detailed description of implementations, the appended claims, and the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The description herein refers to the drawings described below, wherein like reference numerals refer to like parts throughout the several views.

[0009] Figure 1 is a schematic diagram of an example of a video encoding and decoding system.

[0010] Figure 2 is a block diagram of an example of a computing device that may implement a sending station or a receiving station.

[0011] Figure 3 is a diagram of an example of a video stream to be encoded and then decoded.

[0012] Figure 4 is a block diagram of an example of an encoder.

[0013] Figure 5 is a block diagram of an example of a decoder.

[0014] Figure 6 is a diagram of examples of features exhibited by residual blocks.

[0015] Figure 7 is a diagram of an example of a codebook according to an implementation of the present disclosure.

[0016] Figure 8 is a diagram of the functionality of a prediction residual encoding stage using vector quantization according to an implementation of the present disclosure.

[0017] Figure 9 is a diagram of a flowchart of a technique for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure.

[0018] Figure 10 is a diagram of a flowchart of a technique for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure.

[0019] Figure 11 is a diagram of a flowchart of a technique for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure.

[0020] Figure 12 is a diagram of a flowchart of a technique for decoding a current block according to implementations of the present disclosure.

[0021] Figure 13 is a diagram of a flowchart of a technique for encoding a current block using vector quantization according to an implementation of the present disclosure. DETAILED DESCRIPTION

[0022] A video compression scheme may include breaking down a corresponding image or frame into smaller parts, such as blocks, and using techniques to generate a coded bitstream to limit the information included for its corresponding blocks. The coded bitstream can be decoded to recreate the source image with limited information. For example, a video compression scheme may include transforming the prediction residuals (i.e., residual blocks) of a current block of a video stream from the pixel domain into transform coefficients of a transform block in the frequency domain. The transform coefficients are quantized and entropy coded into a coded bitstream. A decoder decodes or decompresses the coded bitstream using the coded transform coefficients to prepare the video stream for viewing or further processing. To reconstruct the prediction residuals, the decoder may receive the quantized transform coefficients, dequantize the transform coefficients, and inverse transform the dequantized transform coefficients.

[0023] There may be many different transform types and transform sizes that can be used to transform the prediction residual for a given block. There may be up to 16 or even more transform types available, such as discrete cosine transform (DCT) or asymmetric discrete sine transform (ADST). There may be a different number of transform sizes available, such as based on the size of the block representing the prediction residual. For example, an 8x8 block representing the prediction residual can be transformed using one 8x8 transform block or four 4x4 transform blocks.

[0024] In order to achieve optimal compression efficiency, a typical encoder tries many transform types and transform sizes and selects a transform type and transform size combination for transforming the prediction residual that produces the lowest rate-distortion cost. This process is called transform search. However, transform search can be a very time-consuming process because the number of transform types and sizes can be large. Furthermore, in many cases, the encoder may repeat the transform search process multiple times for the same prediction residual (for example, where two different prediction modes result in the same prediction and therefore produce the same prediction residual). In a typical coding process, after the prediction residual is transformed.

[0025] The implementation of the present disclosure solves these problems by omitting the transformation of the prediction residual to the frequency domain. Therefore, the coding steps associated with the transformation (in the encoder) and inverse transformation (in the decoder) of the prediction residual can be omitted (i.e., bypassed).

[0026] As further described below, the current block can be predicted using intra prediction or inter prediction. Some prediction residuals (i.e., prediction signals) generated by intra prediction can exhibit certain characteristics. Due to the high frequencies in the transform coefficients, encoding the transform coefficients of such prediction residuals (i.e., prediction residuals that exhibit certain characteristics) often requires a higher bit rate.

[0027] Figure 66 is a diagram of an example 600 of features exhibited by residual blocks. Example 600 includes illustrative prediction residual blocks 602 to 612. Prediction residual blocks 602 to 612 are examples of 4x4 residual blocks. However, prediction residual blocks can be of any size. Prediction residual blocks 602 to 604 are examples of prediction residuals that exhibit local features. The contrast of pixel values ​​indicates local features. Prediction residual blocks 606 to 608 are examples of prediction residuals that exhibit non-smooth features. Prediction residual blocks 610 to 612 are examples of prediction residuals that exhibit diagonal pattern features. In this case, the diagonal pattern means a northeast-southwest diagonal (e.g., prediction residual block 610) or a northwest-southeast diagonal (e.g., prediction residual block 611).

[0028] According to an implementation of the present disclosure, the encoder may search for the closest match to the prediction residual in a codebook (i.e., a lookup table) instead of transforming the prediction residual into the frequency domain. The codebook may include a prototype prediction residual. The prototype prediction residual may be referred to as a codeword or code vector in this document. The encoder may encode the index of the closest match in the compressed bitstream. Thus, the approximation of the prediction block may be indicated in the compressed bitstream (via an index). Therefore, the prediction residual is said to be quantized to the closest matching code vector. The decoder may decode the index, search for the prototype prediction residual in the codebook, and reconstruct the indicated prototype prediction residual into the prediction residual. As further described below, the lookup table may be more than one lookup table. For example, the lookup table may include a shape codebook and a gain codebook. As further described below, the index may be more than one index. For example, the index may include a shape index (i.e., an indicator of an entry in the shape codebook) and a gain index (i.e., an indicator of an entry in the gain codebook).

[0029] By using a codebook of prediction residuals, transform coding can be bypassed. Thus, the bit rate can be reduced, especially for those prediction residuals that exhibit high frequencies, which would otherwise require a high bit rate to code the transform coefficients. Additionally, the decoding process can be faster than a decoding process involving an inverse transform of the transform coefficients.

[0030] Further details of techniques for vector quantization in video compression are described herein first with reference to systems in which they may be implemented, such as Figures 1 to 6 shown.

[0031] Figure 1 is a diagram of an example of a video encoding and decoding system 100. For example, the transmitting station 102 may be a system having a Figure 2 The hardware described herein is configured internally as a computer. However, other implementations of the sending station 102 are also possible. For example, the processing of the sending station 102 may be distributed among multiple devices.

[0032] Network 104 can connect the sending station 102 and the receiving station 106 to encode and decode the video stream. Specifically, the video stream can be encoded in the sending station 102, and the encoded video stream can be decoded in the receiving station 106. Network 104 can be, for example, the Internet. In this example, network 104 can also be a local area network (LAN), a wide area network (WAN), a virtual private network (VPN), a cellular telephone network, or any other device that transmits the video stream from the sending station 102 to the receiving station 106 in this example.

[0033] In one example, the receiving station 106 may be a computer having a Figure 2 The hardware described herein is configured internally as a computer. However, other suitable implementations of the receiving station 106 are also possible. For example, the processing of the receiving station 106 may be distributed among multiple devices.

[0034] Other implementations of the video encoding and decoding system 100 are also possible. For example, an implementation may omit the network 104. In another implementation, the video stream may be encoded and then stored for transmission to a receiving station 106 or any other device with memory at a later time. In one implementation, the receiving station 106 receives the encoded video stream (e.g., via the network 104, a computer bus, and / or some communication path) and stores the video stream for later decoding. In an example implementation, the Real-time Transport Protocol (RTP) is used to transmit the encoded video over the network 104. In another implementation, a transport protocol other than RTP may be used, (e.g., a video streaming protocol based on the Hypertext Transfer Protocol (HTTP)).

[0035] For example, when used in a video conferencing system, as described below, the transmitting station 102 and / or the receiving station 106 may include the capability to encode and decode video streams. For example, the receiving station 106 may be a video conference participant that receives an encoded video bitstream from a video conferencing server (e.g., the transmitting station 102) for decoding and viewing, and then encodes and transmits his or her own video bitstream to the video conferencing server for decoding and viewing by other participants.

[0036] In some implementations, the video encoding and decoding system 100 may alternatively be used to encode and decode data other than video data. For example, the video encoding and decoding system 100 may be used to process image data. The image data may include blocks of data from an image. In such an implementation, the sending station 102 may be used to encode the image data, and the receiving station 106 may be used to decode the image data. Alternatively, the receiving station 106 may represent a computing device that stores the encoded image data for later use (such as after receiving the encoded or pre-encoded image data from the sending station 102). As yet another alternative, the sending station 102 may represent a computing device that decodes the image data, such as before sending the decoded image data to the receiving station 106 for display.

[0037] Figure 2 is a block diagram of an example of a computing device 200 that may implement a sending station or a receiving station. For example, the computing device 200 may implement Figure 1 The computing device 200 may be in the form of a computing system including multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0038] The processor 202 in the computing device 200 can be a conventional central processing unit. Alternatively, the processor 202 can be another type of device or devices, now existing or hereafter developed, that is capable of manipulating or processing information. For example, although the disclosed implementations can be practiced with one processor (e.g., the processor 202) as shown, advantages in speed and efficiency can be achieved by using more than one processor.

[0039] In an implementation, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. However, other suitable types of storage devices may be used as the memory 204. The memory 204 may include code and data 206 accessed by the processor 202 using a bus 212. The memory 204 may also include an operating system 208 and application programs 210, which include at least one program that allows the processor 202 to perform the techniques described herein. For example, the application programs 210 may include applications 1 through N, which also include a video coding application that performs the techniques described herein. The computing device 200 may also include an auxiliary storage device 214, which may be, for example, a memory card used with a mobile computing device. Because a video communication session may contain a large amount of information, it may be stored in whole or in part in the auxiliary storage device 214 and loaded into the memory 204 as needed for processing.

[0040] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the processor 202 via the bus 212. In addition to or as an alternative to the display 218, other output devices that allow a user to program or otherwise use the computing device 200 may be provided. When the output device is or includes a display, the display may be implemented in various ways, including by a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light emitting diode (LED) display, such as an organic LED (OLED) display.

[0041] The computing device 200 may also include or communicate with an image sensing device 220 (e.g., a camera) or any other image sensing device 220 now existing or hereafter developed that can sense an image, such as an image of a user operating the computing device 200. The image sensing device 220 may be positioned so that it is directed toward the user operating the computing device 200. In an example, the position and optical axis of the image sensing device 220 may be configured so that the field of view includes an area directly adjacent to the display 218, and from which the display 218 is visible.

[0042] The computing device 200 may also include or communicate with a sound sensing device 222 (e.g., a microphone) or any other sound sensing device now existing or hereafter developed that can sense sounds in the vicinity of the computing device 200. The sound sensing device 222 may be positioned so that it is directed toward a user operating the computing device 200 and may be configured to receive sounds, such as voice or other utterances, uttered by the user while the user is operating the computing device 200.

[0043] although Figure 2 The processor 202 and memory 204 of the computing device 200 are depicted as being integrated into a single unit, but other configurations may be used. The operations of the processor 202 may be distributed across multiple machines (where a single machine may have one or more processors), which may be directly coupled or coupled across a local area network or other network. The memory 204 may be distributed across multiple machines, such as network-based storage or storage in multiple machines that perform the operations of the computing device 200. Although depicted here as a single bus, the bus 212 of the computing device 200 may be composed of multiple buses. Further, the auxiliary storage device 214 may be directly coupled to other components of the computing device 200, or may be accessed via a network, and may include an integrated unit such as a memory card or multiple units such as multiple memory cards. Thus, the computing device 200 may be implemented in a variety of configurations.

[0044] Figure 3 3 is a diagram of an example of a video stream 300 to be encoded and subsequently decoded. Video stream 300 includes a video sequence 302. At the next level, video sequence 302 includes multiple adjacent frames 304. Although three frames are depicted as adjacent frames 304, video sequence 302 can include any number of adjacent frames 304. Adjacent frames 304 can then be further subdivided into individual frames, such as frame 306. At the next level, frame 306 can be divided into a series of planes or fragments 308. For example, fragment 308 can be a subset of a frame that allows parallel processing. Fragment 308 can also be a subset of a frame that can separate video data into individual colors. For example, a frame 306 of color video data can include a luma plane and two chroma planes. Fragments 308 can be sampled at different resolutions.

[0045] Regardless of whether the frame 306 is divided into segments 308, the frame 306 can be further subdivided into blocks 310, which can contain data corresponding to, for example, 16x16 pixels in the frame 306. The blocks 310 can also be arranged to include data from one or more segments 308 of pixel data. The blocks 310 can also have any other suitable size, such as 4x4 pixels, 8x8 pixels, 16x8 pixels, 8x16 pixels, 16x16 pixels, or larger. Unless otherwise specified, the terms block and macroblock are used interchangeably herein.

[0046] Figure 4 is a block diagram of an example of an encoder 400. As described above, the encoder 400 may be implemented in the sending station 102, such as by providing a computer software program stored in a memory (e.g., the memory 204). The computer software program may include machine instructions that, when executed by a processor such as the processor 202, cause the sending station 102 to generate a signal in a manner that is consistent with the preceding text. Figure 4 The video data is encoded in the manner described. The encoder 400 may also be implemented as dedicated hardware, for example, included in the transmission station 102. In one particularly desirable implementation, the encoder 400 is a hardware encoder.

[0047] The encoder 400 has the following stages to perform various functions in a forward path (shown by solid connecting lines) to produce an encoded or compressed bitstream 420 using the video stream 300 as input: an intra / inter prediction stage 402, a transform stage 404, a quantization stage 406, and an entropy coding stage 408. The encoder 400 may also include a reconstruction path (shown by dashed connecting lines) to reconstruct frames for encoding future blocks. Figure 4, encoder 400 has the following stages to perform various functions in the reconstruction path: dequantization stage 410, inverse transform stage 412, reconstruction stage 414, and loop filtering stage 416. Other structural variations of encoder 400 may be used to encode video stream 300.

[0048] When the video stream 300 is presented for encoding, corresponding adjacent frames 304 (such as frame 306) can be processed in units of blocks. In the intra / inter prediction stage 402, the corresponding blocks can be encoded using intra prediction (also known as intra prediction) or inter prediction (also known as inter prediction). In either case, prediction blocks can be formed. In the case of intra prediction, the prediction blocks can be formed from samples in the current frame that have been previously encoded and reconstructed. In the case of inter prediction, the prediction blocks can be formed from samples in one or more previously constructed reference frames.

[0049] Next, the prediction block can be subtracted from the current block in the intra / inter prediction stage 402 to produce a residual block (also referred to as a residual). A transform stage 404 transforms the residual into transform coefficients, for example, in the frequency domain, using a block-based transform. A quantization stage 406 converts the transform coefficients into discrete quantum values, referred to as quantized transform coefficients, using a quantizer value or quantization level. For example, the transform coefficients can be divided by the quantizer value and truncated.

[0050] The quantized transform coefficients are then entropy encoded by an entropy encoding stage 408. The entropy encoded coefficients, along with other information used to decode the block (e.g., which may include syntax elements such as for indicating the prediction type used, the transform type, motion vectors, and quantizer values, are then output to a compressed bitstream 420. The compressed bitstream 420 may be formatted using various techniques, such as variable length coding (VLC) or arithmetic coding. The compressed bitstream 420 may also be referred to as an encoded video stream or an encoded video bitstream, and these terms will be used interchangeably herein.

[0051] The reconstruction path (shown by the dashed connecting line) can be used to ensure that the encoder 400 and the decoder 500 (described below) Figure 5 The reconstruction path performs the same functions that occur during the decoding process (described below) Figure 5 4. The intra / inter prediction stage 402 performs similar functions as described above, including dequantizing the quantized transform coefficients in a dequantization stage 410 and inverse transforming the dequantized transform coefficients in an inverse transform stage 412 to produce a derivative residual block (also referred to as a derivative residual). In a reconstruction stage 414, the prediction block predicted in the intra / inter prediction stage 402 can be added to the derivative residual to create a reconstructed block. A loop filtering stage 416 can be applied to the reconstructed block to reduce distortion such as blocking artifacts.

[0052] Other variations of encoder 400 may be used to encode compressed bitstream 420. In some implementations, a non-transform based encoder may directly quantize the residual signal without requiring a transform stage 404 for certain blocks or frames. In some implementations, the encoder may have quantization stage 406 and dequantization stage 410 combined in a common stage.

[0053] Figure 5 is a block diagram of an example of a decoder 500. The decoder 500 may be implemented in the receiving station 106, for example, by providing a computer software program stored in the memory 204. The computer software program may include machine instructions that, when executed by a processor such as the processor 202, cause the receiving station 106 to Figure 5 Decoder 500 may also be implemented in hardware included in, for example, transmitting station 102 or receiving station 106.

[0054] Similar to the reconstruction path of the encoder 400 discussed above, in one example, the decoder 500 includes the following stages to perform various functions to produce an output video stream 516 from the compressed bitstream 420: an entropy decoding stage 502, a dequantization stage 504, an inverse transform stage 506, an intra / inter prediction stage 508, a reconstruction stage 510, a loop filtering stage 512, and a deblocking filtering stage 514. Other structural variations of the decoder 500 may be used to decode the compressed bitstream 420.

[0055] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 may be decoded by an entropy decoding stage 502 to produce a set of quantized transform coefficients. A dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and an inverse transform stage 506 inverse transforms the dequantized transform coefficients to produce derivative residuals that may be the same as the derivative residuals created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 may use an intra / inter prediction stage 508 to create the same prediction blocks as were created in the encoder 400 (e.g., in the intra / inter prediction stage 402).

[0056] In the reconstruction stage 510, the predicted block can be added to the derivative residual to create a reconstructed block. A loop filter stage 512 can be applied to the reconstructed block to reduce block artifacts. Other filtering can be applied to the reconstructed block. In this example, a deblocking filter stage 514 is applied to the reconstructed block to reduce block distortion, and the result is output as an output video stream 516. The output video stream 516 can also be referred to as a decoded video stream, and these terms will be used interchangeably in this article. Other variations of the decoder 500 can be used to decode the compressed bitstream 420. In some implementations, the decoder 500 can generate the output video stream 516 without the deblocking filter stage 514.

[0057] As mentioned above, the prediction residual can be quantized to the closest prototype prediction residual using a codebook. The codebook for the prediction residual can be derived in any number of ways. In an example, the codebook can be derived using vector quantization (VQ). VQ is a quantization technique that allows modeling the probability distribution of prototype data (i.e., training samples).

[0058] During the offline training phase, many prediction residual blocks can be used as training samples. VQ divides the training samples into groups. VQ then represents each group of sample prediction residuals by its centroid. The centroid becomes the code vector (e.g., prototype prediction residual). That is, the centroid becomes an entry in the codebook. Several known techniques can be used to cluster the training prediction residual samples and derive the centroid (i.e., code vector). For example, the Lloyd algorithm, the generalized Lloyd algorithm, the Linde-Buzzo-Gray (LBG) algorithm, K-means, or some other techniques can be used.

[0059] In the example of designing a codebook for luma 4x4 prediction residuals, training samples of 4x4 prediction residuals are used. Each of the training samples can be converted into an array (e.g., a vector) of 16 values ​​(i.e., pixel values) using a raster scan of the training samples. The input to the VQ algorithm used can be the number N (e.g., 256 or some other number) of desired code vectors (e.g., the size of the codebook). The resulting codebook will include N 16-value code vectors. Typically, a codebook of size P for MxN prediction residuals includes P M*N value code vectors. The same technique can be used to design a codebook for prediction residuals of size (e.g., MxN) for any color component (chroma U, chroma Y, etc.).

[0060] Figure 7700 is a diagram of an example of a codebook 700 according to an implementation of the present disclosure. The codebook 700 illustrates learned (i.e., during a training phase) code vectors. As mentioned above, the code vectors of the codebook can be represented as a one-dimensional array of pixel values. For ease of visualization, the code vectors of the codebook 700 are illustrated as two-dimensional residual blocks, which are sorted from the highest selection frequency during training (i.e., starting from the upper left corner of the codebook 700) to the lowest selection frequency during training (i.e., ending at the lower right corner of the codebook 700). The codebook 700 includes 256 code vectors. However, only the top 32 and bottom 32 are illustrated.

[0061] In an example, gain shape vector quantization (GSVQ) can be used to derive the magnitude (i.e., gain) codebook and the shape (i.e., normalized signal) codebook. For ease of reference, GSVQ may also be referred to as VQ in the following description. In an example, the codebook for the 4x4 luma (Y) prediction residual may be such that the gain codebook includes 16 scalar codewords and the shape codebook may include 256 code vectors.

[0062] In GSVQ, during training, the norm of each of the training prediction residuals is extracted, and a gain (i.e., amplitude) codebook is derived for the extracted norm. The norm may be the Euclidean norm (i.e., length) of the training prediction residual. As is known to all, a vector x = (x0, x1, ..., x 15 ) is given by given.

[0063] Unit-norm training prediction residuals are training prediction residuals where each value in the training prediction residuals is divided by the norm of the training prediction residuals. That is, given a training prediction residual x, the unit-norm training prediction residual is given by x / ||x||. Unit-norm training prediction residuals are so called because they have a length of 1.

[0064] The gain codebook can be derived using the K-means algorithm on the norm of the training prediction residuals. The shape codebook can be derived by the following operations: first, generating a Voronoi diagram from the random unit norm training prediction residuals; and then iteratively applying the K-means algorithm. In each iteration, the steps of the K-means algorithm may include calculating a sign-invariant distance indicator; clustering the training samples based on the distance; and calculating the normalized cluster mean. The sign-invariant distance indicator is calculated between at least some of the training prediction residuals and each current centroid. In the example, for the unit norm training prediction residual p and the centroid q, the sign-invariant distance indicator d(p,q) can be calculated as d(x,y)=min{||xy||,||x+y||}. That is, the sign-invariant distance indicator d(p,q) can be the minimum of the first length of the vector (xy) and the second length of the vector (x+y).

[0065] Figure 8 is a diagram of the functionality of a prediction residual encoding stage 800 using vector quantization according to an implementation of the present disclosure. The prediction residual encoding stage 800 quantizes the prediction residual into a closest gain value and a closest shape vector.

[0066] In an example, the prediction residual encoding stage 800 may be or may be included in Figure 4 In another example, the prediction residual encoding stage 800 can be the same as Figure 4 The conversion stage 404 is separated from the stage. In any case, and although not in Figure 4 Specifically shown in FIG, when the encoder 400 (or one or more components thereof) determines that the VQ (ie, GSVQ) will be Figure 4 When the prediction residual (ie, prediction block) generated by the intra / inter prediction stage 402 is performed, then at least Figure 4 The quantization stage 406, dequantization stage 410, and inverse transform stage 412 are bypassed. In an example, it can be determined that VQ is used (or at least tested) only when the prediction residual is from intra prediction. In an example, it can be determined that VQ is used if the prediction block is less than or equal to a certain threshold block size. In an example, for luma prediction blocks, the threshold block size is 4x4.

[0067] The prediction residual encoding stage 800 includes a normalization stage 802 and a GSVQ stage 806. The normalization stage 802 may receive the prediction residual x generated by the intra / inter prediction stage 402. In an example, the prediction residual x may be the result of an intra prediction. In an example, the size of the prediction residual x may be 4x4 or some other block size. The prediction residual x may be a luma (Y) block, a chroma (U or V) residual block, or some other color component residual block. For ease of explanation, the notation x for the prediction residual may be used to refer to either or both a two-dimensional MxN (e.g., 4x4) residual block or an equivalent one-dimensional raster scan M*N (e.g., 4*4=16) vector.

[0068] The normalization stage 802 generates (e.g., calculates, extracts, etc.) a norm (e.g., Euclidean length ||x||) of the prediction residual x. The norm ||x|| is scalar-quantized using a gain codebook. The gain index (i.e., the index of the value in the gain codebook to which the norm ||x|| is quantized) can be encoded in the compressed bitstream, such as Figure 4 The compressed bit stream 420 is transmitted as indicated by arrow 804.

[0069] The unit norm prediction residual x / ||x|| is input to the GSVQ stage 806, which quantizes the unit norm vector separately using the shape codebook. That is, the GSVQ stage 806 identifies the code vector that is closest to the unit norm vector. A distance (e.g., error) metric can be used to identify the closest code vector. In an example, as described above, a sign-invariant distance metric between the unit norm prediction residual x / ||x|| and at least some code vectors of the shape codebook can be used. The distance metric can be a mean squared error. The distance metric can be a sum of absolute difference errors. Any other suitable distance metric can be used.

[0070] The shape index (i.e., the index of the code vector in the shape codebook to which the unit norm x / ||x|| is quantized) can be encoded in the compressed bitstream, such as Figure 4 The compressed bitstream 420 is obtained as indicated by arrow 808. Thus, the prediction residual can be approximated (eg, encoded to, compressed to, etc.) by the product of the optimal (eg, closest) gain and the optimal (eg, closest) shape.

[0071] In an implementation, a flag may be additionally encoded in the compressed bitstream. The flag may be used to reduce the size of the codebook. The flag may be a binary symbol representing a value of +1 or -1. For example, a flag value of 0 may represent a value of -1, and a flag value of 1 may represent a value of +1, and vice versa. Thus, the prediction residual may be represented by a gain index, a shape index, and a flag.

[0072] In the case where the flag is not used, the prediction residual can be represented by gain and shape, where gain is a positive scalar and shape is a unit norm vector.

[0073] For illustration purposes only, assume the following two vectors of length 4: v1 = [-1, 1, 1, 1] and v2 = [1, -1, -1, -1], which can be represented by (gain = 2, shape = [-0.5, 0, 5, 0.5, 0.5]) and (gain = 2, shape = [0.5-0.5, -0.5, -0.5]), respectively. In this definition, the shapes of vectors v1 and v2 can be mapped to different shape codewords, even though vectors v1 and v2 are just flipped versions of each other. In implementations, the shapes of vectors v1 and v2 can be treated as the same shape and their signs are decoupled.

[0074] Thus, in an implementation, vectors v1 and v2 can be represented by the triples (gain = 2, sign = 1, shape = [-0.5, 0.5, 0.5]) and (gain = 2, sign = -1, shape = [-0.5, 0.5, 0.5, 0.5]), respectively. Thus, the size of the shape codebook can be halved. That is, for example, if the codebook includes [-0.5, 0.5, 0.5, 0.5], then it does not need to include the code vector [0.5, -0.5, -0.5, -0.5]. Thus, in an implementation using a sign, the search for code vectors in the codebook can be faster because the searches for gain, sign, and shape can be done independently.

[0075] In summary, using GSVQ, the prediction residual can be represented by a gain index and a shape index; alternatively, the prediction residual can be represented by a gain index, a shape index and a flag (which can be a binary symbol).

[0076] As is known, some codecs use what may be referred to as a transform skip mode. In transform skip mode, the encoder may decide to skip transforming the residual block into the frequency domain, but may still perform a quantization step on the residual block.

[0077] Transform skipping can be selected by the encoder when the residual block has a small enough energy that the energy can be ignored. Therefore, in skip block mode, the prediction signal itself is encoded together with the SKIP flag. Skip mode may be particularly useful for text, graphics and / or screen content videos, because these videos often contain large areas of flat and / or smooth areas, so that intra-frame and inter-frame predictions in such areas produce almost perfect predictions with no (or very little) prediction residuals. In an example, to decide whether a transform block should be skipped, the encoder can compare the RD cost of skipping and not skipping, and then can choose the one with the smaller RD cost.

[0078] For the current prediction residual, the encoder according to the implementation of the present disclosure must communicate to the decoder (via the compressed bitstream) how the prediction residual is encoded (i.e., the prediction residual coding mode). That is, the encoder needs to communicate whether transform coding is skipped, whether transform coefficients are generated for the prediction residual, or whether the prediction residual is encoded via GSVQ (as described in relation to the encoding mode of the decoder). Figure 8 description).

[0079] At least two (2) syntax elements may be used to describe the prediction residual coding mode. The first syntax element (i.e., USE_VQ) indicates whether GSVQ is used to encode the residual block. The second syntax element (i.e., SKIP) indicates whether the prediction residual is encoded in transform skip mode. The at least two syntax elements may be written to the compressed bitstream in a header of a prediction block, a header of a block containing the prediction block, a header of a macroblock, a header of a transform block, some other header, or a combination thereof.

[0080] In an example, USE_VQ and SKIP can each be a binary symbol. In an example, a USE_VQ value of 1 indicates that the prediction residual is encoded using GSVQ, and a value of 0 indicates that the prediction residual is not encoded using GSVQ. In an example, a SKIP value of 1 indicates that the prediction residual is encoded using transform skipping, and a value of 0 indicates that the prediction residual is not encoded using transform skipping. However, other values ​​are possible for the USE_VQ and / or SKIP syntax elements.

[0081] Figure 9 is a diagram of a flow chart of a technique 900 for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure. In an example, the technique 900 may be performed (e.g., performed, implemented, etc.) for prediction residuals of size 4x4 and generated by intra prediction. However, the present disclosure is not limited in this regard. The technique 900 may write syntax elements describing the mode of encoding residual blocks into the compressed bitstream, such as Figure 4 The compressed bit stream 420.

[0082] Technique 900 can be implemented, for example, as a software program that can be executed by a computing device such as sending station 102. For example, the software program can include machine-readable instructions that can be stored in a memory such as memory 204 or secondary storage device 214 and, when executed by a processor such as processor 202, can cause the computing device to perform technique 900. Technique 900 can be implemented using dedicated hardware or firmware. For example, hardware components can be configured to perform technique 900. As explained above, some computing devices can have multiple memories or processors, and the operations described in technique 900 can be distributed using multiple processors, memories, or both.

[0083] At 902, the technique 900 may receive a macroblock. A macroblock may be a block of size 128x128, 64x64, or some other MxN size. A macroblock may be partitioned into smaller blocks, which may be referred to as coded blocks. A coded block may also be partitioned into one or more prediction blocks. The prediction blocks are then predicted, as described above with respect to Figure 4 As described above, the intra / inter prediction stage 402 of FIG. 4 is described. As also described above, a prediction residual (e.g., a prediction block) is generated. For example, the prediction block can be a 4x4 luma block, and the residual block can be the corresponding 4x4 prediction block generated by the intra prediction block. The block currently being encoded by the encoder is referred to as the current block.

[0084] Technique 900 can perform a rate-distortion search between GSVQ and transform coding to determine which results in a better RD cost. In an example, the search can be an exhaustive search. In an example, some (e.g., a subset) of possible modes (e.g., transform type, code vector) can be searched. If GSVQ does not result in a better RD cost, technique 900 writes a USE_VQ of 0 to the compressed bitstream and proceeds to 904; otherwise, technique 900 writes a USE_VQ of 1 to the compressed bitstream and proceeds to 910.

[0085] At 904, technique 900 determines whether transforming the prediction residual to the frequency domain is to be skipped, corresponding to encoding the prediction residual using transform skip mode. If the prediction residual is to be encoded using transform skip mode, technique 900 writes a 1 for the syntax element SKIP into the compressed bitstream and ends at 906. In this case, the prediction residual itself can be quantized. In another example, the prediction residual itself can be written into the compressed bitstream. If the prediction residual is not to be encoded using transform skip mode, technique 900 writes a 0 for the syntax element SKIP and proceeds to 908.

[0086] At 908, technique 900 may select (e.g., choose, identify, search, etc.) a transform type that technique 900 uses to convert the prediction residual into a transform block of transform coefficients, as described above with respect to Figure 4 The technique 900 then encodes (ie, writes) the transform type and transform coefficients into the compressed bitstream, as described above with respect to the transform stage 404 of FIG. Figure 4 Descriptive.

[0087] At 910, technique 900 quantizes the prediction residual to the closest shape vector and the closest gain value, as described with respect to Figure 8 The technique 900 writes (e.g., encodes) the shape index and the gain index to the compressed bitstream. In an example, a flag may also be used, such as with respect to Figure 8 Thus, technique 900 may also write flags into the compressed bitstream.

[0088] Bitstream portions 920, 930, 940, and 945 illustrate different organizations of portions of a compressed bitstream.

[0089] Bitstream portion 920 illustrates the organization of the compressed bitstream when technique 900 executes sequence 902-904-906. Thus, bitstream portion 920 includes a 0 flag (eg, a binary value) for the USE_VQ syntax element, followed by a 1 flag (eg, a binary value) for the SKIP syntax element.

[0090] Bitstream portion 930 illustrates the organization of the compressed bitstream when technique 900 executes sequence 902-904-908. Thus, bitstream portion 930 includes a 0 flag for the USE_VQ syntax element, followed by a 0 for the SKIP syntax element, followed by a bit for the coded transform type, and then the entropy coded transform coefficients.

[0091] Bitstream portion 940 illustrates the organization of the compressed bitstream when technique 900 performs sequence 902-910. Thus, bitstream portion 940 includes a 1 flag for the USE_VQ syntax element, followed by a bit for the coding gain index, and then a bit for the coding shape index. Note that the bits for the coding shape index may precede the bits for the coding gain index in bitstream portion 940.

[0092] Bitstream portion 945 illustrates the organization of the compressed bitstream when technique 900 executes sequence 902-910 and the flag is also written into the compressed bitstream. Thus, bitstream portion 945 includes a 1 flag for the USE_VQ syntax element, followed by a bit encoding the gain index, followed by a bit encoding the shape index, followed by a binary symbol indicating the flag. Note that the gain index, shape index, and flag can be written in any order.

[0093] Decoder (such as Figure 5 The decoder 500) can receive a compressed bit stream such as Figure 5 The decoder includes or has access to the same lookup tables (ie, shape codebook and gain codebook) as the encoder that produced the compressed bitstream.

[0094] The decoder decodes the first syntax element (USE_VQ).

[0095] In an implementation, if USE_VQ is equal to 1, the decoder decodes the gain index and the shape index. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook and uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as a·y.

[0096] In another implementation, if USE_VQ is equal to 1, the decoder decodes the gain index, shape index, and sign. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook and uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as sign·a·y.

[0097] On the other hand, if USE_VQ is equal to 0, the decoder continues decoding another syntax element (SKIP) from the compressed bitstream.

[0098] If SKIP is equal to 1, the decoder can decode the quantized prediction residual from the compressed bitstream to obtain the residual block. If SKIP is equal to 0, the decoder can decode the transform type and transform coefficients from the compressed bitstream to obtain the prediction block, as described above for Figure 5 Descriptive.

[0099] The decoder can use the residual block to reconstruct the current block, as described above. Figure 5 Descriptive.

[0100] Figure 10 is a diagram of a flow chart of a technique 1000 for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure. In an example, the technique 1000 may be performed (e.g., performed, implemented, etc.) for a prediction residual of size 4x4 and generated by intra prediction. However, the present disclosure is not limited in this regard. The technique 1000 may write syntax elements describing a mode of encoding a residual block into the compressed bitstream, such as Figure 4 The compressed bit stream 420.

[0101] Technique 1000 can be implemented, for example, as a software program that can be executed by a computing device such as sending station 102. For example, the software program can include machine-readable instructions that can be stored in a memory such as memory 204 or secondary storage device 214 and, when executed by a processor such as processor 202, can cause a computing device to perform technique 1000. Technique 1000 can be implemented using dedicated hardware or firmware. For example, hardware components can be configured to perform technique 1000. As explained above, some computing devices can have multiple memories or processors, and the operations described in technique 1000 can be distributed using multiple processors, memories, or both.

[0102] At 1002, the technique 1000 may receive a macroblock. A macroblock may be a block of size 128x128, 64x64, or some other MxN size. A macroblock may be partitioned into smaller blocks, which may be referred to as coded blocks. A coded block may also be partitioned into one or more prediction blocks. The prediction blocks are then predicted, as described above with respect to Figure 4 As described above, the intra / inter prediction stage 402 of FIG. 4 is described. As also described above, a prediction residual (e.g., a prediction block) is generated. For example, the prediction block can be a 4x4 luma block, and the residual block can be the corresponding 4x4 prediction block generated by the intra prediction block. The block currently being encoded by the encoder is referred to as the current block.

[0103] At 1002, technique 1000 determines whether transforming the prediction residual to the frequency domain is to be skipped, corresponding to encoding the prediction residual using a transform skip mode. If the prediction residual is to be encoded using a transform skip mode, technique 1000 writes a 1 for the syntax element SKIP into the compressed bitstream and ends at 1004. In this case, the prediction residual itself may be quantized or may be written into the compressed bitstream without quantization. If the prediction residual is not to be encoded using a transform skip mode, technique 1000 writes a 0 for the syntax element SKIP and proceeds to 1006.

[0104] At 1006, technique 1000 may perform a rate-distortion search between GSVQ and transform coding to determine which results in a better RD cost, as described above with respect to Figure 9 If GSVQ does not result in a better RD cost, the technique 1000 writes USE_VQ 0 into the compressed bitstream and proceeds to 1010 ; otherwise, the technique 1000 writes USE_VQ 1 into the compressed bitstream and proceeds to 1008 .

[0105] At 1008, technique 1000 quantizes the prediction residual to the closest shape vector and the closest gain value, as described with respect to Figure 8 As described. Technique 1000 writes (e.g., encodes) the shape index and the gain index to the compressed bitstream. In the example, and also as described with respect to Figure 8 As described, technique 1000 can also write flags.

[0106] At 1010, technique 1000 may select (e.g., choose, identify, search, etc.) a transform type that technique 1000 uses to convert the prediction residual into a transform block of transform coefficients, as described above with respect to Figure 4 The technique 1000 then encodes (ie, writes) the transform type and transform coefficients into the compressed bitstream, as described above with respect to the transform stage 404 of FIG. Figure 4 Descriptive.

[0107] Bitstream portions 1020, 1025, 1030, and 1040 illustrate different organizations of portions of a compressed bitstream.

[0108] Bitstream portion 1020 illustrates the organization of the compressed bitstream when technique 1000 executes sequence 1002-1006-1008. Thus, bitstream portion 1020 includes a 0 flag for the SKIP syntax element, followed by a 1 for the USE_VQ syntax element, followed by a bit for the coding gain index, and then a bit for the coding shape index. Note that the bit for the coding shape index may precede the bit for the coding gain index in bitstream portion 1020.

[0109] Bitstream portion 1025 illustrates the organization of the compressed bitstream when technique 1000 executes sequence 1002-1006-1008 including writing flags. Thus, bitstream portion 1025 includes a 0 flag for the SKIP syntax element, followed by a 1 for the USE_VQ syntax element, followed by a bit encoding the gain index, followed by a bit encoding the shape index, and then the flag bit. Note that the gain index, shape index, and flag can be written to bitstream portion 1025 in any order.

[0110] Bitstream portion 1030 illustrates the organization of the compressed bitstream when technique 1000 performs sequence 1002-1006-1010. Thus, bitstream portion 1030 includes a 0 flag for the SKIP syntax element, followed by a 0 for the USE_VQ syntax element, followed by bits for the coded transform type, and then the entropy coded transform coefficients.

[0111] Bitstream portion 1040 illustrates the organization of the compressed bitstream when technique 1000 performs sequence 1002 - 1004. Thus, bitstream portion 1040 includes a 1 flag for the SKIP syntax element.

[0112] Decoder (such as Figure 5 The decoder 500) can receive a compressed bit stream such as Figure 5 The decoder includes or has access to the same lookup tables (ie, shape codebook and gain codebook) as the encoder that produced the compressed bitstream.

[0113] The decoder decodes the first syntax element (SKIP). If SKIP is equal to 1, the decoder can decode the quantized prediction residual from the compressed bitstream to obtain the residual block.

[0114] If SKIP is equal to 0, the decoder decodes the second syntax element (USE_VQ).

[0115] In an implementation, if USE_VQ is equal to 1, the decoder decodes the gain index and the shape index. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook and uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as a·y.

[0116] In another implementation, if USE_VQ is equal to 1, the decoder decodes the gain index, shape index, and sign. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook and uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as sign·a·y.

[0117] On the other hand, if USE_VQ is equal to 0, the decoder can decode the transform type and transform coefficients from the compressed bitstream to obtain the predicted block, as described above for Figure 5 Descriptive.

[0118] The decoder can then use the residual block to reconstruct the current block, as described above for Figure 5 Descriptive.

[0119] Figure 11 is a diagram of a flow diagram of a technique 1100 for bitstream syntax coding when vector quantization is used in an encoder according to an implementation of the present disclosure. The technique 1100 includes a method similar to Figure 10 Block of 1000 of technology. About Figure 11 , descriptions of those similar blocks (ie, similarly numbered blocks) are omitted.

[0120] Technique 1100 differs from technique 1000 in that, instead of transforming the prediction residual to the frequency domain to obtain a transform block in 1008, technique 1100 obtains (i.e., calculates, etc.) a difference prediction residual by subtracting the VQ prediction residual from the prediction block and then transforming the difference prediction residual to the frequency domain. The VQ prediction residual corresponds to GSVQ quantization of the prediction residual.

[0121] Thus, before writing the transform type and transform coefficients in 1102, technique 1100 determines a gain scalar (i.e., a) from the shape code vector and a shape vector (i.e., y) from the gain codebook and shape codebook, respectively, for the prediction residual; calculates the VQ prediction residual as a·y; obtains the VQ prediction residual in 1104; transforms the VQ prediction residual using the transform type to obtain a transformed VQ prediction residual; encodes (i.e., writes) the gain index of the gain scalar and the shape index of the shape code vector into the compressed bitstream in 1008; and encodes (i.e., writes) the transform type and the transformed VQ prediction residual into the compressed bitstream in 1102. As described above, in some examples, technique 1100 may additionally determine a sign for the prediction residual. Thus, technique 1100 calculates the VQ prediction residual as sign·a·y.

[0122] Bitstream portions 1020, 1025, 1130, and 1040 illustrate different organizations of portions of a compressed bitstream. Bitstream portion 1020, bitstream portion 1025, and bitstream portion 1040 are as described with respect to Figure 10 Descriptive.

[0123] Bitstream portion 1130 illustrates the organization of the compressed bitstream when technique 1000 executes sequence 1002-1006-1102. Thus, bitstream portion 1130 includes a 0 marker for the SKIP syntax element, followed by a 0 for the USE_VQ syntax element, followed by a bit for the coding gain index (first bit), followed by a bit for the coding shape index (second bit), optionally followed (as illustrated by the dashed line and depending on the implementation) by a flag bit, followed by a bit for the coding transform type (third bit), followed by a bit for the entropy coded coefficients of the transformed VQ prediction residual (fourth bit). Note that the flag bit (if included), the first bit, the second bit, the third bit, and the fourth bit can be arranged in any order in the compressed bitstream.

[0124] Decoder (such as Figure 5 The decoder 500) can receive a compressed bit stream such as Figure 5 The decoder includes or has access to the same lookup tables (ie, shape codebook and gain codebook) as the encoder that produced the compressed bitstream.

[0125] The decoder decodes the first syntax element (SKIP). If SKIP is equal to 1, the decoder can use the transform skip mode to decode the quantized prediction residual from the compressed bitstream to obtain the residual block.

[0126] If SKIP is equal to 0, the decoder decodes the second syntax element (USE_VQ).

[0127] In an implementation, if USE_VQ is equal to 1, the decoder decodes the gain index and the shape index. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook. The decoder uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as a·y.

[0128] In another implementation, if USE_VQ is equal to 1, the decoder decodes the gain index, shape index, and flag. The decoder uses the shape index to look up the shape vector (i.e., y) in the shape codebook. The decoder uses the gain index to look up the gain scalar value (i.e., a) in the gain codebook. The decoder calculates the residual block as sign·a·y.

[0129] If USE_VQ is equal to 0, the decoder can decode the gain index, shape index, transform type and VQ transform coefficients from the compressed bitstream. The VQ transform coefficients correspond to the entropy coded coefficients of the transformed VQ prediction residual.

[0130] Using the transform type and the VQ transform coefficients, the decoder can obtain (such as via a stage including entropy decoding, dequantization, and inverse transform, or a subset thereof) a difference prediction residual. The decoder can use the gain index and the shape index to obtain (e.g., lookup, etc.) a gain value (i.e., a) from the gain codebook and a shape code vector (i.e., y) from the shape codebook, respectively. Using the gain value and the shape code vector, the decoder obtains the VQ prediction residual (i.e., a y). The decoder can then reconstruct the prediction residual by adding the VQ prediction residual to the difference prediction residual.

[0131] The decoder can then use the residual block to reconstruct the current block, as described above for Figure 5 Descriptive.

[0132] Figure 12 is a diagram of a flow diagram of a technique 1200 for decoding a current block according to an implementation of the present disclosure. The technique 1200 may be implemented in a decoder such as Figure 5 The decoder 500 of the present invention is provided. The technique 1200 decodes the residual block (i.e., the prediction residual) from the compressed bit stream, which may be Figure 5 to reconstruct the current block, as described with respect to Figure 5 The reconstruction level 510 is described.

[0133] Technique 1200 can be implemented, for example, as a software program that can be executed by a computing device such as receiving station 106. For example, the software program can include machine-readable instructions that can be stored in a memory such as memory 204 or secondary storage 214 and, when executed by a processor such as processor 202, can cause a computing device to perform technique 1200. Technique 1200 can be implemented using dedicated hardware or firmware. For example, hardware components can be configured to perform technique 1200. As explained above, some computing devices can have multiple memories or processors, and the operations described in technique 1200 can be distributed using multiple processors, memories, or both.

[0134] Technique 1200 can also be implemented in an encoder, such as Figure 4 The encoder 400. The technique 1200 can be implemented in the reconstruction path of the encoder.

[0135] At 1202, technique 1200 may decode a flag from a compressed bitstream indicating whether a residual block of a current block is coded using vector quantization (VQ). The flag may be the USE_VQ syntax element described above.

[0136] At 1204, technique 1200 determines whether the residual block is encoded using VQ. For example, if the residual block is not encoded using VQ (such as, for example, if the USE_VQ syntax element is equal to 0), technique 1200 proceeds to 1208; if the residual block is encoded using VQ (such as, for example, if the USE_VQ syntax element is equal to 1), technique 1200 proceeds to 1210.

[0137] In 1206, technique 1200 decodes parameters indicating entries in the codebook. As mentioned above, decoding parameters indicating entries in the codebook may include decoding a gain index and decoding a shape index. In an implementation, technique 1200 may also decode a flag. The gain index, shape index, and flag (if decoded) may be as described above. Using the gain index, technique 1200 may select a gain value (gain scalar a) from the gain codebook, as described above. That is, the gain index may be used to look up a gain value in the gain codebook. Using the shape index, technique 1200 may select a shape vector (i.e., a shape code vector y) from the shape codebook. As described above, the shape vector may be a unit norm vector (i.e., a unit norm prediction residual). The decoder uses the same gain codebook and shape codebook (e.g., a copy of) used by the encoder to produce the compressed bitstream.

[0138] At 1208, technique 1200 uses the entry to decode the residual block. That is, in one example, the residual block can be obtained by multiplying the gain shape by the shape vector (i.e., a·y). In another example, where the sign is also encoded, the residual block can be obtained by multiplying the sign, the gain shape, and the shape vector (i.e., sign·a·y).

[0139] At 1212, technique 1200 reconstructs the current block using the prediction block. The current block can be reconstructed using the prediction block described above. Figure 5 The residual block described is used to reconstruct.

[0140] At 1210, the technique 1200 decodes the residual block based on a skip flag indicating whether the current block is encoded using transform skip (ie, transform skip mode). In the example, and as with respect to Figure 9 As described, decoding the residual block based on a skip flag indicating whether the current block is encoded using transform skipping may include: in response to the skip flag indicating that the current block is not encoded using skipping transform, decoding the transform type, decoding the transform block, and generating the residual block using the transform type and the transform block; and in response to the skip flag indicating that the current block is encoded using transform skipping, decoding the residual block without performing an inverse transform operation.

[0141] Figure 13is a diagram of a flow diagram of a technique 1300 for encoding a current block using vector quantization (VQ) according to an implementation of the present disclosure. The technique 1300 may be implemented in an encoder, such as Figure 4 The encoder 400 of the present invention. The technique 1300 encodes the current block into a compressed bit stream such as Figure 4 More specifically, the technique 1300 obtains a residual block for the current block and writes (ie, encodes) information related to the residual block to the compressed bitstream, which a decoder can use to decode the residual block.

[0142] Technique 1300 can be implemented, for example, as a software program that can be executed by a computing device such as sending station 102. For example, the software program can include machine-readable instructions that can be stored in a memory such as memory 204 or secondary storage device 214 and, when executed by a processor such as processor 202, can cause a computing device to perform technique 1300. Technique 1300 can be implemented using dedicated hardware or firmware. For example, hardware components can be configured to perform technique 1300. As explained above, some computing devices can have multiple memories or processors, and the operations described in technique 1300 can be distributed using multiple processors, memories, or both.

[0143] At 1302, technique 1300 determines whether the current block is encoded without using a transform skip mode. For example, the current block is determined to be encoded without using a transform skip mode described with respect to SKIP=0 (i.e., when technique 1000 writes SKIP=0 to the compressed bitstream, Figure 10 For example, the current block is determined not to be encoded using the transform skip mode described with respect to SKIP=0 (i.e., when the technique 1100 writes SKIP=0 into the compressed bitstream, Figure 11 1002 of 1002).

[0144] In response to determining that the current block is not encoded using transform skip mode, the technique 1300 obtains (at 1304) a residual block (i.e., a prediction residual) for the current block. In an example, the prediction residual may be obtained using intra prediction. For example, the prediction residual may be obtained from / using Figure 4The intra / inter prediction stage 402 obtains the result. In 1306, the technique 1300 selects a VQ gain index and a VQ shape index using the residual block. That is, and as described above, the technique 1300 can obtain the norm (e.g., length) of the prediction residual and can obtain a unit norm prediction residual (i.e., a normalized residual block) by partitioning the prediction block (which can be considered as a one-dimensional or two-dimensional array). Using the gain codebook, the norm of the norm (e.g., amplitude, length, etc.) can be quantized to the closest value in the gain codebook. The closest value in the gain codebook corresponds to the gain index (VQ gain index, first index) in the gain codebook. The norm unit prediction residual can be quantized to the closest code vector in the shape codebook. The closest code vector in the shape codebook corresponds to the shape index (VQ shape index, second index) in the shape codebook.

[0145] At 1308, technique 1300 encodes the VQ gain index in the compressed bitstream. At 1310, technique 1300 encodes the VQ shape index in the compressed bitstream.

[0146] In an example, and as mentioned above, the current block can be a 4x4 luma (i.e., luminance) block. However, the current block can be a luma block or chroma block of any other size. In an example, the gain codebook can include 16 entries, and the shape codebook can include 256 entries. In an example, the encoder can use different gain codebook and shape codebook pairs for different current block sizes. In an example, the encoder can use different gain codebook and shape codebook pairs for different color components. As mentioned above, a pair of shape codebook and gain codebook can be obtained during offline training using residual blocks as training samples.

[0147] In the example, and as about Figure 10 and 11 As described, technique 1300 may include encoding a first flag in a compressed bitstream indicating that the current block is not encoded using a transform skip mode (e.g., SKIP=0); and encoding a second flag in the compressed bitstream indicating whether the current block is to be encoded using VQ (e.g., USE_VQ=0 or USE_VQ=1).

[0148] In the example, and as about Figure 11 As described in 1102, encoding the current block using the VQ gain index and the VQ shape index may include determining whether VQ is used to encode the current block; and in response to determining that VQ is not used to encode the current block, obtaining a VQ residual block using the VQ gain index and the VQ shape index, obtaining a difference prediction residual as a difference between the VQ residual block and the residual block, transforming the difference prediction residual to obtain a transformed difference prediction residual, and encoding the transformed difference prediction residual.

[0149] In an example, encoding a current block using a VQ gain index and a VQ shape index may include determining whether VQ is used to encode the current block; and in response to determining that VQ is to be used to encode the current block, encoding a first flag indicating that the current block is not encoded using a transform skip mode (i.e., SKIP=0), and encoding a second flag indicating that the current block will be encoded using VQ (i.e., USE_VQ=1).

[0150] In an example, in addition to using the residual block to select the VQ gain index and the VQ shape index, technique 1300 may also select a flag, as described above with respect to Figure 8 Thus, technique 1300 may also include encoding a flag in the compressed bitstream.

[0151] Using VQ for prediction residual coding can improve performance. For example, decoder time is shown to be reduced to 96.35% for key frames and 97.78% overall compared to not using VQ.

[0152] For simplicity of explanation, techniques 900, 1000, 1100, 1200, and 1300 are depicted and described as a series of blocks, steps, or operations, respectively. However, blocks, steps, or operations according to the present disclosure may occur in various orders and / or simultaneously. Additionally, other steps or operations not presented and described herein may be used. Furthermore, not all illustrated steps or operations may be required to implement the techniques according to the disclosed subject matter.

[0153] The above encoding and decoding aspects illustrate some examples of encoding and decoding techniques. However, it is to be understood that encoding and decoding (as those terms used in the claims) can refer to compression, decompression, transformation or any other processing or change of data.

[0154] The word "example" is used herein to mean serving as an example, instance, or illustration. Any aspect or design described herein as an "example" is not necessarily to be construed as being more preferred or advantageous than other aspects or designs. On the contrary, the use of the word "example" is intended to present the concept in a specific way. As used in this application, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clearly indicated by the context, the statement "X includes A or B" is intended to mean any natural inclusive arrangement thereof. That is, if X includes A; X includes B; or X includes both A and B, then "X includes A or B" is satisfied under any of the aforementioned instances. In addition, the articles "a" and "an" used in this application and the appended claims should generally be interpreted to mean "one or more," unless otherwise specified or clearly indicated by the context to point to the singular form. Moreover, throughout this disclosure, unless so described, the use of the term "implementation" or the term "an implementation" is not intended to mean the same implementation.

[0155] The implementation of the sending station 102 and / or receiving station 106 (and the algorithms, methods, instructions, etc. stored thereon and / or executed by them (including by the encoder 400 and the decoder 500)) can be implemented in hardware, software, or any combination thereof. The hardware may include, for example, a computer, an intellectual property (IP) core, an application specific integrated circuit (ASIC), a programmable logic array, an optical processor, a programmable logic controller, microcode, a microcontroller, a server, a microprocessor, a digital signal processor, or any other suitable circuit. In the claims, the term "processor" should be understood to cover any of the foregoing hardware, either individually or in combination. The terms "signal" and "data" are used interchangeably. Further, parts of the sending station 102 and the receiving station 106 do not necessarily have to be implemented in the same manner.

[0156] Further, in one aspect, for example, the sending station 102 or the receiving station 106 can be implemented using a general-purpose computer or a general-purpose processor with a computer program that, when executed, performs any of the corresponding methods, algorithms, and / or instructions described herein. Additionally or alternatively, for example, a dedicated computer / processor can be used, which can include other hardware for executing any method, algorithm, or instruction described herein.

[0157] For example, sending station 102 and receiving station 106 can be implemented on the computer in the video conferencing system. Alternatively, sending station 102 can be implemented on a server, and receiving station 106 can be implemented on a device (such as a handheld communication device) separated from the server. In this example, sending station 102 can use encoder 400 to encode content into a coded video signal, and send the coded video signal to the communication device. Then, the communication device can use decoder 500 to decode the coded video signal. Alternatively, the communication device can decode the content stored locally on the communication device, such as not the content sent by sending station 102. Other suitable sending and receiving implementations are also available. For example, receiving station 106 can be a roughly fixed personal computer, rather than a portable communication device, and / or the device comprising encoder 400 can also include decoder 500.

[0158] Furthermore, all or part of the implementation of the present disclosure may take the form of a computer program product accessible from, for example, a computer-usable or computer-readable medium. A computer-usable or computer-readable medium may be, for example, any device that can tangibly contain, store, convey, or transmit a program for use by or in conjunction with any processor. The medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media may also be useful.

[0159] The above implementations and other aspects have been described to facilitate easy understanding of the present disclosure and are not intended to limit the present disclosure. On the contrary, the present disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, which should be given the broadest interpretation allowed by law to cover all such modifications and equivalent arrangements.

Claims

1. An apparatus for decoding a current block of a video, comprising: Memory; as well as a processor configured to execute instructions stored in the memory for: Decoding a flag indicating whether a residual block of the current block is encoded using vector quantization (VQ) from a compressed bitstream; In response to the flag indicating that the residual block is encoded using VQ, instructions are executed for: The parameters indicating an entry in the codebook are decoded by: decoding a gain index, the gain index indicating an entry in a gain codebook and indicating a magnitude of the residual block and being used to look up a quantized magnitude of the residual block in the gain codebook; as well as decoding a shape index, the shape index indicating an entry in a shape codebook and used to obtain a code vector to which a unit norm of the residual block is quantized from the shape codebook; as well as obtaining the residual block by multiplying the quantization amplitude by the code vector; In response to the flag indicating that the residual block is not encoded using VQ, instructions are executed for: decoding the residual block based on a skip flag indicating whether the current block is encoded using a transform skip mode; and The current block is reconstructed using the residual block.

2. The apparatus of claim 1 , wherein the instructions further comprise: A decoding flag is provided, wherein the residual block is further multiplied by the flag.

3. The device according to claim 2, wherein The gain codebook and the shape codebook are obtained through offline training using residual blocks as training samples.

4. The apparatus according to any one of claims 1 to 3, wherein decoding the residual block based on a skip flag indicating whether the current block is encoded using transform skipping further comprises: In response to the skip flag indicating that the current block is not encoded using transform skipping, instructions are executed for: Decode transform type; Decode the transform block; as well as generating the residual block using the transform type and the transform block; and In response to the skip flag indicating that the current block is encoded using transform skipping, the residual block is decoded without performing an inverse transform operation.

5. An apparatus for encoding a current block using vector quantization (VQ), comprising: Memory; as well as a processor configured to execute instructions stored in the memory for: determining whether the current block is to be encoded without using a transform skip mode; as well as In response to determining that the current block is not to be encoded using a transform skip mode, determining whether the current block is to be encoded using vector quantization; In response to determining that the current block is to be encoded using vector quantization, instructions are executed to: Obtaining a residual block of the current block; Using the residual block to select a VQ gain index and a VQ shape index, the gain index indicating the magnitude of the residual block and used to look up the quantization magnitude of the residual block in a gain codebook, and the shape index used to obtain a code vector to which a unit norm of the residual block is quantized from the shape codebook; encoding the VQ gain index in a compressed bitstream; as well as encoding the VQ shape index in the compressed bitstream; Otherwise, in response to determining that the current block is not to be encoded using vector quantization, instructions are executed to: Obtaining a residual block of the current block; selecting a transform type for a transform block that converts prediction residuals into transform coefficients; and The transform type and the transform coefficients are encoded in the compressed bitstream. 6 . The apparatus of claim 5 , wherein the VQ gain index is a first index in the gain codebook, and the VQ shape index is a second index in the shape codebook.

7. The apparatus of claim 6, wherein the gain codebook comprises 16 entries and the shape codebook comprises 256 entries.

8. The apparatus of claim 6, wherein using the residual block to select the VQ gain index and the VQ shape index comprises: Calculating the norm of the residual block; identifying a first closest entry to the norm in the gain codebook, wherein the VQ gain index corresponds to the first closest entry; Obtaining a normalized residual block by dividing the residual block by the norm; as well as A second closest entry to the normalized residual block in the shape codebook is identified, wherein the VQ shape index corresponds to the second closest entry. 9 . The apparatus according to claim 6 , wherein the gain codebook and the shape codebook are obtained by offline training using residual blocks as training samples.

10. The apparatus of claim 5, wherein the instructions further comprise instructions for: encoding a first flag in the compressed bitstream, the first flag indicating that the current block is not to be encoded using the transform skip mode; and A second flag is encoded in the compressed bitstream, the second flag indicating whether the current block is to be encoded using VQ.

11. The apparatus of any one of claims 5 to 10, wherein encoding the current block using the VQ gain index and the VQ shape index comprises: determining whether VQ is to be used to encode the current block; as well as In response to determining that a VQ is not to be used for encoding the current block, instructions are executed to: Obtain a VQ residual block using the VQ gain index and the VQ shape index; obtaining a difference prediction residual as a difference between the VQ residual block and the residual block; transforming the difference prediction residual to obtain a transformed difference prediction residual; and The transformed difference prediction residual is encoded in the compressed bitstream.

12. The apparatus of any one of claims 5 to 10, wherein encoding the current block using the VQ gain index and the VQ shape index comprises: determining whether VQ is to be used to encode the current block; as well as In response to determining that a VQ is to be used to encode the current block, instructions are executed to: encoding a first flag in the compressed bitstream, the first flag indicating that the current block is not to be encoded using the transform skip mode; as well as A second flag is encoded in the compressed bitstream, the second flag indicating that the current block is to be encoded using VQ.

13. A method for encoding a current block using vector quantization (VQ), comprising: determining whether the current block is encoded using a transform skip mode; as well as In response to determining that the current block is not to be encoded using a transform skip mode, determining whether the current block is to be encoded using vector quantization; In response to determining that the current block is to be encoded using vector quantization, instructions are executed to: Obtaining a residual block of the current block; Using the residual block to select a VQ gain index and a VQ shape index, the gain index indicating the magnitude of the residual block and used to look up the quantization magnitude of the residual block in a gain codebook, and the shape index used to obtain a code vector to which a unit norm of the residual block is quantized from the shape codebook; encoding the VQ gain index in a compressed bitstream; as well as encoding the VQ shape index in the compressed bitstream; Otherwise, in response to determining that the current block is not to be encoded using vector quantization, instructions are executed to: Obtaining a residual block of the current block; selecting a transform type for a transform block that converts prediction residuals into transform coefficients; and The transform type and the transform coefficients are encoded in the compressed bitstream.

14. The method of claim 13, wherein the VQ gain index is a first index in the gain codebook, and the VQ shape index is a second index in the shape codebook.

15. The method of claim 14, wherein the gain codebook comprises 16 entries, and wherein the shape codebook comprises 256 entries.

16. The method of claim 14, wherein using the residual block to select the VQ gain index and the VQ shape index comprises: Calculating the norm of the residual block; identifying a first closest entry to the norm in the gain codebook, wherein the VQ gain index corresponds to the first closest entry; Obtaining a normalized residual block by dividing the residual block by the norm; as well as A second closest entry to the normalized residual block in the shape codebook is identified, wherein the VQ shape index corresponds to the second closest entry. 17 . The method according to claim 14 , wherein the gain codebook and the shape codebook are obtained by offline training using residual blocks as training samples.

18. The method according to claim 13, further comprising: encoding a first flag in the compressed bitstream, the first flag indicating that the current block is not to be encoded using the transform skip mode; as well as A second flag is encoded in the compressed bitstream, the second flag indicating whether the current block is to be encoded using VQ.

19. The method according to any one of claims 13 to 18, wherein encoding the current block using the VQ gain index and the VQ shape index comprises: determining that VQ will not be used for encoding the current block; Obtain a VQ residual block using the VQ gain index and the VQ shape index; obtaining a difference prediction residual as a difference between the VQ residual block and the residual block; transforming the difference prediction residual to obtain a transformed difference prediction residual; and The transformed difference prediction residual is encoded in the compressed bitstream.

20. The method of any one of claims 13 to 18, wherein encoding the current block using the VQ gain index and the VQ shape index comprises: determining that a VQ is to be used for encoding the current block; encoding a first flag in the compressed bitstream, the first flag indicating that the current block is not to be encoded using the transform skip mode; as well as A second flag is encoded in the compressed bitstream, the second flag indicating that the current block is to be encoded using VQ.

Images