Transform block level scan order selection for video coding

Abstract

The present invention relates to transform block level scan order selection for video coding. A set of candidate scan orders is processed by identifying block end positions within a transform block for the candidate scan orders. A cost value is determined for each of the candidate scan orders to reflect a number of coefficients of the transform block that are located before the respective block end position. In particular, the cost value of a candidate scan order reflects a number of zero value coefficients that are located before the block end position of the candidate scan order. One of the candidate scan orders is then selected based on the cost values. The selected scan order is used for scanning the coefficients in the transform block, such as for encoding the coefficients into a bitstream or for decoding the coefficients into an output video stream.

Description

[0001] Case Analysis

[0002] This application is a divisional application of Chinese Invention Patent Application No. 201880036916.9, filed on September 14, 2018. Background Technology

[0003] Digital video streams can represent video using a series of frames or still images. Digital video can be used in a variety of applications, including video conferencing, high-definition video entertainment, video advertising, or sharing user-generated video. Digital video streams can contain large amounts of data and consume significant computing or communication resources for processing, transmitting, or storing the video data. Various schemes have been proposed to reduce the amount of data in video streams, including encoding and decoding techniques. Summary of the Invention

[0004] According to embodiments of this disclosure, a method for decoding a current transform block based on a scan order selected from the transform coefficients of previously decoded transform blocks includes identifying a region within a previously decoded transform block containing the last non-zero transform coefficient as a first block end position according to a first candidate scan order. The method further includes identifying a region within a previously decoded transform block containing the last non-zero transform coefficient as a second block end position according to a second candidate scan order. The method also includes determining a cost value for the first candidate scan order based on the number of transform coefficients preceding the first block end position. The method further includes determining a cost value for the second candidate scan order based on the number of transform coefficients preceding the second block end position. The method also includes selecting either the first candidate scan order or the second candidate scan order as the scan order for decoding the current transform block based on the cost value of the first candidate scan order and the cost value of the second candidate scan order. The method further includes decoding the current transform block using the selected scan order.

[0005] According to embodiments of this disclosure, an apparatus for decoding a current transform block based on a scan order selected according to the transform coefficients of a previously decoded transform block includes a memory and a processor configured to execute instructions stored in the memory. The instructions include instructions for identifying a region within a previously decoded transform block containing the last non-zero transform coefficient among the transform coefficients as a first block end position according to a first candidate scan order. The instructions also include instructions for identifying a region within a previously decoded transform block containing the last non-zero transform coefficient among the transform coefficients as a second block end position according to a second candidate scan order. The instructions further include instructions for determining a cost value of the first candidate scan order based on the number of transform coefficients preceding the first block end position. The instructions also include instructions for determining a cost value of the second candidate scan order based on the number of transform coefficients preceding the second block end position. The instructions further include instructions for selecting either the first candidate scan order or the second candidate scan order as the scan order for decoding the current transform block based on the cost value of the first candidate scan order and the cost value of the second candidate scan order. Finally, the instructions include instructions for decoding the current transform block using the selected scan order.

[0006] A method for decoding a current transform block according to embodiments of the present disclosure includes identifying a first number of zero-value transform coefficients preceding the last non-zero coefficient of a previously decoded transform block based on a first candidate scan order. The method further includes identifying a second number of zero-value transform coefficients preceding the last non-zero coefficient of a previously decoded transform block based on a second candidate scan order. The method further includes determining that the first number of zero-value transform coefficients is less than the second number of zero-value transform coefficients. The method further includes decoding the current transform block using the first candidate scan order in response to determining that the first number of zero-value transform coefficients is less than the second number of zero-value transform coefficients.

[0007] An apparatus for decoding a current transform block according to an embodiment of the present disclosure includes a memory and a processor configured to execute instructions stored in the memory. The instructions include instructions for: identifying a first number of zero-value transform coefficients preceding the last non-zero coefficient of a previously decoded transform block according to a first candidate scan order; identifying a second number of zero-value transform coefficients preceding the last non-zero coefficient of a previously decoded transform block according to a second candidate scan order; determining that the first number of zero-value transform coefficients is less than the second number of zero-value transform coefficients; and decoding the current transform block using the first candidate scan order in response to determining that the first number of zero-value transform coefficients is less than the second number of zero-value transform coefficients.

[0008] These and other aspects of this disclosure are disclosed in the following detailed description of embodiments, the appended claims, and the accompanying drawings. Attached Figure Description

[0009] The description herein refers to the accompanying drawings described below, wherein similar reference numerals in several views refer to similar parts.

[0010] Figure 1 This is a schematic diagram of a video encoding and decoding system.

[0011] Figure 2 This is a block diagram of an example computing device that can implement a sending station or a receiving station.

[0012] Figure 3 This is a diagram of a regular video stream that is to be encoded and subsequently decoded.

[0013] Figure 4 This is a block diagram of an encoder according to an embodiment of the present disclosure.

[0014] Figure 5 This is a block diagram of a decoder according to an embodiment of the present disclosure.

[0015] Figure 6 This is a block diagram representing a portion of a frame according to an embodiment of the present disclosure.

[0016] Figure 7 An example of a transform block according to an embodiment of the present disclosure and an example of a scan sequence for encoding or decoding the transform block are shown.

[0017] Figure 8 This is a flowchart illustrating an example of a technique for selecting the scan order used to encode or decode transform blocks.

[0018] Figure 9 This is a flowchart illustrating an example of a technique for determining the number of zero-valued coefficients before the end position of a block in a transform block.

[0019] Figure 10 This is a flowchart illustrating an example of a technique that uses the accumulated cost value of adjacent transform blocks to select the scan order.

[0020] Figure 11 A set of candidate scan orders according to an embodiment of this disclosure is shown.

[0021] Figure 12 The current transform block and its adjacent transform blocks within a video frame according to an embodiment of the present disclosure are shown. Specific Implementation

[0022] Video compression schemes may include encoding a video stream by dividing each image or frame of a video sequence into smaller parts (such as blocks), and generating an output bitstream using techniques that limit the information included in each block in the output. The encoded bitstream can be decoded to recreate the blocks and source images based on the limited information. Common video compression and decompression techniques for encoding video streams or decoding encoded bitstreams involve encoding quantized transform coefficients in transform blocks according to a scan order. The scan order transforms a two-dimensional array of quantized transform coefficients (e.g., transform blocks from a two-dimensional planar rectangle) into a one-dimensional sequence. The coefficients are then processed according to their order in the one-dimensional sequence.

[0023] To scan coefficients to be encoded or decoded, there may be many available scan orders. For example, a scan order can scan the transform block line by line, starting from the top (e.g., raster or horizontal scan), or column by column, starting from the left (e.g., vertical scan). In another example, a scan order can process coefficients in directions that are not perfectly horizontal or vertical (e.g., diagonal scan, zigzag scan, etc.). Some scan orders may be better suited for encoding or decoding coefficients in certain situations than others. For example, a transform block may end with a series of zeros. In this case, the block end position is encoded to signal that all coefficients following the block end position are zero-value coefficients. However, the location of the block end position depends on the scan order used. This is because the patterns used to scan the coefficients in the transform block differ between scan orders, and some patterns will locate the last non-zero coefficient faster than others. The efficiency of the encoder or decoder is improved by using a scan order that maximizes the number of zero-value coefficients located after the block end position in the transform block.

[0024] One solution using this scan order involves processing a superblock (e.g., a 64×64 block within a frame) by identifying each non-zero coefficient in each transform block of the superblocks to the left and above. For each of these non-zero coefficients, the values ​​of neighboring coefficients (e.g., to the left or above the current non-zero coefficient in the same transform block) are processed and aggregated to determine the non-zero coefficient's location relative to other non-zero coefficients in the transform block. The values ​​of each non-zero coefficient in each transform block are aggregated according to orientation, and then weighted according to the relative strength of the orientation. The orientation with the lowest weighted aggregated value can then be selected. However, this solution is complex because it requires processing coefficients at the superblock level and is burdensome for the encoder or decoder because it requires processing all non-zero coefficients in all transform blocks of adjacent superblocks to select the scan order of the current superblock. Furthermore, this solution requires that the scan order ultimately selected for the current superblock is used to scan the coefficients in all transform blocks of the current superblock.

[0025] Implementations of this disclosure include systems and techniques for transform block-level selection of scan sequences. A set of candidate scan sequences is processed by identifying block end positions within a transform block for each candidate scan sequence. A cost value is determined for each candidate scan sequence to reflect the number of coefficients in the transform block preceding the corresponding block end position. Specifically, the cost value of a candidate scan sequence reflects (e.g., may indicate or specify) the number of zero-value coefficients preceding the block end position of that candidate scan sequence. The cost value may indicate the computational cost (e.g., the amount of computational resources) used to encode the coefficients. The cost value may depend on or indicate one or more parameters selected from a set of parameters comprising: time used for encoding the coefficients, space used for encoding the coefficients (e.g., the size of computer memory), the number of operations used for encoding the coefficients (e.g., arithmetic operations), the number of coefficients to be encoded, and the bit cost (e.g., the number of bits) of encoding (e.g., entropy coding) of the coefficients. One of the candidate scan sequences is then selected based on these cost values. The selected scan sequence is used to scan coefficients in a transform block, such as for encoding these coefficients into a bitstream or for decoding these coefficients into an output video stream. In some implementations, the selection of the scan order may include the processing cost value or ranking of candidate scan orders of the current transform block’s adjacent transform blocks (e.g., within the same block, superblock, etc.).

[0026] This paper first describes further details of the techniques for transform block level scan order selection by referring to systems in which transform block level scan order selection can be implemented. Figure 1 This is a schematic diagram of a video encoding and decoding system 100. The transmitting station 102 can be, for example, having... Figure 2 The computer internally configured with the hardware described herein. However, other implementations of the sending station 102 are possible. For example, the processing of the sending station 102 may be distributed across multiple devices.

[0027] Network 104 can connect sending station 102 and receiving station 106 for encoding and decoding video streams. Specifically, the video stream can be encoded in sending station 102, and the encoded video stream can be decoded in receiving station 106. Network 104 can be, for example, the Internet. Network 104 can also be a local area network (LAN), wide area network (WAN), virtual private network (VPN), cellular telephone network, or any other device that transfers video streams from sending station 102 to receiving station 106 (in this example).

[0028] In one example, receiving station 106 may have the following characteristics: Figure 2 The internal configuration of the hardware described herein is that of a computer. However, other suitable implementations of the receiving station 106 are possible. For example, the processing of the receiving station 106 can be distributed across multiple devices.

[0029] Other implementations of the video encoding and decoding system 100 are possible. For example, network 104 may be omitted in the implementation. In another implementation, the video stream may be encoded and then stored for later transmission to receiving station 106 or any other device with memory. In one implementation, receiving station 106 receives (e.g., via network 104, a computer bus, and / or some communication path) the encoded video stream and stores it for later decoding. In an example implementation, Real-Time Transport Protocol (RTP) is used to transmit the encoded video over network 104. In another implementation, a transport protocol different from RTP may be used, such as a video streaming protocol based on Hypertext Transfer Protocol (HTTP).

[0030] For example, when used in a video conferencing system, sending station 102 and / or receiving station 106 may include the ability to both encode and decode video streams as described below. For example, receiving station 106 may be a video conferencing participant who receives an encoded video bitstream from a video conferencing server (e.g., sending station 102) for decoding and viewing, and further encodes and sends his or her own video bitstream to the video conferencing server for other participants to decode and view.

[0031] Figure 2 This is a block diagram illustrating an example of a computing device 200 that can be implemented as a transmitting station or a receiving station. For example, the computing device 200 can be implemented... Figure 1 The computing device 200 may be one or both of the transmitting station 102 and the receiving station 106. The computing device 200 may be in the form of a computing system comprising multiple computing devices, or in the form of a single computing device, such as a mobile phone, tablet computer, laptop computer, notebook computer, desktop computer, etc.

[0032] The processor 202 in computing device 200 can be a conventional central processing unit. Alternatively, processor 202 can be another type of device or multiple devices capable of manipulating or processing information that is currently existing or will be developed later. For example, although the disclosed implementation can be implemented with a single processor (e.g., processor 202) shown, advantages in speed and efficiency can be achieved by using more than one processor.

[0033] In an implementation, the memory 204 in the computing device 200 may be a read-only memory (ROM) device or a random access memory (RAM) device. However, other suitable types of storage devices may be used as memory 204. Memory 204 may include code and data 206 accessed by the processor 202 using bus 212. Memory 204 may also include an operating system 208 and an application program 210, which includes at least one program that allows the processor 202 to execute the techniques described herein. For example, application program 210 may include applications 1 to N, which also include a video encoding application that executes the techniques described herein. The computing device 200 may also include an auxiliary storage device 214, which may be, for example, a memory card used with a mobile computing device. Because video communication sessions may contain a large amount of information, it may be stored, in whole or in part, in the auxiliary storage device 214 and loaded into memory 204 as needed for processing.

[0034] The computing device 200 may also include one or more output devices, such as a display 218. In one example, the display 218 may be a touch-sensitive display that combines a display with a touch-sensitive element operable to sense touch input. The display 218 may be coupled to the processor 202 via a bus 212. As a complement to or alternative to the display 218, other output devices may be provided that allow the user to program or otherwise use the computing device 200. When the output device is or includes a display, the display may be implemented in a variety of ways, including via a liquid crystal display (LCD), a cathode ray tube (CRT) display, or a light-emitting diode (LED) display such as an organic LED (OLED) display.

[0035] The computing device 200 may also include or communicate with an image sensing device 220, such as a camera or any other image sensing device 220 existing or developed in the future capable of sensing images (such as images of a user operating the computing device 200). The image sensing device 220 may be positioned such that it is directed toward the user operating the computing device 200. In one example, the position and optical axis of the image sensing device 220 may be configured such that the field of view includes an area directly adjacent to and from which the display 218 can be seen.

[0036] The computing device 200 may also include or communicate with a sound sensing device 222, such as a microphone or any other existing or future sound sensing device capable of sensing sound in the vicinity of the computing device 200. The sound sensing device 222 may be positioned such that it is directed toward a user operating the computing device 200, and may be configured to receive sounds, such as speech or other words, emitted by the user when the user operates the computing device 200.

[0037] although Figure 2 The processor 202 and memory 204 of computing device 200 are depicted as integrated into a single unit, but other configurations may also be used. The operation of processor 202 can be distributed across multiple machines (each machine may have one or more processors), which may be directly coupled or coupled across a local area network or other network. Memory 204 can be distributed across multiple machines, such as network-based memory or memory across multiple machines performing the operations of computing device 200. Although described herein as a single bus, bus 212 of computing device 200 may consist of multiple buses. Furthermore, auxiliary memory 214 may be directly coupled to other components of computing device 200 or accessible via a network, and may include integrated units such as memory cards or multiple units such as multiple memory cards. Computing device 200 can therefore be implemented in a variety of configurations.

[0038] Figure 3 This is a diagram illustrating an example of a video stream 300 to be encoded and subsequently decoded. The video stream 300 includes a video sequence 302. At the next level, the video sequence 302 includes multiple adjacent frames 304. While three frames are depicted as adjacent frames 304, the video sequence 302 can include any number of adjacent frames 304. The adjacent frames 304 can then be further subdivided into individual frames, such as frames 306. At the next level, frames 306 can be divided into a series of planes or segments 308. For example, segments 308 can be subsets of frames that allow for parallel processing. Segments 308 can also be subsets of frames that can be divided into individual colors. For example, frames 306 of color video data can include a luma plane and two chroma planes. Segments 308 can be sampled at different resolutions.

[0039] Regardless of whether frame 306 is divided into segments 308, frame 306 can be further subdivided into blocks 310, which can contain data corresponding to, for example, 16×16 pixels in frame 306. Block 310 can also be arranged to include data from one or more segments 308 of the pixel data. Block 310 can also be any other suitable size, such as 4×4 pixels, 8×8 pixels, 16×8 pixels, 8×16 pixels, 16×16 pixels, or larger. Unless otherwise stated, the terms block and macroblock are used interchangeably herein.

[0040] Figure 4 This is a block diagram of an encoder 400 according to an embodiment of the present disclosure. As described above, the encoder 400 can be implemented in a transmitting station 102, such as by providing a computer software program stored in a memory such as memory 204. The computer software program may include machine instructions that, when executed by a processor such as processor 202, cause the transmitting station 102 to... Figure 4 The encoder 400 encodes video data in the manner described. It can also be implemented as dedicated hardware included, for example, in a transmitting station 102. In a particularly desirable implementation, the encoder 400 is a hardware encoder.

[0041] Encoder 400 has the following stages to perform various functions in the forward path (shown by solid connecting lines) to produce an encoded or compressed bitstream 420 using video stream 300 as input: intra / inter-frame prediction stage 402, transform stage 404, quantization stage 406, and entropy coding stage 408. Encoder 400 may also include a reconstruction path (shown by dashed connecting lines) to reconstruct frames for encoding future blocks. Figure 4 In this encoder 400, the following stages are used to perform various functions in the reconstruction path: dequantization stage 410, inverse transform stage 412, reconstruction stage 414, and loop filtering stage 416. Other structural variations of the encoder 400 can be used to encode video stream 300.

[0042] When video stream 300 is presented for encoding, corresponding adjacent frames 304, such as frame 306, can be processed in blocks. At the intra / inter-frame prediction level 402, the corresponding blocks can be encoded using either intra-frame prediction (also known as intra-prediction) or inter-frame prediction (also known as inter-prediction). In either case, prediction blocks can be formed. In the case of intra-frame prediction, prediction blocks can be formed from samples that have been previously encoded and reconstructed in the current frame. In the case of inter-frame prediction, prediction blocks can be formed from samples in one or more previously constructed reference frames.

[0043] Next, the prediction block can be subtracted from the current block in the intra / inter-frame prediction stage 402 to produce a residual block (also called a residual or prediction residual). The transform stage 404 uses a block-based transform to convert the residual into transform coefficients, for example, in the frequency domain. The quantization stage 406 uses a quantizer value or quantization stage to convert the transform coefficients into discrete quantum values, which are called quantized transform coefficients. For example, the transform coefficients can be divided by the quantizer value and truncated.

[0044] The quantized transform coefficients are then entropy encoded by entropy coding level 408. The entropy-coded coefficients, along with other information used to decode the block (e.g., syntax elements indicating the prediction type, transform type, motion vector, quantizer value, etc.), are then output to compressed bitstream 420. Compressed bitstream 420 can be formatted using various techniques, such as variable-length coding (VLC) or arithmetic coding. Compressed bitstream 420 may also be referred to as encoded video stream or encoded video bitstream, and these terms are used interchangeably herein.

[0045] The reconstruction path (shown by the dashed connection line) can be used to ensure the encoder 400 and decoder 500 (see below) Figure 5 (Description) The same reference frame is used to decode the compressed bitstream 420. The reconstructed path performs functions similar to those that occur during decoding (see below). Figure 5 The process includes dequantizing the quantized transform coefficients at dequantization stage 410 and performing an inverse transform on the dequantized transform coefficients at inverse transform stage 412 to produce a derived residual block (also known as a derived residual). At reconstruction stage 414, the predicted block predicted at intra / inter-frame prediction stage 402 can be added to the derived residual to create a reconstructed block. Loop filtering stage 416 can be applied to the reconstructed block to reduce distortions such as block artifacts.

[0046] Other variations of encoder 400 can be used to encode compressed bitstream 420. In some implementations, for certain blocks or frames, a non-transform-based encoder can directly quantize the residual signal without transform stage 404. In some implementations, the encoder can combine quantization stage 406 and dequantization stage 410 in a common stage.

[0047] Figure 5 This is a block diagram of a decoder 500 according to an embodiment of the present disclosure. The decoder 500 can be implemented in a receiving station 106, for example, by providing a computer software program stored in memory 204. The computer software program may include machine instructions that, when executed by a processor such as processor 202, cause the receiving station 106 to... Figure 5 The video data is decoded in the manner described herein. The decoder 500 can also be implemented using hardware included in, for example, a transmitting station 102 or a receiving station 106.

[0048] Decoder 500 is similar to the reconstruction path of encoder 400 discussed above, and in one example includes the following stages that perform various functions to produce output video stream 516 from compressed bitstream 420: entropy decoding stage 502, dequantization stage 504, inverse transform stage 506, intra / inter-frame prediction stage 508, reconstruction stage 510, loop filtering stage 512, and optional post-filtering stage 514. Other architectural variations of decoder 500 can be used to decode compressed bitstream 420.

[0049] When the compressed bitstream 420 is presented for decoding, the data elements within the compressed bitstream 420 can be decoded by the entropy decoding stage 502 to produce a set of quantized transform coefficients. The dequantization stage 504 dequantizes the quantized transform coefficients (e.g., by multiplying the quantized transform coefficients by a quantizer value), and the inverse transform stage 506 performs an inverse transform on the dequantized transform coefficients to produce a derived residual, which can be the same as the derived residual created by the inverse transform stage 412 in the encoder 400. Using the header information decoded from the compressed bitstream 420, the decoder 500 can use the intra / inter-frame prediction stage 508 to create the same prediction block created in the encoder 400 (e.g., in the intra / inter-frame prediction stage 402).

[0050] In reconstruction stage 510, predicted blocks can be added to the derived residuals to create reconstructed blocks. Loop filtering stage 512 can be applied to the reconstructed blocks to reduce block artifacts (e.g., using deblocking filtering, sample adaptive offset filtering, etc., or combinations thereof). Other filtering can be applied to the reconstructed blocks. In this example, post-filtering stage 514 is applied to the reconstructed blocks to reduce block distortion, and the result is output as output video stream 516. Output video stream 516 can also be referred to as decoded video stream, and these terms are used interchangeably herein. Other variations of decoder 500 can be used to decode compressed bitstream 420. In some implementations, decoder 500 can produce output video stream 516 without post-filtering stage 514.

[0051] Figure 6 Such as embodiments according to this disclosure Figure 3The diagram shows a block representation of frame portion 600 of frame 304. As shown, frame portion 600 includes four 64×64 blocks 610, arranged in two rows and two columns in a matrix or Cartesian plane. In some embodiments, the 64×64 block may be a maximum coding unit, N=64. Each 64×64 block may include four 32×32 blocks 620. Each 32×32 block may include four 16×16 blocks 630. Each 16×16 block may include four 8×8 blocks 640. Each 8×8 block 640 may include four 4×4 blocks 650. Each 4×4 block 650 may include 16 pixels, which may be represented in four rows and four columns in each corresponding block in the Cartesian plane or matrix. Pixels may include information representing the image captured in the frame, such as brightness information, color information, and location information. In some implementations, a block such as the 16×16 pixel block shown may include: a luminance block 660, which may include luminance pixels 662; and two chrominance blocks 670 and 680, such as a U or Cb chrominance block 670 and a V or Cr chrominance block 680. Chroma blocks 670 and 680 may include chrominance pixels 690. For example, a luminance block 660 may include 16×16 luminance pixels 662, and each chrominance block 670 and 680 may include 8×8 chrominance pixels 690, as shown. Although one arrangement of the blocks is shown, any arrangement can be used. Figure 6 An N×N block is shown, but in some implementations, an N×M block can be used. For example, 32×64 blocks, 64×32 blocks, 16×32 blocks, 32×16 blocks, or any other block size can be used. In some implementations, N×2N blocks, 2N×N blocks, or combinations thereof can be used.

[0052] In some implementations, video coding may include ordered block-level coding. Ordered block-level coding may include encoding blocks of a frame in an order such as raster scan order, wherein blocks may be identified and processed starting with the block at the top left corner of the frame or a portion of the frame, and proceeding sequentially along rows from left to right and from top to bottom, identifying and processing each block in turn. For example, the 64×64 block in the top row and left column of the frame may be the first block to be encoded, and the 64×64 block immediately to the right of the first block may be the second block to be encoded. The second row, starting from the top, may be the second row to be encoded, such that the 64×64 block in the left column of the second row may be encoded after the 64×64 block in the rightmost column of the first row.

[0053] In some implementations, encoding a block may include using quadtree encoding, which may involve encoding smaller block units within the block in raster scan order. For example, quadtree encoding can be used to encode... Figure 6The 64×64 block shown in the bottom left corner of the frame portion can be encoded as follows: The top left 32×32 block, then the top right 32×32 block, then the bottom left 32×32 block, and then the bottom right 32×32 block. Each 32×32 block can be encoded using quadtree coding, where the top left 16×16 block, then the top right 16×16 block, then the bottom left 16×16 block, and then the bottom right 16×16 block. Each 16×16 block can be encoded using quadtree coding, where the top left 8×8 block, then the top right 8×8 block, then the bottom left 8×8 block, and then the bottom right 8×8 block. Each 8×8 block can be encoded using quadtree coding, where the top left 4×4 block, then the top right 4×4 block, then the bottom left 4×4 block, and then the bottom right 4×4 block. In some implementations, for a 16×16 block, the 8×8 block can be omitted, and the 16×16 block can be encoded using quadtree coding, wherein the top left 4×4 block can be encoded, and then the other 4×4 blocks in the 16×16 block can be encoded in raster scan order.

[0054] In some implementations, video coding may include compressing information included in the original or input frame by, for example, omitting some information from the original frame in the corresponding coded frame. For example, coding may include reducing spectral redundancy, reducing spatial redundancy, reducing temporal redundancy, or a combination thereof.

[0055] In some implementations, reducing spectral redundancy may include using a color model based on a luminance component (Y) and two chrominance components (U and V or Cb and Cr), which may be referred to as the YUM or YCbCr color model or color space. Using the YUM color model may involve using a relatively large amount of information to represent the luminance component of a portion of a frame and using a relatively small amount of information to represent each corresponding chrominance component of that portion of the frame. For example, a portion of a frame may be represented by a high-resolution luminance component and two lower-resolution chrominance components, the high-resolution luminance component comprising a 16×16 pixel block, and each of the lower-resolution chrominance components representing a portion of the frame as an 8×8 pixel block. Pixels may indicate values ​​in, for example, the range of 0 to 255 and may be stored or transmitted using, for example, eight bits. Although this disclosure is described with reference to the YUM color model, any color model may be used.

[0056] In some implementations, reducing spatial redundancy may include transforming the block to the frequency domain using, for example, a discrete cosine transform (DCT). For example, encoder units, such as... Figure 4 The transform stage 404 shown can perform DCT using transform coefficient values ​​based on spatial frequency.

[0057] In some implementations, reducing temporal redundancy may include using the similarity between frames to encode frames with a relatively small amount of data based on one or more reference frames, which may be previously encoded, decoded, and reconstructed frames of the video stream. For example, a block or pixel in the current frame may resemble a spatially corresponding block or pixel in the reference frame. In some implementations, a block or pixel in the current frame may resemble a block or pixel in the reference frame at a different spatial location, and reducing temporal redundancy may include generating motion information indicating the spatial differences or translations between the locations of blocks or pixels in the current frame and their corresponding locations in the reference frames.

[0058] In some implementations, reducing temporal redundancy may include identifying a portion of a reference frame corresponding to the current block or pixel of the current frame. For example, a reference frame or a portion of a reference frame, possibly stored in memory, may be searched to identify a portion for generating a predictor to encode the current block or pixel of the current frame with maximum efficiency. For example, the search may identify a portion of a reference frame for which the difference in pixel values ​​between the current block and a predicted block generated based on that portion of the reference frame is minimized, and may be referred to as a motion search. In some implementations, the portion of the reference frame being searched may be finite. For example, a portion of the reference frame being searched, which may be referred to as a search region, may include a finite number of rows of the reference frame. In one example, identifying a portion of the reference frame for generating the predictor may include calculating a cost function, such as the sum of absolute differences (SAD) between pixels in the search region and pixels in the current block.

[0059] In some implementations, the spatial difference between the location of the reference frame portion used to generate the predictor in the reference frame and the current block in the current frame can be represented as a motion vector. The difference in pixel values ​​between the predictor block and the current block can be referred to as differential data, residual data, prediction error, or residual block. In some implementations, generating the motion vector can be referred to as motion estimation, and the pixels of the current block can be represented using Cartesian coordinates such as f. x,y Location-based indication. Similarly, the pixels of the search area in the reference frame can be indicated using Cartesian coordinates such as r. x,y Location-based indication. The motion vector (MV) of the current block can be determined based on, for example, the SAD between pixels in the current frame and corresponding pixels in a reference frame.

[0060] Although this document describes frames using a matrix or Cartesian representation for clarity, frames can be stored, transmitted, processed, or any combination thereof in any data structure that allows for efficient representation of pixel values ​​for a frame or image. For example, frames can be stored, transmitted, processed, or any combination thereof in a two-dimensional data structure such as a matrix as shown, or in a one-dimensional data structure such as a vector array. In one embodiment, the representation of a frame, such as the two-dimensional representation shown, can correspond to the physical location of the frame when it is rendered as an image. For example, the location of the top left corner of a block at the top left corner of the frame can correspond to the physical location of the top left corner of the frame when it is rendered as an image.

[0061] In some implementations, block-based coding efficiency can be improved by dividing the input block into one or more prediction partitions, which can be rectangular (including square) partitions used for predictive coding. In some implementations, video coding using prediction partitions can include selecting a prediction partitioning scheme from a plurality of candidate prediction partitioning schemes. For example, in some implementations, candidate prediction partitioning schemes for a 64×64 coding unit can include rectangular prediction partitions ranging in size from 4×4 to 64×64, such as 4×4, 4×8, 8×4, 8×8, 8×16, 16×8, 16×16, 16×32, 32×16, 32×32, 32×64, 64×32, or 64×64. In some implementations, video coding using prediction partitions can include a full prediction partition search, which can include selecting a prediction partitioning scheme by encoding the coding unit using each available candidate prediction partitioning scheme and selecting the best scheme, such as the one that produces the minimum rate-distortion error.

[0062] In some implementations, encoding a video frame may include identifying a prediction partitioning scheme for encoding the current block, such as block 610. In some implementations, identifying the prediction partitioning scheme may include determining whether to encode the block as a single prediction partition of the maximum coding unit size (as shown, the maximum coding unit size could be 64×64), or to divide the block into multiple prediction partitions, which may correspond to sub-blocks, such as the 32×32 block 620, the 16×16 block 630, or the 8×8 block 640 shown, and may include determining whether to divide it into one or more smaller prediction partitions. For example, a 64×64 block may be divided into four 32×32 prediction partitions. Three of the four 32×32 prediction partitions may be encoded as 32×32 prediction partitions, and the fourth 32×32 prediction partition may be further divided into four 16×16 prediction partitions. Three of the four 16×16 prediction partitions may be encoded as 16×16 prediction partitions, and the fourth 16×16 prediction partition may be further divided into four 8×8 prediction partitions, each of which may be encoded as an 8×8 prediction partition. In some implementations, identifying a predictive partitioning scheme may include using a predictive partitioning decision tree.

[0063] In some implementations, video coding of the current block may include identifying the best predictive coding mode from a plurality of candidate predictive coding modes. This can provide flexibility when processing video signals with various statistical properties and can improve compression efficiency. For example, the video encoder may evaluate each candidate predictive coding mode to identify the best predictive coding mode, which may be, for example, the predictive coding mode that minimizes the error metric (e.g., rate-distortion cost) of the current block. In some implementations, the complexity of searching for candidate predictive coding modes can be reduced by limiting the set of available candidate predictive coding modes based on the similarity between the current block and the corresponding predictive block. In some implementations, the complexity of searching for each candidate predictive coding mode can be reduced by performing a targeted refinement pattern search. For example, metrics may be generated for a finite set of candidate block sizes (such as 16×16, 8×8, and 4×4); the error metric associated with each block size may be in descending order; and additional candidate block sizes, such as 4×8 and 8×4 block sizes, may be evaluated.

[0064] In some implementations, block-based coding efficiency can be improved by dividing the current residual block into one or more transform partitions, which can be rectangular (including square) partitions for transform coding. In some implementations, video coding using transform partitions can include selecting a uniform transform partitioning scheme. For example, the current residual block, such as block 610, can be a 64×64 block and can be transformed without using a 64×64 transform for partitioning.

[0065] Despite Figure 6While not explicitly stated, the residual block can be transformed and partitioned using a uniform transform partitioning scheme. For example, a uniform transform partitioning scheme can be used with four 32×32 transform blocks, sixteen 16×16 transform blocks, sixty-four 8×8 transform blocks, or 256 4×4 transform blocks to transform and partition the 64×64 residual block.

[0066] In some implementations, video coding using transform partitioning may include using multiple forms of transform partitioning coding to identify multiple transform block sizes for the residual block. In some implementations, multiple forms of transform partitioning coding may include recursively determining whether to transform the current block using the current block size or by segmenting the current block and performing multiple forms of transform partitioning coding on each segment. For example, Figure 6 The bottom left block 610 shown may be a 64×64 residual block, and various forms of transform partitioning encoding may include determining whether to encode the current 64×64 residual block using a 64×64 transform, or to encode the 64×64 residual block by dividing it into partitions such as four 32×32 blocks 620 and performing various forms of transform partitioning encoding on each partition. In some implementations, determining whether to perform transform partitioning on the current block may be based on a comparison of the cost of transforming and encoding the current block using the current block size with the sum of the costs of transforming and encoding each partition using the partition size.

[0067] Figure 7 A diagram illustrates an example of a transform block 700 according to an embodiment of the present disclosure and an example of a scan sequence 702-708 for encoding or decoding the transform block 700. In some embodiments, the encoder elements, such as Figure 4 The encoder 400 shown has a transform stage 404 that can generate a transform block 700. In some embodiments, the encoder elements, such as Figure 4 The quantization level 406 of the encoder 400 shown can quantize transform block 700 to generate quantized transform blocks. As used herein, unless otherwise explicitly indicated, the term "transform block" can refer to a block, matrix, or other data structure of transform coefficients or quantized transform coefficients. Although a 4×4 block is shown for simplicity, blocks of any size can be used. For example, 64×64 blocks, 64×32 blocks, 32×64 blocks, 32×32 blocks, 32×16 blocks, 16×32 blocks, 16×16 blocks, 16×8 blocks, 8×16 blocks, 8×8 blocks, 8×4 blocks, or 4×8 blocks can be used.

[0068] exist Figure 7In the illustrated transform block 700, the value shown for each location represents the transform coefficient value for the corresponding location. For clarity, the location of the transform coefficients of a block may be referred to as the “position,” “location,” or a variation thereof. As used herein, references to “proximity,” “spatial proximity,” or “distance” between transform coefficients may indicate proximity or distance in the transform coefficient matrix representation of the block’s transform coefficients.

[0069] Some of the coefficients in transform block 700 are non-zero coefficients, such as those that do not have a value of "0". Other coefficients in transform block 700 are zero-value coefficients, such as those that have a value of "0". In some implementations, transform block 700 may be processed in scan order to improve entropy coding efficiency. For example, the scan order may tend to group zero-value coefficients at the end of the block, and consecutive zero-value coefficients at the end of the block (i.e., the tail of zero coefficients) may be omitted from the output bitstream in scan order without data loss.

[0070] In some implementations, the encoded coefficients of transform block 700 may include encoding these coefficients in a scan order, such as a horizontal scan order 702, a vertical scan order 704, a diagonal scan order 706, or a zigzag scan order 708. The values ​​displayed in each block of the scan order indicate the order in which the corresponding coefficients are entropy encoded. Although a 4×4 block is shown for simplicity, any block size can be used. For example, 64×64 blocks, 64×32 blocks, 32×64 blocks, 32×32 blocks, 32×16 blocks, 16×32 blocks, 16×16 blocks, 16×8 blocks, 8×16 blocks, 8×8 blocks, 8×4 blocks, or 4×8 blocks can be used.

[0071] Depending on the scan order used, one of the non-zero coefficients will be the last non-zero coefficient scanned using that scan order. For example, when using horizontal scan order 702, coefficient 701 of transform block 700 is the last non-zero coefficient. Therefore, the block end position in transform block 700 used for horizontal scan order 702 is identified as the location of coefficient 701. In another example, when using vertical scan order 704, coefficient 703 of transform block 700 is the last non-zero coefficient. Therefore, the block end position in transform block 700 of vertical scan order 704 is identified as the location of coefficient 703.

[0072] In some implementations, encoding the coefficients of the transform coefficient matrix in scan order may include generating a one-dimensional array of transform coefficients, such as a vector, by including each transform coefficient in a vector in scan order. For example, it may be the DC coefficient of the coefficient at the top left corner of the transform coefficient matrix, the first element of the scan order vector, a transform coefficient matrix location with (0,0), and a scan order position with (0).

[0073] As used herein, the terms “order,” “scan position,” “vector position,” or variations thereof for transform coefficients refer to the relative position or index of the transform coefficients in the scan order or scan order vector. Although Figure 7 An example of a sequential scan pattern is shown, but non-sequential scan patterns can be used to encode coefficients.

[0074] Now refer to Figure 8-10 Describes the techniques used to encode or decode video frames. Figure 8 This is a flowchart illustrating an example of technique 800 for selecting the scan order used to encode or decode transform blocks. Figure 9 This is a flowchart illustrating an example of technique 900 for determining the number of zero-value coefficients prior to the end position of a block in a transform block. Figure 10 This is a flowchart illustrating an example of technique 1000, which uses the accumulated cost value of adjacent transform blocks to select the scan order.

[0075] One or more of techniques 800, 900, or 1000 can be implemented as, for example, a software program that can be executed by a computing device such as transmitting station 102 or receiving station 106. For example, the software program may include machine-readable instructions that can be stored in a memory such as memory 204 or auxiliary storage device 214, and when executed by a processor such as processor 202, can cause the computing device to perform one or more of techniques 800, 900, or 1000. One or more of techniques 800, 900, or 1000 can be implemented using dedicated hardware or firmware. As explained above, some computing devices may have multiple memories or processors, and the operations described in one or more of techniques 800, 900, or 1000 can be allocated using multiple processors, memories, or both.

[0076] For ease of explanation, techniques 800, 900, and 1000 are each depicted and described as a series of steps or operations. However, the steps or operations according to this disclosure may occur in various orders and / or simultaneously. Furthermore, other steps or operations not presented and described herein may be used. Moreover, not all of the steps or operations shown may be necessary to implement the techniques according to the disclosed subject matter.

[0077] First refer to Figure 8The diagram illustrates a flowchart of an example of a technique 800 for selecting the scan order for encoding or decoding transform blocks. At 802, a block end position is identified for each candidate scan order in the set of candidate scan orders. The block end position for a given candidate scan order represents the location within the transform block of the last non-zero transform coefficient according to that candidate scan order. For example, the set of candidate scan orders may include a first candidate scan order and a second candidate scan order. The first block end position may be identified as the location within the transform block of the last non-zero transform coefficient according to the first candidate scan order, and the second block end position may be identified as the location within the transform block of the last non-zero transform coefficient according to the second candidate scan order.

[0078] The location of the block end position within a transform block refers to the index of the last non-zero coefficient of the transform block in the one-dimensional sequence of its transform coefficients. For example, a candidate scan order can transform the transform coefficients of a transform block from a two-dimensional array into a one-dimensional sequence. The order of the transform coefficients in this one-dimensional sequence reflects the order in which those transform coefficients are processed according to the candidate scan order. Depending on the specific type of candidate scan order, the index of the last non-zero coefficient (e.g., the location of the block end position) can be earlier or later in the one-dimensional sequence.

[0079] Identifying the block end position of the candidate scan order may include determining, based on the candidate scan order, whether a specific number of transform coefficients of the transform block are encoded or decoded using execution technique 800. This specific number of transform coefficients includes both non-zero transform coefficients and zero-value transform coefficients to be encoded or decoded. More specifically, this specific number of transform coefficients includes all non-zero transform coefficients of the transform block and each of the zero-value transform coefficients of the transform block preceding the last non-zero coefficient. See below. Figure 9 This describes implementation methods and examples for determining the number of transform coefficients to be encoded or decoded.

[0080] At 804, a cost value is determined for the candidate scan order based on the block end position. For example, the cost value of the first candidate scan order can be determined based on the number of transform coefficients located before the block end position of the first candidate scan order, and the cost value of the second candidate scan order can be determined based on the number of transform coefficients located before the block end position of the second candidate scan order.

[0081] The cost of a candidate scan order is determined based on the number of zero-value coefficients within the transform block preceding the block end position of the candidate scan order. Thus, a candidate scan order with an earlier block end position in the one-dimensional sequence of transform coefficients will have a lower cost than a candidate scan order with a later block end position in that one-dimensional sequence. See below. Figure 9This describes implementation methods and examples for determining the number of zero-value coefficients prior to the end position of a block in a transform block.

[0082] At 806, one of the candidate scan orders is selected based on the cost value. Selection may include choosing the one with the lowest cost value. For example, if the set of candidate scan orders includes the first and second candidate scan orders described above, one of the first or second candidate scan orders is selected as the scan order for encoding or decoding the transform block based on the cost value of the first and second candidate scan orders.

[0083] For example, selecting one of the first or second candidate scan order may include determining whether the cost value of the first candidate scan order is less than the cost value of the second candidate scan order. In response to determining that the cost value of the first candidate scan order is less than the cost value of the second candidate scan order, the first candidate scan order is selected as the scan order for encoding or decoding the transform block. However, in response to determining that the cost value of the first candidate scan order is greater than the cost value of the second candidate scan order (e.g., or the cost value of the second candidate scan order is less than the cost value of the first candidate scan order), the second candidate scan order is selected as the scan order for encoding or decoding the transform block.

[0084] Alternatively, selecting a candidate scan order may involve more than simply determining which of the candidate scan orders has the lowest cost value. For example, the set of candidate scan orders could include the first and second candidate scan values, as well as the third and fourth candidate scan values. The first and second candidate scan values ​​can scan the transform coefficients of the transform block from the same angle but in different directions. For example, the first candidate scan order could have a 45-degree diagonal direction from the bottom left to the top right, and the second candidate scan order could have a 45-degree diagonal direction from the top right to the bottom left. For both the first and second candidate scan orders, the cost value may not be specified. Instead, the first and second candidate scan orders can share a common cost value.

[0085] Therefore, selecting one of the candidate scan orders may include determining that the first candidate scan order has the lowest cost value among the candidate scan order sets. In this case, either the first or second candidate scan order will be selected. Determining which of the first or second candidate scan orders to select may include determining whether the third candidate scan order has a higher cost value than the fourth candidate scan order. For example, the third and fourth candidate scan orders may have different cost values ​​because they process the transform coefficients of the transform block in different orders. The different orders corresponding to the third and fourth candidate scan orders may indicate whether a bottom-left to top-right direction or a top-right to bottom-left direction is better.

[0086] Thus, in response to determining that the third candidate scan order has a higher cost value than the fourth candidate scan order, the first candidate scan order is selected as the scan order for encoding or decoding the transform block. However, in response to determining that the cost value of the third candidate scan order is equal to or less than the cost value of the fourth candidate scan order, the second candidate scan order is selected as the scan order for encoding or decoding the transform block.

[0087] At 808, the transform block is encoded or decoded according to the selected scan order. For example, the transform block can be encoded or decoded by scanning the transform coefficients according to the selected scan order. At 810, the encoded or decoded transform block is output to an output element. For example, during the encoding operation, the encoded transform block is output to a bitstream, such as a bitstream that can be decoded later. In another example, during the decoding operation, the decoded transform block is output for further processing by the decoder and then output to an output video stream, such as a computing device at the receiving station (e.g., ...). Figure 2 The computing device 200 shown and Figure 1 The video stream is displayed on the receiving station 106 shown in the figure.

[0088] In some implementations, technique 800 may include determining cost values ​​for at least some of the candidate scan orders based on the transform coefficients of adjacent transform blocks. Those cost values ​​determined based on the transform coefficients of adjacent transform blocks can then be used to select a scan order for encoding or decoding the transform blocks. For example, the cost values ​​used for determining the candidate scan order based on the transform coefficients of adjacent transform blocks may be summed to produce a summed cost value. In such an implementation, selecting a scan order for encoding or decoding the transform blocks may include determining that one of the candidate scan orders has the lowest summed cost value and selecting that candidate scan order as the scan order for encoding or decoding the transform blocks.

[0089] In another example, the cost values ​​for candidate scan orders, determined based on the transform coefficients of adjacent transform blocks, can be ranked and accumulated to produce an accumulated ranking. In such an implementation, selecting the scan order for encoding or decoding transform blocks may include determining one of the candidate scan orders that has the lowest accumulated ranking (e.g., closest to "1" when the ranking is expressed as a number), and selecting that candidate scan order as the scan order for encoding or decoding transform blocks. See below. Figure 10 The document describes implementation methods and examples for selecting the scan order using the cumulative cost value or cumulative ranking of adjacent transform blocks.

[0090] Next reference Figure 9The diagram illustrates a flowchart of an example of technique 900 for determining the number of zero-valued coefficients prior to the end position of a block in a transform block. At 902, the total number of transform coefficients to be encoded or decoded is determined. The total number of transform coefficients represents the number of transform coefficients located prior to the end position of a block according to a given candidate scan order. The total number of transform coefficients includes both non-zero and non-zero coefficients.

[0091] In step 904, the total number of non-zero coefficients in the transform block is determined. For each candidate scan order, the total number of non-zero coefficients in the transform block is the same because each candidate scan order must scan each non-zero coefficient before the end position of the block where the candidate scan order can be identified.

[0092] In 906, the total number of zero-valued coefficients located within the transform block and preceding the block end position is determined. Determining the total number of zero-valued coefficients located within the transform block and preceding the block end position involves calculating the difference between the total number of transform coefficients to be encoded or decoded and the total number of non-zero coefficients in the transform block.

[0093] For example, the following pseudocode can be used to execute technique 900, where the cost value of the candidate scan order reflects the total number of zero-value coefficients that are located within the transform block and before the end of the block for that candidate scan order:

[0094] initialize scan_cost as array of size S to all 0

[0095] determine nnz / / number of non-zero coefficients in the transform block

[0096] for each i in{0,1,…,S-1}: / / for each candidate scan order in a set ofS>1scan orders

[0097] determine eob[i] / / number of coefficients to beencoded / decoded

[0098] scan_cost[i]=eob[i]–nnz / / cost value

[0099] (Initialize scan_cost to an array where all values ​​of size S are 0)

[0100] Determine the number of non-zero coefficients in the nnz / / transform block

[0101] For each i in {0, 1, ..., S-1}, for each candidate scan order in the set of scan orders S>1

[0102] Determine eob[i] / / the number of coefficients to be encoded / decoded

[0103] scan_cost[i] = eob[i] – nnz / / cost value)

[0104] Therefore, the cost of a given candidate scan order is positively correlated with the total number of zero coefficients in that candidate scan order that are located within the transform block and before the end of the block.

[0105] In some implementations, the pseudocode may differ from the implementation of technique 900. For example, the set of candidate scan orders may include more or fewer than six candidate scan orders. The cost value of a given candidate scan order may be a function of the total number of zero-value coefficients located within the transform block and before the end position of the block in the transform block (e.g., equal to the total number of zero-value coefficients located within the transform block and before the end position of the block in the transform block).

[0106] Next reference Figure 10 The diagram illustrates a flowchart of an example of a technique 1000 for selecting a scan order using the accumulated cost values ​​of adjacent transform blocks. In 1002, a cost is determined for the candidate scan order of the current transform block. For example, the cost could be as referenced... Figure 8 The cost value is determined as described in technique 800. At 1004, the cumulative cost of the candidate scan order of the left and top adjacent transform blocks of the current transform block is determined. For example, there may be one or more left adjacent transform blocks of the current transform block. In another example, there may be one or more top adjacent transform blocks of the current transform block. In yet another example, a combination of left and top adjacent transform blocks of the current transform block may exist.

[0107] The cost of accumulating candidate scan sequences for the left and top adjacent transform blocks involves determining the cost value for each candidate scan sequence for each adjacent transform block, and then summing the cost values ​​for the same candidate scan sequences to produce an accumulated cost. For example, the cost value of the first candidate scan sequence for the left adjacent transform block could be 1, and the cost value of the same first candidate scan sequence for the top adjacent transform block could be 2. Therefore, the accumulated cost of this first candidate scan sequence would be 3. In step 1006, the candidate scan sequence with the lowest accumulated cost is selected. In another example, the cost values ​​can be weighted by a factor (e.g., the corresponding transform block size) before being used to generate the accumulated cost.

[0108] In some implementations, technique 1000 may include using the cumulative ranking of adjacent transform blocks instead of a cumulative cost value to select a scan order. For example, instead of determining a cost value for candidate scan orders for a given transform block, the available candidate scan orders for that given transform block may be ranked based on one or more of the block end position, the number of zero-value coefficients preceding the block end position, etc. For example, the ranking may be expressed numerically, such as "1" reflecting the best ranking and "N" reflecting the lowest ranking, where N is the number of candidate scan orders in the candidate scan order set. The rankings of the given candidate scan orders at each adjacent transform block may then be summed to produce a cumulative ranking of the candidate scan orders. The cumulative rankings of each of the candidate scan orders may then be compared. Thus, one of the candidate scan orders selected for the current transform block has the lowest of these cumulative rankings.

[0109] In some implementations, technique 1000 may omit the cost of determining the candidate scan order for the current transform block. For example, the scan order selected for encoding or decoding the current transform block may be based solely on the cost determined for the candidate scan orders of adjacent transform blocks. In such an implementation, the operation of accumulating the cost of the candidate scan orders does not include the cost of using the candidate scan order determined for the current transform block.

[0110] In some implementations, cost values ​​may not be available for some candidate scan orders among some adjacent transform blocks. For example, the current transform block may have already been processed using a first candidate scan order, a second candidate scan order, and a third candidate scan order (e.g., such that a cost value for each of the first, second, and third candidate scan orders has been determined for the current transform block). A first adjacent transform block may have been processed using the first and second candidate scan orders, while a second adjacent transform block may have been processed using both the second and third candidate scan orders. Thus, the cost value of the second candidate scan order can be used from both the first and second adjacent transform blocks. However, the cost value of the first candidate scan order can only be used from the first adjacent transform block, and the cost value of the third candidate scan order can only be used from the second adjacent transform block.

[0111] Figure 11A candidate scan sequence set 1100 according to an embodiment of the present disclosure is shown. The candidate scan sequence set includes candidate scan sequences 1102, 1104, 1106, 1108, 1110, and 1112. Each of the candidate scan sequences 1102, 1104, 1106, 1108, 1110, and 1112 has a different angle and / or direction for scanning coefficients in the transform block. For example, candidate scan sequence 1102 has a horizontal 0-degree angle, candidate scan sequence 1104 has a horizontal 22.5-degree angle from bottom left to top right, candidate scan sequence 1106 has a 45-degree angle from bottom left to top right, candidate scan sequence 1108 has a 45-degree angle from top right to bottom left, candidate scan sequence 1110 has a vertical 22.5-degree angle from top right to bottom left, and candidate scan sequence 1112 has a vertical 0-degree angle.

[0112] The candidate scan order set 1100 can be represented as an array or other data object including elements referenced by indices. For example, the candidate scan order set 1100 can be referred to as candidate [x], where x = {0, 1, 2, 3, 4, 5}. For example, candidate scan order 1102 has index [0] and is therefore referenced at candidate [0], candidate scan order 1104 has index [1] and is therefore referenced at candidate [1], and so on. In some cases, the size of the candidate scan order set 1100 can be based on the size of the transform block for which the scan order is selected. For example, when the transform block is large, there may be a larger number of candidate scan orders.

[0113] However, the set of candidate scan orders from which the scan order for encoding or decoding transform blocks is selected can be different from the set of candidate scan orders 1100. For example, other scan orders, such as..., can supplement or replace one or more of candidate scan orders 1102, 1104, 1106, 1108, 1110, or 1112. Figure 7 One or more of the scan orders 702, 704, 706, or 708 shown can be included in the candidate scan order set 1100. In another example, besides Figure 7 Alternatively, a scan order other than the scan order shown in 11 may be included in the set of candidate scan orders, such as as a supplement or alternative to one or more of candidate scan orders 1102, 1104, 1106, 1108, 1110, or 1112.

[0114] Figure 12The illustration shows a current transform block 1200 and its neighboring transform blocks 1202-1208 within a video frame according to an embodiment of this disclosure. Some embodiments of this disclosure include selecting a scan order for encoding or decoding transform blocks based on a cost value determined only with respect to the current transform block (e.g., current transform block 1200). However, other embodiments include selecting a scan order based on the cost value or ranking of neighboring transform blocks (e.g., neighboring transform blocks 1202-1208) from such a current transform block. For example, the above-described... Figure 10 The described technique 1000 generates a cumulative cost or cumulative ranking based on cost values ​​or rankings determined for individual blocks in adjacent transform blocks 1202-1208, and can select a candidate scan order for the current transform block 1200 based on these cumulative costs or cumulative rankings. In this way, adjacent transform blocks 1202-1208 can substantially vote on the candidate scan order for encoding or decoding the current transform block 1200, such as based on cumulative cost values ​​or cumulative rankings from adjacent transform blocks 1202-1208.

[0115] The foregoing aspects of encoding and decoding illustrate some examples of encoding and decoding techniques and hardware components configured to perform all or part of these examples of encoding and / or decoding techniques. However, it should be understood that, as used in the claims, encoding and decoding can mean data compression, decompression, transformation, or any other processing or alteration.

[0116] The term “example” as used herein refers to something used as an example, instance, or illustration. Any aspect or design described herein as an “example” is not necessarily to be construed as preferred or superior to other aspects or designs. Rather, the use of the terms “example” or “implementation” is to present concepts in a specific manner. As used in this application, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless otherwise stated or clearly indicated by the context, the statement “X includes A or B” is intended to naturally include such an arrangement. That is, if X includes A; X includes B; or X includes both A and B, then “X includes A or B” is satisfied in any of the foregoing cases. Furthermore, the article “a / an” used in this application and the appended claims should generally be interpreted as meaning “one or more” unless otherwise stated or clearly indicated from the context to refer to the singular form. In addition, unless so described, the use of the terms “implementation” or “one implementation” throughout this disclosure does not imply the same embodiment or implementation.

[0117] The implementation of transmitting station 102 and / or receiving station 106 (and the algorithms, methods, instructions, etc. stored thereon and / or executed thereon (including by encoder 400 and decoder 500) can be implemented in hardware, software, or a combination thereof. Hardware (e.g., hardware component 1000 and / or hardware component 1100) can include, for example, a computer, intellectual property (IP) core, application-specific integrated circuit (ASIC), programmable logic array, optical processor, programmable logic controller, microcode, microcontroller, server, microprocessor, digital signal processor, or any other suitable circuitry. In the claims, the term "processor" should be understood to encompass any of the foregoing hardware, whether alone or in combination. The terms "signal" and "data" are used interchangeably. Furthermore, portions of transmitting station 102 and receiving station 106 may not necessarily be implemented in the same manner.

[0118] Furthermore, in one aspect, for example, transmitting station 102 or receiving station 106 may be implemented using a general-purpose computer or general-purpose processor via a computer program that, when executed, implements any of the corresponding methods, algorithms, and / or instructions described herein. Alternatively or supplemented, for example, a dedicated computer / processor may be used, which may contain additional hardware for implementing any of the methods, algorithms, or instructions described herein.

[0119] Sending station 102 and receiving station 106 can be implemented, for example, on a computer in a video conferencing system. Alternatively, sending station 102 can be implemented on a server, and receiving station 106 can be implemented on a device separate from the server, such as a handheld communication device. In this case, using encoder 400, sending station 102 can encode content into an encoded video signal and send the encoded video signal to the communication device. The communication device can then decode the encoded video signal using decoder 500. Alternatively, the communication device can decode content stored locally on the communication device, such as content not sent by sending station 102. Other suitable sending and receiving implementations are available. For example, receiving station 106 can be a typically stationary personal computer instead of a portable communication device, and / or the device including encoder 400 can also include decoder 500.

[0120] Furthermore, all or part of the embodiments of this disclosure may take the form of a computer program product accessible from, for example, a computer-usable medium or a computer-readable medium. A computer-usable medium or a computer-readable medium may be any device capable of, for example, tangibly containing, storing, communicating, or transmitting a program used by or in conjunction with any processor. Such a medium may be, for example, an electronic, magnetic, optical, electromagnetic, or semiconductor device. Other suitable media are also available.

[0121] The embodiments, implementations, and aspects described above are intended to facilitate a clear understanding of this disclosure and are not intended to limit it. Rather, this disclosure is intended to cover various modifications and equivalent arrangements included within the scope of the appended claims, and the scope shall be given the broadest interpretation permitted by law to cover all such modifications and equivalent arrangements.

Claims

1. A method for decoding the current transform block, comprising: For one or more previously decoded transform blocks, determine the accumulated cost value for each candidate scan order in a set of candidate scan orders that can be used for block-level scan order selection, wherein determining the accumulated cost value for a given candidate scan order in the set of candidate scan orders includes: The first value and the second value are added together. The first value represents the cost value of the given candidate scan order based on the block end position within the first previously decoded transform block in the one or more previously decoded transform blocks. The second value represents the cost value of the given candidate scan order based on the block end position within the second previously decoded transform block in the one or more previously decoded transform blocks. The candidate scan order with the lowest accumulated cost value among the candidate scan order sets is selected as the scan order; and Use the selected scan order to decode the current transform block.

2. The method according to claim 1, wherein, At least one of the first previously decoded transform block or the second previously decoded transform block is an adjacent transform block of the current transform block.

3. The method according to claim 1, wherein, The selected scan order is the first scan order, wherein the method further includes: The next transform block is decoded using a second scan order selected from the set of candidate scan orders.

4. The method according to claim 1, wherein, The accumulated cost value includes the accumulated cost value determined for the current transform block.

5. The method according to claim 1, wherein, The first value is weighted based on the size of the first previously decoded transform block, and the second value is weighted based on the size of the second previously decoded transform block.

6. The method according to claim 1, wherein, The cumulative cost value is represented as the cumulative ranking of the corresponding candidate scan order in the candidate scan order set, wherein the candidate scan order selected as the scan order has the lowest cumulative ranking among the cumulative rankings.

7. The method according to claim 1, wherein, The first previously decoded transform block is the left neighbor transform block of the current transform block, and the second previously decoded transform block is the upper neighbor transform block of the current transform block.

8. A method for encoding a current transform block, comprising: For one or more previously encoded transform blocks, determine the accumulated cost value for each candidate scan order in a set of candidate scan orders that can be used for block-level scan order selection, wherein determining the accumulated cost value for a given candidate scan order in the set of candidate scan orders includes: The first value and the second value are added together, where the first value represents the cost value of the given candidate scan order based on the block end position within the first previously encoded transform block in the one or more previously encoded transform blocks, and the second value represents the cost value of the given candidate scan order based on the block end position within the second previously encoded transform block in the one or more previously encoded transform blocks. The candidate scan order with the lowest accumulated cost value among the candidate scan order sets is selected as the scan order; and The selected scan order is used to encode the current transform block into the output bitstream.

9. The method according to claim 8, wherein, Determining the cumulative cost value of a given candidate scan order in the candidate scan order set includes: The cost value determined based on the block end position within the current transform block and the block end position within each of the one or more previously encoded transform blocks is added together.

10. The method according to claim 8, wherein, The selected scan order is the first scan order, wherein the method further includes: The next transform block is encoded using a second scan order selected from the set of candidate scan orders.

11. The method according to claim 8, wherein, The accumulated cost value includes the accumulated cost value determined for the current transform block.

12. The method according to claim 8, wherein, The first value is weighted based on the size of the first previously encoded transform block, and the second value is weighted based on the size of the second previously encoded transform block.

13. The method according to claim 8, wherein, The cumulative cost value is represented as the cumulative ranking of the corresponding candidate scan order in the candidate scan order set, wherein the candidate scan order selected as the scan order has the lowest cumulative ranking among the cumulative rankings.

14. The method according to claim 8, wherein, The first previously encoded transform block is the left neighbor transform block of the current transform block, and the second previously encoded transform block is the upper neighbor transform block of the current transform block.

15. A method for decoding the current transform block, comprising: The current transform block is decoded using a scan order selected at the block level, based on the accumulated cost value determined for one or more previously decoded transform blocks. Specifically, for each candidate scan order in the set of candidate scan orders available for block-level scan order selection, the accumulated cost value is determined by adding a first value and a second value, where the first value represents the cost value of the candidate scan order based on the block end position within a first previously decoded transform block of the one or more previously decoded transform blocks, and the second value represents the cost value of the candidate scan order based on the block end position within a second previously decoded transform block of the one or more previously decoded transform blocks, and wherein the candidate scan order having the lowest accumulated cost value is selected as the scan order.

16. The method according to claim 15, wherein, The first value is weighted based on the size of the first previously decoded transform block, and the second value is weighted based on the size of the second previously decoded transform block.

17. The method according to claim 15, wherein, Determining the cumulative cost value of a given candidate scan order in the candidate scan order set includes: The cost value determined based on the block end position within the current transform block and the block end position within at least one of the one or more previously decoded transform blocks is added together.

18. The method according to claim 15, wherein, The accumulated cost value includes the accumulated cost value determined for the current transform block.

19. The method according to claim 15, wherein, The cumulative cost value is represented as the cumulative ranking of the corresponding candidate scan order in the candidate scan order set, wherein the candidate scan order selected as the scan order has the lowest cumulative ranking among the cumulative rankings.

20. The method of claim 15, wherein, The first previously decoded transform block is the left neighbor transform block of the current transform block, and the second previously decoded transform block is the upper neighbor transform block of the current transform block.

Images