Methods and systems for exponential partitioning

Abstract

Providing a method and system for exponential partitioning. [Solution] The decoder includes a circuit configured to receive a bitstream, determine whether an exponential division mode is enabled, divide a block into a first region and a second region according to a curve, and reconstruct the pixel data of the block using the curve, wherein the first region and the second region are non-rectangular. In one embodiment, the exponential division mode is signaled within the bitstream.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 739,446, entitled "EXPONENTIAL PARTITIONING", filed on October 1, 2018; U.S. Provisional Patent Application No. 62 / 739,677, entitled "PREDICTING EXPONENTIAL PARTITIONING PARAMETERS", filed on October 1, 2018; and U.S. Provisional Patent Application No. 62 / 739,531, entitled "SHAPE ADAPTIVE DISCRETE COSINE TRANSFORMATION FOR EXPONENTIALLY PARTITIONED BLOCKS", filed on October 1, 2018, each of which is hereby incorporated by reference in its entirety.

[0002] (Field of the Invention) The present invention generally relates to the technical field of compressing and decompressing digital video, including decoding and encoding. In particular, the present invention is directed to methods and systems for exponential partitioning of coding units.

Background Art

[0003] (Background) A video codec may include an electronic circuit or software for compressing or decompressing digital video. It can convert uncompressed video into a compressed format or vice versa. In the context of video compression, a device that compresses video (and / or performs some function thereof) may typically be called an encoder, and a device that decompresses video (and / or performs some function thereof) may be called a decoder.

[0004] The format of the compressed data may conform to standard video compression specifications. Compression can be lossy in that the compressed video loses some information present in the original video. This may result in the decompressed video having less information than the original uncompressed video, as it may not accurately reconstruct the original video.

[0005] There can be complex relationships between video quality, the amount of data used to represent the video (determined, for example, by the bitrate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay (e.g., latency), and similar factors. [Overview of the Initiative] [Means for solving the problem]

[0006] (Summary of Disclosure) In one aspect of the present invention, the decoder may include a circuit that can be configured to receive a bitstream. In an embodiment, the circuit may further be configured to determine whether an exponential division mode is enabled and to divide a block into a first region and a second region according to a curve. In an embodiment, the circuit may also be configured to reconstruct the pixel data of the block using the curve, and the first and second regions may be non-rectangular.

[0007] The decoder may further include one or more of the following features, which may be utilized individually or in combination: In an embodiment, an exponential partitioning mode may be signaled within the bitstream. In an embodiment, the curve dividing a block into a first and second region may be characterized by a predefined template. In another embodiment, the curve dividing a block into a first and second region may be characterized by predefined coefficient values. Furthermore, in an embodiment, the exponential partitioning mode is available for block sizes larger than or equal to 8x8 lumens. In another embodiment, reconstructing the pixel data may include computing a predictor for the first region using associated motion vectors contained within the bitstream. In another embodiment, the decoder may also include an entropy decoder processor which may be configured to receive a bitstream and decode the bitstream into quantized coefficients. The decoder may also include an inverse quantization and inverse transform processor which may be configured to process the quantized coefficients, including performing an inverse discrete cosine transform. Furthermore, the decoder may include a deblocking filter, a frame buffer, and an intra-prediction processor. In one embodiment, the bitstream may include a parameter indicating whether the exponential partitioning mode is enabled for the block. In another embodiment, the block may form part of a quad-tree plus binary decision tree. Alternatively, the block may be a non-leaf node of the quad-tree plus binary decision tree. In one embodiment, the block may be a coding tree unit or a coding unit. In yet another embodiment, the first region may be a coding unit or a prediction unit.

[0008] In a further aspect of the present invention, the method may include a decoder receiving a bitstream and the decoder determining whether an exponential partitioning mode is enabled. The method may also include the decoder determining a curve for partitioning a block into a first region and a second region, and the decoder reconstructing the pixel data of the block using the curve.

[0009] The method may further include one or more of the following features, which may be utilized individually or in combination: In one embodiment, the exponential partitioning mode may be signaled within the bitstream. In another embodiment, the curve dividing a block into a first and second region may be characterized by a predefined template. In one embodiment, the curve dividing a block into a first and second region may be characterized by predefined coefficient values. In one embodiment, the exponential partitioning mode may be usable for block sizes larger than or equal to 8 × 8 lumens samples. In another embodiment, reconstructing the pixel data may include computing a predictor for the first region using associated motion vectors contained within the bitstream. In another embodiment, the decoder may also include an entropy decoder processor which may be configured to receive a bitstream and decode the bitstream into quantized coefficients. The decoder may also include an inverse quantization and inverse transform processor which may be configured to process the quantized coefficients, including performing an inverse discrete cosine transform. Furthermore, the decoder may encompass a deblocking filter, a frame buffer, and an intra-prediction processor. In one embodiment, the bitstream may include a parameter indicating whether the exponential partitioning mode is enabled for the block. In another embodiment, the block may form part of a quad-tree plus binary decision tree. Alternatively, the block may be a non-leaf node of the quad-tree plus binary decision tree. In yet another embodiment, the block may be a coding tree unit or a coding unit. In one embodiment, the first region may be a coding unit or a prediction unit.

[0010] Non-temporary computer program products that store instructions (i.e., physically embodied computer program products) are also described, such that when an instruction is executed by one or more data processors of one or more computing systems, it causes at least one data processor to perform the operations described herein. Similarly, computer systems that may have one or more data processors and memory coupled to one or more data processors are also described. The memory may temporarily or permanently store instructions that cause at least one processor to perform one or more of the operations described herein. In addition, the method may be implemented by one or more data processors within a single computing system, or by one or more data processors distributed across two or more computing systems. Such computing systems may be connected by direct connections between one or more of the computing systems, by one or more connections including connections over a network (e.g., the Internet, wireless wide area networks, local area networks, wide area networks, wired networks, and similar), etc., and may exchange data and / or commands or other instructions or similar.

[0011] Details of one or more variations of the subject matter described herein are described in the accompanying drawings and the description below. Other features or advantages described herein are evident from the description and drawings, as well as from the claims. The present invention provides, for example, the following: (Item 1) Receiving a bitstream and Determining whether the exponential partitioning mode is enabled, Dividing the block into a first region and a second region according to the curve, Reconstructing the pixel data of the block using the curve, wherein the first region and the second region are non-rectangular. A circuit configured to perform A decoder equipped with a decoder. (Item 2) The decoder according to item 1, wherein the exponential division mode is transmitted within the bitstream. (Item 3) The decoder according to item 1, wherein the curve dividing the block into the first region and the second region is characterized by a predefined template. (Item 4) The decoder according to item 1, wherein the curve dividing the block into the first region and the second region is characterized by a predetermined coefficient value. (Item 5) The decoder described in item 1, wherein the exponential division mode is available for block sizes larger than or equal to 8 x 8 luma samples. (Item 6) The decoder according to item 1, wherein reconstructing the pixel data includes computing a predictor for the first region using associated motion vectors contained within the bitstream. (Item 7) An entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients, An inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine transform, Deblocking filter and Frame buffer and Intra Prediction Processor and The decoder described in item 1, further equipped with the features described in item 1. (Item 8) The decoder according to item 1, wherein the bitstream includes a parameter indicating whether the exponential division mode is enabled for the block. (Item 9) The decoder according to item 1, wherein the block forms part of a quad tree plus binary decision tree. (Item 10) The decoder according to item 9, wherein the block is a non-leaf node of a quad tree plus binary decision tree. (Item 11) The decoder according to item 1, wherein the block is an encoding tree unit or an encoding unit. (Item 12) The decoder according to item 1, wherein the first region is an encoding unit or a prediction unit. (Item 13) A method comprising: a decoder receiving a bitstream; the decoder determining whether an exponential splitting mode is enabled; the decoder determining a curve for splitting a block into a first region and a second region; the decoder reconstructing pixel data of the block using the curve. and. (Item 14) The method according to item 13, wherein the exponential splitting mode is signaled within the bitstream. (Item 15) The method according to item 13, wherein the curve for splitting the block into the first region and the second region is characterized by a predefined template. (Item 16) The method according to item 13, wherein the curve for splitting the block into the first region and the second region is characterized by values of predefined coefficients. (Item 17) The method according to item 13, wherein the exponential splitting mode is applicable to block sizes greater than or equal to 8×8 luma samples. (Item 18) The method according to item 13, wherein reconstructing the pixel data includes calculating a predictor for the first region using associated motion vectors contained within the bitstream. (Item 19) The decoder An entropy decoder processor configured to receive the bitstream and decode the bitstream into quantized coefficients, An inverse quantization and inverse transform processor configured to process the quantized coefficients, including performing an inverse discrete cosine transform, A deblocking filter, A frame buffer, An intra prediction processor and comprising the method according to item 13. (Item 20) The method according to item 13, wherein the bitstream includes a parameter indicating whether the exponential partitioning mode is enabled for the block. (Item 21) The method according to item 13, wherein the block forms part of a quad tree plus binary decision tree. (Item 22) The method according to item 21, wherein the block is a non-leaf node of a quad tree plus binary decision tree. (Item 23) The method according to item 13, wherein the block is a coding tree unit or a coding unit. (Item 24) The method according to item 13, wherein the first region is a coding unit or a prediction unit.

Brief Description of the Drawings

[0012] (Description of the Drawings) [Figure 1] FIG. 1 is a diagram illustrating an example of block partitioning of pixels.

[0013] [Figure 2] Figure 2 illustrates an example of a geometric partition.

[0014] [Figure 3A] Figure 3A illustrates an example of an exponential partition according to several aspects of this subject that can increase compression efficiency.

[0015] [Figure 3B] Figure 3B is a series of diagrams illustrating the exponential partitioning of an example template.

[0016] [Figure 3C] Figure 3C illustrates example curves associated with four predefined coefficients that can define an example exponential function.

[0017] [Figure 3D] Figure 3D illustrates another example block showing different starting P1 and ending P2 indices that divide a rectangular block.

[0018] [Figure 4] Figure 4 is a system block diagram illustrating a video encoder that can perform exponential partitioning as an example.

[0019] [Figure 5A] Figure 5A is a process flow diagram illustrating an example of encoding video with exponential partitioning, following several aspects of this subject that can increase compression efficiency while reducing encoding complexity.

[0020] [Figure 5B] Figure 5B is a process flow diagram illustrating an example of encoding video using partitioning parameters with exponential partitioning, in accordance with several aspects of this subject.

[0021] [Figure 5C]Figure 5C is a process flow diagram illustrating an example of encoding video using shape-adaptive discrete cosine transform with exponential partitioning, in accordance with several aspects of this subject.

[0022] [Figure 6] Figure 6 is a system block diagram illustrating an example of a decoder capable of decoding a bitstream using exponential partitioning.

[0023] [Figure 7] Figure 7 is a process flow diagram illustrating an example of a process for decoding a bitstream using exponential partitioning.

[0024] [Figure 8] Figure 8 illustrates an example of a quad-tree plus binary tree partitioning of a frame.

[0025] [Figure 9] Figure 9 illustrates an example of exponential partitioning at the CU level of the quad tree plus binary tree shown in Figure 8.

[0026] [Figure 10] Figure 10 illustrates an image containing an apple, which may not be efficiently divided by straight line segments.

[0027] [Figure 11] Figure 11 illustrates another example of a block that has been partitioned according to an exponential partitioning method.

[0028] [Figure 12] Figure 12 illustrates the inheritance of exponential partitioning parameters from spatially adjacent blocks by the current block.

[0029] [Figure 13]Figure 13 illustrates an example of spatially adjacent blocks to the current block.

[0030] [Figure 14] Figure 14 illustrates a current block that is an example of a current block that has temporally adjacent blocks and inherits the exponential partitioning parameter.

[0031] Similar reference symbols in various drawings refer to the same elements. [Modes for carrying out the invention]

[0032] (Detailed explanation) Some implementations of this subject relate to exponential partitioning. In exponential partitioning, rectangular blocks can be partitioned into non-rectangular regions with curves, compared to straight line segments. Using curves to partition blocks can enable partitioning that more precisely tracks the target boundary, resulting in lower motion compensation prediction errors, smaller residuals, and therefore improved compression efficiency. In some implementations, the curves can be characterized by exponential functions. The curve (e.g., exponential function) can be determined using predefined coefficients and / or templates that can be signaled within the bitstream for the decoder. In some implementations, exponential partitioning may be usable for samples larger than or equal to 8x8 luma samples. By partitioning rectangular blocks with curves, greater compression efficiency can be achieved for specific objects than techniques limited to straight line segment partitioning, such as those involving geometric partitioning.

[0033] Furthermore, some implementations of this subject involve predicting exponential partitioning parameters using spatial and / or temporal reference blocks. In some implementations, exponential partitioning merge variables can be signaled to a given current block, indicating that the given current block may inherit all or some of these exponential partitioning parameters from another block. This other block may be spatially or temporally adjacent. By enabling blocks to inherit exponential partitioning parameters from other blocks, the amount of signal within the bitstream can be reduced, and greater compression efficiency can be achieved.

[0034] Furthermore, some implementations of this subject may include performing a shape-adaptive discrete cosine transform (SADCT) on regions (e.g., blocks) that are divided into non-rectangular regions with curves, compared to straight line segments. Where a block is exponentially partitioned, the resulting regions (e.g., partitions) can easily be reduced to one region with low prediction error and another with high prediction error. Thus, this subject may include performing a SADCT on the region with low prediction error. By performing a SADCT on the region with low prediction error, compression efficiency can be improved. In some implementations, the inverse SADCT may be performed on one of the regions partitioned by the exponential partitioning during decoding. In some implementations, the inverse SADCT may be signaled within the bitstream as an alternative transform to the full-block discrete cosine transform (DCT) for partitions with low prediction error. In some implementations, the inverse SADCT may be performed based on the exponential partitioning parameters and without requiring explicit signaling within the bitstream on which the inverse SADCT is performed.

[0035] Motion compensation may involve techniques for predicting a video frame or part thereof by capturing the movement of the camera and / or objects in the video, given previous and / or future frames. It may be employed in the encoding and decoding of video data for video compression, such as in the Motion Picture Experts Group (MPEG)-2 (also known as advanced video coding (AVC)) standard. Motion compensation can describe a picture in terms of the conversion of a reference picture to the current picture. The reference picture may be from earlier or later in time compared to the current picture. Compression efficiency can be improved when the image can be accurately synthesized from previously transmitted and / or stored images.

[0036] Block division can refer to a method for finding regions of similar motion in video encoding. Some form of block division can be found in video codec standards, including MPEG-2, H.264 (also known as AVC or MPEG-4 Part 10), and H.265 (also known as High Efficiency Video Coding (HEVC)). In an example of block division techniques, non-overlapping blocks of video frames can be divided into rectangular subblocks to find block divisions containing pixels with similar motion. This technique works well when all pixels in a block division have similar motion. The motion of pixels within a block can be determined in relation to previously encoded frames.

[0037] Figure 1 illustrates an example of pixel block division according to an embodiment. The initial rectangular picture or block 100, which itself may be a subblock (e.g., a node within a coding tree), can be divided into rectangular subblocks. For example, in 110, block 100 is divided into two rectangular subblocks 110a and 110b. Subblocks 110a and 110b can then be processed separately. As another example, in 120, block 100 is divided into four rectangular subblocks 120a, 120b, 120c, and 120d. Subblocks can be further divided until it is determined that the pixels within the subblock share the same motion, that a minimum block size has been reached, or other criteria. When the pixels within a subblock have similar motion, the motion vector can describe the motion of all pixels in that region.

[0038] Some video encoding techniques may involve geometric partitioning, where a rectangular block (as illustrated in Figure 1, for example) is further divided by a straight line segment into two potentially non-rectangular regions. For example, Figure 2 illustrates an example of geometric partitioning. An example rectangular block 200 (represented as M×N pixels, possibly having M pixels wide and N pixels high) can be divided into two regions (region 0 and region 1) along a straight line segment P1 P2 205. When pixels in region 0 have similar motion, the motion vector can explain the motion of all pixels in that region. The motion vector can be used to compress region 0. Similarly, when pixels in region 1 have similar motion, the associated motion vector can explain the motion of pixels in region 1. Such geometric partitioning can be signaled to a receiver (e.g., decoder) by encoding positions P1 and P2 (or alternatives to positions P1 and P2) in the video bitstream.

[0039] When encoding video data using geometric partitioning, a linear segment 205 (or more specifically, P1 and P2) can be determined. However, linear segments may not be able to partition blocks in a way that reflects the object's boundary. As a result, partitioning with linear segments may not be efficient in partitioning blocks (for example, in a way that results in small residuals). This can be the case when a block may contain an object or pixels representing a boundary (e.g., luma samples) that has a curved (e.g., non-linear) boundary. For example, now referring to Figure 10, we see an image containing an apple that may not be efficiently partitioned by linear segments. If the apple were partitioned by linear segments according to geometric partitioning, it would contain several rectangular blocks pointing to parts of the image where the partitioning may not closely follow the boundary of the object (e.g., the apple).

[0040] Some implementations of this subject involve dividing a rectangular block into non-rectangular regions with curves, compared to straight line segments. Using curves to divide a block can allow the division to more closely follow the target boundary, resulting in lower prediction errors, smaller residuals, and therefore improved compression efficiency. In some implementations, the curve can be characterized by an exponential function. The curve (e.g., exponential function) can be represented using predefined coefficients and / or indices that can be signaled within the bitstream for the decoder. In some implementations, exponential division may be usable for blocks with a size larger than or equal to 8x8 luma samples. By dividing rectangular blocks with curves, this subject can achieve greater compression efficiency for specific objects than techniques limited to straight line segment division, such as those involving geometric division.

[0041] Figure 3A illustrates an example of exponential partitioning according to several aspects of this subject that can increase compression efficiency. Rectangular blocks 300 contain pixels (e.g., luma samples). Rectangular blocks 300 may have a size of, for example, 8 × 8 pixels (e.g., luma samples) or larger.

[0042] In Figure 3A, the rectangular block 300 can be divided into two regions (for example, region 0 represented by 310 and region 1 represented by 315) by the curve 305. All luma samples within region 310 may be considered to have the same or similar motion and may be represented by the same motion vector. Similarly, all luma samples within region 315 may be considered to have the same or similar motion and may be represented by the same motion vector. In some implementations, all luma samples to the left or above the curve segment 305 dividing the rectangular block 300 may be considered to belong to region 0 (310). In some implementations, all luma samples to the right or below the curve segment 305 dividing the rectangular block 300 may be considered to belong to region 1 (315). In some implementations, all luma samples through which the curve segment dividing the rectangular block 300 passes belong to region 0 (310). In some implementations, all Luma samples traversed by the curved line segments divided by the rectangular block 300 may be considered to belong to region 1 (315). Other implementations may be possible.

[0043] In some implementations, performing exponential partitioning can reduce the number of possible partitions (compared to, for example, geometric partitioning), which can reduce the computational requirements for evaluating motion estimation to identify appropriate partitions (e.g., to identify the best line segment to partition a block). In some implementations, performing exponential partitioning allows non-rectangular regions (e.g., 310 and 315) to follow the target boundary more closely, thereby reducing prediction errors and residual size and increasing compression efficiency compared to encoding the image using geometric partitioning.

[0044] Exponential partitioning can be represented within a bitstream. In some implementations, exponential partitioning modes may be utilized, and appropriate parameters may be signaled within the bitstream. For example, exponential partitioning can be represented within a bitstream by signaling a predetermined exponential partitioning template. Figure 3B is a series of diagrams illustrating example template partitions according to an embodiment. These regular exponential partitions may specify a set of predetermined orientations. In some implementations, signaling may be performed by including one or more indices of these predefined regular exponential partitions (e.g., templates). For example, Figure 3C illustrates example curves associated with four predefined templates (1, 2, 3, 4) according to an embodiment. The number of template curves may vary in some implementations.

[0045] As another example, exponential partitioning can be represented within a bitstream by signaling predetermined coefficients that indicate the degree of curvature, which may allow for additional exponential functions.

[0046] In some implementations, a predefined template used in exponential partitioning mode may refer to a straight line segment. For example, in Figure 3C, the line segment indexed by coefficient 1 is a straight line, which can be considered a special case of exponential partitioning and can be reduced to an outcome similar to that of geometric partitioning.

[0047] In some implementations, both orientation templates (an example of which is illustrated in Figure 3B) and predefined templates (an example of which is illustrated in Figure 3C) can be used to efficiently signal a large number of potential exponential partitions.

[0048] In some implementations, the start and end indices can be predetermined. For example, Figure 3A illustrates a curved line segment that starts at the lower left corner of a rectangular block 300 and ends at the upper right corner of the rectangular block 300, according to an embodiment. In some implementations, the start and end indices can be explicitly signaled within the bitstream. For example, Figure 3D illustrates another example block showing different start P1 and end P2 indices that divide a rectangular block 300. The start P1 and end P2 indices can be signaled directly or they can point to a set of predetermined values ​​by the indices. In some embodiments, other parameters are possible.

[0049] In some implementations, the exponential partitioning parameters do not need to be included in the bitstream for each block undergoing exponential partitioning (for example, for blocks to which the exponential partitioning mode applies). For example, for a given current block, the exponential partitioning parameters may be inherited from another block (sometimes called a parent block). The parent block may be spatially and / or temporally adjacent. The parent block may be pointed to in the bitstream by an index to a predetermined list, and / or by constructing at least a candidate list, the index to the candidate list being signaled in the bitstream.

[0050] Figure 4 is a system block diagram illustrating an example embodiment of a video encoder 400 capable of performing exponential partitioning, such as with SADCT. The example video encoder 400 receives an input video 405, which can be initially partitioned or divided according to a processing scheme such as a tree-structured macroblock partitioning scheme (e.g., a quad-tree plus binary tree (QTBT)). An example of a tree-structured macroblock partitioning scheme may involve partitioning a picture frame into large block elements called coding tree units (CTUs). In some implementations, each CTU may be further partitioned once or multiple times into a number of subblocks called coding units (CUs). The result of this partitioning may include a group of subgroups that may be called prediction units (PUs). Transformation units (TUs) may also be used. Such partitioning schemes may involve performing exponential partitioning according to some aspects of this subject. For example, Figure 8 illustrates an example of QTBT partitioning of a frame, and Figure 9 illustrates an example of exponential partitioning at the CU level of the QTBT illustrated in Figure 8.

[0051] An example video encoder 400 includes an intra-predictive processor 415, a motion prediction / compensation processor 420 (also called an inter-predictive processor) capable of supporting exponential partitioning, a transform / quantization processor 425, an inverse quantization / inverse transform processor 430, an in-loop filter 435, a decoding picture buffer 440, and an entropy coding processor 445. In some implementations, the motion prediction / compensation processor 420 may perform exponential partitioning, including determining whether the current block can inherit exponential partitioning parameters from another block, and which blocks inherit them from which. In some implementations, the transform / quantization processor 425 may perform SADCT. Bitstream parameters signaling the exponential partitioning mode and inheritances may be input to the entropy coding processor 445 for computation in the output bitstream 450.

[0052] During operation, it can be determined whether each block of frames in the input video 405 is processed via intra-picture prediction or using motion prediction / compensation. The block can be provided to the intra-prediction processor 410 or the motion prediction / compensation processor 420. If the block is to be processed via intra-prediction, the intra-prediction processor 410 may perform processing to output a predictor. If the block is to be processed via motion prediction / compensation, the motion prediction / compensation processor 420 may perform processing, including the use of exponential partitioning, to output a predictor.

[0053] The residual can be formed by subtracting the predictor from the input image. The residual can be received by a transform / quantization processor 425 (which may perform a transform process (e.g., SADCT)) to produce coefficients (which may be quantized). The quantized coefficients and any associated signal transmission information can be provided to an entropy encoding processor 445 for entropy encoding and inclusion in the output bitstream 450. The entropy encoding processor 445 may support encoding of signal transmission information related to exponential partitioning. In addition, the quantized coefficients can be provided to an inverse quantization / inverse transform processor 430 to reconstruct pixels, which can be combined with predictors and processed by an in-loop filter 435. The output of the in-loop filter 435 is stored in a decoded picture buffer 440 for a motion prediction / compensation processor 420 capable of supporting exponential partitioning.

[0054] Figure 5A is a process flow diagram illustrating process 500A, which is an example of encoding video with exponential partitioning in accordance with several aspects of this subject that can reduce encoding complexity while increasing compression efficiency. In 510A, video frames may undergo initial block partitioning using a tree-structured macroblock partitioning scheme, which may include, for example, dividing picture frames into CTUs and CUs. In 520, blocks may be selected for exponential partitioning. Selection may include identifying, according to metric rules, that blocks will be processed according to the exponential partitioning mode.

[0055] In 530A, an exponential partitioning can be determined. A curve (e.g., 305) can be determined to separate pixels contained within a block into two non-rectangular regions (e.g., region 0 and region 1) according to the interframe movement of pixels, such that pixels (e.g., luma samples) within one region (e.g., region 0) have similar movement, and pixels within the other region (e.g., region 1) have similar movement. In 550A, the determined exponential partitioning can be signaled within a bitstream. Signaling within a bitstream may include, for example, indexing to one or more predetermined templates and / or coefficients.

[0056] Figure 5B is a process flow diagram illustrating process 500B, which is an example of encoding video with exponential partitioning in accordance with several aspects of this subject that can reduce encoding complexity while increasing compression efficiency. In 510B, video frames may undergo initial block partitioning using a tree-structured macroblock partitioning scheme, which may include, for example, dividing picture frames into CTUs and CUs. In 520B, blocks may be selected for exponential partitioning. Selection may include identifying, according to metric rules, that blocks will be processed according to the exponential partitioning mode.

[0057] In 530B, an exponential partitioning can be determined. A curve (e.g., 305) can be determined to separate the pixels contained within a block into two non-rectangular regions (e.g., region 0 and region 1) according to the interframe movement of the pixels, such that pixels (e.g., luma samples) inside one region (e.g., region 0) have similar movement, and pixels inside the other region (e.g., region 1) have similar movement.

[0058] In 540B, the partition parameter representation may be determined, which may include determining whether the current block inherits exponential partition parameters from another block, and from which other block the current block inherits them.

[0059] In 550B, a predetermined exponential partition can be signaled within a bitstream. Signaling within a bitstream may include, for example, indexing to a predetermined list of spatially and temporally adjacent blocks.

[0060] Figure 5C is a process flow diagram illustrating process 500C, an example of encoding video with exponential partitioning and SADCT, following several aspects of this subject that can reduce encoding complexity while increasing compression efficiency. In 510C, video frames may undergo initial block partitioning using a tree-structured macroblock partitioning scheme, which may include, for example, dividing picture frames into CTUs and CUs. In 520C, blocks may be selected for exponential partitioning. Selection may include identifying, according to metric rules, that blocks will be processed according to the exponential partitioning mode.

[0061] In 530C, an exponential partitioning can be determined. A curve (e.g., 305) can be determined to separate pixels contained within a block into two non-rectangular regions (e.g., region 0 and region 1) according to the interframe movement of pixels, such that pixels (e.g., luma samples) within one region (e.g., region 0) have similar movement, and pixels within the other region (e.g., region 1) have similar movement. In 540C, an appropriate transformation can be determined for one or more of region 0 and region 1. For example, it can be determined that region 0 or region 1 has a low prediction error. In response to determining that region 0 has a low prediction error, region 0 can be encoded using SADCT. In response to determining that region 1 has a low prediction error, region 1 can be encoded using SADCT. In some implementations, a region may be considered to have a low prediction error if its prediction error is below a predetermined threshold.

[0062] In 550C, a set of predetermined exponential partitioning and transformation options may be signaled within the bitstream. Signaling within the bitstream may include, for example, indices to one or more predetermined templates and / or coefficients. Signaling within the bitstream may include, for example, a signaled SADCT as an option for transformations applied to a full-block DCT for regions with low prediction error (e.g., region 0 or region 1).

[0063] Figure 6 is a system block diagram illustrating a decoder 600 that is an example capable of decoding bitstream 670 using exponential partitioning and / or inverse SADCT. Decoder 600 includes an entropy decoder processor 610, an inverse quantization and inverse transform processor 620, a deblocking filter 630, a frame buffer 640, a motion compensation processor 650, and an intra-prediction processor 660. In some implementations, bitstream 670 includes parameters that signal the exponential partitioning mode. In some implementations, bitstream 670 includes parameters that signal the type of inverse transform to be applied (e.g., inverse blocking DCT or inverse SADCT). The motion compensation processor 650 can reconstruct pixel information using exponential partitioning and / or inverse SADCT as described herein.

[0064] During operation, the bitstream 670 may be received by the decoder 600 and input to the entropy decoder processor 610, which entropy-decodes the bitstream into quantized coefficients. The quantized coefficients may be provided to the inverse quantization and inverse transform processor 620, which may perform inverse quantization and inverse transform using inverse SADCT and according to the signals in the bitstream. The inverse transform may produce a residual signal, which may be added to the output of the motion compensation processor 650 or the intra-prediction processor 660, depending on the processing mode. The outputs of the motion compensation processor 650 and the intra-prediction processor 660 may include block predictions based on previously decoded blocks. The sum of the predictions and residuals may be processed by the deblocking filter 630 and stored in the frame buffer 640. When bitstream 670 signals that the partitioning mode is exponential partitioning for a given block (e.g., CU or PU), motion compensation processor 650 may construct a prediction based on the exponential partitioning scheme described herein, which includes extracting from the bitstream indices of spatially and temporally adjacent blocks to a predetermined list for the current block, and using exponential partitioning parameters for the indicated block to reconstruct the current block.

[0065] Figure 7 is a process flow diagram illustrating process 700, which is an example of decoding a bitstream using exponential partitioning, in which inverse SADCT may be used in some implementations. In 710, blocks (e.g., CTU, CU, PU) are received. Receiving may include extracting and / or parsing the blocks and associated signaling information from the bitstream. In 720, it may be determined whether (e.g., whether) the exponential partitioning mode is enabled for the blocks. If the exponential partitioning mode is not enabled (e.g., whether it is not applicable), the decoder may process the blocks using an alternative partitioning mode, such as geometric partitioning. If the exponential partitioning mode is enabled (e.g., whether it is applicable), in 730, the decoder may extract and / or determine one or more parameters that characterize the exponential partitioning and transformation. These parameters may include, for example, the indices of the exponential coefficients, the values ​​of the exponential coefficients, the indices of the orientation template, and / or the indices of the start and end of the curve (e.g., P1P2). Extraction parameters may include identifying and retrieving parameters from the bitstream (e.g., parsing the bitstream). These parameters may include, for example, transformation parameters that indicate whether to process a block using inverse SADCT. Furthermore, determining one or more parameters that characterize the exponential partitioning may include determining whether exponential partitioning merge is being signaled and using indices contained in the bitstream (which determine the adjacent blocks from which the current block inherits the exponential partitioning parameters). In 740, blocks may be processed according to exponential partitioning (e.g., to produce predictions), including determining associated motion information for each region. In some implementations, blocks may be further processed in 740 using inverse SADCT.

[0066] While several variations are described in detail above, other modifications or additions are possible. For example, in some implementations, exponential partitioning can be applied to symmetric blocks (8x8, 16x8, and similar) as well as various asymmetric blocks (8x8, 16x16, 32x32, 64x64, 128x128, and similar).

[0067] In some implementations, spatial and temporal exponential partitioning predictions can be performed for 16x16 or larger Luma block sizes, such as 64x64 and / or 128x128. In some implementations, a minimum block size of 16x16 may be imposed.

[0068] The partitions can be transmitted within the bitstream based on rate distortion determination within the encoder. Encoding can be based on a combination of regular, predefined partitions (e.g., templates), temporal and spatial predictions of the partitions, and additional offsets. Each region subjected to exponential partitioning can utilize motion-compensated or intra-predictions. The boundaries of the predicted regions can be smoothed before residuals are added. For residual coding, the encoder can choose between a regular rectangular DCT for the entire block and a shape-adaptive DCT for each region.

[0069] In some implementations, a Quad Tree Plus Binary Decision Tree (QTBT) can be implemented. In a QTBT, at the coding tree unit level, the QTBT partition parameters are dynamically derived to adapt to local characteristics without transmitting any overhead. Then, at the coding unit (CU) level, the joint classifier decision tree structure can eliminate unnecessary iterations and control the risk of false predictions. In some implementations, exponential partitioning may be available as an additional partitioning option available per leaf node of the QTBT. In some implementations, exponential partitioning can be used as an additional coding tool at the CU level of the QTBT partition. For example, Figure 8 illustrates an example of a QTBT partition of a frame, and Figure 9 illustrates an example of exponential partitioning at the CU level of the QTBT illustrated in Figure 8.

[0070] In some implementations, the decoder includes an exponential partitioning processor that generates an exponential partition for the current block and provides all partition-related information to the dependent processes. The exponential partitioning processor can directly affect motion compensation because, when a block is exponentially partitioned, it can be executed piecewise. Furthermore, the partitioning processor can provide shape information to the intra-predictive and transform coding processors.

[0071] In some implementations, additional syntax elements may be signaled at different hierarchical levels of the bitstream. To enable exponential partitioning for the entire sequence, an enable flag may be encoded within the sequence parameter set (SPS). Furthermore, a coding tree unit (CTU) flag may be encoded at the CTU level to indicate whether any coding unit (CU) uses exponential partitioning. The CU flag may be encoded to indicate whether the current coding unit utilizes exponential partitioning. Parameters specifying curves on blocks may be encoded. For each region, a flag may be decoded, and the decoded flag specifies whether the current region is inter-predicted or intra-predicted.

[0072] In some implementations, the minimum memory size can be specified.

[0073] Referring to Figure 11, a diagram illustrating block 1100, another example of a block partitioned according to exponential partitioning, is shown. Since exponential partitioning is quite likely to be used for blocks containing objects with curved object boundaries (e.g., coding units), it is quite likely that the partitioning will result in one region with low prediction error and another region with high prediction error. For example, as illustrated in Figure 11, the block is partitioned with a curve according to exponential partitioning. Assuming that the luma samples inside the block represent the ball and the background, the two regions (S0 and S1) contain luma samples corresponding to the background and the ball, respectively. As a result, region S1 has low prediction error because it relates to the ball, while region S0 has high prediction error because it relates to the background. Therefore, some aspects of this subject may involve performing SADCT on the region with low prediction error. Compression efficiency can be improved by performing SADCT instead of full-block DCT on the region with low prediction error.

[0074] In some implementations, parameters for performing the inverse SADCT during decoding can be inferred from the exponential partitioning parameters. For example, the transformation size can be determined from the index of the exponential partitioning template.

[0075] In some implementations, SADCT can be implemented for 64x64 and / or 128x128 sized blocks. In some implementations, SADCT can be signaled as an alternative transformation to full-block DCT for segments with low prediction error.

[0076] Referring to Figure 12, a diagram is shown illustrating the inheritance of exponential partitioning parameters by the current block 1205 from a spatially adjacent block 1210. The curve points to the target boundary 1215 in the image. The current block 1205 and the spatially adjacent block 1210 point to coding unit or prediction unit blocks in a quad-tree plus binary tree (QTBT). As illustrated, the target boundary 1215 generally contains a relatively uniform curvature. Both the adjacent block 1210 and the current block 1205 are partitioned using exponential partitioning. In some implementations of this subject, instead of sending all exponential partitioning parameters (e.g., indices to shape and / or orientation templates, coefficients, start index, end index, and / or similar), the exponential partitioning merge may be signaled in the bitstream along with the index to the adjacent block 1210. During the decoding of the current block, the current block may inherit some or all of the exponential partitioning parameters from the pointed-to adjacent block. Figure 13 illustrates an example of spatially adjacent blocks to the current block. Spatially adjacent blocks may include blocks (e.g., coding units or prediction units) located in the same (e.g., overlapping) locations as A0 (bottom left), A1 (left), B0 (top right), B1 (top), and B2 (top left).

[0077] In addition, adjacent blocks that inherit the exponential partitioning parameters of a given current block can be temporally adjacent. Figure 14 illustrates current block 1405 as an example, with temporally adjacent block 1410 that inherits the exponential partitioning parameters of current block 1405. As illustrated in Figure 14, reference picture 1415 contains adjacent block 1410, adjacent block 1410 has an associated motion vector 1420 that characterizes the movement of adjacent block 1410 from reference picture 1415 to current picture 1425, and current picture 1425 contains current block 1405. Using several implementations of this subject, instead of sending all exponential partitioning parameters (e.g., indices, coefficients, and / or similar to shape and / or orientation templates), the exponential partition merge can be signaled in a bitstream along with the index to adjacent block 1410. During the decoding of the current block 1405, the current block 1405 may inherit some or all of the exponential partitioning parameters from the adjacent block 1410 that it points to.

[0078] In some implementations, the parent block does not have to be an adjacent block; for example, the parent block could be another block in the current frame that has been previously decoded.

[0079] In some implementations, the current block may inherit only a few exponential partitioning parameters from another block. For example, the current block may inherit a first parameter (e.g., shape template) from its first parent block, while additional parameters (e.g., start point, end point, orientation template, and similar) may be included in the bitstream.

[0080] The subject matter described herein offers many technical advantages. For example, some implementations of this subject matter may provide block partitioning that increases compression efficiency. In some implementations, effective visual effects can be achieved by implementing partitioning in a way that more closely follows the target boundary. Similarly, in some implementations, blocking artifacts at the target boundary can be reduced by implementing partitioning in a way that more closely follows the target boundary. In some implementations, compression efficiency can be increased and complexity reduced by implementing SADCT as an alternative transformation to full-block DCT and applying SADCT to regions with exponential partitioning that have low prediction error.

[0081] One or more aspects or features of the subject matter described herein may be realized in digital electronic circuits, integrated circuits, specially designed application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof. These various aspects or features may include implementations in one or more computer programs executable and / or interpretable on a programmable system including a storage system and at least one programmable processor (which may be dedicated or general-purpose) coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device. The programmable system or computing system may include clients and servers. Clients and servers are generally geographically separated from each other and typically interact via a communication network. The relationship between clients and servers arises from computer programs running on each computer that have a client-server relationship with each other.

[0082] These computer programs (which may also be called programs, software, software applications, applications, components, or code) include machine language instructions to a programmable processor and may be implemented in high-level procedural languages, object-oriented programming languages, functional programming languages, logic programming languages, and / or assembly language / machine language. As used herein, the term “machine-readable medium” refers to any computer program product, apparatus, and / or device that includes a machine-readable medium that receives machine language instructions as machine-readable signals, such as magnetic disks, optical disks, memory, and programmable logic devices (PLDs) used to provide machine language instructions and / or data to a programmable processor. The term “machine-readable signal” refers to any signal used to provide machine language instructions and / or data to a programmable processor. A machine-readable medium may store such machine language instructions non-temporarily, such as non-transient solid-state memory or magnetic hard drives or any equivalent storage medium. Machine-readable media may store such machine language instructions in a non-temporary manner, for example, processor cache memory or other random-access memory associated with one or more physical processor cores.

[0083] To provide user interaction, one or more aspects or features of the subject matter described herein may be implemented in a computer having a display device for displaying information to the user, such as a cathode ray tube (CRT), liquid crystal display (LCD), or light-emitting diode (LED) monitor, and a keyboard and a pointing device, such as a mouse or trackball, to which the user can provide input to the computer. Other types of devices may be used in the same way to provide user interaction. For example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, speech, or tactile input. Other possible input devices include touchscreens or other touch-sensitive devices (single or multipoint resistive or capacitive trackpads, speech recognition hardware and software, optical scanners, optical pointers, digital image capture devices and associated interpretation software, and similar).

[0084] In the above description and in the claims, phrases such as “at least one of” or “one or more of” may appear after a parallel list of elements or features. The term “and / or” may also appear in a list of two or more elements or features. Unless implicitly or explicitly negated by the context in which such phrases are used, such phrases are intended to mean either one of the individually listed elements or features, or any of the listed elements or features in combination with any of the other listed elements or features. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are intended to mean “A alone, B alone, or A and B together,” respectively. A similar interpretation is intended for lists containing three or more items. For example, the phrases “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, and / or C” are intended to mean, respectively, “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A, B, and C together.” In addition, the use of the term “based on” above and in the claims is intended to mean “at least partially based,” so as to allow for elements or features that are not enumerated.

[0085] The subject matter described herein may be implemented in systems, apparatus, methods, and / or articles, depending on the desired configuration. The embodiments described above do not represent all embodiments consistent with the subject matter described herein. Rather, the embodiments described above are merely some examples consistent with aspects related to the subject matter described herein. While several variations are described in detail above, other modifications or additions are possible. In particular, further features and / or variations may be provided in addition to the features and / or variations described herein. For example, the embodiments described above may be directed to various combinations and partial combinations of the disclosed features, and / or combinations and partial combinations of some further features disclosed above. In addition, the logic flows depicted in the accompanying figures and / or described herein do not necessarily require a specific order or sequence shown in order to achieve the desired result. Other implementations may be within the scope of the following claims.

Claims

1. A video encoder for encoding a bitstream representing a video signal, wherein the video encoder is A circuit that receives video signals, A circuit that receives the aforementioned video signal and generates an encoded bitstream including the encoded picture. Equipped with, The encoded picture includes an encoded tree unit and signal transmission information, the encoded tree unit comprises a plurality of encoded units, the signal transmission information includes a first index for determining the start point of the nonlinear division boundary within the encoded tree unit and a second index for determining the end point of the nonlinear division boundary within the encoded tree unit, the nonlinear division boundary within the encoded tree unit defines at least a non-rectangular first region and a non-rectangular second region within the encoded tree unit, The bitstream is configured to be decoded by decoding the coding tree unit using the first index and the second index, Decoding the aforementioned coding tree unit is Using the first index, determine the starting point of the boundary of a nonlinear, non-rectangular partition within the coding tree unit, wherein the starting point lies on a first edge of the coding tree unit at a first offset distance from a first corner of the coding tree unit. Using the second index, determine the endpoint of the boundary of the nonlinear, non-rectangular partition within the coding tree unit, wherein the endpoint lies on a second edge of the coding tree unit at a second offset distance from a second corner of the coding tree unit. To generate a first predicted pixel value in the first non-rectangular region, To generate a second predicted pixel value in the non-rectangular second region, The residual pixel value is added to the first predicted pixel value and the second predicted pixel value. A video encoder that performs the operation.

2. The video encoder according to claim 1, wherein the encoded picture includes an encoding unit for intra-predictive pixels, and the decoder includes an intra-predictive processor for decoding blocks of the intra-predictive pixels.

3. A video decoder, wherein the video decoder is A circuit configured to receive an encoded bitstream containing an encoded picture, A circuit configured to perform a method for decoding the encoded bitstream, Equipped with, The encoded picture includes an encoded tree unit and signal transmission information, the encoded tree unit comprises a plurality of encoded units, the signal transmission information includes a first index for determining the start point of a nonlinear division boundary and a second index for determining the end point of the nonlinear division boundary, the nonlinear division boundary defines at least a non-rectangular first region and a non-rectangular second region within the encoded tree unit, The method for decoding the encoded bitstream is: Using the first index, determine the starting point of the boundary of a nonlinear partition within the coding tree unit, wherein the starting point lies on a first edge of the coding tree unit at a first offset distance from a first corner of the coding tree unit. Using the second index, determine the endpoint of the nonlinear partition boundary within the coding tree unit, wherein the endpoint lies on a second edge of the coding tree unit at a second offset distance from a second corner of the coding tree unit. To generate a first predicted pixel value in the first non-rectangular region, To generate a second predicted pixel value in the non-rectangular second region, Smoothing the first predicted pixel value and the second predicted pixel value across the boundary of the nonlinear division, The residual pixel values ​​are added to the smoothed first predicted pixel values ​​and the smoothed second predicted pixel values. A video decoder, including...

4. The video decoder according to claim 3, wherein the encoded picture includes an encoding unit for intra-predictive pixels, and the decoder includes an intra-predictive processor for decoding blocks of intra-predictive pixels.

Images