Inter prediction for exponential partitions

By adopting exponential partitioning mode and advanced motion vector prediction technology in video compression, the problem of low inter prediction efficiency in the prior art is solved, and higher compression efficiency and lower coding complexity are achieved, suitable for processing with bending Objects at the boundary.

CN118972556APending Publication Date: 2024-11-15DOLBY INTERNATIONAL AB
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411176779.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2019-01-28
Filing Date
2020-01-28
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

The existing video compression technology has problems such as low compression efficiency and high encoding complexity in inter-frame prediction, especially when dealing with objects with curved boundaries, the flat line segment partition cannot be effectively partitioned, resulting in large residuals and insufficient compression efficiency. good.

Method used

The exponential partitioning mode is adopted to partition the rectangular blocks into non-rectangular areas through non-linear curves, and the curve is characterized by exponential functions, and the motion vector is determined through advanced motion vector prediction (AMVP) and merge mode to reduce motion compensation prediction errors and improve compression efficiency.

Benefits of technology

By partitioning more closely along the object boundary, the motion compensation prediction error is reduced, the compression efficiency is improved, the encoding complexity is reduced, and the video compression effect is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118972556A_ABST
    Figure CN118972556A_ABST
Patent Text Reader

Abstract

A decoder includes circuitry configured to: receive a bitstream; partitioning the current block into a first region and a second region via an exponential partitioning mode; determining a motion vector associated with the first region or the second region, the determining including constructing a candidate list; and decoding the current block using the determined motion vector. Related apparatus, systems, techniques, and articles of manufacture are also described.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Divisional application

[0002] This application is a divisional application of the Chinese prior invention patent application with application number 2020800258021. The name of the invention of the prior invention patent application is “Inter-frame prediction of index partitioning” and the application date is January 28, 2020.

[0003] CROSS-REFERENCE TO RELATED APPLICATIONS

[0004] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 797,816, filed on January 28, 2019, entitled “INTERPREDICTION IN EXPONENTIAL PARTITIONING,” which is incorporated herein by reference in its entirety. Technical Field

[0005] The present invention relates generally to the field of video compression. In particular, the present invention relates to exponential partitioned inter-frame prediction. Background Art

[0006] A video codec may include electronic circuitry or software that compresses or decompresses digital video. It can convert uncompressed video into a compressed format, and vice versa. In the context of video compression, a device that compresses video (and / or performs some of these functions) may generally be referred to as an encoder, while a device that decompresses video (and / or performs some of these functions) may be referred to as a decoder.

[0007] The format of the compressed data may conform to standard video compression specifications. Compression may be lossy in that the compressed video lacks some information present in the original video. The consequences of this may include that the decompressed video is of lower quality than the original uncompressed video because there is insufficient information to accurately reconstruct the original video.

[0008] There can be a complex relationship between video quality, the amount of data used to represent the video (e.g., determined by the bit rate), the complexity of the encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end delay (e.g., latency), and so on. Summary of the invention

[0009] In one aspect, a decoder includes circuitry configured to: receive a bitstream; partition a current block into a first region and a second region by an exponential partitioning pattern; determine a motion vector associated with a region of the first region or the second region, wherein the determination includes constructing a candidate list; and decode the current block using the determined motion vector.

[0010] In another aspect, a method includes: receiving a bitstream by a decoder; partitioning a current block into a first region and a second region by the decoder through an exponential partitioning mode; determining, by the decoder, a motion vector associated with a region of the first region or the second region, the determining including constructing a candidate list; and decoding the current block by the decoder using the determined motion vector.

[0011] The details of one or more variations of the subject matter described herein are set forth in the following drawings and detailed description.Other features and advantages of the subject matter described herein will be apparent from the following description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] For the purpose of illustrating the invention, the accompanying drawings show one or more embodiments of the invention. However, it should be understood that the invention is not limited to the precise construction and instrumentality shown in the accompanying drawings, in which:

[0013] Figure 1 A schematic diagram showing an example of pixel block partitioning;

[0014] Figure 2 A schematic diagram showing an example of geometric partitioning;

[0015] Figure 3 shows a picture including an apple that is not effectively partitioned by flat straight line segments;

[0016] Figure 4A is a schematic diagram illustrating an example of index partitioning that can increase compression efficiency according to some aspects of the current subject matter;

[0017] Figure 4B is a series of schematic diagrams illustrating exemplary template index partitioning;

[0018] Figure 4C An exemplary curve associated with four predetermined coefficients that may define an exemplary exponential function is shown;

[0019] Figure 4D Another exemplary block is shown showing different starting P1 and ending P2 indexes for partitioning a rectangular block;

[0020] Figure 5 is a schematic diagram showing an example of the position of potential motion vector candidates relative to a current block instance partitioned according to an exponential partitioning method;

[0021] Figure 6 Shown with annotations Figure 5 , thereby showing the brightness position including the upper left brightness position of the first area and the upper right brightness position of the second area; Figure 7A schematic diagram showing the positions of potential motion vector candidates relative to a current block instance partitioned according to an exponential partitioning method;

[0022] Figure 8 Shown with annotations Figure 7 , thereby showing the brightness position including the lower left brightness position of the second region and the upper right brightness position of the second region;

[0023] Fig. 9 To illustrate a system block diagram of an example video encoder capable of encoding video using inter-frame prediction and index partitioning;

[0024] Fig.10 An example of frame QTBT partitioning is shown;

[0025] Fig.11 Shows Figure 8 An example of index partitioning at the CU level of the QTBT is shown;

[0026] Fig.12 is a process flow diagram illustrating an example process for encoding a video through index partitioning and inter-frame prediction according to some aspects of the current subject matter, which process example can reduce encoding complexity while increasing compression efficiency.

[0027] Fig.13 is a system block diagram illustrating an example of a decoder capable of decoding a bitstream using exponential partitioning and inter-frame prediction according to some aspects of the current subject matter;

[0028] Fig.14 is a process flow diagram illustrating an example process for decoding a bitstream using index partitioning and inter-frame prediction according to some aspects of the current subject matter; and

[0029] Fig.15 is a block diagram of a computing system that can be used to implement any one or more of the methods disclosed herein, and any one or more portions thereof.

[0030] The accompanying drawings are not necessarily drawn to scale and may be illustrated by dashed lines, schematic representations, and fragmentary views. In some cases, details that are not necessary for understanding the embodiments or that make other details difficult to perceive may be omitted. Similar reference numerals in the various figures indicate similar elements. DETAILED DESCRIPTION

[0031] Some embodiments of the current subject matter include performing inter prediction with non-rectangular regions that have been partitioned with a curve; the curve may or may not be a straight line. Performing inter prediction with non-rectangular blocks that have been partitioned with a curve may allow the partitioning to more closely follow object boundaries, resulting in lower motion compensated prediction errors, smaller residuals, and thus improved compression efficiency. During inter prediction, motion compensation may be performed using motion vectors predicted for blocks (e.g., coding units, prediction units, etc.) determined according to an exponential partitioning pattern. Motion vectors may be predicted using advanced motion vector prediction (AMVP) and / or via merge mode, where a motion vector is selected from a list of motion vector candidates without encoding motion vector differences.

[0032] In exponential partitioning, a rectangular block may be partitioned into non-rectangular regions by a curve; the non-rectangular regions may include straight line segments in the case of geometric partitioning, or may be non-linear curves in the more general case. Partitioning a block using a non-linear curve may allow the partitioning to follow object boundaries more closely, resulting in lower motion compensated prediction errors, smaller residuals, and thus improved compression efficiency. In some embodiments, the curve may be characterized by an exponential function. The curve (e.g., an exponential function) may be determined using predetermined coefficients that may be signaled in the bitstream for use by a decoder. In some embodiments, exponential partitioning may be available for greater than or equal to 8×8 luma samples. By partitioning a rectangular block with a curve, the present subject matter may achieve greater compression efficiency for certain objects compared to techniques limited to partitioning with straight line segments (such as geometric partitioning).

[0033] Motion compensation may include a method of predicting a video frame or portion thereof that takes into account previous and / or future frames to account for the motion of the camera and / or objects in the video. Motion compensation may be used for encoding and decoding of video data for video compression, such as for encoding and decoding using the Moving Picture Experts Group (MPEG)-2 (also known as Advanced Video Coding (AVC)) standard. Motion compensation may describe an image with respect to a transformation from a reference image to a current image. The reference image may be previous or future when compared to the current image. Compression efficiency may be improved when pictures may be accurately synthesized from previously transmitted and / or stored pictures.

[0034] Block partitioning can be a method of finding areas of similar motion in video coding. Some form of block partitioning can be found in video codec standards, including MPEG-2, H.264 (also known as AVC or MPEG-4 Part 10), and H.265 (also known as High Efficiency Video Coding (HEVC)). In an example of a block partitioning approach, non-overlapping blocks of a video frame can be partitioned into rectangular sub-blocks to find block partitions that contain pixels with similar motion. This approach works well when all pixels of a block partition have similar motion. The motion of pixels in a block can be determined with respect to previously encoded frames.

[0035] Figure 1 Schematic diagram showing an example of block partitioning of pixels. An initial rectangular image or block 100 may itself be a sub-block (e.g., a node within a coding tree) that may be partitioned into rectangular sub-blocks. For example, at 110, block 100 is partitioned into two rectangular sub-blocks 110a and 110b. Sub-blocks 110a and 110b may then be processed separately. As another example, at 120, block 100 is partitioned into four rectangular sub-blocks 120a, 120b, 120c, and 120d. The sub-blocks themselves may be further divided until it is determined that the pixels within the sub-block share the same motion, a minimum block size is reached, or another criterion. When the pixels in a sub-block have similar motion, a motion vector may describe the motion of all pixels in the area.

[0036] Still refer to Figure 1 , some approaches to video encoding may include geometric partitioning, which may be exponential partitioning where rectangular blocks (e.g., Figure 1 As shown) it is also divided into two regions (which may be non-rectangular) by a straight line segment. For example, Figure 2 205 is a schematic diagram illustrating an example of geometric partitioning. An example rectangular block 200 (which may have a width of M pixels and a height of N pixels, represented as M×N pixels) may be divided into two regions (region 0 and region 1) along a straight line segment P1P2 205. When pixels in region 0 have similar motion, a motion vector may describe the motion of all pixels in the region. The motion vector may be used to compress region 0. Similarly, when pixels in region 1 have similar motion, the associated motion vector may describe the motion of pixels in region 1. Such geometric partitioning may be signaled to a receiver (e.g., a decoder) by coded positions P1 and P2 (or representations of positions P1 and P2) in a video bitstream.

[0037] Continue to refer Figure 2, when encoding video data using geometric partitioning, straight line segments 205 (or more specifically, points P1 and P2) may be determined. However, straight line segments may not be able to partition blocks in a manner that reflects object boundaries. Therefore, partitioning with straight line segments may not be able to partition blocks in an efficient manner (e.g., such that any resulting residuals are small). This may be true in the case where a block may contain pixels (e.g., luminance samples) that represent objects or boundaries with curved (e.g., non-straight) boundaries. For example, Figure 3 An exemplary embodiment of a picture containing an apple is shown, which may not be effectively partitioned by straight line segments; the picture of the apple shown includes some rectangular blocks, which indicate some parts of the picture. In the case where the picture is partitioned using straight line segments according to geometric partitioning, the partitions may not closely follow the boundaries of the object (e.g., apple), such as Figure 3 Shown bent.

[0038] Figure 4A Schematic diagram illustrating a non-limiting example of exponential partitioning using a non-linear curve, which is defined for purposes of this disclosure as a non-linear curve in accordance with some aspects of the current subject matter, which can increase compression efficiency. Rectangular block 400 can include pixels (e.g., luma samples). As a non-limiting example provided for illustration purposes only, rectangular block 400 can have a size of 8×8 pixels (e.g., luma samples) or larger.

[0039] exist Figure 4A In FIG. 4 , the rectangular block 400 may be partitioned into two or more regions, for exemplary purposes. Figure 4A405. In the figure, region 0 and region 1 are shown as region 0 and region 1, respectively denoted as 410 and 415, separated by curved line 405. All luma samples in each region so defined may be considered to be similar in motion and therefore representable using the same motion vector. For exemplary purposes, all luma samples within region 410 may be considered to have the same or similar motion and may be represented by the same motion vector. Similarly, all luma samples within region 415 may be considered to have the same or similar motion and may be represented by the same motion vector. As described more fully below, the corresponding motion vectors may be determined according to an AMVP mode or a merge mode. In some embodiments and for the purpose of discussion, all luma samples to the left or above the curved line segment 405 that divides the rectangular block 400 may be considered to belong to region 0 (410). In some embodiments, all luma samples on or above the curved line segment 405 that divides the rectangular block 400 may be considered to belong to region 1 (415). In some embodiments, all brightness samples passed through by the curved line segments that divide the rectangular block 400 (i.e., brightness samples on the line segments and / or intersecting the line segments) belong to region 0 (410). In some embodiments, all brightness samples passed through by the curved line segments that divide the rectangular block 400 may be considered to belong to region 1 (415). Other embodiments may be possible, as will be understood by those skilled in the art after reviewing the entire text of this disclosure.

[0040] Still refer to Figure 4A , exponential partitioning may be represented in the bitstream. In some embodiments, an exponential partitioning mode may be utilized and appropriate parameter signals may be issued in the bitstream. For example, exponential partitioning may be represented in the bitstream by signaling a predetermined exponential partitioning template. Figure 4B 4 is a series of schematic diagrams illustrating non-limiting examples of template partitions 420-435. In some embodiments, signaling may be performed by including an index to one or more of these predetermined rule (e.g., template) index partitions. These rule index partitions may specify a set of predetermined orientations. For example, Figure 4C Non-limiting exemplary curves associated with four predetermined targets (1, 2, 3, 4) are shown. In some embodiments, multiple target curvatures may vary.

[0041] Continue to refer Figure 4B And as another non-limiting example, exponential partitioning may be standardized in the bitstream by emitting predetermined coefficients, such as coefficients of an exponential function, that indicate a degree of curvature that may allow for additional exponential functions.

[0042] In some embodiments and still with reference to Figure 4B , the predetermined template of the plurality of templates 420-435 used in the exponential partition mode may indicate a flat line segment. Figure 4C, the line segment indexed by the coefficient 1 is a straight line, which can be regarded as a specific case of an exponential partition, which is a geometric partition, as described above.

[0043] In some embodiments, an orientation template (such as Figure 4B as shown) and predetermined templates (such as Figure 4C As shown) both can be used to effectively signal any exponential pattern of a large number of potential exponential partitions; for example, the template may provide a curved portion 440, including but not limited to a line segment 1 and / or one or more nonlinear curves 2-4, the curved portion 440 may include an exponential curve, any of which may be selected by an encoder, a user and / or an automatic process to form a partition as described in the present disclosure.

[0044] In some embodiments and herein referred to Figure 4A , the start and end indexes may be predetermined. For example, 4A shows an exemplary curved line segment starting at the lower left corner of rectangular block 400 and ending at the upper right corner of rectangular block 400. Such predetermined start and end indexes may be stored in a memory of a decoder. Alternatively or in addition, in some embodiments, the start and end index signals may be explicitly sent out in the bitstream. For example, 4D shows another example block, which shows different start P1 and end P2 indexes for partitioning rectangular block 400. The start P1 and end P2 index signals may be sent directly, or may be indicated as a set of predetermined values ​​by indexing. Other parameters are possible, as will be understood by those skilled in the art after reviewing the full text of this disclosure.

[0045] Still refer to Figure 4A , inter-frame prediction can be performed using regions that have been index partitioned. Motion vectors for motion compensation can be generated by using AMVP or merge mode. In AMVP, motion vector prediction can be performed by issuing an index to a motion vector candidate list, and the motion vector difference (e.g., residual) can be encoded and included in the bitstream. In merge mode, the motion vector is selected from the motion vector candidate list without encoding the motion vector difference, allowing the current block to adopt the motion information of another previously decoded block. In both AMVP and merge mode, the candidate list can be constructed by both the encoder and the decoder, and the index in the candidate list can be signaled in the bitstream.

[0046] Figure 51 is a schematic diagram showing a non-limiting example of the location of potential spatial motion vector candidates relative to an exemplary current block 1100 partitioned according to the exponential partitioning method. The potential spatial motion vector candidates can be considered for constructing a motion vector candidate list during AMVP mode or merge mode. As a non-limiting example, the current block 1100 can be partitioned into two regions, region S0 and region S1, by a curve between points P0 and P1. Each of region S0 and region S1 can be unidirectionally or bidirectionally predicted. The spatial candidates for the first region (region S0) are shown for exemplary purposes in Figure 5 and may include but is not limited to a lower left candidate A0, a left side candidate A1, an upper left candidate B2, an upper side candidate B1, and an upper right candidate B0.

[0047] like Figure 5 As shown and in some embodiments, each position (A0, A1, B2, B1, and B0) may represent a block at the corresponding position. For example, but not limitation, the upper left candidate B2 may represent a block that resides at a position immediately to the left of and immediately above the region S0; for example, if the luminance position of the upper left corner of S0 is (0,0), then the upper left candidate B2 may reside at position (-1,-1). The lower left candidate A0 may be located immediately to the left and below P1; for example and not limitation, if the luminance position of P1 is (P1x, P1y), then the lower left candidate A0 may reside at position (Plx-1, Ply+1). The left candidate A1 may be located immediately to the left of P1; for example, the left candidate A1 may reside at position (Plx-1, Ply). The upper candidate B1 may be positioned immediately above P0; for example, if the luminance position of P0 is (P0x, P0y), then the upper candidate B1 is at (P0x, P0y-1). The upper right candidate B0 may be positioned immediately above the upper and rightmost luminance position in the second region S1; for example, if the upper right corner of S1 is at (S0_width+Sl_width-1,0), then the upper right candidate B0 may be at (S0_width+Sl_width-1,-1), where M=S0_width+Sl_width. Other positions are possible, as will be understood by those skilled in the art after reviewing the full text of this disclosure. Figure 6 Shown with annotations Figure 5 , thereby showing the brightness position including the upper left brightness position of the first region S0 and the upper right brightness position of the second region S1.

[0048] In some embodiments and still with reference to Figure 6, when constructing the candidate list for region S0, some potential candidates may be automatically marked as unavailable and may be removed from the list; because in the presence of index partitioning, such partitioning may be performed to partition regions (or objects) within the frame with different motion information. Therefore, it can be inferred that the blocks associated with these candidates also represent another object with different motion, and therefore these candidates may be automatically marked as unavailable (e.g., not further considered, removed from the list, etc.). As a non-limiting example and as described above Figure 5 As shown, with respect to region S0, the lower left candidate A0 may be automatically marked as unavailable because it is possible that region S0 does not share motion information with the block located at the lower left candidate A0. Similarly, with respect to region S0, the upper right candidate B0 may be automatically marked as unavailable because it is possible that region S0 does not share motion information with the block located at the upper right candidate B0.

[0049] Figure 7 1 is a schematic diagram showing a non-limiting example of the location of potential motion vector candidates relative to an exemplary current block 1400 partitioned according to the exponential partitioning method. The potential motion vector candidates may be considered for building a candidate list during AMVP mode or merge mode. The current block 1400 may have been partitioned into two regions, region S0 and region S1, by a curve between points P0 and P1. Each of region S0 and region S1 may be unidirectionally or bidirectionally predicted. The candidates for the second region (region S1) are shown in Figure 7 and may include a lower left candidate A0, a left side candidate A1, an upper left candidate B2, an upper side candidate B1, and an upper right candidate B0.

[0050] like Figure 7As shown, each position (A0, A1, B2, B1, and B0) may represent a block at a corresponding position. For example, the upper left candidate B2 may be a block residing at a luminance position immediately to the left of and immediately above region S1; for example, if the luminance position of the upper left corner of S1 is adjacent to P0 of the luminance position coordinate (P0x+1, P0y), then the upper left candidate B2 may reside at position (P0x, P0y-1), and the lower left candidate A0 may be positioned immediately to the left of and below the lower left luminance position of S1; for example, if the lower left luminance position of S1 is the luminance position (0, S0_height+Sl_height-1), then the lower left candidate A0 may reside at position (-1, S0_height+Sl_height), where N=S0_height+S1_height. The left candidate A1 may be positioned immediately to the left of the lower left luminance position of S1 (e.g., the lower left corner of S1); for example, if the lower left luminance position of S1 is luminance position (0, S0_height+Sl_height-1), then the left candidate A1 may reside at luminance position (-1, S0_height+1_height-1). The upper candidate B1 may be positioned immediately above the upper right luminance position of S1; for example, if the upper right luminance position of S1 is (S0_width+Sl_width-1,0), then the upper candidate B1 may be located at (S0_width+Sl_width-1,-1), where M=S0_width+Sl_width. The upper right candidate item B0 can be positioned immediately above and to the right of the upper right brightness position of the second region S1; for example, if the upper right brightness position of S1 (e.g., the upper right corner) is at (S0_width+Sl_width-1,0), then the upper right candidate item B0 can be located at (S0_width+Sl_width-1,-1). Figure 8 Shown with annotations Figure 7 , thereby showing the brightness positions including the lower left brightness position of the second region S1 and the upper right brightness position of the second region S1.

[0051] In some embodiments and still with reference to Figure 8 , when constructing the candidate list for region S1, some potential candidates may be automatically marked as unavailable and may be removed from the list; because in the presence of index partitioning, such partitioning may be performed to partition regions (or objects) within the frame with different motion information. Therefore, it can be inferred that the blocks associated with these candidates also represent another object with different motion, and therefore these candidates may be automatically marked as unavailable (e.g., not further considered, removed from the list, etc.). Figure 7In the example of , with respect to region S1 , the top-left candidate B2 may be automatically marked as unavailable because it is possible that region S1 does not share motion information with the block located at the top-left candidate B2 .

[0052] Fig. 9 A system block diagram is shown for illustrating a non-limiting example of a video encoder 900 that is capable of encoding video using inter-frame prediction and exponential partitioning. The example video encoder 900 receives an input video 905, which may be initially segmented or partitioned according to a processing scheme, such as a tree-structured coding block partitioning scheme (e.g., a quadtree plus binary tree (QTBT)). An example of a tree-structured coding block partitioning scheme may include partitioning an image frame into large block elements called coding tree units (CTUs). In some embodiments, each CTU may also be partitioned once or multiple times into multiple sub-blocks called coding units (CUs). The final result of this partitioning may include a group of sub-blocks that may be called prediction units (PUs). A transform tree (TU) may also be utilized. Such a partitioning scheme may include performing exponential partitioning according to some aspects of the current subject matter. Figure 8 shows an example of a QTBT partition of a frame, and Fig.11 Shown in Figure 8 An example of index partitioning at the CU level of a QTBT is shown.

[0053] Still refer to Fig. 9 , the example video encoder 900 may include an intra prediction processor 915, a motion estimation / compensation processor 920 (also referred to as an inter prediction processor), a transform / quantization processor 925, an inverse quantization / inverse transform processor 930, an in-loop filter 935, a decoded picture buffer 940, and an entropy coding processor 945; the motion estimation / compensation processor 920 is capable of supporting exponential partitioning including AMVP and merge modes. In some embodiments, the motion estimation / compensation processor 920 may perform inter prediction using exponential partitioning and including the use of AMVP mode and merge mode. Bitstream parameters signaling the exponential partitioning mode, AMVP mode, and merge mode may be input to the entropy coding processor 945 for inclusion in the output bitstream 950.

[0054] In operation, and continue to refer to Fig. 9 , for each block of a frame of input video 905, whether or not the block is to be processed via intra-frame image prediction or using motion estimation / compensation. The block may be provided to an intra-frame prediction processor 910 or a motion estimation / compensation processor 920. If the block is to be processed via intra-frame prediction, the intra-frame prediction processor 910 may perform processing to output a predictor. If the block is to be processed via motion estimation / compensation, the motion estimation / compensation processor 920 may perform processing to output a predictor, the processing including the use of index partitioning with AMVP mode and merge mode.

[0055] Still refer to Fig. 9 , the residual may be formed by subtracting the predictor from the input video. The residual may be received by a transform / quantization processor 925, which may perform a transform process (e.g., a discrete cosine transform (DCT)) to produce coefficients, which may be quantized. The quantized coefficients and any related signaling information may be provided to an entropy coding processor 945 for entropy coding and included in an output bitstream 950. The entropy coding processor 945 may support encoding of signaling information related to exponential partitioning mode, AMVP mode, and merge mode. In addition, the quantized coefficients may be provided to an inverse quantization / inverse transform processor 930, which may reproduce the pixels, may be combined with the predictor and processed by an in-loop filter 935, the output of which may be stored in a decoded picture buffer 940 for use by a motion estimation / compensation processor 920, which may be capable of supporting exponential partitioning mode, AMVP mode, and merge mode.

[0056] Fig.12 1 is a process flow diagram illustrating an example process 1200 for encoding a video with exponential partitioning and inter-frame prediction according to some aspects of the current subject matter, which can reduce encoding complexity while increasing compression efficiency. At step 1210, a video frame can undergo initial block segmentation, such as using a tree-structured coding block partitioning scheme, which can include partitioning the image frame into CTUs and CUs. At step 1220, blocks can be selected for exponential partitioning; the exponential partitioning can include geometric partitioning or can include exponential partitioning using nonlinear curves. The selection can include identifying blocks to be processed according to the exponential partitioning mode based on a metric rule.

[0057] At step 1230 and continuing with reference Fig.12 , an index partition may be determined. The curved line and / or line segment (e.g., 405) and / or the straight line and / or line segment may determine that the pixels included in the block are separated into two non-rectangular regions (e.g., region 0 and region 1) according to their inter-frame motion, so that pixels (e.g., luminance samples) within one of the regions (e.g., region 0) have similar motion, and pixels within the other region (e.g., region 1) have similar motion.

[0058] At step 1240 and continuing with reference Fig.12, the motion information of each non-rectangular area can be determined and processed using AMVP mode or merge mode. When processing a region using AMVP mode, a candidate list can be constructed by considering both spatial candidates and temporal candidates (including the spatial candidates described above) and including marking some candidates as unavailable. The motion vector can be selected from a list of motion vector candidates as motion vector predictions, and the motion vector difference (e.g., residual) can be calculated. The index of the candidate list can be determined. In merge mode, a candidate list can be constructed by considering both spatial candidates and temporal candidates (including the spatial candidates described above) and including marking some candidates as unavailable. The motion vector can be selected from a list of motion vector candidates for the region to adopt the motion information of another block. The index of the candidate list can be determined.

[0059] At step 1250 and continuing with reference Fig.12 , the determined index partition and motion information may be signaled in the bitstream. Signaling the index partition in the bitstream may include, for example, an index to one or more predetermined templates and / or coefficients. The signaling of motion information when processing a region using AMVP may include a motion vector difference (e.g., a residual) and an index to a motion vector candidate list in the bitstream. The signaling of motion information when processing a region using merge mode may include an index to a motion vector candidate list in the bitstream.

[0060] Fig.13 1 is a system block diagram, showing an example decoder 600 capable of decoding a bitstream 1370 using exponential partitioning and inter-frame prediction according to some aspects of the current subject matter. The decoder 600 may include an entropy decoder processor 1310, an inverse quantization and inverse transform processor 1320, a deblocking filter 1330, a frame buffer 1340, a motion compensation processor 1350, and an intra-frame prediction processor 1360. In some embodiments, the bitstream 1370 includes parameters that signal an exponential partitioning mode, an AMVP mode, and a merge mode. Using exponential partitioning and inter-frame prediction as described in the present disclosure, the motion compensation processor 1350 can reconstruct pixel information.

[0061] In operation, the bitstream 1370 may be received by the decoder 600 and input to the entropy decoder processor 1310, which decodes the bitstream into quantized coefficients. The quantized coefficients may be provided to the inverse quantization and inverse transform processor 1320, which may perform inverse quantization and inverse transform to form a residual signal, which may be added to the output of the motion compensation processor 1350 or the intra-frame prediction processor 1360 depending on the processing mode. The output of the motion compensation processor 1350 and the intra-frame prediction processor 1360 may include a block prediction value based on a previously decoded block. The sum of the prediction value and the residual may be processed by the deblocking filter 1330 and stored in the frame buffer 1350. With respect to a given block (e.g., CU or PU), when the bitstream 1370 signals that the partition mode is exponential partitioning, the motion compensation processor 1350 may construct a prediction value based on the exponential partitioning method described herein and using AMVP or merge mode as described herein.

[0062] Fig.14 To illustrate a process flow diagram of an example process 1400, the example process 1400 decodes a bitstream using index partitioning and using inter-frame prediction according to some aspects of the current subject matter. At step 1410, a bitstream is received. The receiving may include extracting and / or parsing the bitstream and related signaling information of the bitstream, thereby including parsing the current block and related signaling information of the bitstream.

[0063] At step 1420 and still referring to Fig.14 , the current block may be partitioned into a first region and a second region via an exponential partitioning mode. The partitioning may include determining whether exponential partitioning is enabled for the block (e.g., correct), thereby indicating that the exponential partitioning utilizes the use of a nonlinear curve. If the exponential partitioning mode is not enabled (e.g., wrong), the decoder may process the block using an alternative exponential partitioning mode (such as geometric partitioning); parameters of the geometric partitioning (including but not limited to segment endpoints, coefficients, etc.) may be received from the bitstream, as described above. If the exponential partitioning mode is enabled (e.g., true), the decoder may extract or determine one or more parameters characterizing the exponential partitioning. These parameters may include, for example, exponential coefficient indices, exponential coefficient values, orientation template indices, and / or start and end indices of a curved line (e.g., P1P2). Extracting or determining may include identifying and retrieving parameters of the bitstream (e.g., parsing the bitstream).

[0064] At step 1430 and continuing with reference Fig.14, a motion vector associated with a region of the first region or the second region may be determined. Determining the motion vector may include determining whether motion information for the region is determined using AMVP mode or merge mode. When processing the region using AMVP mode, a candidate list may be constructed by considering both spatial candidates and temporal candidates (including the spatial candidates described above) and including marking some candidates as unavailable. The motion vector may be selected from the list of motion vector candidates because motion vector predictions and motion vector differences (e.g., residuals) may be calculated. In merge mode, the determination may include constructing a candidate list of spatial candidates and temporal candidates for each region. With respect to each region, the spatial candidates may be as described above with respect to Figures 5 to 8 The spatial candidates described. Constructing the candidate list may include automatically marking the candidate as unavailable and removing the unavailable candidate from the candidate list. The index of the constructed candidate list may be parsed from the bitstream and used to select the final candidate from the candidate list. The motion information of the current region may be determined to be the same as the motion information of the final candidate (e.g., the motion vector of the region may be adopted from the final candidate).

[0065] At step 1440 and still referring to Fig.14 , the current block can be decoded using the determined motion vector.

[0066] Still refer to Fig.14 Although a few variations have been described in detail above, other modifications or additions are possible. For example, in some embodiments, index partitioning may be applicable to symmetric blocks (8×8, 16×16, 32×32, 64×64, 128×128, etc.) as well as various asymmetric blocks (8×4, 16×8, etc.).

[0067] Continue to refer Fig.14 , the partitions can be signaled in the bitstream based on rate-distortion decisions in the encoder. The encoding can be based on a combination of regular predetermined partitions (e.g., templates), temporal and spatial predictions of the partitions, and additional offsets. Each exponential partition region can utilize motion compensated prediction or intra-frame prediction. The boundaries of the prediction regions can be smoothed before adding the residual. For residual coding, the encoder can choose between conventional rectangular DCT for the entire block and shape adaptive DCT for each region.

[0068] Still refer to Fig.14, in some embodiments, a quadtree plus binary decision tree (QTBT) may be implemented. In QTBT, at the coding tree unit level, the partitioning parameters of the QTBT may be dynamically derived to adapt to local characteristics without transmitting any overhead. Subsequently, at the coding unit (CU) level, the joint classifier decision tree structure may eliminate unnecessary iterations and control the risk of incorrect predictions. In some embodiments, exponential partitioning may be available as an additional partitioning option available at each leaf node of the QTBT. In some embodiments, exponential partitioning is available as an additional coding tool at the CU level for QTBT partitioning. For example, 8 shows an example of QTBT partitioning of a frame; and Fig.11 Shown Figure 8 An example of CU-level index partitioning of a QTBT is shown.

[0069] In some embodiments, the decoder includes an exponential partitioning processor that can generate exponential partitions of the current block and provide all partition-related information to subordinate processes. The exponential partitioning processor can directly affect motion compensation because it can be performed piece by piece in the case of exponential partitioning of the block. In addition, the partitioning processor can provide shape information to the intra prediction processor and the transform coding processor.

[0070] In some embodiments, additional syntax elements may be signaled at different hierarchical levels of the bitstream. To enable exponential partitioning for the entire sequence, an enable flag may be encoded in the sequence parameter set (SPS). Additionally, a CTU flag may be encoded at the coding tree unit (CTU) level to indicate whether any coding unit (CU) utilizes exponential partitioning. A CU flag may be encoded to indicate whether the current coding unit utilizes exponential partitioning. Parameters that specify warping lines on blocks may be encoded. For each region, a flag may be decoded that specifies whether the current region is inter- or intra-predicted.

[0071] In some implementations, a minimum region size may be specified.

[0072] The subject matter described herein provides a number of technical advantages. For example, some implementations of the current subject matter can provide for partitioning of blocks that increases compression efficiency. In some implementations, by implementing the partitioning in a manner that more closely follows object boundaries, effective visual effects can be achieved. Similarly, in some implementations, by implementing the partitioning in a manner that more closely follows object boundaries, blocking artifacts at object boundaries can be reduced.

[0073] It is noted that any one or more of the aspects and embodiments described herein may be conveniently implemented using digital electronic circuitry, integrated circuits, specially designed application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof, as implemented and / or implemented in one or more machines (e.g., one or more computing devices (the one or more computing devices are used as user computing devices for electronic files), one or more server devices (such as file servers), etc.), which are programmed according to the teachings of this specification, as will be apparent to those skilled in the computer arts. These various aspects or features may include implementation in one or more computer programs and / or software, which are executable and / or interpretable on a programmable system including at least one programmable processor, which is connectable for special and general purposes to receive data and instructions from a storage system, at least one input device, and at least one output device and transmit data and instructions to it. Appropriate software coding can be readily prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software arts. The aspects and embodiments discussed above that employ software and / or software modules may also include appropriate hardware for facilitating the implementation of machine-executable instructions of the software and / or software modules.

[0074] Such software may be a computer program product using a machine-readable storage medium. A machine-readable storage medium may be any medium capable of storing and / or encoding a sequence of instructions executed by a machine (e.g., a computing device) and causing the machine to execute any of the methods and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state storage devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. As used herein, machine-readable media is intended to include a single medium and a set of physically independent media, such as, for example, a set of a combination of a compressed disk or one or more hard disk drives and a computer memory. As used herein, a machine-readable storage medium does not include a temporary form of signal transmission.

[0075] Such software may also include information (e.g., data) carried as a data signal on a data carrier (such as a carrier wave). For example, machine executable information may be included as a data bearing signal embodied in a data carrier, wherein the signal encodes a sequence of instructions or a portion thereof executed by a machine (e.g., a computing device), and any associated information (e.g., data structures and data); the associated information causes the machine to perform any of the methods and / or embodiments described herein.

[0076] Examples of computing devices include, but are not limited to, electronic book reading devices, computer workstations, terminal computers, server computers, handheld devices (e.g., tablet computers, smart phones, etc.), network appliances, network routers, network switches, network bridges, any machine capable of executing a sequence of instructions that specifies actions to be taken by the machine, and any combination thereof. In one example, the computing device may include and / or may be included in a kiosk.

[0077] Fig.15 A schematic representation of one embodiment of a computing device in the exemplary form of a computer system 1500 is shown, within which a set of instructions may be executed, the set of instructions being used to cause a control system to perform any one or more of the aspects and / or methods of the present disclosure. It is also contemplated that multiple computing devices may be used to implement a specially configured set of instructions for causing one or more of the devices to perform any one or more of the aspects and / or methods of the present disclosure. The computer system 1500 includes a processor 1504 and a memory 1508, which communicate with each other and with other components via a bus 1512. The bus 1512 may include any of several types of bus structures, including but not limited to a memory bus, a memory controller, a peripheral bus, a local bus, and any combination thereof, utilizing any of a variety of bus architectures.

[0078] The memory 1508 may include various components (e.g., machine-readable media), including, but not limited to, random access memory components, read-only components, and any combination thereof. In one example, a basic input / output system 1516 (BIOS), including basic paths that facilitate the transfer of information between elements within the computer system 1500, such as during startup, may be stored in the memory 1508. The memory 1508 may also include instructions (e.g., software) 1520 (e.g., stored on one or more machine-readable media) that implement any one or more of the aspects and / or methods of the present disclosure. In another example, the memory 1508 may also include any number of program modules, including, but not limited to, an operating system, one or more application programs, other program modules, program data, and any combination thereof.

[0079] The computer system 1500 may also include a storage device 1524. Examples of storage devices (e.g., storage device 1524) include, but are not limited to, hard disk drives, disk drives, optical disk drives combined with optical media, solid-state memory devices, and any combination thereof. The storage device 1524 may be connected to the bus 1512 via an appropriate interface (not shown). Example interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (Firewire), and any combination thereof. In one example, the storage device 1524 (or one or more of its components) may be removably connected to the computer system 1500 (e.g., via an external port connector (not shown)). In particular, the storage device 1524 and associated machine-readable media 1528 may provide non-volatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data to the computer system 1500. In one example, the software 1520 may reside entirely or partially in the machine-readable medium 1528. In another example, software 1520 may reside completely or partially within processor 1504 .

[0080] Computer system 1500 may also include input device 1532. In one example, a user of computer system 1500 may enter commands and / or other information into computer system 1500 via input device 1532. Examples of input device 1532 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices, joysticks, game pads, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touch pads, optical scanners, video capture devices (e.g., still cameras, video cameras), touch screens, and any combination thereof.

[0081] The input device 1532 may be connected to the bus 1512 via any of a variety of interfaces (not shown), including, but not limited to, a serial interface, a parallel interface, a game port, a USB interface, a FireWire interface, a direct interface to the bus 1512, and any combination thereof. The input device 1532 may include a touch screen interface, which may be part of or separate from the display 1536 discussed further below. The input device 1532 may be used as a user selection device for selecting one or more graphical representations of the graphical interface as described above.

[0082] The user may also input commands and / or other information to the computer system 1500 via a storage device 1524 (e.g., a removable disk drive, a flash drive, etc.) and / or a network interface device 1540. A network interface device (such as network interface device 1540) may be used to connect the computer system 1500 to one or more of a variety of networks, such as network 1544, and one or more remote devices 1548 connected thereto. Examples of network interface devices include, but are not limited to, a network interface card (e.g., a mobile network interface card, a LAN card), a modem, and any combination thereof. Examples of networks include, but are not limited to, a wide area network (e.g., the Internet, an enterprise network), a local area network (e.g., a network associated with an office, a building, a campus, or other smaller geographic space), a telephone network, a data network associated with a telephone / voice provider (e.g., a mobile communications provider data and / or voice network), a direct connection between two computing devices, and any combination thereof. A network (such as network 1544) may employ wired and / or wireless communication modes. In general, any network topology may be used. Information (eg, data, software 1520 , etc.) may be communicated to and / or from computer system 1500 via network interface device 1540 .

[0083] The computer system 1500 may also include a video display adapter 1552 for communicating displayable images to a display device, such as a display device 1536. Examples of display devices include, but are not limited to, liquid crystal displays (LCDs), cathode ray tubes (CRTs), plasma displays, light emitting diode (LED) displays, and any combination thereof. The display adapter 1552 and the display device 1536 may be used in conjunction with the processor 1504 to provide graphical representations of some aspects of the present disclosure. In addition to the display device, the computer system 1500 may also include one or more other peripheral output devices, including, but not limited to, audio speakers, printers, and any combination thereof. Such peripheral output devices may be connected to the bus 1512 via a peripheral interface 1556. Examples of peripheral interfaces include, but are not limited to, serial ports, USB connections, FireWire connections, parallel connections, and any combination thereof.

[0084] The foregoing has described the exemplary embodiments of the present invention in detail. Various modifications and additions may be made without departing from the spirit and scope of the present invention. The features of each of the various embodiments described above may be combined with the features of other described embodiments where appropriate to provide a variety of feature combinations of related new embodiments. In addition, although the foregoing describes a plurality of independent embodiments, the embodiments described herein only illustrate the application of the principles of the present invention. In addition, although the specific methods herein may be shown and / or described as being performed in a particular order, the order is highly variable within the technical scope of the embodiments disclosed herein. Therefore, this description is intended to be carried out only in an example manner, rather than to limit the scope of the present invention in other ways.

[0085] In the above description and claims, phrases such as "at least one of ..." or "one or more of ..." may appear after a list of linked elements or features. The term "and / or" may also appear in a list of two or more elements or features. Unless implicitly or explicitly contradictory to the context of use, such phrases are intended to mean any one of the listed elements or features alone, or to mean any one of the narrated elements or features combined with any other narrated elements or features. For example, the phrases "at least one of A and B", "one or more of A and B", and "A and / or B" are each intended to mean "single A, single B, or A and B together". Similar explanations are also intended to be used for lists including three or more items. For example, the phrases "at least one of A, B, and C", "one or more of A, B, and C", and "A, B, and / or C" are each intended to mean "single A, single B, single C, A and B together, A and C together, B and C together, or A and B and C together". Furthermore, the term “based on” as used above and in the claims is intended to mean “based at least in part on” such that unlisted features and elements are also permissible.

[0086] The subject matter described herein can be implemented in systems, devices, methods and / or articles, depending on the desired configuration. The embodiments described in the foregoing description do not represent all embodiments consistent with the subject matter described herein. On the contrary, they are only some examples consistent with the aspects related to the described subject matter. Although a few variations have been described in detail above, other modifications or additions are possible. In particular, in addition to those described herein, other features and / or variations may also be provided. For example, the embodiments described above may relate to various combinations and subcombinations of the disclosed features and / or combinations and subcombinations of some other features disclosed above. In addition, the logical flow shown in the drawings and / or described herein does not necessarily require the specific order or sequential order shown to achieve the desired result. Other embodiments may fall within the scope of the following claims.

Claims

1. An encoder configured to generate an encoded bitstream for decoding by a compatible decoder, the bitstream comprising an encoded image, the encoded image comprising an encoded pixel block and signaling information, the signaling information comprising a first partition index for determining a starting point of a non-rectangular non-rectangular partition, and a second partition index for determining an ending point of the non-rectangular non-rectangular partition, the decoder receiving the bitstream and configured to: determining a starting point of a non-linear non-rectangular partition in a block using the first partition index, the starting point being located on a first side of the block at a first offset distance from a first corner of the block; determining an end point of a non-linear, non-rectangular partition in the block using the second partition index, the end point being located on a second side of the block at a second offset distance from a second corner of the block; generating a first predicted pixel value in a first area on a first side of the non-linear non-rectangular partition; generating a second predicted pixel value in a second area on a second side of the non-linear non-rectangular partition; as well as The block is decoded using the predicted pixel values.

2. The encoder according to claim 1, in, A first motion vector selected from the first set of candidate motion vectors is used to generate the first region of predicted pixel values, and a second motion vector selected from the second set of candidate motion vectors is used to generate the second region of predicted pixel values.

3. The encoder according to claim 1, in, The decoder smoothes the first predicted pixel value and the second predicted pixel value on the non-linear non-rectangular partition; and The coded block is decoded by adding the residual pixel value to the smoothed first predicted pixel value and the second predicted pixel value. The encoder of claim 1 , wherein the block is a coding tree unit.

5. The encoder of claim 1, wherein the size of the block is NxN, and N is one of 128, 64, or 32. The encoder of claim 1 , wherein the non-linear non-rectangular partitions are curved.

7. An encoder configured to generate an encoded bitstream for decoding by a compatible decoder, the bitstream comprising an encoded picture, the encoded picture comprising a coding tree unit, the coding tree unit comprising a plurality of coding units and signaling information, the signaling information comprising a first partition index for determining a starting point of a non-rectangular non-rectangular partition in the coding tree unit, and a second partition index for determining an ending point of the non-rectangular non-rectangular partition in the coding tree unit, the decoder receiving the bitstream and configured to: Determine a starting point and an ending point of a non-linear non-rectangular partition in the coding tree unit using the first partition index and the second partition index; generating a first predicted pixel value in a first area on a first side of the non-linear non-rectangular partition using a first motion vector from a first set of motion vector candidates; generating a second predicted pixel value in a second area on a second side of the non-linear non-rectangular partition using a second motion vector from a second set of motion vector candidates; smoothing the first predicted pixel value and the second predicted pixel value on the non-linear non-rectangular partition; as well as The coding tree unit is decoded by adding the residual pixel value to the smoothed first predicted pixel value and the second predicted pixel value.

8. The encoder according to claim 7, in, The bitstream includes a first motion vector candidate index for identifying the first motion vector from a first set of motion vector candidates; and The bitstream comprises a second motion vector candidate index for identifying the second motion vector from a second set of motion vector candidates.

9. A non-transitory computer-readable medium encoded with a bitstream decoded by a compatible decoder, the bitstream comprising an encoded image, the encoded image comprising an encoded pixel block and signaling information, the signaling information comprising a first partition index for determining a starting point of a non-rectangular non-rectangular partition, and a second partition index for determining an ending point of the non-rectangular non-rectangular partition, the decoder receiving the bitstream and configured to: determining a starting point of a non-linear non-rectangular partition in a block using the first partition index, the starting point being located on a first side of the block at a first offset distance from a first corner of the block; determining an end point of a non-linear, non-rectangular partition in the block using the second partition index, the end point being located on a second side of the block at a second offset distance from a second corner of the block; generating a first predicted pixel value in a first area on a first side of the non-linear non-rectangular partition; generating a second predicted pixel value in a second area on a second side of the non-linear non-rectangular partition; as well as; The block is decoded using the predicted pixel values.