Efficient geometric partition mode video coding

Abstract

Various schemes for implementing efficient geometric partition mode (GPM) video coding are described. A device receives pixel data to be encoded as a current block. The apparatus also selects a partition pattern from the set of partition pattern candidates. The apparatus also determines two merge candidates, each merge candidate corresponding to a respective GPM portion of the current block determined by the selected partition mode. The apparatus also excludes one or more mixing width candidates from the set of mixing width candidates based on the selected partition mode and the two merge candidates or auxiliary data of the current block. After exclusion, the apparatus also selects a mixing width from the set of mixing width candidates. The apparatus also encodes and decodes the current block using the selected partition mode, the two merge candidates, and the selected mixing width.

Description

[0001] Cross-references

[0002] The present invention is a non-provisional application of and claims priority to U.S. Provisional Patent Application No. 63 / 336,397 filed on April 29, 2022. The entire contents of the U.S. Provisional Patent Application are incorporated herein by reference. Technical Field

[0003] The present disclosure relates generally to video decoding and, more particularly, to methods and apparatus for efficiently signaling a geometric partitioning mode (GPM) within a video codec. Background Art

[0004] Unless otherwise indicated herein, the approaches described in this section are not prior art to the claims listed below and are not admitted to be prior art by inclusion in this section.

[0005] Video coding generally involves encoding a video (i.e., source video) into a bitstream by an encoder, transmitting the bitstream to a decoder, and decoding the video from the bitstream by parsing and processing the bitstream by a decoder to produce a reconstructed video. Video decoders (i.e., encoders and decoders) may employ various decoding modes or tools when encoding and decoding video, one of the purposes of which is to achieve efficient video decoding, which manifests itself, for example, as a high decoding gain. That is, a video encoder aims to reduce the total size of the bitstream that needs to be transmitted from the encoder to the decoder, while still providing the decoder with enough information about the original video so that a reconstructed video that is satisfactorily faithful to the original video can be generated by the decoder.

[0006] Video is usually presented as a series of pictures or frames displayed in time order. Many coding tools are block-based coding tools, where the pictures or frames to be encoded are divided into a number of non-overlapping rectangular areas, called "blocks", and the video encoder processes the video sequence one block at a time in a pipeline. That is, blocks constitute the basic elements processed by the coding tools, as is common in intra-image prediction and inter-image prediction, the two main techniques used in video coding and decoding by removing spatial and temporal redundancy in the source video, respectively. In general, video redundancy is removed by searching through multiple coded blocks and finding one or more reference blocks that are most similar to the current block to be encoded. The coded blocks are usually stored in the reconstructed picture buffer (RPB) of the encoder and the decoded picture buffer (DPB) of the decoder, so that they can be used by the encoder and decoder for subsequent processing. Once the reference block is found, the current block can be encoded or otherwise represented using the reference block itself and the difference between the reference block and the current block (called the "residual"), thereby eliminating redundancy. Intra-picture prediction utilizes reference blocks found in the same picture as the current block to remove redundancy, whereas inter-picture prediction utilizes reference blocks that are not found in the same picture as the current block but in another picture of the source video.

[0007] For inter-picture prediction, the reference block is called the "predictor" of the current block (i.e., the direct block being encoded), and the picture containing the reference block is called the "reference picture" of the current block. The predictor is found by a motion estimation (ME) process, where the block with the smallest residual is determined to be the predictor because it is most similar to the current block. The current block is therefore represented by the residual, the predictor (usually indicated by an index), and a motion vector (MV) indicating the positional offset between the reference block and the current block, all of which are signaled (i.e., encoded into the bitstream by the encoder). On the decoder side, the bitstream is parsed to recover the residual, the index indicating the predictor, and the MV, and the current block is subsequently reconstructed based on the residual through a motion compensation (MC) process.

[0008] This inter-picture prediction technique used by video encoders is generally referred to as motion compensated prediction (MCP). To further enhance the accuracy of the ME and MC processes, MCP can utilize fractional sample interpolation (i.e., sub-pixel interpolation), where reference picture sample data can be interpolated to sub-pixel resolution, such as half-pixel (1 / 2-pel), quarter-pixel (1 / 4-pel), or finer resolution. That is, the MV can point to an integer, half-sample, quarter-sample, or even finer position in the reference picture. In addition, the MV is generally not directly encoded, but differentially encoded as an MV difference (MVD) with the MV of the neighboring block of the current block. Specifically, the MV is represented as an MV predictor (MVP) plus an MVD, where the MVP can be one of the MVs of the neighboring blocks, or can be derived from a combination of multiple MVs of the neighboring blocks. That is, the MVP is related to the MV of the neighboring blocks, and the correlation can be indicated by an index. Encoding the combination of the MVD and the MVP instead of the MV itself can further reduce the bitstream size. The MV (or equivalently the MVD and the MVP) is generally referred to as the "motion data" or "motion information" of the current block. To encode the current block, the encoder entropy encodes the reference ID (i.e., the index indicating the reference block), the residual (after transformation and quantization), and the motion information into the bitstream, and the decoder parses (i.e., entropy decodes) the bitstream to recover the reference ID, motion information, and the residual, thereby reconstructing the current block.

[0009] Various techniques demonstrated in the present disclosure can be used to enhance the efficiency and performance of MCP, especially for blocks containing object boundaries that are substantially non-parallel to the edges of the blocks. The techniques of this article can be applied to codecs that comply with contemporary video coding standards such as High Efficiency Video Coding (HEVC) or Versatile Video Coding (VVC), and thus extend or enhance their capabilities. HEVC is an international video coding standard developed by the Joint Collaborative Group on Video Coding (JCT-VC). HEVC is based on a hybrid block-based motion compensation class DCT (DCT stands for discrete cosine transform) transform coding architecture. The basic unit of compression is called a coding unit (CU), which is a 2Nx2N square block of pixels, and each CU can be recursively divided into four smaller CUs until a predefined minimum size is reached. Each CU contains one or more prediction units (PUs). Versatile Video Coding (VVC) is a codec designed to meet the upcoming needs of video conferencing, OTT streaming, mobile phones, etc. VVC is designed to provide a variety of functions to meet all video needs from low resolution and low bitrate to high resolution and high bitrate, high dynamic range (HDR), 360 omnidirectional, etc. VVC supports YCbCr color space with 4:2:0 sampling, 10 bits per component, YCbCr / RGB4:4:4, and YCbCr 4:2:2 with bit depths up to 16 bits per component, with HDR and wide gamut color, and auxiliary channels for transparency, depth, etc. Summary of the invention

[0010] The following summary is illustrative only and is not intended to be limiting in any way. That is, the following summary is provided to introduce the concepts, highlights, benefits, and advantages of the novel and non-obvious technologies described herein. Selected but not all embodiments are further described in the detailed description below. Therefore, the following summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine the scope of the claimed subject matter.

[0011] The purpose of the present disclosure is to provide schemes, concepts, designs, techniques, methods and devices related to efficient GPM video decoding. It is believed that various embodiments in the present disclosure achieve benefits including enhanced coding gain, lower signaling overhead and / or reduced hardware cost.

[0012] In one aspect, a method for performing efficient GPM encoding by applying candidate set size reduction and / or candidate reordering is proposed. The method may involve receiving pixel data of a current block and pixel data of one or more neighboring blocks of the current block. In some embodiments, the one or more neighboring blocks may include a neighboring block immediately above the current block and / or a neighboring block immediately to the left of the current block. The method may also involve selecting a partition mode from a partition mode candidate set. The partition mode candidate set may include one or more partition mode candidates, and each partition mode candidate is defined or otherwise characterized by a corresponding partition boundary within the current block. In some embodiments, the partition boundary may be a straight line that partitions the current block into two parts called "GPM parts". The straight partition line may be defined by an angle parameter and an offset parameter. The method may also involve determining two merge candidates for the current block, each of which is a reference block in a reference picture and corresponds to a corresponding one of the two GPM parts. Two merge candidates may be selected from a merge candidate set having two or more merge candidates. The two merge candidates may not be the same reference block. The method may also involve determining a mixed width from a mixed width candidate set based on the selected partition mode and auxiliary data of the two merge candidates or the current block. The step of determining the mixed width may also involve reducing the size of the mixed width candidate set including two or more mixed width candidates. That is, the codec may exclude one or more mixed width candidates from the mixed width candidate set. The mixed width determines the mixing strength of mixing two merge candidates into one GPM prediction. The codec may do so by excluding some mixed width candidates based on the selected partition mode, the auxiliary data of the two merge candidates, and / or the auxiliary data of the current block. The auxiliary data may include the MV of the current block, the MV of one or more adjacent blocks of the current block, the sample values ​​of the two GPM parts, the index of the two merge candidates, the sample values ​​of the template area adjacent to the current block, and / or the sample values ​​of the template area adjacent to one of the two merge candidates. The step of determining the mixed width may also involve selecting the mixed width from the reduced size mixed width candidate set, that is, after excluding one or more mixed width candidates. Finally, the method may also involve encoding and decoding the current block using the selected partition mode, the determined two merge candidates, and the selected mixed width.

[0013] In some embodiments, the step of determining the mixing width may further include reordering the reduced size mixing width candidate set before selecting the mixing width. The reordering is based on some guess cost calculations, which are much less computationally intensive than the actual rate distortion (RD) cost calculations. Specifically, for each mixing width candidate retained in the mixing width candidate set, a corresponding guess cost of the associated GPM prediction is calculated. The guess cost calculation may employ template matching and / or boundary matching techniques.

[0014] In some embodiments, the method may further include constructing a GPM candidate set having a plurality of GPM candidates, each of which represents a corresponding combination of one of the partition mode candidates and one of the merge candidates. In some embodiments, the GPM candidate set may also be reordered based on a guessing cost calculation method. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are included to provide a further understanding of the present disclosure and are incorporated into and constitute a part of the present disclosure. The accompanying drawings illustrate embodiments of the present disclosure and, together with the description, are used to explain the principles of the present disclosure. It is worth noting that the accompanying drawings are not necessarily drawn to scale, because in order to clearly illustrate the concepts of the present disclosure, some components may be shown as being out of proportion to the sizes in actual implementation.

[0016] Figure 1 is a diagram of an example design according to an embodiment of the present disclosure.

[0017] Figure 2 is a diagram of an example design according to an embodiment of the present disclosure.

[0018] Figure 3 is a diagram of an example design according to an embodiment of the present disclosure.

[0019] Figure 4 is a diagram of an example design according to an embodiment of the present disclosure.

[0020] Figure 5 is a diagram of an example design according to an embodiment of the present disclosure.

[0021] Figure 6 is a diagram of an example design according to an embodiment of the present disclosure.

[0022] Figure 7 is a diagram of an example design according to an embodiment of the present disclosure.

[0023] Figure 8 is a diagram of an example video encoder according to an embodiment of the present invention.

[0024] Fig. 9 yes Figure 8 Figure 2 is a diagram of a portion of an example video encoder.

[0025] Fig.10 is a diagram of an example video decoder according to an embodiment of the present disclosure.

[0026] Fig.11 yes Fig.10 Figure 2 is a diagram of a portion of an example video decoder.

[0027] Fig.12 is a flow chart of an example process according to an implementation of the present disclosure.

[0028] Fig.13 is a diagram of an example electronic system according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] In the detailed description below, many specific details are set forth by way of example in order to provide a thorough understanding of the relevant teachings. Any variation, derivation and / or extension based on the teachings described herein are within the scope of protection of the present disclosure. In some cases, well-known methods, processes, components and / or circuits related to one or more example implementations disclosed herein may be described at a relatively high level without detail to avoid unnecessarily obscuring aspects of the teachings of the present disclosure.

[0030] Embodiments according to the present invention relate to various technologies, methods, schemes and / or solutions related to efficient GPM decoding, including signaling schemes between encoders and decoders of video codecs. According to the present disclosure, multiple possible solutions can be implemented individually or in combination. That is, although these possible solutions can be described separately below, two or more of these possible solutions can be implemented in one combination or another combination.

[0031] I. Motion Compensated Prediction Using Merge Mode

[0032] As described elsewhere above, by applying MCP to the current block, the motion information of the current block is encoded in the bitstream. However, the MV coding efficiency of MCP can be improved by adopting the so-called "merge mode", in which the motion information is inherited or otherwise obtained from spatially neighboring blocks (spatial candidates) or temporally co-located blocks (temporal candidates). That is, in merge mode, motion information is not encoded. Instead, a candidate index is selected and signaled to indicate which candidate in a merge candidate set is used for merging, where the merge candidate set includes a merge candidate list, which includes spatial candidates, temporal candidates, and some additional / derived candidates.

[0033] The current block can be a PU, so the MV, prediction direction and reference picture index of the selected candidate are reused. In merge mode, multiple PUs can refer to the same candidate, and each of the multiple PUs (called merged PUs) therefore inherits the same MV, prediction direction and reference picture index from the candidate.

[0034] A special case of merge mode is called "skip mode" where the residual of the current block is inferred to be zero and is not encoded (ie, not signaled).

[0035] Figure 11 is a diagram of an exemplary design according to an embodiment of the present disclosure, in which several motion candidates of a merge mode are shown. Each motion candidate may be a candidate in a merge candidate set. Specifically, a video codec is encoding or decoding a current block 100 of a video picture or frame. As shown in the figure, among the motion candidates, a maximum of four spatial MV candidates are derived from spatial neighbors A0, A1, B0, and B1, and a maximum of four spatial MV candidates are derived from T BR or T CTR Derive a temporal MV candidate, where T BR Indicates the time when the target is located at the lower right neighbor. CTR represents the center block of the current block 100, and T CTR represents the center block of temporally co-located blocks. If T BR If available, use T BR As a candidate, if T BR If not available, use T CTR . If any of the four spatial MV candidates is not available, position B2 is used to derive an MV candidate as an alternative. In some embodiments, after the derivation process of four spatial MV candidates and one temporal MV candidate, a "pruning" operation, essentially a redundancy check, is performed to remove any redundant MV candidates. After the redundancy check, if the number of available MV candidates is less than a predefined number, such as 5, three types of additional candidates can be derived and then added to the merge candidate set. The encoder of the video codec can select a final candidate from the merge candidate set for encoding the current block 100 in merge mode. For example, the encoder can select a candidate using a rate-distortion optimization (RDO) decision. That is, the encoder calculates the RD cost of the current block predicted by each candidate in the merge candidate set. The candidate with the lowest RD cost is then selected as the final candidate for encoding the current block. Therefore, the encoder transmits or otherwise signals an index indicating the final candidate to the decoder of the video codec.

[0036] II. Geometric Partitioning Mode

[0037] The merge mode described above (hereinafter referred to as "regular merge mode") is intended to improve MV coding efficiency by not explicitly signaling MCP motion information but rather having the current block inherit the motion information of the selected merge candidate. Regular merge mode works well for blocks that do not include object boundaries or blocks that include object boundaries that are substantially parallel to one side of the current block (i.e., substantially vertical or horizontal). However, for videos containing natural objects that move relative to the background or relative to other moving objects, the boundaries of the natural objects are typically neither vertical nor horizontal, but rather tilted relative to the sides of the current block. Due to the tilt, regular merge mode may not predict such blocks well, resulting in suboptimal coding gain.

[0038] In order to solve the boundary tilt and thus improve the coding gain, a special merge mode called geometric partition mode (GPM) can be used instead of the conventional merge mode. GPM is intended to provide flexible CU partitioning, which tracks the object boundary more closely by dividing the current block into two parts along the partition boundary. In some embodiments, the partition boundary can be a straight partition line that is better aligned with the inclined object boundary of the CU (i.e., the current block) than the horizontal or vertical PU edge. GPM usually produces better MCP than the conventional merge mode. The partition line is interchangeably referred to as the partition boundary below, and the two parts of the CU separated by the partition boundary are referred to as the two GPM parts of the CU. The two GPM parts use their own motion information for inter-frame prediction, respectively, and are encoded and decoded using the conventional merge mode. In addition, the GPM part only allows unidirectional MCP. That is, each of the two GPM parts only allows one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure the same memory bandwidth as traditional bidirectional prediction. That is, only two motion compensated predictions (MCPs) are performed for each CU under GPM. Then, the two MCPs predicted using their respective GPM parts are mixed into a GPM prediction through a blending operation, which is then subtracted from the original samples of the current block to generate a residual.

[0039] Figure 2 An example of a GPM as described above is shown in FIG. Figure 2, where a current block 255 of a current picture 259 is decoded using GPM. As shown in the figure, the current block 255 is partitioned into two parts 250 and 251 along a partition line 253, which is a straight line passing through the current block 255. The partition line 253 can be substantially aligned with the boundary of an object contained in the current block 255. In addition, the partition line 253 is inclined relative to the edge of the current block 255. The video codec performs unidirectional MCP on each of the parts 250 and 251, resulting in the part 250 being inter-predicted from the reference block 205 of the reference picture 209 through MV0, and the part 251 being inter-predicted from the reference block 215 of another reference picture 219 through MV1. That is, the parts 250 and 251 are inter-predicted in a conventional merge mode, respectively, which results in a merge (the reference blocks 205 and 215 are interchangeably referred to as merge candidates 205 and 215 hereinafter). Subsequently, the video codec performs a blending operation to form a GPM prediction 257 based on the merge candidates 205 and 215. The GPM prediction 257 has the same size (i.e., width and height) as the current block 255 and each merge candidate 205 and 215. The blending operation is performed on the pixel samples along or near the blending area 256. That is, the blending operation blends the merge candidates 205 and 215 into the GPM prediction 257 by adjusting the sample values ​​of the reference block 205 and / or the reference block 215 within the blending area 256. The samples within the blending area 256 of the GPM prediction 257 can be a weighted average of one or more samples of the reference block 205 and one or more samples of the reference block 215. The GPM prediction 257 is then subtracted from the current block 255 to generate a residual of the current block 255, and the residual is then transformed and quantized before being encoded into a bitstream.

[0040] It should be noted that adaptive weighting may be applied in the blending process, rather than fixed weighting. For example, the blending weights for samples of reference blocks 205 and 215 may be different when forming a first sample in blend region 256 of GPM prediction 257 than when forming a second sample in blend region 256 of GPM prediction 257. Further details regarding the adaptive blending feature of the present disclosure are provided elsewhere below.

[0041] GPM can support many different partition modes, each of which is defined by a corresponding partition boundary. For example, GPM is able to divide the current block 255 along one of many different partition boundaries, thereby forming different pairs of GPM parts 250 and 251. For straight partition boundaries, each partition boundary is represented by an angle parameter and an offset parameter. The video encoder can determine or otherwise select a specific one of many different partition modes to encode the current block in GPM mode. For example, the encoder can perform an RDO search within a partition mode candidate set, that is, by calculating the respective RD cost of each partition mode candidate, performing an RDO operation on the partition mode candidates in the partition mode candidate set, and then specifying the partition mode with the lowest RD cost as the selected partition mode to be encoded for the CU. After selecting the partition mode, an index indicating the selected partition mode can be sent to the decoder together with two merge indexes, each merge index indicating a reference block of one of the two parts generated by applying the selected partition mode to the current block.

[0042] For a particular partition pattern, the location of the corresponding partition line within the current block is mathematically derived from the angle parameter and the offset parameter, somewhat similar to the way a straight line is defined in the xy plane using a slope and an intercept. Figure 3 FIG. 1 is a diagram of an example design according to an embodiment of the present disclosure, showing various partition lines of a GPM. Specifically, Figure 3 In FIG. 310 , twenty different angles in the range of [0, 2π) (i.e., 0-360 degrees) that the GPM can support are shown. The angle of the partition pattern is defined by the angle parameter. A partition line with a zero offset represents a partition pattern with a partition boundary passing through the center position of the current block. Some partition patterns may have a non-zero offset. That is, a partition line passing through the center position of the current block may be moved in parallel by an offset specified by the offset parameter, resulting in a different partition pattern for the current block, although the angle parameter remains the same value. Figure 3 320 shows multiple groups of parallel partition lines with the same angle parameter value and different offset values, wherein each partition line represents a GPM partition mode. As shown in the figure, each group contains at most four different GPM partition modes parallel to each other.

[0043] GPM is a merge mode signaled using a CU-level flag. The VVC standard test model has a GPM for each possible CU size w×h=2 m ×2 n, supporting up to 64 different partition modes, where m,n∈{3,…,6}, excluding sizes of 8x64 and 64x8. In order to signal a GPM partition mode among the sixty-four possible candidate partition modes, up to 6 bits are required to signal an index indicating the selected GPM partition mode. In some embodiments, the video codec may avoid using GPM to decode elongated CUs, such as blocks with aspect ratios greater than N to 1 or less than 1 to N, where N is an integer threshold (e.g., 4, 6, 8, or 10). Similar to regular merge mode, GPM may have a skip mode as a special case, in which the residual of the current block is inferred to be zero and not encoded (i.e., not signaled).

[0044] III. GPM Mixing Width

[0045] As described elsewhere above, a blending process is performed to blend two merge candidates (e.g., reference blocks 205 and 215) into a GPM prediction (e.g., GPM prediction 257). In general, a blending region, such as blending region 256, in which sample values ​​may be adjusted by the blending process, includes a stripe having a width of 2θ along a partition line (e.g., partition line 253), where the partition line is located along the center line of the stripe. That is, in a direction perpendicular to the partition line, the blending region has a size of 2θ, where 1θ is on one side of the partition line and 1θ is on the other side of the partition line, as Figure 4 410. The parameter θ is referred to as the mixing width parameter of the GPM mixing operation, and is hereinafter interchangeably referred to as "mixing width".

[0046] The mixing operation involves two integer mixing matrices W0 and W1, which are respectively Figure 4 The weights in the GPM mixing matrices W0 and W1 contain values ​​from a predefined range set by two thresholds (e.g., [0,8]) and are based on the distance from the sample position (e.g., position 411) to the GPM partition boundary (e.g., Figure 4 The partition line 253). Figure 4 As shown in 410, the sample position 411 is represented as (x c ,y c ), and the displacement is expressed as d(x c ,y c ). The displacement can be calculated as the quantized distance from the sample position 411 to the GPM partition boundary 253.

[0047] The blending matrices W0 and W1 are sometimes referred to as "blending masks", and each value therein represents the "blending strength" of the corresponding sample of the merge candidate 205 or 215 when forming the GPM prediction 257. Therefore, each blending matrix W0 and W1 has the same size (i.e., width w and height h) as the current block 255 (and the merge candidates 205 and 215). In some embodiments, the weights are given by a discrete ramp function parameterized by a displacement and two thresholds. For example, the two thresholds can be 0 and 8, and the weight values ​​in one of the blending masks W0 and W1 can be given by the following ramp function:

[0048]

[0049] The weight values ​​of other blending masks can be easily calculated based on the following relationship:

[0050] W0+W1=8*J w,h , (Equation 2)

[0051] Among them J w,h is a matrix of size w × h. Therefore, the GPM prediction 257 is calculated as

[0052]

[0053] Among them, P0 represents the merge candidate 205, P1 represents the merge candidate 215, and PG represents the GPM prediction 257.

[0054] In some embodiments, the blend width θ of any GPM-coded CU is a fixed value, regardless of the partitioning mode used. For example, θ can be defined as a fixed value of 2 pixels. In the case of a fixed θ = 2 pixels, the ramp function of equation 1 can be quantized as

[0055]

[0056] However, the fixed value of θ may not be optimal for all CUs. That is, depending on the content of the CU, the fixed value of θ may be too large or too small to achieve optimal mixing.

[0057] In some embodiments, the value of the mixed width θ can be selected from a predefined set of values, which is referred to as a "mixed width candidate set". For example, a video decoder can select a mixed width θ from a mixed width candidate set that includes the following predefined candidates: 1 / 2 pixel, 1 pixel, 2 pixels, 4 pixels, and 8 pixels. That is, θ∈{1 / 2,1,2,4,8}, as shown in 430. In general, the number of candidates in a candidate set is referred to as the "size" of the candidate set. Therefore, the size of the mixed width candidate set (where θ∈{1 / 2,1,2,4,8}) is 5 because there are five candidates in the candidate set. In addition, each candidate in the mixed width candidate set can be indicated or otherwise represented by an index, and in some embodiments, the index of the selected mixed width can be signaled from the encoder to the decoder. Up to 3 bits ( Figure 4 The index of θ selected from the mixed width candidate set having five candidates is signaled (indicated by "bin" in the figure). In order to select a mixed width θ from the mixed width candidate set for GPM encoding, the codec can perform an RDO search in the mixed width candidate set, that is, perform an RDO operation on the mixed width candidates in the mixed width candidate set. For example, for each mixed width candidate in the mixed width candidate set, a corresponding GPM prediction 257 is generated by mixing two merge candidates 205 and 215 using the corresponding mixed width candidate. Subsequently, a corresponding RD cost is calculated between the generated GPM prediction 257 and the current block 255. The mixed width candidate that produces the lowest RD cost is then designated as the selected mixed width used for the current block 255. In general, if two GPM parts are very different from each other, smaller θ values ​​tend to result in lower RD costs than larger θ values ​​and are therefore preferred. On the other hand, if the gradient across the partition boundary is quite small, larger θ values ​​are generally preferred.

[0058] IV. GPM Signaling

[0059] In order to signal a GPM-encoded CU, several syntax elements may be signaled from the encoder to the decoder. The syntax elements may include: (1) two merge candidate indices indicating two merge candidates (e.g., merge candidates 205 and 215); (2) a partition mode index indicating a partition mode (e.g., partition line 253) selected from a partition mode candidate set having multiple partition mode candidates (e.g., various partition modes shown in 320); (3) a mixed width index indicating a mixed width θ (e.g., θ∈{1 / 2,1,2,4,8}, as shown in 430) selected from a mixed width candidate set having multiple mixed width candidates. As described elsewhere above, in the case where the partition mode candidate set has 64 partition mode candidates, up to 6 bits are required to signal the selected partition mode using fixed-length binary encoding. Similarly, in the case where the mixed width candidate set has five mixed width candidates, up to 3 bits are required to signal the selected partition mixed width. In addition, the two merge candidate indices may be encoded using a truncated unary (TU) binarization scheme and require a total of 9 bits to signal. Obviously, the many bits required to signal a GPM-coded CU (ie, 9+6+3=18 bits) is not optimal, since the many bits required may negatively impact the coding gain of the video codec.

[0060] Compared to fixed-length coding schemes, using variable-length binary coding for the index can help reduce signaling overhead. For example, 430 lists the corresponding binary codewords (in the "bin" column) used to signal the θ candidates using a variable-length coding scheme. As shown, the 1-bit codeword "1" is assigned to θ=2, while each of the other four θ candidates is encoded with a corresponding 3-bit codeword. Since the θ=2 candidate is the most commonly selected mixing width for GPM-coded CUs in the video, the corresponding short codeword used to represent the mixing width θ=2 helps to enhance the overall coding gain.

[0061] In some embodiments, for a GPM-encoded CU, the video codec may determine the selected partition mode and the selected hybrid width θ by performing an RDO search operation that requires calculating the RD cost for each possible combination of the partition mode and the hybrid width. For example, using 64 candidates in the partition mode candidate set and 5 candidates in the hybrid width candidate set, the encoder of the video codec will perform an RDO search on 64x5=320 possible partition-θ combinations of the current CU. The encoder may determine or otherwise specify the combination with the lowest RD cost as the pair of the selected partition and the selected hybrid width. Obviously, RDO across all partition θ combinations is a very computationally intensive task for the encoder.

[0062] Various techniques, as described in detail below, can be used for GPM to reduce the computational burden of the encoder as well as the signaling overhead, thereby enhancing coding efficiency without sacrificing much, if any, coding gain.

[0063] V. Candidate Set Size Reduction

[0064] On the one hand, the present disclosure proposes to improve overall coding efficiency by reducing the size of the mixed width candidate set, that is, by excluding one or more candidates from the predefined mixed width candidates that are unlikely to produce satisfactory coding gain. The method of reducing the size of the candidate set has three stages or steps: (i) detecting or otherwise determining the object edge within each of the two GPM merge candidates; (ii) calculating the gradient of the current block based on the gradient of the detected object edge across the two GPM merge candidates; (iii) excluding one or more unlikely candidates based on the calculated gradient of the current block. Steps (i) and (ii) are performed using sample values ​​and / or auxiliary data of the two merge candidates generated by the selected partition mode (e.g., reference blocks 205 and 215). Step (iii) is performed using a statistical model or a lookup table (LUT), which can be history-based. That is, based on the calculated gradient and the statistical model / LUT, the video codec can exclude one or more mixed width candidates from the mixed width candidate set that are unlikely to be used for GPM encoding of the current block, thereby reducing the size of the mixed width candidate set. On the encoder side, after candidate size reduction, an RDO search or calculation based on two merge candidates or auxiliary data of the current block can be performed within the reduced-size candidate set, instead of a predefined, full-size candidate set, for selecting the final candidate of the mixed width for GPM encoding of the current block.

[0065] Figure 5 is a diagram of an example design according to an embodiment of the present disclosure, in which edge detection is performed on reference blocks 205 and 255 to detect object edges 510 and 520, respectively. Typical edge detection algorithms can be used for this task. For example, a sudden change or discontinuity in sample intensity within a block can be detected as an object edge. The most significant discontinuity (i.e., the most sudden change) detected within a reference block 205 or 255 can be designated as an object edge 510 or 520. In some embodiments, samples in a "template" area are part of the auxiliary data and can also be used for edge detection.

[0066] A template of a block is a region adjacent to the block that contains rows and / or columns of samples from neighboring blocks that have been reconstructed using intra prediction, with or without position dependent intraprediction combination. A more detailed description of templates is given elsewhere below. Figure 5As shown, reference block 205 has a template 570 that includes several rows of neighboring blocks immediately above block 205. Similarly, reference block 215 has a template 571 that includes several rows of neighboring blocks immediately above block 205. In the case where the template samples (i.e., the samples in templates 570 and 571) are available to the codec (e.g., still stored in the reconstructed picture buffer of the encoder or the decoded picture buffer of the decoder), when the current block 255 is being encoded, the template samples can also be used in combination with or instead of the samples in reference blocks 205 and 215, respectively, for edge detection in step (i).

[0067] In step (ii), a gradient across each of the detected object edges 510 and 520 may be calculated based on sample values ​​of the corresponding reference block or some auxiliary data thereof (e.g., samples in template 570 or 571). For example, the gradient may be calculated using average sample values ​​within certain regions on both sides of the object edge 510 or 520. For example, the gradient across the object edge 510 may be represented by a first gradient value 511, which is calculated as a difference between average sample values ​​within regions 513 and 514 of the block 205, where regions 513 and 514 are two regions positioned substantially horizontally relative to each other and located near the object edge 510. Alternatively, the gradient edge 510 across the object can be represented by a second gradient value 512, which is calculated as the difference between the average sample values ​​within regions 515 and 516 of the block 205, where regions 515 and 516 are two regions substantially positioned vertically relative to each other and similarly, the gradient across the object edge 520 can be calculated as a horizontal gradient value 521 or a vertical gradient value 522, and the gradient values ​​521 and 522 are calculated in a manner similar to the gradient. In some embodiments, the gradient of the object edge 510 can be calculated by combining the first gradient value 511 and the second gradient value 512, for example as a weighted average thereof. Similarly, the gradient across the object edge 520 can be calculated by combining the gradient values ​​521 and 522 as a weighted average. The weights for calculating the weighted average can depend on the inclinations of the object edges 510 and 520, respectively. For example, the object edge 510 is substantially diagonal within the block 205, so the weights of the gradient values ​​511 and 512 can be substantially the same when calculating the weighted average. As another example, object edge 520 is at a more vertical angle within block 215 , and therefore gradient value 522 may have a higher weight than gradient value 521 when calculating the weighted average.

[0068] In some embodiments, samples in templates 570 and 571 may additionally or alternatively be used to calculate gradients across object edges 510 and 520, respectively. For example, when calculating gradients across object edge 510, horizontal gradient 517 calculated using sample values ​​in template 570 may be used in place of horizontal gradient 511. Alternatively, horizontal gradient 517 may be directly specified as the gradient across object edge 510. When gradients are calculated using sample values ​​in templates 570 and 571, object edges 510 and 520 may be extrapolated or otherwise extended across the vertical dimension of templates 570 and 571, respectively.

[0069] In addition, in step (ii), the gradient of the current block 255 may be determined based on the gradient across the object edge 510 and the gradient across the object edge 520. In some embodiments, the larger of the two gradients may be designated as the gradient of the current block 255. In some embodiments, a weighted average of the two gradients may be calculated and designated as the gradient of the current block 255. For example, the two gradients may be weighted according to a first index difference between the reference block 205 and the current block 255, and the two gradients may be weighted according to a second index difference between the reference block 215 and the current block 255. Each of the first and second index differences may be an indication of a difference between the current block 255 and the corresponding reference block 205 or 215 in a picture order count (POC). The weight may be inversely proportional to the first index difference and the second index difference.

[0070] In step (iii) of reducing the size of the hybrid width candidate set, one or more unlikely candidates are excluded from the candidate set based on the gradient of the current block 255 as calculated in step (ii). In general, larger gradients require smaller GPM hybrid widths θ to achieve higher coding gains, while smaller gradients work better at larger values ​​of θ. For example, for a predefined hybrid width candidate set with five candidates, e.g., θ∈{1 / 2,1,2,4,8} as shown in 430, where step (ii) results in large gradient values ​​across object edges, the video codec can exclude θ=4 and θ=8 from the hybrid width candidate set, leaving only θ∈{1 / 2,1,2} as the reduced size candidate set. On the other hand, if step (ii) produces a relatively small gradient value, the video codec can exclude θ=1 / 2 and θ=1 from the hybrid width candidate set, leaving only θ∈{2,4,8} as the reduced size candidate set. The reduced size candidate set enables the use of shorter codewords to signal the selected hybrid width. For example, when the size of the candidate set is reduced from five to three, the codeword can be reduced from 3 bits to 2 bits. The reduced candidate set also reduces the RDO burden on the codec, thereby improving coding efficiency.

[0071] As mentioned elsewhere above, the relationship between the gradient value of the current block calculated in step (ii) and the candidates excluded in step (iii) can be established by a statistical model, which can be a LUT. In some embodiments, the statistical model / LUT can be history-based. That is, the codec can determine which one or more mixed width candidates to exclude in step (iii) based on the results of a previous RDO operation based on a mature candidate set. For example, before building the model / LUT, the codec can skip step (iii) and perform an RDO search on each predefined candidate in the mixed width candidate set. As the codec encodes more and more CUs in GPM, based on the results of the RDO search exhausted when encoding the CU, the codec can build and refine the statistical model or LUT to make an educated guess about the likelihood of certain candidate mixed widths relative to certain gradient values. Once the LUT is available, the codec can use the LUT to perform step (iii) to exclude one or more unlikely candidates based on the gradient value of the current block calculated in step (ii). In some embodiments, the LUT may be drained and rebuilt when the codec starts encoding a new slice, when a scene change is detected, or when there is a transition between a natural image (NI) and a screen content image (SCI).

[0072] In some embodiments, the codec may exclude one or more candidates from the mix width candidates in step (iii). In some embodiments, the codec may exclude all but one candidate from the predefined mix width candidate set, such that the mix width candidate set has only one mix width candidate after step (iii). If only one candidate is left in the mix width candidate set, then the only candidate is automatically selected without RDO, and the selected mix width does not need to be signaled from the encoder to the decoder, since it has already been implicitly determined on the mix width.

[0073] In some embodiments, the candidate set size reduction method described above can be slightly modified, where step (i) is modified to: (ia) detect or otherwise determine the object edge within the current block, step (ii) is modified to: (ii-a) calculate the gradient of the current block across the determined object edge, and step (iii) remains the same. The modified method is further described below.

[0074] In step (ia), the object edge within the current block is determined. In some embodiments, the partition boundary of the selected partition mode of the current block 255, such as the partition boundary 253, can be designated as the object edge within the current block. In some embodiments, the object edge within the current block can be derived from the object edges detected within the two GPM merge candidates in step (i). Figure 5As shown, the object edge 530 within the current block 255 can be derived from the object edge 510 detected in the merge candidate 205 and the object edge 520 detected in the merge candidate 215. For example, the object edges 510 and 520 can be weighted and averaged to generate the object edge 530. In some embodiments, the weight can be inversely proportional to the aforementioned first index difference and the second index difference, that is, the index difference between the reference block 205 and the current block 255 and the index difference between the reference block 215 and the current block 255.

[0075] In step (ii-a), the gradient across the object edge of the current block determined in step (ia) may be calculated in a manner similar to the calculation of the gradient across the object edge 510 or 520 in step (ii). In some embodiments, other auxiliary data may be used in addition or alternatively to calculate the gradient across the object edge 510, 520, or 530 in step (ii) and step (ii-a), while the basic principle of calculating the gradient remains the same. The auxiliary data may include an MV in the current CU, an MV of a neighboring block or sub-block (e.g., one or more neighboring blocks of the current CU) or a difference between neighboring MVs, a difference in MVs between the current CU and a neighboring CU, a reference sample value or reference index of two GPM parts (e.g., sample values ​​or indices of reference blocks 205 and 215) or a difference thereof, a prediction mode or type of two GPM parts, dimensions (e.g., width, height, area, and / or aspect ratio of the current block), or a combination of two or more of the items listed above. That is, in addition to or in lieu of using sample values ​​of a reference block to determine the gradient across an object edge as described above, any auxiliary data or combination thereof may be used to compute the gradient across an object edge.

[0076] In some embodiments, the auxiliary data may also include template samples of the current CU and template samples of two merge candidates (e.g., template sample values ​​of blocks 255, 205, and 215). The template of a block refers to an area, usually L-shaped, adjacent to the block containing several rows and / or columns of samples of neighboring blocks that have been reconstructed by intra prediction, with or without PDPC. In raster scan coding mode, the template includes samples in neighboring blocks immediately above and / or to the left of the block. Figure 66 is a diagram of an example design according to an embodiment of the present disclosure, in which a template area is shown for a block that is a current block 255, a GPM reference block 205, or a GPM reference block 215. The following description of the template area takes the current block 255 as an example. The same definition of the template area applies to the reference block 205, 255, or any other block. Different embodiments define the template area differently, usually depending on the partition mode considered. In some embodiments, the template area of ​​the current block 205 may include an area 671, an area 672, or both, and the area 671 represents the last M rows of reconstructed samples of the adjacent block immediately above the current block 205, and the area 672 represents the last N columns of reconstructed samples of the adjacent block immediately to the left of the current block 205. In some embodiments, the template area may also include samples represented by an area 673, and the area 673 is located in a block immediately adjacent to the current block 205 in the upper left direction. In some embodiments, M may be equal to N, while in some other embodiments, M may not be equal to N. In some embodiments, M and / or N may be related to the size of the current block. For example, M may be a certain percentage of the height of the current block 205. Likewise, N may be a percentage of the width of the current block 205 .

[0077] Therefore, in step (ii-a), the gradient across the object edge of the current block can be calculated using auxiliary data. For example, the gradient can be calculated using the sample values ​​within the template 575 of the current block 255, and the object edge 530 can be extrapolated or otherwise extended by the vertical dimension of the template 575. As another example, the gradient can be calculated or otherwise estimated using the difference in the indexes of the reference blocks 205 and 215. The larger the index difference, the larger the gradient across the partition boundary 253 or the object edge 530. As another example, the gradient can be calculated or otherwise estimated using the MV in the current block 255, specifically, using the difference between the first group of MVs and the second group of MVs, wherein the first group includes the MV of the current block on one side of the object edge or partition boundary, and the second group includes the MV of the current block on the other side of the object edge or partition boundary. In some embodiments, the first group may additionally or alternatively include the MV of the template area of ​​the current block on one side of the object edge, and the second group may additionally or alternatively include the MV of the template area of ​​the current block on the other side of the object edge. In some embodiments, the first group of MVs and the second group of MVs may additionally or alternatively include MVs of neighboring blocks. That is, the first group may include MVs of neighboring blocks on one side of an object edge, and the second group may include MVs of neighboring blocks on the other side of an object edge. In some embodiments, the first group of MVs and the second group of MVs may include MV differences, i.e., the difference in MVs between the current CU and the neighboring blocks. That is, the first group may include the difference in MVs between the current CU and the neighboring blocks on one side of an object edge, and the second group may include the difference in MVs between the current CU and the neighboring blocks on the other side of an object edge.

[0078] It should be noted that, when executing step (iii), as described above, different LUTs may need to be established for different ways of using sample values ​​or auxiliary data for calculation or estimation.

[0079] In one aspect, the present disclosure proposes to exclude one or more candidates based on any auxiliary data or a combination thereof from the predefined mixed width candidates. For example, for large CU sizes, only larger mixed widths (e.g., θ=2, 4, and 8) are retained. Otherwise, for smaller CU sizes, smaller mixed widths (e.g., θ=1 / 2, 1, and 2) are retained.

[0080] VI. Candidate Re-ranking

[0081] In one aspect, the present disclosure proposes to reduce signaling overhead by reordering candidates within a candidate set, where the candidates are arranged in a specific sequence or order. For example, the candidates within the candidate set can be reordered after the candidate set size has been reduced or not reduced. That is, the step of reordering the candidates in the candidate set can be performed on the original candidate set without performing the step of reducing the size of the candidate set. Specifically, the video codec can use auxiliary data of the current block (e.g., current CU 255), auxiliary data of two merge candidates (e.g., merge candidates 205 and 215) generated by the selected partition mode, or auxiliary data of the GPM prediction (e.g., GPM prediction 257) to perform guess cost calculation. The guess cost calculation gives an estimate or guess of the actual RD cost, while being relatively less computationally intensive than a mature RDO search operation. The codec can rearrange the order of the candidates based on the calculated guess cost associated with each candidate. Candidates with lower guess costs will produce better predictions when encoding the current CU and are therefore placed in front of the sequence. That is, the codec reorders the candidates in ascending order of guess cost, where a candidate with a lower guess cost is placed in front of another candidate with a higher guess cost in the sequence. In addition, a candidate occupying a front position in the sequence is usually assigned a shorter (i.e., represented using fewer bits) signaling codeword than another candidate occupying a back position in the sequence. As a result, coding gain is enhanced while signaling overhead is reduced.

[0082] Figure 7is a diagram of an example design according to an embodiment of the present disclosure, in which a template matching (TM) technique is used for guessing cost calculation. Specifically, the codec uses a template 750 of the current block 255, a template 700 of the merge candidate 205, and a template 710 of the merge candidate 215 to perform guessing cost calculation. Although shown as an L-shaped area, each of the templates 700, 710, and 750 may optionally have only one or two portions of the L-shaped area, such as one or two but not all three of the areas 671, 672, and 673. For example, each of the templates 700, 710, and 750 may alternatively include several adjacent rows of adjacent blocks immediately above the blocks 205, 215, and 255, respectively.

[0083] Figure 7 One GPM predicted 257 and another GPM predicted 757. Figure 7 The GPM predictions 257 and 757 are both GPM predictions 257 and another GPM prediction 757. Figure 7 represents a blend of two merge candidates 205 and 215, while prediction 257 represents a blend with a blend width θ1, and prediction 757 represents a blend with a blend width θ2 different from θ1, where θ1 and θ2 are both blend width candidates. The blend regions of the two predictors are also shown as Figure 7 256 and 756 in.

[0084] The video codec can determine which of predictions 257 and 757 is a better candidate for decoding the current block 255 in GPM. Instead of actually generating the two predictions and performing RD cost calculation between the current block 255 and each of predictions 257 and 757, the codec can perform guess cost calculation using templates 700, 710, and 750. Specifically, template 770 and template 780 are generated by blending template 700 and template 710 within a region that extends along partition line 253 in the blend region 256 and overlaps with template 770, with a blend width of θ1. Similarly, template 780 is generated by blending template 700 and template 710 with template 770. The blend region 756 has a blend width of θ2 within a region that extends along partition line 253 and overlaps with template 780. Similar to the blend region 256, the blend region 756 includes a stripe centered along the partition line 253, with a width of only 2*θ2 instead of 2*θ1. After generating the blend templates 770 and 780, the codec can perform guess cost calculation using template matching between template 750 of the current block 255 and each of the blend templates 770 and 780. For example, the codec can compare template 750 and template 770 and specify the difference in sample values between the two templates, presented in the form of the sum of absolute differences (SAD) or the sum of absolute transform differences (SATD), as the guess cost for prediction 257. Similarly, the codec can also compare template 750 and template 780 and specify the difference in sample values between them as the guess cost for prediction 757. Between predictions 257 and 757, the one with the lower guess cost is the better prediction, and the associated blend width is arranged in front of the other blend width to reorder the set of blend width candidates. As Figure 7 shown, if SAD1 > SAD2, then θ2 is reordered before θ1 in the set of blend width candidates. On the other hand, if SAD1 < SAD2, then θ1 is reordered before θ2 in the set of blend width candidates.

[0085] Figure 7 The boundary matching (BM) technique can be used instead of template matching for guess cost calculation to determine which of predictions 257 and 757 is a better candidate for encoding the current block 255 in GPM. Specifically, the blend templates 770 and 780 are compared with the boundary region 760 of the current block 255, respectively. The boundary region 760 can include some rows or columns of the current block 255 adjacent to the template region 750. As Figure 6As shown, the boundary region 760 may include two or three of regions 661, 662, and 663. Basically, the shape of the boundary region 760 should match the shape of the template region 750. That is, if the template region 750 consists of region 671 and may also include 673, then the boundary region 760 may consist of regions 661 and 663. If the template region includes region 672 and may also include 673, then the boundary region 760 may consist of regions 662 and 663. If the template region consists of regions 671 and 672 and may additionally consist of 673, then the boundary region 760 may consist of regions 661, 662, and 663. As Figure 6 shown, each of regions 661, 662, and 663 may consist of m rows and / or n columns of samples, while each of regions 671, 672, and 673 may consist of M rows and / or N columns of samples. It is not required that M = m or N = n, but M = m and N = n are typically used for boundary matching.

[0086] After generating the hybrid templates 770 and 780 as described above, the codec may perform guess cost calculation using boundary matching between the boundary region 760 of the current block 255 and each of the hybrid templates 770 and 780. For example, the codec may compare the boundary region 760 and the template 770 and specify SAD or SATD as the guess cost for the prediction 257. Similarly, the codec may also compare the boundary region 760 and the template 780 and specify the difference in sample values between them as the guess cost for the prediction 757. Between the predictions 257 and 757, the prediction with the lower guess cost is a better prediction, and the associated hybrid width is arranged in front of another hybrid width in the hybrid width candidate set reordering. As Figure 7 shown, if SAD3 > SAD4, then θ2 is reordered before θ1 in the hybrid width candidate set. On the other hand, if SAD3 < SAD4, then θ1 is reordered before θ2 in the hybrid width candidate set.

[0087] In addition to the template matching and boundary matching techniques, other techniques may also be used to estimate the guess cost for each hybrid width candidate and reorder the hybrid width candidate set accordingly. For example, various gradients estimated from auxiliary materials (as described elsewhere above for candidate set size reduction) may also be used for candidate set reordering.

[0088] It should be noted that the mixed-width candidate order is partition-mode dependent. That is, the order of mixed-width candidates in a candidate set may be different for different partition modes. For example, while θ1 is reordered to be before θ2 for some partition modes, θ1 may be reordered to be after θ2 for another partition mode. It should also be noted that the above reordering method can be applied to predefined, mature mixed-width candidate sets, as well as to mixed-width candidate sets that have undergone downsizing with candidate size reduction as described elsewhere.

[0089] In some embodiments, the reordering of the mixed-width candidates may follow the size reduction of the mixed-width candidate set. That is, after excluding the one or more unlikely mixed-width candidates described above, the codec may perform candidate reordering on the mixed-width candidates in the reduced-size candidate set.

[0090] In some embodiments, the size reduction of the mixed width candidate set can be performed after the reordering of the mixed width candidates. That is, the codec can reorder the candidates in the predefined mixed width candidate set and then perform the size reduction to exclude one or more impossible mixed width candidates, as described above. In some embodiments, the candidate set size reduction can be based on the result of the candidate reordering. For example, when performing the candidate set size reduction, the codec can retain the first N candidates in the reordered candidate set and exclude the remaining candidates. In some embodiments, N can be equal to 1; that is, only the first candidate in the reordered mixed width candidate set is retained, and the current CU uses the first candidate for GPM encoding and decoding.

[0091] The concept of candidate reordering may not be limited to mixed-width candidate sets. Instead, the same concept can be applied to another candidate set of GPM. For example, the partition mode candidate set can be reordered in a similar manner using TM or BM techniques. In addition, the merged candidate set can also be reordered similarly.

[0092] On the one hand, the present disclosure proposes to construct a GPM candidate set by combining a partition mode candidate set and a merge candidate set. Subsequently, the guess cost calculation as described above can be performed across the candidates in the GPM candidate set to reorder the GPM candidate set. The GPM candidate with the lowest guess cost is used to encode the current CU in the GPM. That is, the partition mode-merge candidate pair combination indicated by the GPM candidate with the lowest guess cost is used to encode the current CU.

[0093] Specifically, each candidate in the GPM candidate set represents a possible combination of a partition mode and a pair of merge candidates, where each merge candidate is used for one of the two GPM parts. For example, the partition mode candidate set can have 64 different partition mode candidates, and the merge candidate set can have 6 different merge candidates. This results in a GPM candidate set with up to 64*

[0094] P(6,2)=64*6*(6-1)=1920 GPM candidates. The codec can then calculate the guess cost for each of the 1920 GPM candidates and reorder them accordingly. In some embodiments, size reduction can be performed to reduce the GPM candidate set to a number less than 1920 by excluding some unlikely candidates in a manner similar to how to reduce the mixed width candidate set as described elsewhere above. The first candidate in the reordered GPM candidate set indicates a combination of a partition mode and a pair of merge candidates that together give the lowest guess cost. This combination is the combination that the codec uses to encode the current CU in the GPM.

[0095] For one or more candidates in the reordered GPM candidate set, the mixed width candidates in the reduced mixed width candidate set may be reordered to save signaling bits. For example, for the first candidate in the GPM candidate set after the GPM candidate set is reordered, the mixed weight candidates in the mixed width candidate set are reordered using horizontal and / or vertical gradients as described above. That is, at most 6x5 (merged candidate pairs)*64

[0096] The (partition mode)*k(mixing weight) candidates are reordered together. The best N GPM candidates (i.e., N sets of merged candidate pairs, partition modes, and mixing weights) are retained. The index is explicitly determined by the RD cost to indicate the best GPM candidate among the best N GPM candidates and is sent to the decoder. Alternatively, the index is implicitly determined by selecting the index of the GPM candidate with the minimum template matching cost without signaling.

[0097] VII. Example Implementation

[0098] Figure 8An example video encoder 800 is marked, in which various embodiments and schemes described elsewhere above can be adopted or otherwise implemented. As shown, the video encoder 800 receives an input video signal from a video source 805 and encodes the signal into a bitstream 895. The video encoder 800 has several components or modules for encoding the signal from the video source 805, including at least some components selected from the following: a transform module 810, a quantization module 811, an inverse quantization module 814, an inverse transform module 815, an intra-frame estimation module 820, an intra-frame prediction module 825, a motion compensation module 830, a motion estimation module 835, a loop filter 845, a reconstructed picture buffer 850, an MV buffer 865, an MV prediction module 875, and an entropy encoder 890. The motion compensation module 830 and the motion estimation module 835 are part of an inter-frame prediction module 840. The inter-frame prediction module 840 may include an integer motion estimation (IME) kernel configured to perform an integer pixel search and a fractional motion estimation (FME) kernel configured to perform a fractional pixel search. Both integer pixel search and fractional pixel search are basic functions of the motion compensation module 830 and the motion estimation module 835.

[0099] In some embodiments, modules 810-890 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device or electronic device. In some embodiments, modules 810-890 are hardware circuit modules implemented by one or more integrated circuits (ICs) of an electronic device. Although modules 810-890 are shown as separate modules, some modules can be combined into a single module.

[0100] The video source 805 provides an uncompressed raw video signal that presents pixel data for each video frame. That is, the video source 805 provides pictures or frames presented in a time series. The subtractor 808 calculates the difference between the raw video pixel data of the video source 805 and the predicted pixel data 813 from the motion compensation module 830 or the intra-frame prediction module 825. The transform module 810 converts the difference (or residual pixel data or residual signal 808) into transform coefficients (e.g., by performing a discrete cosine transform or DCT). The quantization module 811 quantizes the transform coefficients into quantized data (or quantized coefficients) 812, which are encoded into a bitstream 895 by the entropy encoder 890.

[0101] The inverse quantization module 814 inversely quantizes the quantized data (or quantized coefficients) 812 to obtain transform coefficients, and the inverse transform module 815 inversely transforms the transform coefficients to generate a reconstructed residual 819. The reconstructed residual 819 is added to the predicted pixel data 813 to generate reconstructed pixel data 817. In some embodiments, the reconstructed pixel data 817 is temporarily stored in a line buffer (not shown) for intra-frame prediction and spatial MV prediction. The reconstructed pixels are filtered by the loop filter 845 and stored in the reconstructed picture buffer 850. In some embodiments, the reconstructed picture buffer 850 is a memory external to the video encoder 800. In some embodiments, the reconstructed picture buffer 850 is a memory internal to the video encoder 800.

[0102] The intra estimation module 820 performs intra prediction based on the reconstructed pixel data 817 to generate intra prediction data. The intra prediction data is provided to the entropy encoder 890 to be encoded into a bitstream 895. The intra prediction data is also used by the intra prediction module 825 to generate predicted pixel data 813.

[0103] The motion estimation module 835 performs inter-frame prediction by generating MVs to refer to pixel data of a previously decoded frame stored in the reconstructed picture buffer 850. These MVs are provided to the motion compensation module 830 to generate predicted pixel data.

[0104] Instead of encoding the complete actual MV in the bitstream, the video encoder 800 uses MV prediction to generate a predicted MV, and the difference between the MV used for motion compensation and the predicted MV is encoded as residual motion data and stored in the bitstream 895.

[0105] The MV prediction module 875 generates a predicted MV based on a reference MV generated for a previously encoded video frame, i.e., a motion compensated MV for performing motion compensation. The MV prediction module 875 retrieves a reference MV from a previous video frame from the MV buffer 865. The video encoder 800 stores the MV generated for the current video frame in the MV buffer 865 as a reference MV for generating a predicted MV.

[0106] The MV prediction module 875 uses the reference MV to create a predicted MV. The predicted MV can be calculated by spatial MV prediction or temporal MV prediction. The entropy encoder 890 encodes the difference (residual motion data) between the predicted MV and the motion compensation MV (MC MV) of the current frame into the bitstream 895.

[0107] The entropy encoder 890 encodes various parameters and data into a bitstream 895 by using an entropy coding technique such as context adaptive binary arithmetic coding (CABAC) or Huffman coding. The entropy encoder 890 encodes various header elements, flags along with quantized transform coefficients 812 and residual motion data as syntax elements into the bitstream 895. The bitstream 895 is in turn stored in a storage device or transmitted to a decoder over a network communication medium, such as a network.

[0108] The in-loop filter 845 performs a filtering or smoothing operation on the reconstructed pixel data 817 to reduce encoding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0109] Fig. 9 The figure illustrates a portion of the video encoder 800 that utilizes the candidate set reordering and size reduction proposed in the present invention to achieve efficient GPM decoding. Specifically, the figure illustrates some components of the inter-frame prediction module 840 of the video encoder 800, which calculates various guess costs, gradient values, and RD costs, and maintains and manages various candidate sets. The inter-frame prediction module 840 may include a processor for calculating matching costs (e.g., Figure 7 Calculator 910 for calculating guess costs SAD1-SAD4) and gradients (e.g., gradients 511, 512, and 517) in the image processing module 800. Calculator 910 includes an edge detection module 912 for detecting object edges (e.g., object edges 510, 520, and 530) using sample values ​​or auxiliary data, wherein the sample values ​​are provided by a reconstructed picture buffer (RPB) 850 and / or directly from a video source 805 and auxiliary data provided by a template / boundary retriever 911, and the template / boundary retriever 911 is capable of obtaining template regions (e.g., regions 700, 710, and 750) and boundary regions (e.g., region 760) based on reference or neighboring blocks from the RPB 850 and MVs from the MV buffer 865.

[0110] The inter prediction module 840 may include a candidate manager 920 for managing various candidate sets, including a merge candidate set 921, a partition mode candidate set 922, and a mixed width candidate set 924. The candidate manager 920 is also capable of generating a GPM candidate set 923 based on the candidates in the merge candidate set 921 and the partition mode candidate set 922. Each candidate in the GPM candidate set 923 represents a corresponding combination of a merge candidate of the merge candidate set 921 and a partition mode of the partition mode candidate set 922. The candidate manager 920 may reorder the candidates in any one of the candidate sets 921-924 based on the calculated gradient and / or guess cost provided by the calculator 910, and exclude certain candidates from any one of the candidate sets 921-924. The candidate manager 920 may also assign a codeword to the index of a candidate in any one of the candidate sets 921-924, regardless of whether the corresponding candidate set is before or after size reduction or reordering. In some embodiments, the candidate manager 920 may send the index of the selected or remaining candidate to the entropy encoder 890 for signaling to the decoder. Furthermore, the candidate manager 920 may send the index of the selected candidate to the motion compensation module 830 for encoding the current block in GPM.

[0111] In some embodiments, the encoder may signal (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0112] In some embodiments, the encoder may signal (or generate) one or more syntax elements in a bitstream so that the decoder may parse the one or more syntax elements from the bitstream.

[0113] Fig.10 An example video decoder 1000 is illustrated in which various embodiments and schemes described elsewhere above may be employed or otherwise implemented. As shown, the video decoder 1000 is an image decoding or video decoding circuit that receives a bitstream 1095 and decodes the contents of the bitstream into pixel data of a video frame for display. The video decoder 1000 has several components or modules for decoding the bitstream 1095, including some components selected from an inverse quantization module 1011, an inverse transform module 1010, an intra-frame prediction module 1025, a motion compensation module 1030, a loop filter 1045, a decoded picture buffer 1050, an MV buffer 1065, an MV prediction module 1075, and a parser 1090. The motion compensation module 1030 is part of the inter-frame prediction module 1040.

[0114] In some embodiments, modules 1010-1090 are software instruction modules executed by one or more processing units (e.g., processors) of a computing device. In some embodiments, modules 1010-1090 are hardware circuit modules implemented by one or more ICs of an electronic device. Although modules 1010-1090 are shown as separate modules, some modules may be combined into a single module.

[0115] The parser 1090 (or entropy decoder) receives the bitstream 1095 and performs initial parsing according to the syntax defined by the video coding or image coding standard. The parsed syntax elements include various header elements, flags, and quantization data (or quantization coefficients) 1012. The parser 1090 parses out the various syntax elements using entropy coding techniques such as context-adaptive binary arithmetic coding (CABAC) or Huffman coding.

[0116] The inverse quantization module 1011 inversely quantizes the quantized data (or quantized coefficients) 1012 to obtain transform coefficients, and the inverse transform module 1010 inversely transforms the transform coefficients 1016 to obtain a reconstructed residual signal 1019. The reconstructed residual signal 1019 is added to the predicted pixel data 1013 from the intra-frame prediction module 1025 or the motion compensation module 1030 to generate decoded pixel data 1017. The decoded pixel data is filtered by the in-loop filter 1045 and stored in the decoded picture buffer 1050. As shown in the figure, in some embodiments, the decoded picture buffer 1050 is a storage outside the video decoder 1000. In some embodiments, the decoded picture buffer 1050 is a storage inside the video decoder 1000.

[0117] The intra prediction module 1025 receives intra prediction data from the bitstream 1095 and generates prediction pixel data 1013 from decoded pixel data 1017 stored in the decoded picture buffer 1050. In some embodiments, the decoded pixel data 1017 is also stored in a line buffer (not shown) for intra prediction and spatial MV prediction.

[0118] In some embodiments, the contents of the decoded picture buffer 1050 are used for display. The display device 1055 either retrieves the contents of the decoded picture buffer 1050 for direct display or retrieves the contents of the decoded picture buffer to a display buffer. In some embodiments, the display device receives pixel values ​​from the decoded picture buffer 1050 via pixel transfer.

[0119] The motion compensation module 1030 generates predicted pixel data 1013 from decoded pixel data 1017 stored in the decoded picture buffer 1050 according to motion compensated MVs (MC MVs). These motion compensated MVs are decoded by adding residual motion data received from the bitstream 1095 to the predicted MVs received from the MV prediction module 1075.

[0120] The MV prediction module 1075 generates a predicted MV, for example, a motion compensated MV for performing motion compensation, based on a reference MV generated for decoding a previous video frame. The MV prediction module 1075 retrieves a reference MV of a previous video frame from the MV buffer 1065. The video decoder 1000 stores the motion compensated MV generated for decoding a current video frame in the MV buffer 1065 as a reference MV for generating a predicted MV.

[0121] The in-loop filter 1045 performs a filtering or smoothing operation on the decoded pixel data 1017 to reduce coding artifacts, particularly at the boundaries of pixel blocks. In some embodiments, the filtering operation performed includes sample adaptive offset (SAO). In some embodiments, the filtering operation includes an adaptive loop filter (ALF).

[0122] Fig.11 The portion of the video decoder 1000 that utilizes the candidate set reordering and size reduction proposed in the present invention to achieve efficient GPM decoding is illustrated. Specifically, the figure shows some components of the inter-frame prediction module 1040 of the video decoder 1000, which calculates various guess costs, gradient values, and RD costs, and maintains and manages various candidate sets. The inter-frame prediction module 1040 may include a processor for calculating matching costs (e.g., Figure 7 Calculator 1110 for calculating guess costs SAD1-SAD4) and gradients (e.g., gradients 511, 512, and 517) in the decoded picture buffer 1050. Calculator 1110 includes an edge detection module 1112 for detecting object edges (e.g., object edges 510, 520, and 530) using sample values ​​or auxiliary data, wherein the sample values ​​are provided by a decoded picture buffer (DPB) 1050, and the auxiliary data are provided by a template / boundary retriever 1111, which can retrieve template regions (e.g., regions 700, 710, and 750) and boundary regions (e.g., region 760) based on reference or neighboring blocks from DPB 1050 and MVs from MV buffer 1065. In some embodiments, calculator 1110 can be the same as calculator 910.

[0123] The inter prediction module 1040 may include a candidate manager 1120 for managing various candidate sets, including a merge candidate set 1121, a partition mode candidate set 1122, and a mixed width candidate set 1124. The candidate manager 1120 is also capable of generating a GPM candidate set 1123 based on the candidates in the merge candidate set 1121 and the partition mode candidate set 1122. Each candidate in the GPM candidate set 1123 represents a corresponding combination of a merge candidate of the merge candidate set 1121 and a partition mode of the partition mode candidate set 1122. The candidate manager 1120 may reorder the candidates in any one of the candidate sets 1121-1124, and reduce the size of the mixed width candidate set 1124 by excluding certain candidates from any one of the candidate sets 1121-1124, based on the calculated gradient and / or guess cost provided by the calculator 1110. The candidate manager 1120 may also assign a codeword to an index of a candidate in any of the candidate sets 1121-1124, regardless of whether the corresponding candidate set is before or after size reduction or reordering. In some embodiments, the candidate manager 1120 may receive an index of a selected or remaining candidate from the entropy decoder 1090, which is signaled from the encoder 800. Moreover, the candidate manager 1120 may send an index of the selected candidate to the motion compensation module 1030 for decoding the current block in the GPM. In some embodiments, the candidate manager 1120 may be the same as the candidate manager 920. Therefore, the size and order of the candidate sets 1121-1124 may be synchronized with the size and order of the candidate sets 921-924, respectively.

[0124] VIII. Illustrative Process

[0125] Fig.12 An example process 1200 is shown according to an implementation of the present disclosure. Process 1200 may represent aspects of implementing the various proposed designs, concepts, schemes, systems, and methods described above. More specifically, process 1200 may represent one aspect of the proposed concepts and schemes related to implementing an efficient GPM decoding process according to the present disclosure. Process 1200 may include one or more operations, actions, or functions as shown in one or more of blocks 1210, 1220, 1230, 1240, 1250, and 1260. Although shown as discrete blocks, the various blocks of process 1200 may be divided into additional blocks, combined into fewer blocks, or eliminated, depending on the desired implementation. In addition, the blocks / sub-blocks of process 1200 may be arranged in a manner that is consistent with the implementation of the present disclosure. Fig.121000, or, optionally, in a different order. In addition, one or more blocks / sub-blocks of process 1200 may be performed repeatedly or iteratively. Process 1200 may be implemented by apparatus 800 or 1000 and any variants thereof. For illustrative purposes only and without limiting the scope, process 1200 is described below in the context of apparatus 800 or 1000. Process 1200 may begin at block 1210.

[0126] At 1210, flow 1200 may involve the inter prediction module 840 or 1040 receiving pixel data for the current block. For example, the inter prediction module 840 may receive pixel data for block 255 from the video source 805 or the RPB 850. Likewise, the inter prediction module 1040 may receive pixel data for block 255 from the DPB 1050. Flow 1200 may proceed from 1210 to 1220.

[0127] At 1220, the process 1200 may involve the candidate manager 920 or 1120 selecting a partition mode (e.g., a partition mode defined by the partition line 253) from a partition mode candidate set (e.g., partition mode candidate set 922 or 1122). The partition mode candidate set may include one or more partition mode candidates. Each partition mode candidate in the partition mode candidate set is defined by a corresponding partition boundary within the current block (e.g., any partition line shown as partition line 253 or 320). For the encoder 800, the candidate manager 920 may select a partition mode by performing an RDO search within the partition mode candidate set 922. For the decoder 1000, the candidate manager 1120 may select a partition mode based on a partition mode index signaled from the encoder 800. The entropy decoder 1090 may parse the bitstream 1095 to obtain the signaled partition mode index and provide the index to the candidate manager 1120 for selection. The process 1200 may proceed from 1220 to 1230.

[0128] At 1230, the process 1200 may involve the candidate manager 920 or 1120 determining two merge candidates for the current block, wherein each of the two merge candidates corresponds to a respective GPM portion of the current block determined by the selected partition mode. For example, the partition mode represented by the partition line 253 divides the current block 255 into two GPM portions 250 and 251. For the encoder 800, the candidate manager 920 may determine the two merge candidates 205 and 215 by performing a unidirectional MCP on each of the GPM portions 250 and 251. For the decoder 1000, the candidate manager 1120 may determine the two merge candidates 205 and 215 by receiving indices of the merge candidates 205 and 215, which are signaled from the encoder 800. The parser 1090 may parse the bitstream 1095 to obtain the signaled merge candidate indices and provide the indices to the candidate manager 1120 to determine the two merge candidates. The process 1200 may proceed from 1230 to 1235.

[0129] At 1235, the process 1200 may involve the candidate manager 920 or 1120 determining a mixed width candidate from the mixed width candidate set based on the auxiliary data of the two merge candidates 205 and 215 or the current block 255. The auxiliary data may be the MV of the current block 255 or the MV of one or more neighboring blocks of the current block. The auxiliary data may also be sample values ​​of the two GPM parts 250 and 251, or indices of the two merge candidates 205 and 215. Step 1235 may also include sub-steps 1240 and 1250.

[0130] At 1240, the process 1200 may involve the candidate manager 920 or 1120 reducing the size of the mixed width candidate set by excluding one or more mixed width candidates from the mixed width candidate set (e.g., 924 or 1124) or reordering the mixed width candidate set. The candidate set has two or more mixed width candidates. For example, the candidate manager 920 or 1120 may exclude candidates based on the selected partition mode 253 and the two merge candidates 205 and 215 or auxiliary data of the current block 255. The auxiliary data may be the MV of the current block 255 or the MV of one or more neighboring blocks of the current block. The auxiliary data may also be sample values ​​of the two GPM parts 250 and 251, or indices of the two merge candidates 205 and 215. In some embodiments, the auxiliary data may be sample values ​​of a template area of ​​the current block (e.g., template area 750), or sample values ​​of a template area of ​​any one of the two merge candidates (e.g., template area 700 or 710). The template area may be determined by the template retriever 911 or 1111. The candidate manager 920 or 1120 may exclude one or more unlikely blend width candidates based on the gradient of the object edge across the current block. The object edge (e.g., edge 530) is detected by the edge detection module 912 or 1112, and the gradient is calculated by the calculator 910 or 1110. Flow 1200 may proceed from 1240 to 1250.

[0131] At 1250, the process 1200 may involve the candidate manager 920 or 1120 selecting a mix width from the mix width candidate set 924 or 1124. For the encoder 800, the candidate manager 920 may select the mix width by performing an RDO search within the mix width candidate set. Alternatively, the candidate manager 920 may select the mix width by performing a guess cost calculation for each candidate in the mix width candidate set. For the decoder 1000, the candidate manager 1120 may select the mix width by receiving an index of a selected mix width candidate, which is signaled from the encoder 800. The entropy decoder 1090 may parse the bitstream 1095 to obtain the mix width selected by the signaled index and provide the index to the candidate manager 1120 for selecting the mix width. For the decoder 1000, the candidate manager 1120 may also implicitly select the mix width based on auxiliary data without signaling. The selection of the mix width includes designating only one mix width candidate as the selected mix width. The process 1200 may proceed from 1250 to 1260.

[0132] At 1260, the process 1200 may involve the motion compensation module 830 or 1130 encoding or decoding the current block using the selected partition mode, the two merge candidates, and the selected blend width. For the encoder 800, the motion compensation module 830 may encode the current block 255 using the selected partition mode 253, the two merge candidates 205 and 215, and the selected blend width θ1. For the decoder 1000, the motion compensation module 1030 may decode the current block 255 using the selected partition mode 253, the two merge candidates 205 and 215, and the selected blend width θ1.

[0133] IX. Example Electronic System

[0134] Many of the above-described features and applications are implemented as software processes that are specified as a set of instructions recorded on a computer-readable storage medium (also referred to as a computer-readable medium). When these instructions are executed by one or more computing or processing units (e.g., one or more processors, processor cores, or other processing units), they cause the processing units to perform the actions indicated in the instructions. Examples of computer-readable media include, but are not limited to, CD-ROMs.

[0135] ROM, flash drive, random access memory (RAM) chip, hard disk drive, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc. Computer-readable media do not include carrier waves and electronic signals transmitted wirelessly or through wired connections.

[0136] In this specification, the term "software" is meant to include firmware residing in a read-only memory or an application stored in a magnetic storage device, which can be read into the memory for processing by the processor. In addition, in some embodiments, multiple software inventions can be implemented as sub-parts of a larger program while retaining different software inventions. In some embodiments, multiple software inventions can also be implemented as separate programs. Finally, any combination of separate programs that implement the software inventions described herein together are within the scope of this disclosure. In some embodiments, when the software program is installed to run on one or more electronic systems, one or more specific machine implementations for executing and performing the operations of the software program are defined.

[0137] Fig.13An electronic system 1300 is conceptually illustrated to implement some embodiments of the present disclosure. The electronic system 1300 may be a computer (e.g., a desktop calculator, a personal calculator, a tablet calculator, etc.), a phone, a PDA, or any other kind of electronic device. Such an electronic system includes various types of computer-readable media and interfaces for various other types of computer-readable media. The electronic system 1300 includes a bus 1305, a processing unit 1310, a graphics processing unit (GPU) 1315, a system memory 1320, a network 1325, a read-only memory 1330, a permanent storage device 1335, an input device 1340, and an output device 1345.

[0138] The bus 1305 collectively represents all system, peripheral, and chipset buses that communicatively connect the numerous internal devices of the electronic system 1300. For example, the bus 1305 communicatively connects the processing unit 1310 and the GPU 1315, the read-only memory 1330, the system memory 1320, and the permanent storage device 1335.

[0139] From these various memory units, the processing unit 1310 retrieves instructions to be executed and data to be processed in order to perform the processes of the present disclosure. In different embodiments, the processing unit may be a single processor or a multi-core processor. Some instructions are passed to and executed by the GPU 1315. The GPU 1315 may offload various calculations or supplement the image processing provided by the processing unit 1310.

[0140] Read-only memory (ROM) 1330 stores static data and instructions used by processing unit 1310 and other modules of the electronic system. On the other hand, permanent storage device 1335 is a read-write storage device. This device is a non-volatile storage unit that stores instructions and data even when electronic system 1300 is turned off. Some embodiments of the present disclosure use a mass storage device (such as a magnetic disk or optical disk and its corresponding magnetic disk drive) as permanent storage device 1335.

[0141] Other embodiments use removable storage devices (e.g., floppy disks, flash memory devices, etc. and their corresponding disk drives) as permanent storage devices. Like permanent storage device 1335, system memory 1320 is a read-write storage device. However, unlike storage device 1335, system memory 1320 is a volatile read-write memory, such as random access memory. System memory 1320 stores some instructions and data used by the processor at runtime. In some embodiments, processing according to the present disclosure is stored in system memory 1320, permanent storage device 1335 and / or read-only memory 1330. For example, various memory units include instructions for processing multimedia clips and some embodiments. From these different memory units, processing unit 1310 retrieves instructions to be executed and data to be processed in order to perform processing of some embodiments.

[0142] The bus 1305 is also connected to input and output devices 1340 and 1345. The input device 1340 enables a user to transmit information and select commands to the electronic system. The input device 1340 includes an alphanumeric keyboard and a pointing device (also called a "cursor control device"), a camera (e.g., a webcam), a microphone, or a similar device for receiving voice commands, etc. The output device 1345 displays images generated by the electronic system or outputs data in other ways. The output device 1345 includes a printer and a display device, such as a cathode ray tube (CRT) or a liquid crystal display (LCD), and a speaker or similar audio output device. Some embodiments include devices that are used as both input and output devices, such as a touch screen.

[0143] Finally, if Fig.13 As shown, bus 1305 also couples electronic system 1300 to a network 1325 via a network adapter (not shown). In this manner, the computer can be part of a network of computers, such as a local area network ("LAN"), a wide area network ("WAN") or an intranet, or a network of networks. Any or all components of electronic system 1300 may be used in conjunction with the present disclosure.

[0144] Some embodiments include electronic components, such as microprocessors, memory, and storage that store computer program instructions in machine-readable or computer-readable media (or alternatively referred to as computer-readable storage media, machine-readable media, or machine-readable storage media). Some examples of such computer-readable media include RAM, ROM, compact disk-read only (CD-ROM), compact disk-recordable (CD-R), compact disk-rewritable (CD-RW), digital versatile disk-read only (e.g., DVD-ROM, dual-layer DVD-ROM), various recordable / rewritable DVDs (e.g., DVD-RAM, DVD-RW, DVD+RW, etc.), flash memory (e.g., SD card, mini-SD card, micro SD card, etc.), magnetic and / or solid-state hard drives, read-only and recordable A computer readable medium may store a computer program executable by at least one processing unit and including a set of instructions for performing various operations. Examples of computer programs or computer code include machine code, such as that generated by a compiler, and files including high-level code that is executed by a computer, an electronic component, or a microprocessor using an interpreter.

[0145] Although the above discussion primarily involves microprocessors or multi-core processors executing software, many of the above functions and applications are performed by one or more integrated circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions stored on the circuits themselves. In addition, some embodiments execute software stored in programmable logic devices (PLDs), ROM, or RAM devices.

[0146] As used in this specification and any claims of this application, the terms "computer", "server", "processor" and "memory" refer to electronic or other technical devices. These terms do not include people or groups of people. For the purposes of this specification, the terms display or display means display on an electronic device. As used in this specification and any claims of this application, the terms "computer readable medium", "computer readable medium" and "machine readable medium" are entirely limited to tangible physical objects that store information in a readable form. These terms do not include any wireless signals, wired download signals, and any other temporary signals.

[0147] Although the present disclosure has been described with reference to numerous specific details, one of ordinary skill in the art will recognize that the present disclosure can be embodied in other specific forms without departing from the spirit of the present disclosure.

[0148] Additional notes

[0149] The subject matter described herein sometimes illustrates different components contained within or connected to different other components. It should be understood that the architectures depicted in this way are merely examples, and that many other architectures that implement the same functionality can actually be implemented. Conceptually, any arrangement of components that implement the same functionality is effectively "associated" to achieve the desired functionality. Therefore, any two components combined herein to achieve a particular functionality regardless of the architecture or intermediate components can be considered to be "associated" with each other to achieve the desired functionality. Similarly, any two components so associated may also be considered to be "operably connected" or "operably coupled" to each other to achieve the desired functionality, and any two components that can be so associated may also be considered to be "operably connected" or "coupled" to each other to achieve the desired functionality. Specific examples of operably coupled include, but are not limited to, physically matable and / or physically interactive components and / or wirelessly interactive and / or wirelessly interactive components and / or logically interactive and / or logically interactive components.

[0150] Furthermore, with respect to the use of substantially any plural and / or singular terms herein, those skilled in the art may translate from the plural to the singular and / or from the singular to the plural, depending on the context and / or application. For the sake of clarity, the various singular / plural arrangements may be expressly set forth herein.

[0151] Furthermore, those skilled in the art will appreciate that, in general, the terms used herein, and particularly the terms used in the appended claims, such as the bodies of the appended claims, are generally intended as "open" terms, e.g., the term "comprising" should be interpreted as "including but not limited to," and "having" should be interpreted as "at least having." Those skilled in the art will further appreciate that if a specific number of claim statements are intended to be introduced, such intent will be explicitly stated in the claim, and such intent is absent in the absence of such statement. For example, to aid understanding, the following appended claims may contain statements that use the introductory phrases "at least one" and "one or more" to introduce the claim statements. However, the use of such phrases should not be interpreted as implying that a claim introduced by the indefinite article "a" or "an" limits any particular claim that contains such introduced claim to an implementation that contains only one such statement, even when the same claim includes the introductory phrases "one or more" or "at least one" and an indefinite article such as "an," which should be interpreted as "at least one" or "one or more." The same applies to the use of definite articles to introduce claim statements. Furthermore, even if a specific number of the claims recited by the introduction is explicitly cited, a person skilled in the art will recognize that such a recitation should be interpreted as indicating at least the number of citations, e.g., a reference to "two iterations," without other modifiers, means at least two iterations, or two or more iterations. Furthermore, in those cases where the convention uses something like "at least one of A, B, and C, etc.", generally, such a construction is intended to be understood by a person skilled in the art in the sense of the convention, e.g., "a system having at least one of A, B, and C" would include, but is not limited to, systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. In those cases where the convention uses something like "at least one of A, B, or C," generally, such a construction is intended to be understood by a person skilled in the art in the sense of the convention, e.g., "a system having at least one of A, B, or C" would include, but is not limited to, systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc. Those skilled in the art will further understand that in practice, any disjunctive words and / or phrases that appear as two or more alternative terms, whether in the specification, claims or drawings, should be understood to include the possibility of one term, one term or two terms. For example, the phrase "A or B" will be understood to include the possibility of "A" or "B" or "A and B".

[0152] It can be understood from the above that various embodiments of the present disclosure have been described herein for illustrative purposes, and various modifications may be made without departing from the scope and spirit of the present disclosure. Therefore, the various embodiments disclosed herein are not intended to be limiting, and the true scope and spirit are indicated by the attached claims.

Claims

1. A video encoding and decoding method, applicable to an encoder or a decoder, comprising: Receiving pixel data to be encoded or decoded as a current block of a current picture of a video; selecting a partition mode from a partition mode candidate set comprising one or more partition mode candidates, each partition mode candidate being defined by a corresponding partition boundary within the current block; determining two merge candidates for the current block, each of the two merge candidates corresponding to a respective geometric partition mode (GPM) portion of the current block determined by the selected partition mode; determining a blend width from a set of blend width candidates based on the selected partition mode and two merge candidates or auxiliary data of the current block; as well as The current block is encoded or decoded using the selected partition mode, the two merge candidates, and the selected blend width.

2. The method according to claim 1, wherein the auxiliary data of the two merge candidates or the current block includes: A motion vector (MV) of the current block; MVs of one or more neighboring blocks of the current block; Sample values ​​of the two GPM parts; reference indices of the two merge candidates; The sample value of the template area of ​​the current block; The sample value of the template region of each of the two merge candidates; or A combination of two or more of the above.

3. The method according to claim 1, wherein determining the mixing width from the mixing width candidate set further comprises: Excluding one or more mixed width candidates from a mixed width candidate set including two or more mixed width candidates based on the selected partition mode and two merge candidates or auxiliary data of the current block; as well as A blending width is selected from the blending width candidates after the elimination step.

4. The method of claim 3, wherein excluding the one or more hybrid width candidates comprises: determining a corresponding object edge of each of the two merging candidates based on the auxiliary data of the two merging candidates; calculating, based on the auxiliary data of the two merge candidates, respective gradients of each of the two merge candidates across respective object edges; Calculating the gradient of the current block according to the gradients of the two merge candidates; as well as One or more blend width candidates are eliminated based on the gradient of the current block.

5. The method of claim 3, wherein excluding the one or more hybrid width candidates comprises: Determining an object edge within the current block according to a selected partitioning mode or auxiliary data of the current block; Calculating the gradient of the current block across the edge of the object according to the auxiliary data of the current block; and One or more blend width candidates are eliminated based on the gradient of the current block. 6 . The method of claim 5 , wherein the elimination of the one or more blend width candidates is further based on a history-based statistical model or a look-up table (LUT).

7. The method according to claim 1, wherein determining the mixing width from the mixing width candidate set further comprises: reordering the mixed width candidate set based on the two merge candidates or the auxiliary data of the current block; as well as The mixing width is selected from the mixing width candidates after the re-ranking step. 8 . The method of claim 7 , wherein reordering the mixed-width candidate set comprises computing, for each mixed-width candidate retained in the mixed-width candidate set, a respective guess cost of an associated GPM prediction.

9. The method of claim 8, wherein calculating the corresponding guess cost comprises template matching (TM) or boundary matching (BM).

10. The method according to claim 3, applied to the encoder, wherein: The selection of a partitioning scheme involves performing a rate-distortion optimization (RDO) search among a set of partitioning scheme candidates. Determining each of the two merge candidates comprises performing unidirectional motion compensated prediction (MCP) on a corresponding GPM portion of the current block, and The selection of the mixed width includes performing an RDO search within the mixed width candidate set after exclusion or performing a guess cost calculation for each candidate in the mixed width candidate set, or the selection of the mixed width is performed based on auxiliary data of the current block or two merge candidates of the current block.

11. The method according to claim 1, applied to the encoder, further comprising: Indices indicating two merge candidates, an index indicating the selected partition mode, and an index indicating the determined mix width are signaled to the decoder.

12. The method according to claim 1, applied to the encoder, further comprising: Indices indicating two merge candidates and indicating a selected partition mode are signaled to the decoder without signaling an index indicating the determined mix width.

13. The method of claim 1, applied to the decoder, wherein: The selecting of the partition mode comprises receiving an index indicating the selected partition mode from the encoder, Determining each of the two merge candidates comprises receiving from the encoder a respective index indicating one of the two merge candidates, and The determination of the mix width includes receiving an index indicating the determined mix width from the encoder.

14. The method according to claim 3, applied to the decoder, wherein: The selecting of the partition mode comprises receiving an index indicating the selected partition mode from the encoder, The determining each of the two merge candidates comprises receiving from the encoder a respective index indicating one of the two merge candidates, The mixed-width candidate set contains only one mixed-width candidate after exclusion, and The selecting of the blending width includes designating only one blending width candidate as the selected blending width.

15. The method according to claim 1, wherein the partition mode candidate set includes a first number of the partition mode candidates, and the method further comprises: constructing a GPM candidate set including a third number of GPM candidates based on the partition mode candidate set and the merge candidate set including the second number of merge candidates, each GPM candidate representing one of the partition mode candidates and a corresponding combined candidate of the two merged; and Re-rank the set of GPM candidates by calculating the guess cost of each GPM candidate, The selecting of the partition mode and the determining of the two merge candidates include selecting or determining the partition mode and the two merge candidates indicated by one of the GPM candidates having the lowest guess cost. The method of claim 15 , wherein calculating the corresponding guess cost comprises template matching (TM) or boundary matching (BM).

17. The method according to claim 15, further comprising: For one or more candidates in the reordered GPM candidate set, the mixed width candidate set is reordered based on two merged candidates or auxiliary data of the current block.

18. An electronic device comprising: The video encoding circuit is configured to perform operations including the following operations: Receiving pixel data to be encoded or decoded as a current block of a current picture of a video; selecting a partition mode from a partition mode candidate set comprising one or more partition mode candidates, each partition mode candidate being defined by a corresponding partition boundary within the current block; determining two merge candidates for the current block, each of the two merge candidates corresponding to a respective geometric partition mode (GPM) portion of the current block determined by the selected partition mode; determining a blend width from a set of blend width candidates based on the selected partition mode and auxiliary data of two merge candidates or the current block; as well as The current block is encoded or decoded using the selected partition mode, the two merge candidates, and the selected blend width.

19. The apparatus of claim 18, wherein the operation of determining the mixing width further comprises: Based on the selected partition mode and two merge candidates or auxiliary data of the current block, excluding one or more mixed width candidates from a mixed width candidate set including two or more mixed width candidates; as well as After eliminating one or more blending width candidates, a blending width is selected from the blending width candidate set.

20. The apparatus of claim 18, wherein the operation of determining the mixing width further comprises: constructing a GPM candidate set including two or more GPM candidates based on the partition mode candidate set and the merge candidate set including one or more merge candidates, each GPM candidate representing a corresponding combination of one of the partition mode candidates and one of the merge candidates; and Re-rank the GPM candidate set by calculating the guess cost of each GPM candidate; The selecting of the partition mode and the determining of the two merge candidates include selecting or determining the partition mode and the two merge candidates indicated by one of the GPM candidates having the lowest guess cost.

Images