Template Matching Based Merge Index Reordering for Geometric Partitioning Mode (GPM)

Abstract

The method includes: dividing the current block into a first partition and a second partition; dividing the template sample of the current block into a first template region adjacent to the first partition and a second template region adjacent to the second partition; determining a plurality of first candidate reference blocks for the first partition; determining a plurality of second candidate reference blocks for the second partition; and reordering at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks based on the size of the first template region of the template sample and the size of the second template region of the template sample. The current block is reconstructed based on the received index value and based on the reordered at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks.

Description

[Technical Field]

[0001] [Related Applications] This application claims the benefit of priority to U.S. Provisional Patent Application No. 63 / 416,401, entitled "Template-Matching Based Merge Index Reordering For Geometric Partition Mode (GPM)," filed October 14, 2022, which claims the benefit of priority to U.S. Patent Application No. 18 / 241,058, entitled "TEMPLATE-MATCHING BASED MERGE INDEX REORDERING FOR GEOMETRIC PARTITION MODE (GPM)," filed August 31, 2023. The disclosures of the foregoing applications are incorporated herein by reference in their entireties.

[0002] [Technical field] This disclosure generally describes embodiments related to video coding. [Background technology]

[0003] The background description provided herein is intended to provide a general background to the present disclosure. The work of the presently named inventors is not expressly or implicitly admitted as prior art to the present disclosure, to the extent that the work described in this background section, as well as aspects of the description that may not be considered prior art at the time of filing, is not admitted as prior art to the present disclosure.

[0004] Image / video compression helps transfer image / video data between various devices, storage, and networks with minimal loss of quality. In some examples, video codec techniques can compress video based on spatial and temporal redundancy. For example, video codecs can use a technique called intra-prediction, which can compress images based on spatial redundancy. For example, intra-prediction can use reference data from the current picture being reconstructed for sample prediction. In another example, video codecs can use a technique called inter-prediction, which can compress images based on temporal redundancy. For example, inter-prediction can predict samples in a current picture from a previously reconstructed picture using motion compensation. Motion compensation is commonly indicated by a motion vector (MV). Summary of the Invention

[0005] Aspects of this disclosure include methods and apparatus for video encoding / decoding. In some examples, an apparatus for video decoding includes a processing circuit.

[0006] According to an aspect of the present disclosure, a video decoding method executed in a video decoder is provided. In the method, a video bitstream including a current block in a current frame and a template sample for the current block is received. The template sample includes a top template along an upper side of the current block and a left template along a left side of the current block. The current block is divided into a first partition and a second partition, and the template sample is divided into a first template region adjacent to the first partition and a second template region adjacent to the second partition. A plurality of first candidate reference blocks is determined for the first partition, and a plurality of second candidate reference blocks is determined for the second partition. Each first candidate reference block has a respective first candidate template corresponding to a shape of the first template region, and each second candidate reference block has a respective second candidate template corresponding to a shape of the second template region. At least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks is reordered based on a size of the first template region of the template sample and a size of the second template region of the template sample. The current block is reconstructed based on the index value received in the bitstream and based on a permutation of at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks.

[0007] In one example, based on the candidate reference blocks indicated by the motion vectors (MVs) in the first reference list and the second reference list, a plurality of first candidate reference blocks of a first partition of the current block and a plurality of second candidate reference blocks of a second partition of the current block are determined.

[0008] In one example, based on one of the non-adjacent blocks and the blocks in the same-position reference frame indicated by the temporal motion vector, multiple first candidate reference blocks for a first partition of the current block and multiple second candidate reference blocks for a second partition of the current block are determined.

[0009] In one aspect, a template-matching (TM) difference between a first template region of the template sample and each of the first candidate templates is determined. A TM difference between a second template region of the template sample and each of the second candidate templates is determined. A plurality of first candidate reference blocks for the first partition are sorted based on an ascending order of the TM difference between the first template region of the template sample and the first candidate template. A plurality of second candidate reference blocks for the second partition are sorted based on an ascending order of the TM difference between the second template region of the template sample and the second candidate template.

[0010] In one embodiment, based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template, A plurality of first candidate reference blocks for the first partition are reordered based on a number of samples in a sample row of the first portion of the top template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on a number of samples in a sample row of the second portion of the top template being greater than a threshold.

[0011] In one embodiment, based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template, A plurality of first candidate reference blocks for the first partition are reordered based on widths of sample rows in the first portion of the upper template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on widths of sample rows in the second portion of the upper template being greater than a threshold.

[0012] In one embodiment, based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template, The plurality of first candidate reference blocks for the first partition are ordered based on a number of samples in the sample sequence of the first portion of the left template being greater than a threshold, and the plurality of second candidate reference blocks for the second partition are reordered based on a number of samples in the sample sequence of the second portion of the left template being greater than a threshold.

[0013] In one embodiment, based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template, A plurality of first candidate reference blocks for the first partition are reordered based on a height of a sample sequence in the first portion of the left template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on a height of a sample sequence in the second portion of the left template being greater than a threshold.

[0014] In one aspect, a TM difference between one of the unsegmented top template or the unsegmented left template and each corresponding region of the first candidate template is determined based on a first template region of the template sample including one of the unsegmented top template or the unsegmented left template and a second template region of the template sample including the other of the segmented top template or the left template. Further, the plurality of first candidate reference blocks of the first partition are sorted based on an ascending order of the TM differences between one of the unsegmented top template or the unsegmented left template and each corresponding region of the first candidate template.

[0015] In one example, a plurality of candidate partitions for a current block are determined based on a plurality of candidate partition methods. A plurality of candidate reference blocks are determined for each of the plurality of candidate partitions. A TM difference between a template sample of the current block and a template region of each of the plurality of candidate reference blocks of the plurality of candidate partitions is determined. The plurality of candidate reference blocks are sorted in ascending order based on the TM difference between the template sample of the current block and the template regions of the plurality of reference blocks for the current block.

[0016] In one example, the video bitstream includes a plurality of indices indicating a subset of a plurality of sorted candidate reference blocks, and the subset of the plurality of sorted candidate reference blocks includes the candidate reference blocks having the first N smallest TM differences.

[0017] According to another aspect of the present disclosure, an apparatus is provided, the apparatus including a processing circuit, the processing circuit being configurable to perform any of the described methods of video decoding / encoding.

[0018] Aspects of the present disclosure also provide a non-transitory computer-readable medium storing instructions that, when executed by a computer, cause the computer to perform a method for video decoding / encoding. [Brief explanation of the drawings]

[0019] Further features, characteristics, and various advantages of the disclosed subject matter will become more apparent from the following detailed description and accompanying drawings.

[0020] [Figure 1] FIG. 1 is a schematic diagram of an exemplary block diagram of a communication system (100).

[0021] [Figure 2] FIG. 2 is a schematic diagram of an exemplary block diagram of a decoder.

[0022] [Figure 3] FIG. 2 is a schematic diagram of an exemplary block diagram of an encoder.

[0023] [Figure 4] 10 illustrates an exemplary partition angle distribution for geometric partition mode (GPM) inter prediction.

[0024] [Figure 5] 1 illustrates exemplary candidate partition lines associated with partition angles of a GPM.

[0025] [Figure 6] FIG. 12 is a schematic diagram of blending at partition edges associated with a GPM.

[0026] [Figure 7] 1 illustrates a first example of merge index sorting based on template matching for GPM, according to some embodiments of the present disclosure.

[0027] [Figure 8] 10 illustrates a second example of merge index sorting based on template matching for GPM, according to some embodiments of the present disclosure.

[0028] [Figure 9] 10 illustrates a third example of merge index sorting based on template matching for GPM, according to some embodiments of the present disclosure.

[0029] [Figure 10] 1 shows a flowchart outlining a decoding process according to some embodiments of the present disclosure.

[0030] [Figure 11] 1 shows a flowchart outlining an encoding process according to some embodiments of the present disclosure.

[0031] [Figure 12] FIG. 1 is a schematic diagram of an exemplary computer system, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0032] 1 shows a block diagram of a video processing system 100 in some examples. The video processing system 100 is a video encoder and video decoder in a streaming environment, which is one example of an application of the disclosed subject matter. The disclosed subject matter is equally applicable to, for example, video conferencing, digital TV, streaming services, storing compressed video on digital media including CDs, DVDs, memory sticks, etc., other video-enabled applications, etc.

[0033] The video processing system 100 includes a video source 101, e.g., a capture subsystem 113, which may include a digital camera, that generates an uncompressed video picture stream 102. In one example, the video picture stream 102 includes samples captured by the digital camera. The video picture stream 102, shown in bold to emphasize its high data volume when compared to the encoded video data 104 (or coded video bitstream), may be processed by an electronic device 120 that includes a video encoder 103 coupled to the video source 101. The video encoder 103 may include hardware, software, or a combination thereof, and may enable or implement aspects of the disclosed subject matter, as described in more detail below. The encoded video data 104 (or coded video bitstream), shown in thin to emphasize its low data volume when compared to the video picture stream 102, may be stored on a streaming server 105 for future use. One or more streaming client subsystems, such as the client subsystems 106 and 108 of FIG. 1, can access the streaming server 105 to retrieve copies 107 and 109 of the encoded video data 104. The client subsystem 106 may include a video decoder 110, for example, within an electronic device 130. The video decoder 110 decodes the input copy 107 of the encoded video data and generates an output video picture stream 111 that can be rendered on a display 112 (e.g., a display screen) or other rendering device (not shown). In some streaming systems, the encoded video data 104, 107, and 109 (e.g., a video bitstream) may be encoded according to a particular video coding / compression standard. Examples of these standards include ITU-T Recommendation H.265. In one example, a video coding standard under development is known informally as Versatile Video Coding (VVC). The disclosed subject matter may be used in the context of VVC.

[0034] It should be noted that electronic devices 120 and 130 may include other components (not shown). For example, electronic device 120 may include a video decoder (not shown), and electronic device 130 may also include a video encoder (not shown).

[0035] 2 shows an exemplary block diagram of a video decoder (210). The video decoder (210) may be included in an electronic device (230). The electronic device (230) may include a receiver (231) (e.g., a receiving circuit). The video decoder (210) may be used in place of the video decoder (110) in the example of FIG. 1.

[0036] The receiver (231) can receive one or more coded video sequences, e.g., included in a bitstream, to be decoded by the video decoder (210). In an embodiment, one coded video sequence is received at a time, with the decoding of each coded video sequence being independent of the decoding of other coded video sequences. The coded video sequences may be received from a channel (201), which may be a hardware / software link to a storage device that stores the coded video data. The receiver (231) may receive the coded video data along with other data, e.g., coded audio data and / or auxiliary data streams, which may be forwarded to respective using entities (not shown). The receiver (231) may separate the coded video sequences from other data. To eliminate network jitter, a buffer memory (215) may be coupled between the receiver (231) and the entropy decoder / parser (220) (hereinafter, "parser (220)"). In certain applications, the buffer memory (215) is part of the video decoder (210). Alternatively, it may be external to the video decoder 210 (not shown). Still alternatively, there may be a buffer memory (not shown) external to the video decoder 210, e.g., to remove network jitter, in addition to another buffer memory 215 internal to the video decoder 210, e.g., to handle playout timing. When the receiver 231 is receiving data controllably from a store / forward device of sufficient bandwidth or from an isosynchronous network, the buffer memory 215 may not be needed or may be small. For use with best-effort packet networks such as the Internet, the buffer memory 215 may be needed, but it may be relatively large, advantageously of adaptive size, and implemented at least in part in an operating system or similar element (not shown) external to the video decoder 210.

[0037] The video decoder (210) may include a parser (220) to reconstruct symbols (221) from the coded video sequence. These symbol categories include information used to manage the operation of the video decoder (210) and information for controlling a rendering device, such as a render device (212) (e.g., a display screen), which may not be an integral part of the electronic device (230) but may be coupled to the electronic device (230) as shown in FIG. 2. The control information for the rendering device may be in the form of a Supplemental Enhancement Information (SEI) message or a Video Usability Information (VUI) parameter set fragment (not shown). The parser (220) may parse / entropy decode the received coded video sequence. The coding of the coded video sequence may follow a video coding technique or standard and may follow various principles, including variable length coding, Huffman coding, arithmetic coding with or without context dependency, etc. The parser (220) may extract from the coded video sequence a set of subgroup parameters for at least one of the subgroups of pixels in the video decoder based on at least one parameter corresponding to the group. The subgroups may include groups of pictures (GOPs), pictures, tiles, slices, macroblocks, coding units (CUs), blocks, transform units (TUs), prediction units (PUs), etc. The parser (220) may also extract information such as transform coefficients, quantization parameter values, motion vectors, etc. from the coded video sequence.

[0038] The parser (220) may perform entropy decoding / parsing operations on the video sequence received from the buffer memory (215) to generate symbols (221).

[0039] The reconstruction of the symbols (221) may include several different units, depending on the type of coded video picture or portion thereof (e.g., inter- and intra-picture, inter- and intra-block) and other factors. Which units are included and how can be controlled by group control information parsed by the parser (220) from the coded video sequence. The flow of such subgroup control information between the parser (220) and the following units is not shown for clarity.

[0040] Beyond the functional blocks already mentioned, the video decoder (210) can be conceptually subdivided into a number of functional units, as described below. In an actual implementation operating under commercial constraints, many of these units will interact closely with each other and may be at least partially integrated with each other. However, for purposes of describing the disclosed subject matter, the following conceptual subdivision into functional units is appropriate.

[0041] The first unit is a scalar / inverse transform unit 251. The scalar / inverse transform unit (251) receives quantized transform coefficients and control information from the parser (220) as symbols (221), including which transform to use, block size, quantization coefficients, quantization scaling matrices, etc. The scalar / inverse transform unit (251) can output blocks containing sample values ​​that can be input to an aggregator (255).

[0042] In some cases, the output samples of the scaler / inverse transform unit (251) may relate to intra-coded blocks. Intra-coded blocks are blocks that do not use prediction information from a previously reconstructed picture but can use prediction information from a previously reconstructed portion of the current picture. Such prediction information can be provided by the intra-picture prediction unit (252). In some cases, the intra-picture prediction unit (252) generates a block of the same size and shape as the block being reconstructed using surrounding, already reconstructed information fetched from the current picture buffer (258). The current picture buffer (258), for example, buffers the reconstructed current picture partially and / or completely. The aggregator (255), in some cases, adds the prediction information generated by the intra-prediction unit (252) to the output sample information provided by the scaler / inverse transform unit (251) on a sample-by-sample basis.

[0043] In other cases, the output samples of the scaler / inverse transform unit (251) may relate to an inter-coded, possibly motion-compensated, block. In such cases, the motion-compensated prediction unit (253) can access the reference picture memory (257) to fetch samples used for prediction. After motion-compensating the fetched samples according to the symbols (221) associated with the block, these samples may be added by the aggregator (255) to the output of the scaler / inverse transform unit (251) to generate output sample information (in this case, referred to as residual samples or residual signals). The addresses in the reference picture memory (257) from which the motion-compensated prediction unit (253) fetches prediction samples can be controlled by the motion-compensated prediction unit (253)'s available motion vectors, e.g., in the form of symbols (221) that may have X, Y, and reference picture components. Motion compensation may include interpolation of sample values ​​fetched from the reference picture memory (257) when sub-sample accurate motion vectors are in use, motion vector prediction mechanisms, etc.

[0044] The output samples of the aggregator (255) may undergo various loop filtering techniques in a loop filter unit (256). Video compression techniques may include in-loop filtering techniques controlled by parameters contained in the coded video sequence (also called the coded video bitstream) and made available to the loop filter unit (256) as symbols (221) from the parser (220). Video compression may not only be responsive to previously reconstructed, loop-filtered sample values, but may also be responsive to meta-information obtained during the decoding of previous portions (in decoding order) of the coded picture or coded video sequence.

[0045] The output of the loop filter unit (256) may be a sample stream that can be output to a render device (212) and stored in a reference picture memory (257) for use in future inter-picture prediction.

[0046] Once a particular coded picture is fully reconstructed, it can be used as a reference picture for future prediction. For example, once the coded picture corresponding to the current picture is fully reconstructed and the coded picture is identified as a reference picture (e.g., by the parser (220)), the current picture buffer (258) can become part of the reference picture memory (257), and a fresh current picture buffer can be reallocated before beginning reconstruction of a subsequent coded picture.

[0047] The video decoder 210 may perform decoding operations in accordance with a standard or predetermined video compression technology, such as ITU-T Rec. H.265. The coded video sequence may conform to the syntax specified by the video compression technology or standard in use, in the sense that the coded video sequence conforms to both the video compression technology or standard and a profile documented in the video compression technology or standard. Specifically, a profile may select certain tools from the full set of tools available in the video compression technology or standard as tools usable only under the profile. Compliance may also require that the complexity of the coded video sequence be within limits defined by the level of the video compression technology or standard. In some cases, the level may limit the maximum picture size, maximum frame rate, maximum reconstruction sample rate (e.g., measured in megasamples per second), maximum reference picture size, etc. The limits set by the level may, in some cases, be further constrained through a Hypothetical Reference Decoder (HRD) specification and metadata for HRD buffer management signaled within the coded video sequence.

[0048] In embodiments, the receiver (231) may receive additional (redundant) data along with the coded video. The additional data may be included as part of the coded video sequence. The additional data may be used by the video decoder (210) to correctly decode the data and / or to more accurately reconstruct the original video data. The additional data may be in the form of, for example, temporal, spatial, or signal-to-noise ratio (SNR) enhancement layers, redundant slices, redundant pictures, forward error correction codes, etc.

[0049] 3 shows an exemplary block diagram of a video encoder (303). The video encoder (303) is included in an electronic device (320). The electronic device (320) includes a transmitter (340) (e.g., a transmitting circuit). The video encoder (303) can be used in place of the video encoder (103) in the example of FIG. 1.

[0050] The video encoder (303) may receive video samples from a video source (301) (which, in the example of FIG. 3, is not part of the electronic device (320)) that may capture video images to be coded by the video encoder (303). In another example, the video source (301) is part of the electronic device (320).

[0051] The video source (301) may provide a source video sequence to be coded by the video encoder (303) in the form of a digital video sample stream of any suitable bit depth (e.g., 8-bit, 10-bit, 12-bit, ...), any color space (e.g., BT.601 Y CrCB, RGB, ...), and any suitable sampling structure (e.g., YCrCb 4:2:0, YCrCb 4:4:4). In a media presentation system, the video source (301) may be a storage device that stores previously prepared video. In a video conferencing system, the video source (301) may be a camera that captures local image information as a video sequence. The video data may be provided as multiple individual pictures that, when viewed sequentially, give the appearance of motion. The pictures themselves may be organized as a spatial array of pixels. Each pixel may contain one or more samples, depending on the sampling structure, color space, etc., in use. The following discussion focuses on samples.

[0052] According to an embodiment, the video encoder (303) may code and compress pictures of a source video sequence into a coded video sequence (343) in real time or under any other required time constraints. Enforcing the appropriate coding rate is one function of the controller (350). In some embodiments, the controller (350) controls and is operatively coupled to other functional units, described below. The coupling is not shown for clarity. Parameters set by the controller (350) may include rate control-related parameters (picture skip, quantizer, lambda value for rate-distortion optimization techniques, ...), picture size, group of pictures (GOP) layout, maximum motion vector search range, etc. The controller (350) may be configured with other appropriate functionality associated with the video encoder (303) optimized for a particular system design.

[0053] In some embodiments, the video encoder (303) is configured to operate within a coding loop. As a highly simplified explanation, in one example, the coding loop may include a source coder (330) (e.g., responsible for generating symbols, such as a symbol stream, based on an input picture to be coded and a reference picture) and a (local) decoder (333) built into the video encoder (303). The decoder (333) reconstructs the symbols to create sample data in a manner similar to that created by a (remote) decoder. The reconstructed sample stream (sample data) is input to a reference picture memory (334). When decoding the symbol stream yields bit-exact results independent of the decoder location (local or remote), the contents of the reference picture memory (334) are also bit-exact between the local and remote encoders. In other words, the predictive portion of the encoder "sees" exactly the same sample values ​​as the decoder "sees" when using prediction during decoding. This basic principle of reference picture synchronism (and the resulting drift when synchronism cannot be maintained, for example due to channel errors) is similarly used in several related techniques.

[0054] The operation of the "local" decoder (333) may be the same as a "remote" decoder, such as the video decoder (210) described in detail above in connection with Figure 2. However, and referring briefly to Figure 2 as well, the entropy decoding portion of the video decoder (210), including the buffer memory (215) and parser (220), may not be fully implemented in the local decoder (333) because symbols are available and the encoding / decoding of the symbols into a coded video sequence by the entropy coder (345) and parser (220) may be lossless.

[0055] In embodiments, decoder techniques, excluding analysis / entropy decoding, present in a decoder are present in the corresponding encoder in the same or substantially the same functional form. Therefore, the subject matter of the described disclosure focuses on decoder operation. A description of the encoder techniques can be omitted, as they are the reverse of the decoder techniques, which are described generically. In certain areas, more detailed descriptions are provided below.

[0056] In operation, in some examples, the source coder (330) may perform motion-compensated predictive coding, which predictively codes an input picture with reference to one or more previously coded pictures from a video sequence designated as "reference pictures." In this method, the coding engine (332) codes differences between pixel blocks of the input picture and pixel blocks of reference pictures that may be selected as prediction references for the input picture.

[0057] The local video decoder (333) may decode coded video data of pictures that may be designated as reference pictures based on symbols generated by the source coder (330). The operation of the coding engine (332) may advantageously be lossy. When the coded video data is decoded in a video decoder (not shown in FIG. 3), the reconstructed video sequence may typically be a copy of the source video sequence with some errors. The local video decoder (333) may replicate the decoding process that may be performed by a video decoder on the reference pictures, resulting in reconstructed reference pictures to be stored in the reference picture memory (334). In this way, the video encoder (303) may store copies of reconstructed reference pictures that have content in common with reconstructed reference pictures obtained by a far-end video decoder (absent transmission errors).

[0058] The predictor (335) may perform a predictive search for the coding engine (332). That is, for a new picture to be coded, the predictor (335) may search the reference picture memory (334) for sample data (such as candidate reference pixel blocks) or specific metadata such as reference picture motion vectors, block shapes, etc. that can serve as appropriate prediction references for the new picture. The predictor (335) may operate on a sample block-pixel block basis to find an appropriate prediction reference. In some examples, an input picture may have prediction references drawn from multiple reference pictures stored in the reference picture memory (334), as determined by the search results obtained by the predictor (335).

[0059] The control unit (350) may manage the coding operations of the source coder (330), including, for example, setting parameters and subgroup parameters used for encoding the video data.

[0060] The output of all of the aforementioned functional units may undergo entropy coding in an entropy coder (345), which converts the symbols produced by the various functional units into a coded video sequence by applying lossless compression to the symbols according to techniques such as Huffman coding, variable length coding, arithmetic coding, etc.

[0061] The transmitter (340) may buffer the coded video sequence produced by the entropy coder (345) for transmission over a communication channel (360), which may be a hardware / software link to a storage device that may store the coded video data. The transmitter (340) may merge the coded video data from the video encoder (303) with other data to be transmitted, such as coded audio data and / or auxiliary data streams (sources not shown).

[0062] The controller (350) may manage the operation of the video encoder (303). During coding, the controller (350) may assign a particular coded picture type to each coded picture, which may affect the coding technique that may be applied to each picture. For example, pictures may often be assigned as one of the following picture types:

[0063] Intra-pictures (I-pictures) may be coded and decoded without using any other picture in the sequence as a source of prediction. Some video codecs allow different types of intra-pictures, including, for example, Independent Decoder Refresh (IDR) pictures.

[0064] A predictive picture (P-picture) may be a picture that can be coded and decoded using intra- or inter-prediction, in most cases using motion vectors and reference indices to predict the sample values ​​of each block.

[0065] A Bi-directionally Predictive Picture (B Picture) may be coded and decoded using intra- or inter-prediction, using two motion vectors and reference indices to predict the sample values ​​of each block. Similarly, a multi-predictive picture can use more than two reference pictures and associated metadata for the reconstruction of a single block.

[0066] A source picture is generally spatially subdivided into multiple sample blocks (e.g., blocks of 4x4, 8x8, 4x8, or 16x16 samples each) and may be coded block by block. Blocks may be predictively coded with reference to other (already coded) blocks determined by the coding assignment applied to each picture of the block. For example, blocks of an I-picture may be non-predictively coded, or they may be predictively coded with reference to already coded blocks of the same picture (spatial prediction or intra-prediction). Pixel blocks of a P-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one previously coded reference picture. Blocks of a B-picture may be predictively coded via spatial prediction or via temporal prediction with reference to one or two previously coded reference pictures.

[0067] The video encoder (303) may perform coding operations according to a predetermined video coding technique or standard, such as ITU-T Rec. H.265. In doing so, the video encoder (303) may perform various compression operations, including predictive coding operations that exploit temporal and spatial redundancy in the input video sequence. The coded video data may therefore conform to a syntax specified by the video coding technique or standard being used.

[0068] In one embodiment, the transmitter (340) may transmit additional data along with the coded video. The source coder (330) may include such data as part of the coded video sequence. The additional data may include temporal / spatial / SNR enhancement layers, other forms of redundant data such as redundant pictures and slices, SEI messages, VUI parameter set fragments, etc.

[0069] Video may be captured as multiple source pictures (video pictures) in a time sequence. Intra-picture prediction (sometimes abbreviated as intra-prediction) exploits spatial correlation within a given picture, while inter-picture prediction exploits correlation (temporal or other) between pictures. In one example, a particular picture being encoded / decoded is called the current picture and is partitioned into blocks. When a block in the current picture is similar to a reference block in a previously coded and still buffered reference picture in the video, the block in the current picture can be coded by a vector called a motion vector. A motion vector points to a reference block within the reference picture and may have a third dimension that identifies the reference picture if multiple reference pictures are in use.

[0070] In some embodiments, bi-prediction techniques can be used in inter-picture prediction. Bi-prediction techniques use two reference pictures, such as a first reference picture and a second reference picture, both of which precede the current picture in the video in decoding order (but may be past and future, respectively, in display order). A block in the current picture can be coded with a first motion vector that points to a first reference block in the first reference picture and a second motion vector that points to a second reference block in the second reference picture. A block can be predicted by a combination of the first and second reference blocks.

[0071] Furthermore, merge mode techniques can be used in inter-picture prediction to improve coding efficiency.

[0072] According to some embodiments of the present disclosure, prediction, such as inter-picture prediction and intra-picture prediction, is performed in units of blocks. For example, according to the HEVC standard, pictures in a video picture sequence are partitioned into coding tree units (CTUs) for compression. CTUs within a picture have the same size, such as 64x64 pixels, 32x32 pixels, or 16x16 pixels. Typically, a CTU includes three coding tree blocks (CTBs): one luma CTB and two chroma CTBs. Each CTU can be recursively quadtree-decomposed into one or more coding units (CUs). For example, a 64x64 pixel CTU can be partitioned into one CU of 64x64 pixels, four CUs of 32x32 pixels, or 16 CUs of 16x16 pixels. In one example, each CU is analyzed to determine the CU's prediction type, such as an inter-prediction type or an intra-prediction type. A CU is divided into one or more prediction units (PUs) depending on temporal and / or spatial predictability. Typically, each PU includes a luma prediction block (PB) and two chroma PBs. In one embodiment, prediction operations in coding (encoding / decoding) are performed in units of prediction blocks. Using a luma prediction block as an example of a prediction block, the prediction block includes a matrix of values ​​(e.g., luma values) for pixels, such as 8x8 pixels, 16x16 pixels, 8x16 pixels, 16x8 pixels, etc.

[0073] It is noted that the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using any suitable technology. In one embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more integrated circuits. In another embodiment, the video encoders (103) and (303) and the video decoders (110) and (210) may be implemented using one or more processors executing software instructions.

[0074] The present disclosure includes aspects related to reordering merge indexes of merge MVs associated with partitions in a geometric partition mode (GPM) based on template matching processing.

[0075] GPM, such as in VVC, can be applied to inter prediction. GPM is only applicable to CUs of 8x8 or larger. GPM can be signaled, for example, using a CU-level flag and can be treated as a merge mode. Other merge modes, such as in VVC, include normal merge mode, merge motion vector differences (MMVD) mode, combined inter intra prediction (CIIP) mode, and sub-block merge mode.

[0076] When GPM mode is used, a CU can be evenly divided into two geometrically shaped partitions using one of 64 different partitioning methods, distinguished by 24 angles (e.g., non-uniform, quantized between 0° and 360°) and up to four edges relative to the center of the CU. FIG. 4 shows exemplary supported angles in VVC. FIG. 5 shows possible (or candidate) partition edges for an angle with index 3. For example, four partition edges can be provided for the angle with index 3. Each geometric partition of a CU can be inter-predicted using its own motion vector. Uni-prediction may only be allowed for each partition. That is, each partition can have one motion vector and one reference index. Uni-prediction motion constraints can ensure that only two motion-compensated predictions are required for each CU, as in traditional bi-prediction.

[0077] If GPM is currently used for the CU, a signal indicating the geometric partition index and two merge indices (one per partition) can be further signaled. The maximum GPM candidate size value can be explicitly signaled at the slice level, specifying the syntax binarization of the GPM merge index. After each of the two geometric partitions is predicted, the sample values ​​along the geometric partition edges can be further adjusted using a blending process with adaptive weights. The blending strength of the GPM can be shown in Figure 6. As shown in Figure 6, the blending strength (or blending region width) can be fixed for all different contents. The weighting value of the blending mask can be given by the ramp function of equation (1) as follows:

number

number

[0078] The blended result can be signaled as a prediction signal for the entire CU, and further transformation and quantization processes can be applied to the entire CU, just like in other prediction modes. Furthermore, the motion field of the CU predicted using GPM can be stored.

[0079] As in VVC, the motion information of a CU can be stored in 4x4 units. The stored motion information can be used for MV prediction and merge list construction for the next coded CU. In GPM, three types of motion information can be stored spanning 4x4 units. The stored motion information can include two geometric partitions P0 and P1 and unidirectional MVs of the two geometric partitions. The blending region between P0 and P1 can be predicted using motion information from these two partitions. This allows the motion information of GPM to be stored according to the partition.

[0080] The motion information of the GPM can signal the merge mode. To avoid additional memory bandwidth access, uniprediction is only allowed for each partition of the GPM. However, regular merge candidates can be unipredictive or bipredictive and cannot be directly used as a GPM merge list. To minimize implementation complexity, an index parity-based method is proposed to directly extract GPM merge candidates from the regular merge list without pruning. For example, for candidates with even values ​​of the GPM merge index, MV0 from reference list 0 with the corresponding regular merge index can be used as a GPM merge candidate. If MV0 is unavailable, MV1 from reference list 1 can be used instead. Conversely, for odd values ​​of the GPM merge index, MV1 can be selected as the default GPM merge candidate.

[0081] To further improve the compression efficiency of the VVC standard, JVET-U0100 and EE2 were conducted between the 21st and 22nd JVET meetings and were planned to evaluate advanced compression tools beyond the capabilities of VVC. JVET-U0100 and EE2 proposed template matching (TM) to refine motion on the decoder side. In TM mode, motion can be refined by constructing a template from neighboring reconstruction samples to the left and above, and finding the closest match between the template in the current picture and the corresponding template in the reference frame.

[0082] In some embodiments, the TM can be applied to a GPM such as JVET-V117. When a CU is coded with the GPM, it can be determined whether each motion of a geometric partition is refined using the TM. If the TM is selected, a template is constructed using neighboring samples to the left and above, and the motion is refined by finding the best match between the current template and a reference region with the same template pattern in the reference frame. The refined motion can be used to perform motion compensation of the geometric partition and can be further stored in a motion field.

[0083] As in ECM, template matching (TM) can be used in GPM to sort the GPM split modes according to the TM cost of each GPM split mode. However, the two merge indexes signaled in the GPM may be configured in a fixed order. The signal cost of these two merge indexes may be high.

[0084] In this disclosure, merge indexes associated with partitions of a GPM may be sorted based on the TM cost of the merge indexes according to a template matching process.

[0085] In one aspect, for a given geometric partitioning mode, the geometric partitioning candidate list is sorted by using the template matching cost of available templates that are reconstruction templates above and / or to the left of the current CU for all possible MVs from the GPM merge candidates, in ascending order of TM cost.

[0086] In one embodiment, for a given geometric partitioning mode, the candidate list of geometric partitions defined by the geometric partitioning mode can be sorted by applying the template matching (TM) costs of the available templates of the current CU to all possible MVs of the GPM merge candidates, and using the TM costs in ascending order. The available templates of the current CU can be reconstructed templates above and / or to the left of the current CU.

[0087] In one embodiment, as shown in FIG. 7, the top template is divided into two partitions, TT P0 and T.T. P1 It can be divided into TT P0 The left template is used to sort the merge indexes of the geometric partition P0 using the TM cost in ascending order, and the left template is used to sort the merge indexes of the geometric partition P0 using the TM cost in ascending order. P1 is used to sort the merge index for partition P1.

[0088] In one example, as shown in Figure 7, a current CU (702) can be partitioned into a first partition P0 and a second partition P1 according to a GPM division edge (or partition line) (704). The current CU (702) can have a top template (706) along the top side of the current CU (702) and a left template (708) along the left side of the current CU (702). The top template (706) is divided into two partitions TT by the partition line (704). P0 (or the first part) and TT P1 (or second part). TT P0The left template can be used as the first template region to sort the merge indexes of the geometric partition P0 using the TM cost in ascending order, and the TT P1 can be used as a second template region to sort merge indexes of partition P1. The merge indexes of partition P0 can indicate candidate MVs of partition P0. Each of the candidate MVs of partition P0 can indicate a first candidate reference block of each of partition P0. The merge indexes of partition P1 can indicate candidate MVs of partition P1. Each of the candidate MVs of partition P1 can indicate a first candidate reference block of each of partition P1. Each merge index can correspond to a candidate MV (or candidate reference block). The candidate MVs (or candidate reference blocks) can be sorted based on the TM cost. For example, TT P0 And the candidate MVs (or candidate reference blocks) of partition P0 can be reordered based on the TM cost (or TM difference) between the left template (708) and the corresponding templates of the candidate reference blocks of partition P0.

[0089] In one aspect, all MVs from Reference List 0 and Reference List 1 of normal merge candidates can be used as MVs of geometric partition for merge index reordering. All MVs are sorted in ascending order using TM cost, and only the N candidates with the smallest TM cost are used and signaled in the GPM merge list with merge index reordering using TM cost in ascending order.

[0090] In one example, all MVs from Reference List 0 (L0) and Reference List 1 (L1) of normal merge candidates can be used as MVs of the geometric partition for merge index reordering. The MVs from L0 and / or L1 can be sorted in an order, such as ascending, using TM costs. The TM costs can indicate the difference between the template of the GPM partition and the template of the MV corresponding to the GPM partition. In one example, only the N candidates with the smallest TM costs can be used and signaled in a GPM merge list with the reordered merge index using ascending TM costs.

[0091] In one aspect, MVs from non-adjacent blocks and temporal motion vectors from co-located reference frames can be used as MVs for merge candidate list reordering of GPMs. In one example, MVs from non-adjacent blocks and / or temporal motion vectors from co-located reference frames can be used as MVs for merge candidate list reordering of GPMs.

[0092] In one embodiment, a template matching based reordering method, such as adaptive reordering of merge candidates with template matching (ARMC-TM), is first applied to the merge candidate list, and the GPM candidate derivation process is performed based on the reordered merge candidate list.

[0093] In one embodiment, a sorting based on template matching may be first applied to the merge candidate list, and the derivation of GPM candidates may be based on the sorted merge candidate list.

[0094] In one aspect, the construction of the GPM merge candidate list for one partition (e.g., P0) can be based on the sorted merge candidate list, while other partitions (e.g., P1) can continue to use the pre-sorted candidate list based on the TM.

[0095] In one example, the construction of a GPM merge candidate list for one partition (e.g., P0) can be based on a sorted merge candidate list, while other partitions (e.g., P1) can continue to use the unsorted candidate list based on the TM.

[0096] In one aspect, for a given geometric partitioning mode, if the corresponding template size is smaller than a threshold S, which is a non-negative value, merge index reordering by using template matching for the geometric partition is not allowed. Figures 8 and 9 show two examples where template matching cannot be applied to templates in geometric partition P1 when the threshold S is 4. In Figures 8 and 9, each square in the template represents a pixel unit.

[0097] In one embodiment, for a given geometric partitioning mode, merge index reordering by using template matching is not allowed for a geometric partition if the template size of the template for the geometric partition is smaller than a threshold S. The threshold S may be a non-negative value such as 0 or other positive integer (e.g., 4).

[0098] The constraints on merge index sorting by using TM can be shown, for example, in Figures 8 and 9. Figures 8 and 9 show two examples in which template matching cannot be applied to templates in geometric partition P1 when the threshold S is 4. In Figures 8 and 9, each square in the template can represent a pixel unit.

[0099] As shown in FIG. 8 , the current CU (802) may include a top template (806) and a left template (808). The current CU (802) may be partitioned into a first partition P0 and a second partition P1 by a GPM partition edge (or partition line) 804. The top template (806) may also be partitioned into first portions (806A) and (806B). The first portions (806A) of the top template (806) and left template (808) may function as a template (or a first template region) for the first partition P0, and the second portion (806B) of the top template (806) may function as a template (or a second template region) for the second partition P1. Because the size of the second portion (806B) is smaller than a threshold (e.g., 4), template matching may not be applied to the template for the geometric partition P1.

[0100] As shown in FIG. 9 , the current CU (902) may include a top template (906) and a left template (908). The current CU (902) may be partitioned into a first partition P0 and a second partition P1 by a GPM division edge (or partition line) 904. The left template (908) may also be partitioned into a first portion (908A) and a second portion (908B). The first portion (908A) and the top template (906) may function as a template (or a first template region) for the first partition P0, and the second portion (908B) may function as a template (or a second template region) for the second partition P1. Because the size of the second portion (908B) is smaller than a threshold (e.g., 4), template matching may not be applied to the template for the geometric partition P1.

[0101] In one aspect, the template size for a given geometric partitioning mode refers to the total number of samples in the corresponding template size of the geometric partition. In one aspect, the template size for a given geometric partitioning mode refers to the width or height of the corresponding top or left template, regardless of the number of rows in the top template or the number of columns in the left template.

[0102] In one example, the template size may indicate the total number of samples in a corresponding template of a geometric partition. For example, as shown in FIG. 8, the first portion (806A) of the top template (806) may have a template size between 7 and 8, and the second portion (806A) of the top template (806) may have a template size less than 1.

[0103] In one example, the template size may indicate the width or height of the corresponding top template (e.g., 806) or left template (e.g., 808), regardless of the number of rows in the top template or the number of columns in the left template. For example, as shown in FIG. 8, the width of the first portion 806A of the top template 806 may indicate the template size of the first portion 806A. In FIG. 9, the height of the first portion 908A of the left template 908 may indicate the template size of the first portion 908A.

[0104] In one aspect, template matching is not allowed if one or more of the following conditions are true: (1) The size of the inner row of the top template or the inner column of the left template of the corresponding geometric partition is less than or equal to a threshold. (2) The size of one of the rows in the top template or one of the columns in the left template of the corresponding geometric partition is less than or equal to a threshold. (3) The size of all rows in the top template or all columns in the left template of the corresponding geometric partition is less than or equal to a threshold.

[0105] In one example, template matching may not be allowed if one or more of the following conditions are true: (1) the size of an inner line (or inner row) of the top template or an inner column of the left template of the corresponding geometric partition is less than or equal to a threshold; (2) the size of one of the lines (or rows) in the top template of the corresponding geometric partition or one of the columns in the left template is less than or equal to a threshold; or (3) the sizes of all lines (or rows) in the top template of the corresponding geometric partition or all columns in the left template are less than or equal to a threshold.

[0106] In one aspect, the construction of the GPM merge list is only applicable to partitions that have at least one template with the full size of the block width or height (unaffected by GPM partitioning), and only the full template is used for template matching-based sorting of the candidates of the corresponding partition. In one aspect, if a partition does not have a full neighboring template, the corresponding GPM candidate list cannot be sorted.

[0107] In one example, the construction of the GPM merge list can be applied only to partitions that have at least one template with a full size, i.e., the width or height of the template is full and is not affected (or split) by the GPM partitioning. Therefore, only full templates are used for sorting candidates based on template matching for the corresponding partition.

[0108] In one example, if a partition does not have a complete neighborhood template, the GPM candidate list corresponding to the partition cannot be reordered.

[0109] In one aspect, the GPM merge list configuration for geometric partitions where template matching is not applicable remains the same as the GPM merge list configuration in VVC or current ECM software.

[0110] In one example, if the GPM merge list configuration for a geometric partition to which template matching is not applicable is not applicable, the GPM merge list configuration remains the same as the GPM merge list configuration in the associated codec (or software), such as VVC or ECM software.

[0111] In one aspect, template matching is applied to all possible combinations of geometric partitioning modes and two (or more) merge indices of two (or more) geometric partitions, and then to an index indicating which geometric partitioning mode and motion vector from the GPM merge list is to be used, sorted based on ascending template matching cost.

[0112] In one embodiment, template matching can be applied to all possible combinations of (i) a geometric partitioning mode and (ii) two (or more) merge indices for two (or more) geometric partitions determined by the geometric partitioning mode. For example, a GPM can have 64 partitioning modes, each partitioning mode can have two geometric partitions, and each geometric partition can have N merge indices. Thus, the number of merge index combinations can be equal to 64×2×N. Each merge index can further correspond to each merge candidate (or reference block). The merge indices can be sorted based on template matching cost in a predefined order (e.g., ascending order) and can indicate which geometric partitioning modes and motion vectors (indicated by the merge indices) from the GPM merge list can be used.

[0113] In one aspect, the GPM merge candidates may be, but are not limited to, regular merge candidates, non-adjacent candidates, TMVP candidates, etc. In one example, the GPM merge candidates may be, but are not limited to, regular merge candidates, non-adjacent candidates, temporal motion vector prediction (TMVP) candidates, etc.

[0114] In one aspect, only the n indices with the smallest TM cost are selected, where n is a non-zero value and is less than the number of all possible combinations of geometric partition modes and two merge indices for two geometric partitions. In one aspect, the value of n can be signaled in a high-level syntax such as a VPS, SPS, PPS, slice header, etc. In one example, an index is signaled to indicate which of the n indices is used to reconstruct the current block.

[0115] In one aspect, instead of signaling both a GPM merge index and a GPM split mode, only one GPM index is signaled, but this signaled GPM index includes information about which GPM merge candidates are used for each GPM partition and which GPM split mode is used for the current block. In one example, instead of signaling both a GPM merge index and a GPM split mode, one GPM index can be signaled.

[0116] 10 shows a flowchart outlining a process (1000) according to one embodiment of the present disclosure. The process (1000) may be used in a video decoder. In various embodiments, the process (1000) is performed by a processing circuit, such as a processing circuit performing the functions of the video decoder (110), a processing circuit performing the functions of the video decoder (210), or the like. In some embodiments, the process (1000) is implemented by software instructions, and thus, the processing circuit performs the process (1000) when it executes the software instructions. The process begins at (S1001) and proceeds to (S1010).

[0117] At (S1010), a video bitstream including a current block in a current frame and template samples for the current block is received. The template samples include a top template along an upper side of the current block and a left template along a left side of the current block. The current block is divided into a first partition and a second partition, and the template sample is divided into a first template region adjacent to the first partition and a second template region adjacent to the second partition.

[0118] For example, for a given geometric partitioning mode, for all possible MVs from the GPM merge candidates, the template matching costs of the available templates that are reconstruction templates above the current CU (e.g., top template) and / or to the left of the current CU (e.g., left template) are used to sort the geometric partition candidate list by using the TM cost in ascending order. Furthermore, as shown in Figure 7, the top template is divided into two partitions, TT P0 and T.T. P1 It can be divided into TT P0 and the left template is used as the first template region to sort the merge indexes of the geometric partition P0 using the TM cost in ascending order; P1 is used as the second template region to sort the merge indexes of partition P1.

[0119] At (S1020), a plurality of first candidate reference blocks are determined for the first partition, and a plurality of second candidate reference blocks are determined for the second partition, each of the first candidate reference blocks having a respective first candidate template corresponding to the shape of the first template region, and each of the second candidate reference blocks having a respective second candidate template corresponding to the shape of the second template region.

[0120] For example, all MVs from Reference List 0 and / or Reference List 1 of a normal merge candidate can be used as MVs of the geometric partition for merge index reordering. All MVs are sorted using TM cost in ascending order, and only the N candidates with the smallest TM costs are used and signaled in the GPM merge list with merge index reordering using TM cost in ascending order. All MVs from Reference List 0 (L0) and Reference List 1 (L1) of a normal merge candidate can be used as MVs of the geometric partition for merge index reordering. The MVs from L0 and / or L1 can be sorted in an order, such as ascending order, using TM cost. The TM cost can indicate the difference between the template of the GPM partition and the template of the MV corresponding to the GPM partition.

[0121] At (S1030), at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks is rearranged based on the size of the first template region of the template sample and the size of the second template region of the template sample.

[0122] For example, for a given geometric partitioning mode, if the corresponding template (e.g., template region) size is smaller than a threshold S, where S is a non-negative value, merge index sorting using template matching for the geometric partition is not permitted. Figures 8 and 9 show two examples in which template matching cannot be applied to the template of geometric partition P1 when the threshold S is 4. In Figures 8 and 9, each square in the template represents a pixel unit. For example, as shown in Figure 8, the current CU (802) can be partitioned into a first partition P0 and a second partition P1 by a GPM partition edge (or partition line) 804. The top template (806) can also be partitioned into first portions 806A and 806B. The first portion 806A of the top template (806) and the left template (808) can correspond to the first template region of the first partition P0, and the second portion 806B of the top template (806) can correspond to the second template region of the second partition P1. Because the size of the second portion (806B) is less than a threshold (eg, 4), template matching may not be applied to the template of geometric partition P1.

[0123] At (S1040), the current block is reconstructed based on the index values ​​received in the bitstream and based on a reordering of at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks. In an example, all MVs are sorted using ascending TM cost, and only the N candidates with the smallest TM costs are used and signaled in a GPM merge list with reordering of merge index values ​​(e.g., indicating index values) using ascending TM cost. In one example, instead of signaling both a GPM merge index and a GPM partition mode, a single GPM index can be signaled. The signaled single GPM index can include information about which GPM merge candidate is used for each GPM partition and which GPM partition mode is used for the current block.

[0124] In one example, based on the candidate reference blocks indicated by the motion vectors (MVs) in the first reference list and the second reference list, a plurality of first candidate reference blocks of a first partition of the current block and a plurality of second candidate reference blocks of a second partition of the current block are determined.

[0125] In one example, based on one of the non-adjacent blocks and the blocks in the same-position reference frame indicated by the temporal motion vector, multiple first candidate reference blocks for a first partition of the current block and multiple second candidate reference blocks for a second partition of the current block are determined.

[0126] In one aspect, a template-matching (TM) difference between a first template region of the template sample and each of the first candidate templates is determined. A TM difference between a second template region of the template sample and each of the second candidate templates is determined. A plurality of first candidate reference blocks for the first partition are sorted based on an ascending order of the TM difference between the first template region of the template sample and the first candidate template. A plurality of second candidate reference blocks for the second partition are sorted based on an ascending order of the TM difference between the second template region of the template sample and the second candidate template.

[0127] In one embodiment, based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template, A plurality of first candidate reference blocks for the first partition are reordered based on a number of samples in a sample row of the first portion of the top template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on a number of samples in a sample row of the second portion of the top template being greater than a threshold.

[0128] In one embodiment, based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template, A plurality of first candidate reference blocks for the first partition are reordered based on widths of sample rows in the first portion of the upper template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on widths of sample rows in the second portion of the upper template being greater than a threshold.

[0129] In one embodiment, based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template, The plurality of first candidate reference blocks for the first partition are ordered based on a number of samples in the sample sequence of the first portion of the left template being greater than a threshold, and the plurality of second candidate reference blocks for the second partition are reordered based on a number of samples in the sample sequence of the second portion of the left template being greater than a threshold.

[0130] In one embodiment, based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template, A plurality of first candidate reference blocks for the first partition are reordered based on a height of a sample sequence in the first portion of the left template being greater than a threshold, and a plurality of second candidate reference blocks for the second partition are reordered based on a height of a sample sequence in the second portion of the left template being greater than a threshold.

[0131] In one aspect, a TM difference between one of the unsegmented top template or the unsegmented left template and each corresponding region of the first candidate template is determined based on a first template region of the template sample including one of the unsegmented top template or the unsegmented left template and a second template region of the template sample including the other of the segmented top template or the left template. Further, the plurality of first candidate reference blocks of the first partition are sorted based on an ascending order of the TM differences between one of the unsegmented top template or the unsegmented left template and each corresponding region of the first candidate template.

[0132] In one example, a plurality of candidate partitions for a current block are determined based on a plurality of candidate partition methods. A plurality of candidate reference blocks are determined for each of the plurality of candidate partitions. A TM difference between a template sample of the current block and a template region of each of the plurality of candidate reference blocks of the plurality of candidate partitions is determined. The plurality of candidate reference blocks are sorted in ascending order based on the TM difference between the template sample of the current block and the template regions of the plurality of reference blocks for the current block.

[0133] In one example, the video bitstream includes a plurality of indices indicating a subset of a plurality of sorted candidate reference blocks, and the subset of the plurality of sorted candidate reference blocks includes the candidate reference blocks having the first N smallest TM differences.

[0134] Next, the process proceeds to (S1099) and ends.

[0135] The process 1000 may be adapted as appropriate. Steps of the process 1000 may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0136] 11 shows a flowchart outlining a process (1100) according to one embodiment of the present disclosure. The process (1100) can be used in a video encoder. In various embodiments, the process (1100) is performed by a processing circuit, such as a processing circuit performing the functions of the video encoder (103), a processing circuit performing the functions of the video encoder (303), etc. In some embodiments, the process (1100) is implemented by software instructions, and thus the processing circuit performs the process (1100) when it executes the software instructions. The process begins at (S1101) and proceeds to (S1110).

[0137] In (S1110), a plurality of candidate first reference blocks are determined for a first partition of a current block, and a plurality of candidate second reference blocks are determined for a second partition of the current block. The current block is coded using a geometric partitioning mode (GPM) in which the current block is partitioned into a first partition and a second partition by a partition line. The current block includes template samples including a top template adjacent to the top side of the current block and a left template adjacent to the left side of the current block. The template samples are divided by the partition line into a first template region adjacent to the first partition and a second template region adjacent to the second partition. Each of the plurality of candidate first reference blocks in the first partition has a respective first candidate template. Each of the plurality of candidate second reference blocks in the second partition has a respective second candidate template.

[0138] An exemplary embodiment related to step (S1110) may be shown in FIG. 7. As shown in FIG. 7, the current CU (702) may be partitioned into a first partition P0 and a second partition P1 according to a GPM division edge (or partition line) (704). The current CU (702) may have a top template (706) located above the current CU (702) and a left template (708) located to the left of the current CU (702). The top template (706) is divided into two partitions TT by the partition line (704). P0 and T.T. P1 It can be divided into TT P0 and the left template (708) can be used to sort the merge indexes of the geometric partition P0 using the TM cost in ascending order, and TT P1 can be used to sort the merge index for partition P1.

[0139] At (S1120), at least one of the plurality of candidate first reference blocks of the first partition and the plurality of candidate second reference blocks of the second partition is sorted based on the size of the first template region of the template sample and the size of the second template region of the template sample.

[0140] For example, as shown in FIG. 8, a current CU (802) can be partitioned into a first partition P0 and a second partition P1 by a GPM division edge (or partition line) 804. A top template (806) can also be partitioned into first portions 806A and 806B. The top template (806) and the first portion 806A of the left template (808) can serve as a template for the first partition P0, and the second portion 806B of the top template (806) can serve as a template for the second partition P1. Because the size of the second portion 806B is smaller than a threshold (e.g., 4), template matching may not be applied to the template for the geometric partition P1.

[0141] At (S1130), the current block is encoded based on a permutation of at least one of a plurality of candidate first reference blocks for the first partition and a plurality of candidate second reference blocks for the second partition. In one example, only n indices with the smallest TM cost may be selected, where n is a non-zero value and is less than the number of all possible combinations of geometric partition modes and two merge indices for two geometric partitions. In one example, the value of n can be signaled in a high-level syntax such as a VPS, SPS, PPS, or slice header. In one example, an index is signaled to indicate which of the n indices is used to reconstruct the current block.

[0142] Next, the process proceeds to (S1199) and ends.

[0143] The process 1100 may be adapted as appropriate. Steps of the process 1100 may be modified and / or omitted. Additional steps may be added. Any suitable order of implementation may be used.

[0144] The techniques described above can be implemented as computer software using computer-readable instructions and physically stored on one or more computer-readable media. For example, Figure 12 illustrates a computer system (1200) suitable for implementing certain embodiments of the subject matter of this disclosure.

[0145] Computer software can be coded using any suitable machine code or computer language that can be processed by mechanisms such as assembly, compilation, linking, etc. to generate code containing instructions that can be executed by one or more computer central processing units (CPUs), graphics processing units (GPUs), etc., directly or through interpretation, microcode execution, etc.

[0146] The instructions may be executed by a variety of computers or components thereof, including, for example, personal computers, tablet computers, servers, smartphones, gaming devices, Internet of Things devices, and the like.

[0147] 12 of the computer system (1200) are exemplary in nature and do not suggest any limitation on the scope of use or functionality of the computer software implementing the embodiments of the present disclosure. Furthermore, the arrangement of components should not be interpreted as having any dependency or requirement relating to any one or combination of components illustrated in the exemplary embodiment of the computer system (1200).

[0148] The computer system (1200) may include certain human interface input devices. Such human interface input devices may be responsive to input by one or more human users, for example, through sensory input (e.g., keystrokes, swipes, data grab actions), audio input (e.g., voice, clapping), visual input (e.g., gestures), or olfactory input (not shown). Human interface devices may also be used to capture certain media that are not necessarily directly related to conscious human input, such as audio (e.g., speech, music, ambient sounds), images (e.g., scanned images, photographic images obtained from a digital camera), and video (including, for example, two-dimensional video, three-dimensional video, and stereoscopic video).

[0149] The input human interface devices may include one or more of a keyboard (1201), a mouse (1202), a trackpad (1203), a touchscreen (1210), a data grab (not shown), a joystick (1205), a microphone (1206), a scanner (1207), and a camera (1208) (only one of which is shown).

[0150] The computer system (1200) may also include certain human interface output devices. Such human interface output devices may stimulate one or more of the human user's senses through, for example, sensory output, sound, light, and smell / taste. Such human interface output devices may include sensory output devices (e.g., sensory feedback via a touchscreen (1210), a data glove (not shown), or a joystick (1205; however, sensory feedback devices that do not function as input devices may also exist), audio output devices (e.g., speakers (1209), headphones (not shown)), and visual output devices (e.g., a screen (1210), including a CRT screen, an LCD screen, a plasma screen, and an OLED screen, each with or without touchscreen input capability and each with or without sensory feedback capability, some of which may be capable of outputting two-dimensional visual output or three-dimensional or higher-dimensional output through means such as stereoscopic output, virtual reality glasses (not shown), holographic displays, and smoke tanks (not shown), and printers (not shown)).

[0151] The computer system (1200) may also include human-accessible storage and associated media such as optical media including CD / DVD ROM / RW (1220) with media such as CD / DVD (1221), thumb drives (1222), removable hard drives or solid state drives (1223), legacy magnetic media such as tape and floppy disks (not shown), dedicated ROM / ASIC / PLD based devices such as security dongles (not shown), etc.

[0152] Those skilled in the art should also understand that the term "computer-readable medium" as used in connection with the subject matter of this disclosure does not encompass transmission media, carrier waves, or other transitory signals.

[0153] The computer system 1200 may also include an interface 1254 to one or more communications networks 1255. Networks may be, for example, wireless, wired, or optical. Networks may further be local, wide-area, metropolitan, vehicular, and industrial, real-time, latency-tolerant, and the like. Examples of networks include local area networks such as Ethernet; cellular networks, including WLAN, GSM, 3G, 4G, 5G, LTE, and the like; TV wired or wireless wide-area digital networks, including cable TV, satellite TV, and terrestrial broadcast TV; and vehicular and industrial networks, including CAN Bus. Particular networks generally require external network interfaces that are attached to particular general-purpose data ports or peripheral buses 1249 (e.g., USB ports on the computer system 1200). Others are generally integrated into the core of the computer system 1200 by attachment to a system bus, as described below (e.g., an Ethernet interface to a PC computer system, or a cellular network interface to a smartphone computer system). Using these networks, the computer system 1200 can communicate with other entities. Such communication may be one-way receive only (e.g., broadcast TV), one-way transmit only (e.g., CANbus to a specific CANbus device), or two-way to other computer systems, for example, using local or wide-area digital networks. Specific protocols and protocol stacks may be used with each of the above networks and network interfaces.

[0154] The aforementioned human interface devices, human accessible storage devices, and network interfaces may be attached to the core (1240) of the computer system (1200).

[0155] The core (1240) may include one or more central processing units (CPUs) (1241), graphics processing units (GPUs) (1242), dedicated programmable processing units (1243) in the form of FPGAs, task-specific hardware accelerators (1244), graphics adapters (1250), etc. These devices may be connected through a system bus (1248), along with read-only memory (ROM) (1245), random access memory (1246), and internal mass storage devices (1247) such as internal non-user-accessible hard drives, SSDs, etc. In some computer systems, the system bus (1248) is accessible in the form of one or more physical plugs to allow expansion with additional CPUs, GPUs, etc. Peripheral devices can be attached directly to the core's system bus (1248) or through a peripheral bus (1249). In an example, a screen (1210) can be connected to the graphics adapter (1250). Peripheral bus architectures include PCI, USB, and the like.

[0156] The CPU (1241), GPU (1242), FPGA (1243), and accelerator (1244) can execute specific instructions that can be combined to generate the aforementioned computer code. The computer code can be stored in ROM (1245) or RAM (1246). Temporary data can also be stored in RAM (1246), while permanent data can be stored, for example, in an internal mass storage device (1247). Rapid storage and retrieval from any of the memory devices can be enabled through the use of cache memory, which can be closely associated with one or more of the CPU (1241), GPU (1242), mass storage device (1247), ROM (1245), RAM (1246), etc.

[0157] The computer-readable medium may bear computer code for performing various computer-implemented operations. The medium and computer code may be those specially designed and constructed for the purposes of the present disclosure, or they may be of the kind well known and available to those skilled in the computer software arts.

[0158] As an example and not by way of limitation, the computer system (1200) having the architecture, and specifically the core (1240), can provide functionality as a result of a processor (including a CPU, GPU, FPGA, accelerator, etc.) executing software embodied in one or more tangible computer-readable media. Such computer-readable media can be specific storage of the core (1240) of a non-transitory nature, such as the core's internal mass storage (1247) or ROM (1245), as well as media associated with user-accessible mass storage devices such as those described above. Software implementing various embodiments of the present disclosure can be stored on such devices and executed by the core (1240). The computer-readable media can include one or more memory devices or chips, depending on the particular needs. The software can cause the core (1240) and specifically the processor (including a CPU, GPU, FPGA, etc.) therein to perform specific processes or portions of specific processes described herein, including defining and modifying data structures stored in RAM (1246) according to software-defined operations. Additionally or alternatively, a computer system may provide functionality as a result of implementation in hardwired logic or other circuitry (e.g., accelerator 1244) that can operate in conjunction with or in place of software to perform particular processes or portions of particular processes described herein. References to software include logic, and vice versa, where appropriate. References to computer-readable media may include, where appropriate, circuitry (such as an integrated circuit (IC)) that stores software for execution, circuitry that implements logic for execution, or both. The present disclosure includes any appropriate combination of hardware and software.

[0159] The use of "at least one" or "one" in this disclosure is intended to include any one or combination of the listed elements. For example, reference to at least one of A, B, or C; at least one of A, B, and C; at least one of A, B, and / or C; and at least one of A through C is intended to include A only, B only, C only, or any combination thereof. Reference to one of A or B, and one of A and B is intended to include A or B or (A and B). The use of "one" does not exclude any combination of the listed elements, where applicable, such as when the elements are not mutually exclusive.

[0160] While this disclosure has described several exemplary embodiments, alterations, permutations, and various substitute equivalents exist, and are encompassed within the scope of this disclosure. Those skilled in the art will appreciate that numerous systems and methods can be devised that, although not explicitly shown or described herein, embody the principles of the present disclosure and therefore are within the spirit and scope of the present disclosure.

Claims

1. 1. A method of video decoding performed in a video decoder, comprising: receiving a video bitstream including a current block in a current frame and template samples for the current block, the template samples including a top template along an upper side of the current block and a left template along a left side of the current block, the current block being partitioned into a first partition and a second partition, and the template samples being divided into a first template region adjacent to the first partition and a second template region adjacent to the second partition; determining a plurality of first candidate reference blocks for the first partition and a plurality of second candidate reference blocks for the second partition, each first candidate reference block having a respective first candidate template corresponding to a shape of the first template region, and each second candidate reference block having a respective second candidate template corresponding to a shape of the second template region; reordering at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks based on a size of the first template region of the template sample and a size of the second template region of the template sample; reconstructing the current block based on index values ​​received in the video bitstream and based on a permutation of at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks; A method comprising:

2. The determining step includes:

2. The method of claim 1, further comprising determining the plurality of first candidate reference blocks of the first partition of the current block and the plurality of second candidate reference blocks of the second partition of the current block based on candidate reference blocks indicated by motion vectors (MVs) in a first reference list and a second reference list.

3. The determining step includes:

2. The method of claim 1, further comprising determining the plurality of first candidate reference blocks for the first partition of the current block and the plurality of second candidate reference blocks for the second partition of the current block based on one of non-adjacent blocks and blocks in a co-located reference frame of the current frame indicated by a temporal motion vector.

4. The sorting step includes: determining a template matching (TM) difference between the first template region of the template sample and each of the first candidate templates, and a TM difference between the second template region of the template sample and each of the second candidate templates; (i) sorting the first candidate reference blocks for the first partition based on an ascending order of TM difference between the first template region of the template sample and the first candidate template, and (ii) sorting the second candidate reference blocks for the second partition based on an ascending order of TM difference between the second template region of the template sample and the second candidate template; The method of claim 1 further comprising:

5. The sorting step includes: based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template; reordering the plurality of first candidate reference blocks for the first partition based on a number of samples in a sample row of the first portion of the top template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a number of samples in a sample row of the second portion of the top template being greater than the threshold; The method of claim 1 further comprising:

6. The sorting step includes: based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template; reordering the plurality of first candidate reference blocks for the first partition based on a width of a sample row of the first portion of the upper template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a width of a sample row in the second portion of the upper template being greater than the threshold; The method of claim 1 further comprising:

7. The sorting step includes: based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template; reordering the plurality of first candidate reference blocks for the first partition based on a number of samples in a sample sequence of the first portion of the left template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a number of samples in a sample sequence of the second portion of the left template being greater than the threshold; The method of claim 1 further comprising:

8. The sorting step includes: based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template; reordering the plurality of first candidate reference blocks for the first partition based on a height of a sample sequence in the first portion of the left template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a height of a sample sequence in the second portion of the left template being greater than the threshold; The method of claim 1 further comprising:

9. The sorting step further comprises: based on the first template region of the template sample including one of the unsegmented top template or the unsegmented left template, and the second template region of the template sample including the other of the segmented top template or the left template, determining a template matching (TM) difference between one of the unsegmented top template or the unsegmented left template and a corresponding region of each of the first candidate templates; sorting the plurality of first candidate reference blocks for the first partition based on an ascending order of TM difference between one of the un-segmented top template or the un-segmented left template and a corresponding region of the first candidate template; The method of claim 1 further comprising:

10. determining a plurality of candidate partitions for the current block based on a plurality of candidate partition methods; determining a plurality of candidate reference blocks for each of the plurality of candidate partitions; determining a TM difference between the template sample of the current block and a template region of each of the plurality of candidate reference blocks of the plurality of candidate partitions; sorting the plurality of candidate reference blocks in ascending order based on TM differences between the template sample of the current block and the template regions of the plurality of candidate reference blocks for the current block; The method of claim 4 further comprising:

11. 11. The method of claim 10, wherein the video bitstream includes a plurality of indices indicating a reordered subset of the plurality of candidate reference blocks, the reordered subset of the plurality of candidate reference blocks including the first N candidate reference blocks having the smallest TM differences.

12. 1. An apparatus, the apparatus including a processing circuit, the processing circuit comprising: receiving a video bitstream including a current block in a current frame and template samples of the current block, the template samples including a top template along an upper side of the current block and a left template along a left side of the current block, the current block being partitioned into a first partition and a second partition, and the template samples being divided into a first template region adjacent to the first partition and a second template region adjacent to the second partition; determining a plurality of first candidate reference blocks for the first partition and a plurality of second candidate reference blocks for the second partition, each first candidate reference block having a respective first candidate template corresponding to the shape of the first template region, and each second candidate reference block having a respective second candidate template corresponding to the shape of the second template region; reordering at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks based on a size of the first template region of the template sample and a size of the second template region of the template sample; reconstructing the current block based on index values ​​received in the video bitstream and based on a permutation of at least one of the plurality of first candidate reference blocks and the plurality of second candidate reference blocks. The equipment is configured to:

13. The processing circuitry 13. The device of claim 12, configured to determine the plurality of first candidate reference blocks of the first partition of the current block and the plurality of second candidate reference blocks of the second partition of the current block based on candidate reference blocks indicated by motion vectors (MVs) in a first reference list and a second reference list.

14. The processing circuitry 13. The device of claim 12, configured to determine the plurality of first candidate reference blocks for the first partition of the current block and the plurality of second candidate reference blocks for the second partition of the current block based on one of a non-adjacent block and a block in a co-located reference frame of the current frame indicated by a temporal motion vector.

15. The processing circuitry determining a template matching (TM) difference between the first template region of the template sample and each of the first candidate templates and a TM difference between the second template region of the template sample and each of the second candidate templates; (i) sorting the first candidate reference blocks for the first partition based on an ascending order of TM differences between the first template region of the template sample and the first candidate template, and (ii) sorting the second candidate reference blocks for the second partition based on an ascending order of TM differences between the second template region of the template sample and the second candidate template.

13. The device of claim 12, configured to:

16. The processing circuitry based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template; reordering the plurality of first candidate reference blocks for the first partition based on a number of samples in a sample row of the first portion of the top template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a number of samples in a sample row of the second portion of the top template being greater than the threshold; 13. The device of claim 12, configured to:

17. The processing circuitry based on the top template being divided into a first portion and a second portion, the first template region of the template sample including the left template and the first portion of the top template, and the second template region of the template sample including the second portion of the top template; reordering the plurality of first candidate reference blocks for the first partition based on a width of a sample row of the first portion of the upper template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a width of a sample row of the second portion of the upper template being greater than the threshold; 13. The device of claim 12, configured to:

18. The processing circuitry based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template; reordering the plurality of first candidate reference blocks for the first partition based on a number of samples in a sample sequence of the first portion of the left template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a number of samples in a sample sequence of the second portion of the left template being greater than the threshold; 13. The device of claim 12, configured to:

19. The processing circuitry based on the left template being divided into a first portion and a second portion, the first template region of the template sample including the top template and the first portion of the left template, and the second template region of the template sample including the second portion of the left template; reordering the plurality of first candidate reference blocks for the first partition based on a height of a sample sequence of the first portion of the left template being greater than a threshold; reordering the plurality of second candidate reference blocks for the second partition based on a height of a sample sequence in the second portion of the left template being greater than the threshold; 13. The device of claim 12, configured to:

20. The processing circuitry based on the first template region of the template sample including one of the unsegmented top template or the unsegmented left template, and the second template region of the template sample including the other of the segmented top template or the left template, determining a template matching (TM) difference between one of the unsegmented top template or the unsegmented left template and a corresponding region of each of the first candidate templates; sorting the plurality of first candidate reference blocks for the first partition based on an ascending order of TM differences between one of the un-segmented top template or the un-segmented left template and corresponding regions of the first candidate template; 13. The device of claim 12, configured to:

21. 1. A method of video encoding performed in a video encoder, comprising: determining a plurality of candidate first reference blocks for a first partition of a current block and a plurality of candidate second reference blocks for a second partition of the current block, the current block being coded using a geometric partitioning mode (GPM) in which the current block is partitioned into the first partition and the second partition by a partition line, the current block including template samples including a top template adjacent to an upper side of the current block and a left template adjacent to a left side of the current block, the template samples being divided by the partition line into a first template region adjacent to the first partition and a second template region adjacent to the second partition, each of the plurality of candidate first reference blocks in the first partition having a respective first candidate template, and each of the plurality of candidate second reference blocks in the second partition having a respective second candidate template; reordering at least one of the plurality of candidate first reference blocks of the first partition and the plurality of candidate second reference blocks of the second partition based on a size of the first template region of the template sample and a size of the second template region of the template sample; encoding the current block based on a permutation of at least one of the plurality of candidate first reference blocks for the first partition and the plurality of candidate second reference blocks for the second partition; A method comprising:

Images