Video encoding and decoding method and apparatus for triangular prediction

Abstract

To provide a method and apparatus for motion compensated prediction with a triangle prediction unit in video encoding and decoding.SOLUTION: A video encoding method includes partitioning a video image into a plurality of coding units (CUs) partitioned into two PUs, at least one further including at least one geometrically shaped prediction unit (PU), configuring a first merge list including a plurality of candidates each including one or more motion vectors, and obtaining a single-prediction merge list for the geometrically shaped PU, including a plurality of single-prediction merge candidates, each containing a motion vector of one of the corresponding candidates in the first merge list.SELECTED DRAWING: Figure 14

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] The application was filed on March 12, 2019 and is entitled "Video with Triangle Prediction." No. 62 / 817,537, filed on March 13, 2019, which is a provisional application for a method of encoding and decoding a digital signal. No. 62,333,535, filed on Dec. 13, 2003, entitled “Video Encoding and Decoding Using Triangular Prediction.” This application claims priority to US Pat. No. 6,313,312 and US Pat. No. 6,313,312, filed on Oct. 13, 2003, and the entire specifications of these patent applications. is incorporated herein by reference. [Technical field]

[0002] This application relates generally to video encoding, decoding and compression, and in particular, but not exclusively, to video A special case of the triangle prediction unit (i.e., the geometric block prediction unit) in encoding and decoding The present invention relates to a method and apparatus for motion compensated prediction using a combination of a first approximation and a second approximation. [Background technology]

[0003] Digital television, laptop or desktop computer, tablet computer computers, digital cameras, digital recording devices, digital media players, Video game consoles, smartphones, video conferencing equipment, video streaming equipment, etc. Various electronic devices support digital video. These electronic devices are used for video compression / expansion. It performs the operations required to receive, transmit, encode, decode and store digital video data. Digital video equipment uses Versatile Video Coding (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPE G-4, ITU-T H.263, ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), ITU-T H.265 / High Efficiency Standards defined by High Efficiency Video Coding (HEVC) and its Such video encoding and decoding techniques may be implemented as described in the standard extensions.

[0004] Video encoding and decoding generally involves prediction due to redundancy present in a video image or sequence. Video encoding and decoding technology uses prediction methods (e.g., inter-prediction, intra-prediction). One of the key goals is to streamline video data while avoiding or minimizing video quality degradation. The goal is to compress video content into a form with lower bit rates. As more and more technologies become available, there is a need for coding / decoding techniques with better coding / decoding efficiency. It is essential.

[0005] Video compression typically involves spatial (intra-frame) prediction and / or temporal (inter-frame) prediction. to reduce or remove redundancy inherent in the video data. In video coding based on CNN, a video frame is divided into coding tree units (CTUs). The image is partitioned into one or more slices, each of which contains multiple video blocks called slice units. Each CTU may contain one coding unit (CU) or may be a predefined It may be recursively split into smaller CUs until the minimum CU size is reached. U (also called leaf CU) contains one or more transform units (TUs). Each of the PUs includes a prediction unit (PU) and one or more prediction units (PUs). A CU can be coded in intra, inter, or IBC mode. Video blocks in an intra-coded (I) slice in a video frame are The image is coded using spatial prediction with reference samples in adjacent blocks in the video frame. Video blocks within an inter-coded (P or B) slice in a video frame. The lock is a spatial prediction algorithm based on reference samples in adjacent blocks in the same video frame. For reference samples in measurements or other previous and / or future reference video frames, Use a time prediction for the

[0006] Spatial or temporal prediction using previously coded reference blocks, e.g., neighboring blocks In the above, a prediction block is obtained for the current video block to be coded. The process of finding the lock can be achieved by a block matching algorithm. Residual data indicating pixel differences between the current block to be coded and the predicted block is The data is called the residual block or prediction error. Inter-coded blocks are the blocks that are generated by the prediction block. A motion vector pointing to a reference block in the reference frame that formed the locked image and a residual block The process of determining motion vectors is usually called motion estimation. Intra-coded blocks are coded according to the intra prediction mode and the residual block. For further compression, the residual block is converted from the pixel domain to a transform domain, e.g., the frequency domain. The resulting residual transform coefficients are then quantified. The quantified transform coefficients arranged in a matrix are scanned to generate a one-dimensional transform coefficient vector. Then, the entropy coding is added to the video bitstream to achieve further compression. It will be encoded.

[0007] and storing the encoded video bitstream on a computer readable storage medium. (e.g. flash memory) to another electronic device with digital video capability and The electronic device may, for example, analyze the encoded video bitstream to generate the encoded video bitstream. Gets syntax elements from the bitstream and returns the and extracting digital video data from the encoded video stream based at least in part on the encoded video data. By reconstructing the data into its original format, video decompression (the opposite of video compression as described above) is achieved. This reconstructed digital video data is then displayed on the display of the electronic device. Reproduced to Ray.

[0008] Digital video quality has progressed from high resolution to 4K x 2K or / and 8K x 4K. As video technology advances, the amount of video data that needs to be encoded / decoded increases exponentially. Title: Well and efficiently encoding video data while maintaining image quality of the decoded video data Decrypting / reading is always a challenge.

[0009] In the Joint Video Experts Team (JVET) meetings, First draft of VVC (Voltage Video Coding) and VVC Test Model 1 (VTM1) 1) The coding method was defined. Binary and ternary split coding block structure nested matrix Quadtrees with multitype trees are included as the first new coding feature in VVC. Since then, the reference software VT for implementing the encoding / decoding method has been developed. The M and Draft VVC decoding processes were developed during the JVET conference. Summary of the Invention

[0010] This disclosure is a special case of geometric partition prediction in video encoding and decoding in general. An example technique for motion compensated prediction using a triangular prediction unit is described.

[0011] According to a first aspect of the present disclosure, a video image is captured by at least one A set of coding units (CUs) partitioned into two PUs, each containing a geometrically shaped PU. The candidate units (CUs) are divided into a plurality of candidate units, each of which includes one or more motion vectors. and constructing a first merge list including: a plurality of single prediction merge candidates including one motion vector of a corresponding candidate for the PU; and obtaining a single prediction merge list for the video coding and decoding in a triangular prediction mode. A method for encoding is provided.

[0012] In accordance with a second aspect of the present disclosure, a processor and instructions executable by the processor and a memory configured to store, when the processor executes the instructions, , the video image is further subjected to at least one geometric shape prediction unit The CU is divided into two PUs, each of which is divided into a plurality of coding units (CUs), each of which is divided into two PUs, each of which is divided into a PU. constructing a first merge list including a plurality of candidates, each of which includes one or more motion vectors; a plurality of merge lists each including a motion vector of one of the corresponding candidates in the first merge list; A uni-prediction merge list for the PU, including the uni-prediction merge candidates, is generated in a triangular partition mode. The present invention provides a computing device configured to derive a

[0013] In accordance with a third aspect of the present disclosure, a non-transitory computer readable memory device having instructions stored thereon is provided. a storage medium, the instructions, when executed by a processor, causing the processor to The video image is then subjected to at least one further geometric shape prediction unit (P The coding unit (CU) is divided into two PUs, each of which is a coding unit (CU), A first merge list is constructed including a plurality of candidates including one or more motion vectors, A plurality of single merge lists each including a motion vector of one of the corresponding candidates in the first merge list. Derive a single prediction merge list for the PU, including prediction merge candidates, in triangular partition mode. and providing a non-transitory computer-readable storage medium for performing operations such as do. [Brief description of the drawings]

[0014] A more particular description of the present disclosure will now be made by reference to specific examples which are illustrated in the accompanying drawings. These drawings are given by way of example only and are therefore not intended to be limiting in scope. While not intended to be limiting, these examples may be illustrated with additional features by the use of the accompanying drawings. The isomerism and details are explained. [Figure 1] FIG. 1 is a block diagram illustrating an example video encoder according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a block diagram illustrating an example video decoder according to an embodiment of this disclosure. [Diagram 3]FIG. 3 is a schematic diagram illustrating a quadtree plus binary tree (QTBT) structure according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a schematic diagram illustrating an example of an image divided into CTUs, according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a schematic diagram illustrating a multi-type tree splitting mode according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a schematic diagram illustrating splitting a CU into triangular prediction units according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram showing the location of adjacent blocks according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a schematic diagram illustrating the locations of spatial merging candidates according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a schematic diagram illustrating motion vector scaling for temporal merging candidates according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a schematic diagram illustrating candidate locations for temporal merging candidates according to an embodiment of the present disclosure. [Figure 11] 11A-11B are schematic diagrams illustrating an example of single predictor motion vector (MV) selection for triangular prediction mode, according to an embodiment of the present disclosure. [Figure 12] 12A to 12D are schematic diagrams illustrating examples of single-prediction MV selection for triangular prediction mode according to an embodiment of the present disclosure. [Figure 13] FIG. 13 is a block diagram illustrating an example apparatus for video encoding and decoding according to an embodiment of this disclosure. [Figure 14] FIG. 14 is a flowchart illustrating an example process of video encoding / decoding for motion compensated prediction by a triangular prediction unit according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the detailed description, a number of non-limiting specific examples are provided to facilitate understanding of the subject matter discussed herein. However, it will be apparent to those skilled in the art that various modifications can be made. For example, the subject matter discussed here applies to many types of electronic devices that have digital video capabilities. It will be apparent to those skilled in the art that this may be implemented in a device.

[0016] In this specification, the terms "one embodiment," "embodiment," "example," "an embodiment," " Reference to "an example" or similar language means that the particular feature, structure or characteristic being described has at least one It means to be included in one embodiment or example. It means to be linked to one or several embodiments. Any features, structures, elements or characteristics described herein may be used interchangeably with other implementations unless expressly indicated otherwise. It is also applicable to the shape.

[0017] Throughout this disclosure, the terms "first," "second," "third," etc., all refer to the respective elements, For example, it is used only to refer to an apparatus, part, configuration, step, etc. No spatial or chronological order is implied unless the context clearly indicates otherwise. For example, a "first device" and a "second device" may refer to two separately formed devices, or or two parts, components, or operating states of the same equipment, arbitrarily named It can be done.

[0018] As used herein, "if..." or "if..." The term "and" shall be understood to mean "when" or "according to," depending on the context. When these terms appear in the claims, they are used to qualify the associated limitation or feature. The term may not be intended to be optional or adjunct.

[0019] "Module", "Submodule", "Circuit", "Subcircuit", "Circuit system", The terms "sub-circuit system," "unit" or "subunit" mean one or Memory that stores code or instructions that can be executed by multiple processors (shared, dedicated, A module contains a program or group of programs. A module is a program that stores or stores code or instructions. A module or circuit may contain one or more circuits that are not directly The components may include one or more components that are connected directly or indirectly to the , may be physically connected to each other, adjacent to each other, or not.

[0020] A unit or module may be implemented entirely in software, It may be implemented by hardware or a combination of hardware and software. A fully software implementation may be implemented by combining the units or modules are linked together directly or indirectly to perform a specific function. may contain functionally related code blocks or software components that It is possible.

[0021] Figure 1 shows the block-based processing used in conjunction with many video encoding and decoding standards. The block diagram of an exemplary block-based mixed video encoder 100 that can be used A block diagram of the encoder 100 is shown. The encoder 100 receives video frames and divides them into multiple video streams for processing. For each particular video block, prediction is performed using an inter prediction approach. In inter prediction, the previously reconstructed image is generated based on a single prediction approach. Based on pixels from the generated frames, motion estimation and motion compensation are performed to generate one or more In intra prediction, a predictor is formed based on the reconstructed pixels in the current frame. Through the mode decision, the best predictor for predicting the current block is determined. The probe can be selected.

[0022] A prediction residual, which represents the difference between the current video block and its predictor, is sent to the transform circuit 102. Then, for entropy reduction, the transform coefficients are quantified from the transform circuit 102. The quantified coefficients are then provided to an entropy coding circuit 106. As shown in Figure 1, the inter-prediction Video block partition information from the prediction circuit and / or intra prediction circuit 112, motion vectors, Prediction related information 110 such as frame rate, reference image index, and intra prediction mode is also stored in the A compressed video bitstream 1 is provided through an entropy coding circuit 106. It will be saved in 14.

[0023] In the encoder 100, decoder-related circuitry is used to reconstruct pixels for prediction purposes. First, the prediction residual is re-transformed via an inverse quantification 116 and an inverse transform circuit 118. This reconstructed prediction residual is combined with the block predictor 120 to produce The filter generates unfiltered reconstructed pixels for the current video block.

[0024] Spatial prediction (also called "intra prediction") is a method to predict the location of a video block that is the same as the current video block. Samples of already coded adjacent blocks in the frame (also called reference samples) Pixels from are used to predict the current video block.

[0025] Temporal prediction (also called "inter prediction") is the process of predicting the time from an already coded video image. The current block is predicted using the pixels of the reconstruction of The temporal prediction signal for a particular CU is usually One or more motion vectors (M V: motion vector). Multiple reference images are supported. If so, it is necessary to identify which reference picture in the reference picture store the temporal prediction signal comes from. One reference image index for the image to be processed is additionally transmitted.

[0026] After spatial and / or temporal prediction, the intra / inter prediction in the encoder 100 The mode decision circuit 121 determines the optimal prediction mode based on, for example, a rate-distortion optimization method. Next, the block predictor 120 is subtracted from the current video block; The obtained prediction residuals are decorrelated by a transform circuit 102 and a quantification circuit 104. The quantified residual coefficients are inversely quantified by the inverse quantification circuit 116 and then transformed into the inverse transform circuit 117. 18 to form the reconstruction residual, which is then block to form the reconstructed signal of this CU. The selected CU is placed in the reference picture storage portion of picture buffer 117 and is used to code future video blocks. Before being used for decoding, the deblocking filter, sample adaptive offset (SAO: sample adaptive offset, and / or adaptive in-loop filter (ALF) An in-loop filter 115, such as a 1-ve in-loop filter, is used for this reconstructed CU. To form the output video bitstream 114, The coding mode (inter or intra), prediction mode information, motion information, and quantified All residual coefficients are sent to the entropy coding unit 106 for further compression and packing. The bitstream is formed.

[0027] For example, current versions of AVC, HEVC, and VVC use deblocking filters. In HEVC, SAO (Sequential Audio Optimization) is used to further improve coding efficiency. An additional in-loop filter called sample adaptive offset is defined. The current version of the VC standard introduces an additional filter called the ALF (Adaptive Loop Filter). In-loop filters are being actively researched and have a high chance of being included in the final standard. There is.

[0028] These in-loop filter operations are selectable. They improve the encoding efficiency and visual quality. It is possible that the power supply 102 is turned off as determined by the reader 100.

[0029] Note that when these filter options are turned on by the encoder 100, Intra prediction is usually based on the pixels of the unfiltered reconstruction, but The center prediction is based on the pixels of the filtered reconstruction.

[0030] FIG. 2 shows an example of a video encoding / decoding standard that can be used in conjunction with many FIG. 2 is a block diagram illustrating an exemplary block-based video decoder 200. 00 is similar to the reconstruction-related portion present in the encoder 100 of FIG. At 00, the input video bitstream 201 is first subjected to entropy decoding 202. The quantified coefficient levels and prediction related information are then derived. The quantified coefficient levels are processed via inverse quantification 204 and inverse transformation 206 to reconstruct The generated prediction residual is obtained. The block predictor mechanism performs intra prediction 208 or The inverse transform 206 is then adapted to perform a motion compensation 210. and the prediction output generated by the block predictor mechanism are added by an adder 214. These are summed together to obtain a set of pixels of the unfiltered reconstruction.

[0031] The reconstructed blocks are stored in an image buffer 213, which acts as a reference image store. Before being passed through the image buffer, the image may be passed through an in-loop filter 209. The reconstructed video in 213 can be sent out to drive a display device or used for future video recordings. It can be used to predict out-of-loop. When 09 is on, the filtering operation is performed on these reconstructed pixels. are performed to derive the final reconstructed video output 222.

[0032] The above mentioned video codes such as VVC, JEM, HEVC, MPEG-4, Part10 The encoding / decoding standards are conceptually similar; for example, they all use block-based processing. The block division schemes in some standards are explained in more detail below. do.

[0033] HEVC is based on a mixed block motion compensation transform coding scheme. The basic unit for coding is called a coding tree unit (CTU). For the 32x32 pixel CTU, the maximum CTU size is 64x64 luma pixels and two 32x32 chroma pixels. Each CTU contains one coding unit (CU) and is defined as a pixel block. is recursively split into 4 smaller CUs until a predefined minimum CU size is reached. Each CU (also called leaf CU) has one or more prediction units (PUs). ) and a tree of one or more transform units (TUs).

[0034] In general, except for monochrome content, a CTU consists of one luminance coding tree block (CT B) and two corresponding chroma CTBs, and the CU contains one luma coding block (CB) and two corresponding chroma CBs, and PU includes one luma prediction block (PB) and The TU contains one luma transform block (TB) and two corresponding chroma transform blocks (PB). It may contain a corresponding chroma TB, but the minimum TB size is 4x4 for both luma and chroma. (i.e. 2x2 chroma TB is not supported in 4:2:0 color format) (a) and each intra-luminance PB is There is always only one intra chroma PB in a chroma CB.

[0035] In intra CU, luminance CB can be predicted by one or four luminance PBs, and two chrominance PBs can be predicted by two chrominance PBs. Each luminance CB is usually predicted by one chroma PB, where each luminance PB is There is an intra luma prediction mode, and two chroma PBs share one intra chroma prediction mode Furthermore, in an intra CU, the TB size cannot be larger than the PB size. In each PB, intra prediction is applied to predict the neighboring TBs in the PB. For each PB, we predict the sample of the TB from the sample of the reconstruction. In addition to the standard intra prediction mode, DC and planar modes are also supported, respectively. Predict flat areas and gradually changing areas.

[0036] For each inter PU, there are three prediction modes, including inter, skip and merge. Generally speaking, motion vector conflict (MVC) is The on-vector competition method selects a particular candidate set that includes spatial and temporal motion candidates. The motion estimation is performed by using the motion estimation algorithm. The optimal reconstructed reference image list (i.e., List0 and List1) You can find references here: Inter mode (AMVP, which stands for Advanced Motion Vector Prediction) In the mode, the inter prediction indicator (List0, List1, or is bidirectional prediction), reference index, motion candidate index, motion vector difference (MVD: The motion vector difference (MVD) and prediction residual are transmitted. For the merge mode, only the merge index is sent and the current PU is Inter prediction indicator from the neighboring PU pointed to by the inter index, reference index For skip-coded CUs, the residual signal is also omitted. It is omitted.

[0037] The Joint Exploration Model (JEM) is built on the HEVC test model. The basic encoding and decoding flow of C has not changed in JEM. However, Structure, intra and inter prediction, residue transformation, loop filter, entropy coding The design elements of the most important modules, including the decoding module, have been slightly changed and additional code has been added. The following new encoding features have been added:

[0038] In HEVC, CTUs are divided into CUs using a quadtree structure, which is referred to as a coding tree. It adapts to various local characteristics by using either inter-picture (temporal) or intra-picture (spatial) prediction. The decision to use to code an image region is made at the CU level. Depending on the type, it can be further divided into one, two or four PUs. Within the PU, the same prediction process is applied and the relevant information is sent to the decoder on the basis of the PU. After obtaining the residual block by applying the prediction process based on the division type, the coding tree of the CU is Partitioning the CU into transform units (TUs) according to another quadtree structure similar to that of Lee. One of the main features of the HEVC structure is that it is divided into multiple regions, including CUs, PUs, and TUs. The concept of a picture is that

[0039] FIG. 3 is a schematic diagram illustrating a quad-tree plus binary tree (QTBT) structure according to an embodiment of the present disclosure. This is a schematic diagram.

[0040] The QTBT structure removes the concept of multiple partition types, i.e. CU, PU, ​​and T Removes the separation of the U concept and supports more flexibility in CU partition shapes. In the square structure, the CU can have a square or rectangular shape. As shown in Figure 3, First, a coding tree unit (CTU) is partitioned according to a quadtree structure. The tree leaf nodes can be further partitioned by a binary tree structure. There are two types of partitions in a binary tree: symmetric horizontal partitions and symmetric vertical partitions. A binary tree leaf node is , called coding units (CUs), and the segmentation can be further divided This is because CU, PU, ​​and TU are QTBT coded. In JEM, a CU is a set of CUs that have different block sizes. For example, the 4:2:0 chroma format For P and B slices, one CU has one luma CB and two chroma CBs. A CU may contain a single component CB, e.g., for an I slice, In this case, a CU contains only one luminance CB or two chroma CBs. This may happen.

[0041] For the QTBT partition method, the following parameters are defined: -CTU size: The root node size of the quadtree, which is the same concept as HEVC; -MinQTSize: The minimum allowed quadtree leaf node size; -MaxBTSize: maximum allowed binary tree root node size; -MaxBTDepth: maximum allowed binary tree depth; -MinBTSize: The minimum allowed binary tree leaf node size.

[0042] In one example of a QTBT partition structure, the CTU size is two corresponding 64x64 chroma samples. 128x128 luma samples with 4:2:0 chroma format, MinQTS size is 16x16, MaxBTSize is 64x64, MinBTSize (both width and height) is 4x4, M axBTDepth is set to 4. The quadtree partition is first applied to the CTU, and then the quadtree leaf is A quadtree leaf node is generated by dividing the quadtree leaf node by 16x16 (i.e. MinQTSize). It is possible for the size to be up to 28x128 (i.e. the CTU size). If the node is 128x128, the size exceeds MaxBTSize (i.e. 64x64). Therefore, the quad-tree leaf node is not further split by the binary tree. If so, the quadtree leaf node can be further partitioned by a binary tree. Therefore, the quadtree leaf node is also the binary tree root node and has binary tree depth 0. When the binary tree depth reaches MaxBTDepth (i.e., 4), no further splits are considered. If the binary tree node has a width equal to MinBTSize (i.e., 4), no more water will be added. Flat splits are not considered. Similarly, if a binary tree node has a height equal to MinBTSize, then it Any vertical partitions beyond this are not considered. Binary tree leaf nodes are partitioned without further partitioning. The maximum CTU size in JEM is 2 56x256 luma samples.

[0043] Figure 3 shows an example of a block partition and the corresponding tree representation according to the QTBT method. The solid lines indicate quadtree division, and the dotted lines indicate binary tree division. As shown in Figure 3, The CTU 300 was initially partitioned into four quadtree structures. Three of the nodes 302, 304, 306, and 308 are quadtree or binary tree structures. For example, the quadtree leaf node 306 is further partitioned by the quadtree partition. The quadtree leaf node 304 is further partitioned into two leaves by binary tree splitting. The quadtree leaf node 302 is also divided into two parts, 304a and 304b. At each partition (i.e., non-leaf) node of a binary tree, A flag is signaled indicating the type of split being performed (i.e., horizontal or vertical), where where 0 indicates a horizontal split and 1 indicates a vertical split. For example, for quadtree leaf node 304, In this case, a 0 is signaled to indicate a horizontal split, and in the case of a quadtree leaf node 302 A 1 is signaled to indicate a vertical split. A quadtree split always splits a block horizontally. Since the image is divided both vertically and horizontally to generate four sub-blocks of equal size, the division type There is no need to specify a group.

[0044] The QTBT method also supports the ability to have separate QTBT structures for luma and chroma. Currently, for P slices and B slices, the luminance CTB in one CTU is The luminance and chroma CTBs share the same QTBT structure. However, in the case of the I slice, the luminance and chroma CTBs The chromatic CTB is divided into CUs by one QTBT structure, and the saturation CTB is divided into CUs by another QTBT structure. This is because the CU in the I slice is a coding block of the luma component. A P slice or B slice consists of a coded block of two chroma components. This means that U consists of coded blocks of all three color components.

[0045] At a meeting of the Joint Video Experts Team (JVET), the JVET decided to develop a versatile video codec. Defines the first draft of the VVC coding standard and the VVC Test Model 1 (VTM1) coding method We have a nested multi-type tree with binary and ternary partition coding block structure. It was decided that the quadtree would be included as the first new coding feature in VVC.

[0046] In VVC, the input video is divided into coding tree units (CTUs) according to the image partition structure. The CTU is a quadtree with a nested multi-type tree structure. Thus, the leaf coding unit (CU) is divided into It defines regions that share the same prediction mode (e.g., intra or inter). In this paper, the term "unit" defines a region of an image that contains all of its components. The term "lock" is used to define a region that contains a particular component (e.g., luminance). If a chroma sampling format such as 4:2:0 is considered, the spatial position may vary depending on the location. Partitioning of the CTU in the image

[0047] FIG. 4 is a schematic diagram illustrating an example of an image segmented into CTUs, according to an embodiment of the present disclosure. do.

[0048] In VVC, an image is divided into a set of CTUs, where the concept of a CTU is different from that of HEVC. For an image with three sample arrays, the CTU is an N × N luminance sub-sampler. The CTU40 consists of a chroma sample block and two corresponding chroma sample blocks. An example of an image 400 divided into two is shown.

[0049] The maximum allowed size of a luminance block in the CTU is specified as 128x128 ( However, the maximum size of a luminance transformation block is 64x64. Compartmentalization of CTU using a tree structure

[0050] FIG. 5 is a schematic diagram illustrating a multi-type tree splitting mode according to an embodiment of the present disclosure. be.

[0051] In HEVC, the CTU is represented as a coding tree to adapt to various local characteristics. The image is divided into CUs by a quadtree structure that is generated by the inter-picture (temporal) prediction or intra-picture (spatial) prediction. The prediction or coding of an image region is decided at the leaf CU level. The CU is further divided into 1, 2, or 4 PUs depending on the PU division type. Within one PU, the same prediction process is applied, and the relevant information is decoded on the basis of the PU. The residual block is obtained by applying the prediction process based on the PU partition type. After that, the leaf CU is assigned to a transformation unit according to another quadtree structure similar to the coding tree of this CU. One of the main features of the HEVC structure is the division of CUs, PUs, and , and there are multiple compartment concepts including TU.

[0052] VVC supports nested multi-types with binary and ternary split segmentation structures. The quadtree with the tree replaces the concept of multiple partition unit types, i.e., CU, Removed the separation of PU and TU concepts (in the case of CU, the size of the maximum transformation length is too large). In the coding tree structure, CUs are square. A coding tree unit (CTU) is a block that can be 3D or rectangular in shape. First, it is partitioned by a quadtree structure. Then, the leaf nodes of this quadtree are partitioned by a multitype tree. As shown in Figure 5, the multi-type tree structure can be further divided into The structure includes a vertical binary split 502 (SPLIT_BT_VER), a horizontal binary split 504 (SPLIT_BT_HOR ), vertical ternary split 506 (SPLIT_TT_VER), and horizontal ternary split 508 (SPLIT_TT_HOR) There are four types of partitioning. The leaf nodes of the multitype tree are coding units (CUs). ), and this segmentation is called It is used for prediction and conversion without partitioning. This is mostly done in nested In a quadtree with a multi-type tree coding block structure, CU, PU, ​​and TU are The exception is that the maximum transformation length supported by In VTM1, the CU is smaller than the width or height of the different color components. Each CU consists of a coding block (CB). For example, one CU has one luminance CB and two chrominance CBs. The CB includes the color component CB (if the video is monochrome, i.e., if there is only one color component). outside). Partitioning a CU into multiple prediction units

[0053] In VVC, for each CU divided based on the above structure, Prediction is performed for the entire CU block, or in a sub-block manner as described below. Such an operational unit of prediction is called a Prediction Unit (or PU).

[0054] In the case of intra prediction (or intra-frame prediction), the size of the PU is usually CU In other words, prediction is performed for the entire CU block. In the case of interframe prediction (or interframe prediction), the size of the PU is equal to or larger than the size of the CU. In other words, if we split a CU into multiple PUs to be predicted, There is.

[0055] Examples where the PU size is smaller than the CU size include affine prediction mode, advanced temporal level Advanced Temporal Level Motion Vector Prediction (ATMVP) ) mode, and triangular prediction mode.

[0056] In affine prediction mode, a CU can be divided into multiple 4x4 PUs to be predicted. It is possible. A motion vector can be derived for each 4x4 PU, and the motion vectors of this 4x4 PU can be calculated accordingly. In ATMVP mode, a CU can perform motion compensation on one or more predictions. The target object can be divided into 8x8 PUs. For each 8x8 PU, a motion vector is calculated. We can then derive the rule and perform motion compensation for this 8x8 PU accordingly. In mode, a CU can be divided into two triangular shaped prediction units. Derive a motion vector for each U and perform motion compensation accordingly. The triangular prediction modes are , supported for inter prediction. The details of triangle prediction modes are shown below. . Triangle prediction mode (or triangle partition mode)

[0057] FIG. 6 illustrates splitting a CU into triangular prediction units according to an embodiment of the present disclosure. FIG.

[0058] The concept of triangular prediction mode introduces triangular partitions for motion compensation prediction. The prediction mode is also called a triangular prediction unit mode or a triangular partition mode. As shown, the CU 602 or 604 has two triplet assemblies arranged diagonally or anti-diagonally. The pixel prediction unit PU1 is divided into pixel prediction units PU2 and PU3 (i.e., as shown in CU602). It is divided from the upper left corner to the lower right corner, or from the upper right corner to the lower left corner as shown in CU604. Each triangle prediction unit in a CU is assigned its own prediction candidate list derived from the single prediction candidate list. The motion vectors are inter predicted using the single predicted motion vector and reference frame index. After predicting these triangle prediction units, adaptive weighting is applied to the diagonal edges. Then, the transformation and quantification processes are applied to the entire CU. applies only to the skip and merge modes of the current VVC. As shown in Figure 6, In general, CUs are shown as square blocks, but the triangular prediction mode can be used to predict non-square (i.e. This may also be applied to CUs with a rectangular shape.

[0059] The single prediction candidate list includes one or more candidates, each of which is a motion vector. Therefore, in the entire disclosure, the term "single prediction candidate list", "single prediction The terms "motion vector candidate list" and "uni-prediction merge list" are used interchangeably. and the terms "uni-predictive merge candidate list" and "uni-predictive motion vector" are used to refer to can be applied interchangeably. Single predictor motion vector candidate list

[0060] FIG. 7 is a schematic diagram showing the location of adjacent blocks according to an embodiment of the present disclosure.

[0061] In one example, the single predictor motion vector candidate list includes 2 to 5 single predictor motion vectors. In another example, other numbers are possible. The single predictor motion vector candidate list is derived from the block as shown in FIG. There are five spatially adjacent blocks (1 to 5) and two temporally co-located blocks. It is derived from the seven adjacent blocks that contain block (6 to 7). The motion vectors of the overlapping blocks are collected in a first merge list. Then, in a predetermined order, forming a single prediction candidate list based on the motion vectors of the first merge list according to Based on the order, the single predicted motion vector from the first merge list is selected first. The motion vectors are then added to the uni-prediction motion vector candidate list, and then to the bi-prediction motion vector reference image list. L0 or L1 motion vectors, and L2 or L3 reference picture list for bi-predictive motion vectors 1 motion vector, followed by the averaging of the L0 and L1 motion vectors of the bi-predictive motion vector The motion vectors obtained are successively added to the list. At that point, the number of candidates reaches the target number (the current In VVC, if the number is less than 5), zero motion vectors are added to the link to meet the target number. will be added to the list.

[0062] For each triangle PU, a predictor is derived based on its motion vector. Note that there is an overlapping region of the two predictors along the shared diagonal edge of the two triangles PU. As such, the derived predictor covers a larger area than the actual triangle PU. To derive the final prediction, the weighting process is applied to the diagonal edge region between the two predictors. Currently, the weighting factors applied to luma and chroma samples are {7 / 8, 6 / 8, 5 / 8, 4 / 8, 3 / 8, 2 / 8, 1 / 8} and {6 / 8, 4 / 8, 2 / 8}. Normal merge mode motion vector candidate list

[0063] According to the current VVC, normal merging is predicted without splitting the entire CU into multiple PUs. In the mode, the motion vector candidate list or merge candidate list is The procedure is different for the

[0064] First, refer to FIG. 8, which is a schematic diagram showing the locations of spatial merge candidates according to an embodiment of the present disclosure. As shown, based on the motion vectors from the neighboring blocks, spatial motion vector candidates In deriving spatial merge candidates for the current block 802, as shown in FIG. Select up to four merge candidates from the candidates in the best position. The order of derivation is A1 → B1 → B0 → A0 → (B2). Location B2 cannot use any PU at locations A1, B1, B0, or A0. This is taken into account only when intra-coding or intra-decoding is performed.

[0065] Next, we derive temporal merge candidates. Picture Order Count (POC) of the current image in the reference image list Based on the co-located PUs belonging to the image with the smallest difference, the scaled motion vector is calculated. The reference image list used to derive the co-located PU is slice The scaled motion for temporal merge candidates is explicitly signaled in the header of The motion vectors are the motion vector sequences for the temporal merge candidates according to an embodiment of the present disclosure. As shown by the dotted line in Fig. 9, which indicates the scaling, The scaled motion vectors of are at the same position using the POC distances tb and td. Scaled from the motion vector of PUcol_PU, where tb is the reference of the current picture curr_pic. is defined as the POC difference between the reference image curr_ref and the current image curr_pic, and td is the same position POC between the reference image col_ref of the image col_pic in the same position and the image col_pic in the same position The reference image index of the temporal merge candidate is set to 0. The practical implementation of the filtering process is described in the HEVC draft specification. , one motion vector is for reference picture list 0 and the other motion vector is Obtain and combine two motion vectors such as those for reference image list 1, Create bi-predictive merge candidates.

[0066] FIG. 10 is a schematic diagram illustrating candidate positions for temporal merge candidates, according to an embodiment of the present disclosure. This is a schematic diagram.

[0067] The position of the PU in the same position is selected from two candidate positions C3 and H, as shown in FIG. The PU at position H is not available, has been intra-coded, or is currently If it is outside the CTU, then location C3 is used to derive the temporal merge candidates. Otherwise, position H is used to derive temporal merging candidates.

[0068] As mentioned above, both spatial and temporal motion vectors are added to the merge candidate list. After the insertion, history-based merge candidates are added. The supplementary information is stored in a separate motion vector list and managed according to certain rules. It contains motion vectors from previously encoded / decoded CUs.

[0069] After the history-based candidate is inserted, if the merge candidate list is not full, the pairwise merge The size-averaged motion vector candidates are further added to this list. As the name suggests, Candidates of type are constructed by averaging the candidates already in the current list. Specifically, we select two candidates from the merge candidate list based on a certain order or rule. at a time and add the average motion vector of the two candidates to the current list.

[0070] After the pairwise average motion vectors are inserted, the merge candidate list is not yet full. If not, zero motion vectors are added until the list is full. Create the first merge list for triangle prediction using the normal merge list creation process

[0071] The current triangular prediction mode in VVC has the following overall procedure for forming the predictor: It shares some commonalities with the regular merge prediction mode. For example, both prediction modes In this mode, at least the neighboring spatial motion vectors of the current CU and the motion vectors at the same position are The merge list must be created based on the rule. There are some differences from the merge prediction mode.

[0072] For example, in both triangular prediction mode and regular merge prediction mode, a merge list is created. Although the detailed procedures for obtaining such a list differ,

[0073] These differences add cost to the implementation of the codec because they require additional logic. The procedure and logic for creating the merge list is the same for both the triangle prediction mode and the normal merge prediction mode. The measurement modes can be integrated and shared between the measurement modes.

[0074] In one example, unidirectional prediction (also called single prediction) merged prediction for triangular prediction modes When forming the merge list, before adding a new motion vector to the merge list, New motion vectors are completely truncated relative to those already in the list. In other words, the new motion vector is added to each of the motion vectors already in the uni-prediction merge list. The merge list is compared with the motion vector, and if it is different from all the motion vectors in the merge list, Otherwise, the new motion vector is not added to this list. Not added.

[0075] According to an example of the present disclosure, in triangular prediction mode, the unidirectional prediction merge list is Obtained or created from the normal merge mode motion vector candidate list, called the merge list It is possible to do so.

[0076] More specifically, to create a merge candidate list for the triangle prediction mode, we first Then, a first merge list is created based on the merge list creation process for normal merge prediction. The first merge list includes a number of candidates, each of which is a motion vector. Then, the motion vectors in this first merge list are used to find the optimal motion vector for the triangle prediction mode. A unidirectional prediction merge list is further created or derived.

[0077] In this case, the first merge list created will be in the general merge mode. or select a list size different from the list size for the normal merge mode. In one example of the present disclosure, the first merge list is a list of general merge modes. In another example of the present disclosure, the first merge list created has the same size as the general The list in merge mode has a different list size. Creating a unidirectional prediction merge list from the first merge list

[0078] According to an example of the present disclosure, a unidirectional prediction merge list for a triangular prediction mode may be: The first merge list may be generated or derived from the first merge list according to one of the following methods: It is possible.

[0079] In one example of this disclosure, to create or derive this unidirectional prediction merge list, first: The prediction list 0 motion vectors of the candidates in the first merge list are checked, and the unidirectional After this process, the unidirectional prediction merge list is filled with If this is not the case (e.g., the number of candidates in this list is still less than the target number), Then, the prediction list 1 motion vectors of the candidates in the first merge list are checked, and the unidirectional Selected for prediction merge list, but the unidirectional prediction merge list is not full In this case, the prediction list 0 zero vector is added to this unidirectional prediction merge list. But if this unidirectional prediction merge list is not full, the prediction list 1 zero vector is added to this unidirectional prediction merge list.

[0080] In another example of the present disclosure, for each candidate in the first merge list, its predicted list 0 motion Vector and Prediction List 1 Motion vectors are interleaved and unidirectionally predicted merged list More specifically, for each candidate in the first merge list, If it is a directional prediction motion vector, it is added directly to the unidirectional prediction merge list. Otherwise, if the candidate is a bidirectional motion vector predictor in the first merge list, First add the prediction list 0 motion vector to the forward prediction merge list, then add the prediction list 1 motion vector to the forward prediction merge list. Add the first motion vector. All motion vector candidates in the first merge list are checked. If a frame is checked and added, but the unidirectional prediction merge list is not yet full, It is possible to add a unidirectional prediction zero motion vector. For example, each reference frame For an index, we put the unidirectional prediction merge list into Adding Prediction List 0 Zero Motion Vector and Prediction List 1 Zero Motion Vector separately It is possible.

[0081] In yet another example of the present disclosure, first, a unidirectional predicted motion vector from the first merge list is After this process, the unidirectional prediction merge list is If not, for each bidirectional predictive motion vector in the first merge list, First add the prediction list 0 motion vector to the directional prediction merge list, then add the prediction list 1 motion vector to the directional prediction merge list. Add step 1 motion vector. After this process, the unidirectional prediction merge list is still full. If not, it is possible to add a unidirectional prediction zero motion vector. For example, For a reference frame index, the unidirectional prediction merge list is filled up. The prediction list 0 zero motion vector and the prediction list 1 zero motion vector are separately It is possible to add

[0082] In the above description, when a unidirectional prediction motion vector is added to the unidirectional prediction merge list, If the new motion vectors to be added are already in the unidirectional prediction merge list, Perform a motion vector truncation process to ensure that the motion vector is different. Such motion vector truncation processing is partially performed to reduce complexity. It is also possible to run it on all the merged images already in the unidirectional prediction merge list, e.g. Checks for new motion vectors that are added only for some motion vectors, not for others. In the extreme case, the motion vector truncation (i.e., the motion vector comparison operation) is not performed in the process. It will not be carried out. Creating a unidirectional prediction merge list from the first merge list based on an image prediction configuration

[0083] In one example of the present disclosure, the uni-prediction merge list is generated based on whether the current image uses backward prediction. For example, the uni-prediction merge list can be created adaptively based on ,The current image can be created in different ways depending on whether it uses backward prediction or not. The Picture Order Count (POC) values ​​of all reference images are multiplied by the POC value of the current image. means that the current picture does not use backward prediction.

[0084] In one example of the present disclosure, the current image does not use backward prediction or the current image is backward predicted. If it is decided not to use the prediction list 0 motion vector of the candidate in the first merge list, The vectors are checked and selected for the unidirectional prediction merge list, and then the prediction list of those candidates is The step 1 motion vector is selected for this unidirectional prediction merge list. If the merge list is not already full, add a single predicted zero motion vector. Otherwise, if the current image uses backward prediction, the first mark The Prediction List 0 and Prediction List 1 motion vectors of each candidate in the are checked and selected into the unidirectional prediction merge list in an interleaved manner as described above. That is, the prediction list 0 motion vector of the first candidate in the first merge list is The prediction list 1 motion vector of the first candidate is added, then the prediction list 2 motion vector of the second candidate is added, then the prediction list 3 motion vector of the first candidate is added, then the prediction list 4 motion vector of the second candidate is added, The prediction list 0 motion vector of the candidate is added, followed by the prediction list 1 motion vector of the second candidate. vectors are added, etc. At the end of the process, the unidirectional prediction merge list is still full. If not, it is possible to add a single predictive zero vector.

[0085] In another example of the present disclosure, if the current image does not use backward prediction, the first merge The candidate prediction list 1 motion vectors in the list are checked and selected for the unidirectional prediction merge list. The prediction list 0 motion vectors of these candidates are selected into the unidirectional prediction merge list. Also, if the unidirectional prediction merge list is not yet full, the single prediction zero motion vector is It is possible to add vectors to the current image, otherwise the current image will use backward prediction. In the case of the first merge list, the prediction list 0 motion vector and the prediction list 1 Motion vectors are checked and unidirectional prediction merged in an interleaved manner as described above. The first merge list can be selected as the first candidate, i.e., the first candidate in the first merge list can be selected as the first candidate in the first merge list. The predicted list 0 motion vector of the first candidate is added, then the predicted list 1 motion vector of the first candidate is added. The prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate. The prediction list 1 motion vector is added, etc. At the end of the process, the unidirectional prediction merge list If the buffer is not already full, it is possible to add a single predictive zero vector. .

[0086] In yet another example of the present disclosure, if the current image does not use backward prediction, first Only prediction list 0 motion vectors of candidates in the page list are checked, and unidirectional prediction merge Also, if the unidirectional prediction merge list is not already full, It is possible to add one prediction zero motion vector. Otherwise, the current picture will If one-way prediction is used, the prediction list 0 motion vectors of each candidate in the first merge list are and Prediction List 1 motion vectors are checked and are split in an interleaved fashion as above. The directional prediction merge list is selected, that is, the prediction list of the first candidate in the first merge list. The Prediction List 0 motion vector of the first candidate is added, then the Prediction List 1 motion vector of the first candidate is added. Then the prediction list 0 motion vector of the second candidate is added, followed by the prediction list 1 motion vector of the second candidate. The prediction list 1 motion vector is added, etc. At the end of the process, the unidirectional prediction merge list If ,is not already full, it is possible to add a single predictive zero vector.

[0087] In yet another example of the present disclosure, if the current image does not use backward prediction, first Only prediction list 1 motion vectors of candidates in the page list are checked, and unidirectional prediction merge Also, if the unidirectional prediction merge list is not already full, It is possible to add one prediction zero motion vector. Otherwise, the current picture will If one-way prediction is used, the prediction list 0 motion vectors of each candidate in the first merge list are and Prediction List 1 motion vectors are checked and are split in an interleaved fashion as above. It can be selected for the directional prediction merge list, i.e., the first merge list. The prediction list 0 motion vector of the second candidate is added, then the prediction list 1 motion vector of the first candidate is added. vector is added, then the prediction list 0 motion vector of the second candidate is added, followed by The second candidate prediction list 1 motion vector is added, and so on. If the prediction merge list is not already full, add a single prediction zero vector. is possible. Using the first merge list for triangle prediction without creating a unidirectional prediction merge list

[0088] In the above example, the unidirectional prediction merge list for triangle prediction is A unidirectional prediction merge list is created by selecting a motion vector from the However, in practice, the method is based on the assumption that the unidirectional (or single-prediction) merge list is physically formed. It is possible for the implementation to take different forms, whether or not they are implemented. So, we can directly use the first merge list without physically creating a unidirectional prediction merge list. For example, the list 0 and / or list 1 motion vectors of each candidate in the first merge list can be The vectors can be easily indexed based on some particular order, and can be directly extracted from the first merge list. It is possible to be accessed.

[0089] For example, the first merge list may be obtained from a decoder or other electronic device / component. In another example, the merge list creation process for normal merge prediction can be used. A first merge list including a plurality of candidates, each of which is one or more motion vectors, based on the first merge list. After creating the list, we do not create a unidirectional prediction merge list, but instead, A prediction vector includes a number of reference indices, which are references to candidate motion vectors in the merge list of the candidate vectors. Using the index list defined for the unidirectional merge candidate for the triangle prediction mode, The index list is a table of unidirectional prediction merge lists for triangular prediction. The unidirectional prediction merge list contains at least the reference index. The merge list contains a subset of the candidates in the first merge list that correspond to the index. The order of the selections should follow one of the selection orders mentioned in the example of creating a unidirectional prediction merge list. In practice, such an index list can be realized in a variety of forms. For example, it may be implemented as an explicit list. It may be realized or obtained by specific logic or program functions without creating a test.

[0090] In one example of the present disclosure, the index list is configured to determine whether the current image uses backward prediction. For example, the number of references in the index list can be determined adaptively. The reference index depends on whether the current picture uses backward prediction, i.e. The images are arranged based on the picture order count (POC) of the image and the POC of the reference image. The Picture Order Count (POC) values ​​of all reference images may be multiplied by the POC of the current image. If it is less than or equal to the value, it means that the current picture does not use backward prediction.

[0091] In one example of the present disclosure, if the current image does not use backward prediction, the first merge list The prediction list 0 motion vector of the candidate in the first merge list is the index as it appears in the first merge list. The unidirectional predictive merge candidates are indexed according to the same order as the If it is determined that the POC of the current image is greater than each of the POCs of the reference images, The indexes are the same as the order of the list 0 motion vectors of the candidates in the first merge list. Otherwise, if the current picture uses backward prediction, the first The list 0 motion vector and list 1 motion vector of each candidate in the merge list are List 0 motion vector of the first candidate in the image list, List 1 motion vector of the first candidate tor, list 0 motion vectors of the second candidate, then list 1 motion vectors of the second candidate , etc., with unidirectional prediction merges indexed based on the interleaving scheme The POC of the current image is used as a candidate, i.e., if the POC of the current image is at least 1 / 2 the POC of the reference image, If it is determined that the reference index is smaller than one of the bidirectional predictions in the first merge list, Each list 0 motion vector and list 1 motion vector of the candidate motion vectors The candidates in the first merge list are arranged according to the interleaving method of unidirectional motion vectors. If the candidate is a vector, then the zero motion vector is the unidirectional predictor that follows the candidate motion vector. It is indexed as a merge candidate if the current image uses backward prediction. For each candidate in the first merge list, a bidirectional predictive motion vector or a unidirectional predictive motion vector is selected. Provides two unidirectional motion vectors as unidirectional prediction merge candidates regardless of the vector do.

[0092] In another example of the present disclosure, if the current image does not use backward prediction, the first merge list is The prediction list 0 motion vectors of the candidates in the merge list are indexed as they are in the first merge list. The unidirectional predictive merge candidates are indexed according to the same order as the frame order. Otherwise, if the current image uses backward prediction, each candidate in the first merge list The list 0 motion vector and the list 1 motion vector of List 0 motion vectors of candidate 1, List 1 motion vectors of candidate 2, The list 0 motion vectors, then the list 1 motion vectors of the second candidate, etc. Used as unidirectional prediction merge candidates indexed based on the interleaving method. If a candidate in the first merge list is a unidirectional motion vector, the motion vector is used. The unidirectional prediction vector follows this candidate's motion vector, adding a certain motion offset. are indexed as potential merge candidates.

[0093] Therefore, if the candidate in the first merge list is a unidirectional motion vector, If it is determined that the POC of the image is smaller than at least one of the POCs of the reference images, The index is the motion vector of each candidate in the first merge list, and the zero motion vector or or the sum of this motion vector and the offset. do.

[0094] In the above process, new motions that are to be added to the unidirectional prediction merge list are When checking a vector, truncation can be performed completely or partially. If it is partially executed, the new motion vector is already in the uniprediction merge list. This means that the motion vectors in the image are compared to some, but not all, of the motion vectors in the image. In this process, the motion vector truncation (i.e., the motion vector comparison process) is performed. To carry out.

[0095] Also, when forming a single prediction merge list, we check whether the current image uses backward prediction. It is possible to adaptively perform motion vector pruning based on whether the image In the disclosed example of index list determination based on prediction configuration, the current image is a backward predicted image. If the estimation is not used, the motion vector truncation process is performed either completely or partially. If the image uses backward prediction, the motion vector truncation process is not performed. Selection of single prediction merge candidates for triangle prediction mode

[0096] In addition to the above examples, other methods for creating single-prediction merge lists or selecting single-prediction merge candidates may be used. A method is disclosed.

[0097] In one example of the present disclosure, a first merge list for a normal merge mode is created as follows: It is possible to select single prediction merge candidates for triangle prediction according to the rules below: .

[0098] For the motion vector candidates in the first merge list, the list 0 motion vector and the list Only one of the first motion vectors is used for triangulation prediction;

[0099] For a particular motion vector candidate in the first merge list, If the merge index value of the candidate motion vector is even, the list 0 motion vector is If available, it is used for triangle prediction, and this motion vector candidate is the List 1 motion vector. If there is no list 0 motion vector, then the list 0 motion vector is used for triangle prediction; and

[0100] For a particular motion vector candidate in the first merge list, If the merge index value of the candidate vector is odd, the list 1 motion vector is If available, it is used for triangle prediction, and this motion vector candidate is the List 1 motion vector. If there is no list 0 motion vector, then the list 0 motion vector is used for triangle prediction.

[0101] FIG. 11A illustrates a single predictor motion vector (MV) selection (or single predictor MV) for triangular prediction mode. In this example, the first merge list is The first 5 merge MV candidates produced are indexed from 0 to 4. Each row contains the list 0 and list 1 motion vectors of a candidate in the first merge list. There are two columns, each representing a different predictor. Each candidate in this list can be either uni-predictive or bi-predictive. In the case of a single prediction candidate, the list 0 motion vector and the list 1 motion vector can be It is possible to have only one motion vector, not both. In FIG. 11A, each merge event has both list 0 and list 1 motion vectors. For indexes, the motion vectors marked with an "x" are the first, if available. Used for triangle prediction when motion vectors marked with "x" are not available. , the unmarked motion vectors corresponding to the same merge index are then triangulated. It is used in.

[0102] The above concept can be extended to other examples. FIG. 11B shows the 3D 3D model for triangular prediction mode. FIG. 11B shows another example of single-predictor motion vector (MV) selection. The rules for selecting single prediction merge candidates for estimation are as follows:

[0103] For the motion vector candidates in the first merge list, the list 0 motion vector and the list Only one of the first motion vectors is used for triangulation prediction;

[0104] For a particular motion vector candidate in the first merge list, If the merge index value of the candidate motion vector is even, the list 1 motion vector is If available, it is used for triangle prediction, and this motion vector candidate is the List 1 motion vector. If there is no list 0 motion vector, then the list 0 motion vector is used for triangle prediction; and

[0105] For a particular motion vector candidate in the first merge list, If the merge index value of a candidate motion vector is odd, the list 0 motion vector is If available, it is used for triangle prediction, and this motion vector candidate is added to list 0 motion vectors. If there is no list 1 motion vector, then the list 1 motion vector is used for triangle prediction.

[0106] In one example, other different orders are defined and their motion vectors in the first merge list are The ML candidate is used to select a single prediction merge candidate for triangle prediction from the candidate candidates. Specifically, for a particular motion vector candidate in the first merge list, Deciding whether motion vectors or list 1 motion vectors are used for triangle prediction when available The determination does not necessarily depend on the parity of the index values ​​of the candidates in the first merge list as described above. For example, the following rule may be used:

[0107] For the motion vector candidates in the first merge list, the list 0 motion vector and the list Only one of the first motion vectors is used for triangulation prediction;

[0108] A plurality of motion vectors in the first merge list are selected based on a predetermined pattern. For candidates, their list 0 motion vectors are used for triangle prediction, if available. If a list 0 motion vector does not exist, the corresponding list 1 motion vector is used as the triangle prediction vector. Used in measurement;

[0109] Based on the same predetermined pattern, the remaining motion vectors in the first merge list are For candidates, their list 1 motion vectors are used for triangle prediction, if available. If a list 1 motion vector does not exist, the corresponding list 0 motion vector is used as the triangle prediction vector. Used in measurement;

[0110] 12A to 12D show a single predictive motion vector (MV) selection for triangular prediction mode. For each merge index, the motion vectors marked with an "x" are If available, it is first used to predict the triangle. Moves marked with an "x" If the vector is not available, the unmarked motion vectors corresponding to the same merge index are The resulting vector is then used for triangle prediction.

[0111] In FIG. 12A, the first three motion vector candidates in the first merge list are first If a list 0 motion vector is not available, then their list 0 motion vectors are checked. The corresponding list 1 motion vector is used for triangle prediction only if For the fourth and fifth motion vector candidates in the list, first, the list 1 motion vector Only if the list 1 motion vector is not available, the corresponding list 0 motion vector is used for triangle prediction. We show three other patterns for selecting single prediction merge candidates from the The examples shown are not limiting and there are further examples. For example, the examples shown in Figures 12A to 12D Horizontally and / or vertically mirrored versions of those patterns may also be used.

[0112] The selected uni-prediction merge candidate is indexed and selected from the first merge list as These selected single prediction merge candidates can be accessed directly; can be put into a single prediction merge list for triangle prediction. The uni-prediction merge list includes multiple uni-prediction merge candidates, each of which is , including one motion vector of the corresponding candidate in the first merge list. According to this, each candidate in the first merge list has a list 0 motion vector and a list 1 motion vector. Each single prediction merge candidate includes at least one of the vectors in the first merge list. The corresponding candidate list 0 motion vector and the list 1 motion vector of the Each uni-predictive merge candidate is associated with an integer-valued merge index. The list 0 and list 1 motion vectors are the predefined unipredictive merge candidates. The selection is based on the set rules.

[0113] In one example, for each uni-prediction merge candidate with an even merge index value, List 0 movement of corresponding candidates with the same merge index in the merge list of Select the vector as a single-prediction merge candidate; and select odd merge index values For each single prediction merge candidate with the same merge index in the first merge list, In another example, we select the corresponding candidate list 1 motion vector with even index. For each single prediction merge candidate having a merge index value, Select the corresponding candidate list 1 motion vectors with the same merge index in and for each uni-predictive merge candidate with an odd merge index value, List 0 of corresponding candidates with the same merge index in the merge list Select the vector.

[0114] In yet another example, for each single prediction merge candidate, If it is determined that the candidate list 1 motion vector is available, the list 1 motion vector is select the file as a uni-predictive merge candidate; and if list-1 motion vectors are not available select the list 0 motion vector of the corresponding candidate in the first merge list do.

[0115] In yet another example, each single prediction merge candidate having a merge index value within the first range may be For each candidate, the list 0 motion vector of the corresponding candidate in the first merge list is used as the single predictor. and select each single merge index value within the second range as a merge candidate. For a predicted merge candidate, the list 1 motion vector of the corresponding candidate in the first merge list Select .

[0116] In the above process, motion vector truncation can be performed as well. Such truncation can be done completely or partially. The new motion vector is compared to all the motion vectors already included in the single prediction merge list. This means that the merging candidates for triangle prediction are compared with some of the candidates, not all of them. Before being used as a new motion vector, only some, but not all, of the new motion vectors are used for truncation. This also means that the second motion vector needs to be checked. The first motion vector is compared with the second motion vector before being used as a merging candidate for triangle prediction. Only eye motion vectors are checked for truncation, all other motion vectors are truncated. In the extreme case, the motion vector is not checked for discarding. Do not perform motion vector truncation (i.e., motion vector comparison).

[0117] The method of forming a uni-predictive merge list in this disclosure is described with respect to triangular prediction mode. Although the above methods have been described, these methods are applicable to other prediction modes of a similar type. For example, A more general geometrical case in which a CU is partitioned into two PUs along a line that is not an exact diagonal. In partition prediction mode, two PUs are arranged in a geometric shape, such as a triangle, a wedge, or a trapezoid. In such a case, the prediction of each PU is performed in the same manner as the triangular prediction mode. The methods described herein are equally applicable.

[0118] FIG. 13 is a block diagram illustrating an apparatus for video encoding and decoding according to an embodiment of the present disclosure. The device 1300 may be a mobile phone, a tablet computer, or a digital broadcast terminal. , a tablet device, or a terminal such as a personal digital assistant.

[0119] As shown in FIG. 13, the device 1300 includes a processing unit 1302, a memory 1304, a power supply unit, 1306, a multimedia section 1308, an audio section 1310, an input / output (I / O) One or more of the interface 1312, the sensor unit 1314, and the communication unit 1316 It may include.

[0120] The processing unit 1302 normally handles display, telephone calls, data communications, camera operations, and recording operations. The processing unit 1302 controls the overall operation of the device 1300, including operations related to the above. One or more devices for carrying out instructions for implementing all or part of the steps of the method. The processor 1302 may include a processor 1320. One or more modules that contribute to the interaction between 1302 and other components For example, the processing unit 1302 may include a multimedia unit 1308 and a processing unit. 1302 includes a multimedia module for contributing to the interaction between the Good too.

[0121] The memory 1304 stores different types of data to support the operation of the device 1300. Examples of such data include any data stored in the device 1300. instructions for applications or methods, contact data, phone book data, messages , images, videos, etc. The memory 1304 may include any type of volatile or non-volatile The memory 1304 may be implemented by a static run-time memory device, a static storage device, or a combination thereof. Static Random Access Memory (SRAM), electrically erasable Programmable Read-Only Memory (EEPROM) Erasable Read-Only Memory, Erasable Programmable Read-Only Memory (EPROM) sable Programmable Read-Only Memory (PRO M: Programmable Read-Only Memory, ROM: Read-Only Memory y), magnetic memory, flash memory, magnetic disk, or compact disk. This is also fine.

[0122] The power supply unit 1306 supplies power to each component of the device 1300. A management system, one or more power sources, and a device 1300 for generating, managing, and and other components related to dispensing.

[0123] The multimedia section 1308 provides an output interface between the device 1300 and the user. In one example, the screen includes a liquid crystal display (LCD). The screen may include a touch panel (TP: Touch Panel). If the screen includes a touch panel, the screen may be a touch panel that receives input signals from a user. The touch panel may be realized as a touch screen. , one or more touch sensors for sensing slides, and gestures. The touch sensor not only detects the boundary of the touch or slide motion, but also The duration and pressure associated with a touch or slide action can also be detected. In an example, the multimedia section 1308 may include a front camera and / or a rear camera. When the device 1300 is in an operational mode, such as an imaging mode or a video mode, The front and / or rear camera can receive external multimedia data. can be done.

[0124] The audio section 1310 is configured to output and / or input audio signals. For example, the audio unit 1310 includes a microphone (MIC). The phone can be used to select an operating mode of the device 1300, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal is configured to receive an external audio signal when the received audio signal is in the The signal may be further stored in memory 1304 or transmitted via communication unit 1316. In one example, the audio unit 1310 may be a switch for outputting an audio signal. It further includes a peaker.

[0125] The I / O interface 1312 connects the processing unit 1302 to the peripheral interface module. The peripheral interface module provides an interface between the peripherals and the These buttons may include a keyboard, a click wheel, or buttons. This includes, but is not limited to, the power button, volume buttons, start button, and lock button. Not determined.

[0126] The sensor unit 1314 may include one or more sensors for providing status assessments at different aspects of the device 1300. For example, the sensor unit 1314 may detect the on / off state of the device 1300 and For example, the components may be located at the same position as the device 1300. The sensor section 1314 may also be a display and a keypad. The position of the components of the device 1300 changes, the presence or absence of a user's touch on the device 1300, It can also detect the orientation or acceleration / deceleration of the sensor 1300, as well as changes in temperature of the sensor 1300. The sensor unit 1314 is configured to detect the presence of a nearby object without physical contact. The sensor unit 1314 may include a CCD sensor for use in imaging applications. It may further include an optical sensor, such as a MOS or CCD image sensor. The sensor unit 1314 includes an acceleration sensor, a gyro sensor, a magnetic sensor, and a pressure sensor. , or a temperature sensor.

[0127] The communication unit 1316 is adapted to facilitate wired or wireless communication between the device 1300 and other devices. The device 1300 may be configured to communicate with a communication network such as WiFi, 4G, or a combination thereof. In one example, the communication unit 131 can access a wireless network based on a standard. 6 receives a notification signal or notification-related information from an external notification management system via a notification channel. In one example, the communication unit 1316 may be a near field communication (NFC) communication unit for facilitating short-range communication. For example, the NFC module may include a radio frequency identification (NFC) module. (RFID) technology, Infrared Data Association (IrDA) technology, Ultra-Wideband (UWB) technology, It may be implemented based on Bluetooth (BT) technology as well as other technologies.

[0128] In one example, the apparatus 1300 may include an application specific integrated circuit (ASIC) for performing the above method. IC), digital signal processor (DSP), digital signal processor (DSPD), Programmable logic devices (PLDs), field programmable gate arrays (FPGAs), A controller, microcontroller, microprocessor, or other electronic This may be achieved by one or more of the elements.

[0129] The non-transitory computer readable storage medium is, for example, a hard disk drive (HDD). DD), solid-state drives (SSD), flash memory, hybrid drives and solid-state high Hybrid Drive (SSHD), Read Only Memory (ROM), Compact Disc Reader It may be a read-only memory (CD-ROM), magnetic tape, floppy disk, etc. stomach.

[0130] FIG. 14 illustrates a video encoding scheme for motion compensated prediction with triangular prediction according to an embodiment of the present disclosure. 1 is a flowchart illustrating an example process for encoding and decoding.

[0131] In step 1402, the processor 1320 processes the video image by at least one The prediction unit (PU) is further divided into multiple coding units (CUs), which are then divided into two prediction units (PUs). The two PUs may include at least one geometrically shaped PU. For example, the geometric shape of the PU may be a pair of triangular PUs, a pair of wedge-shaped PUs, or other It is possible to include PU of the geometric shape.

[0132] In step 1404, the processor 1320 selects one or more motion vectors, each of which is a vector. For example, processor 13 may construct a first merge list that includes a plurality of candidates that include the vector. 20, based on the merge list construction process for normal merge prediction, the first merge The processor 1320 may also include other electronic devices or memory units. It is also possible to obtain the first merge list from

[0133] In step 1406, the processor 1320 performs single prediction for the triangular PU. obtain or derive a uni-prediction merge list, where the uni-prediction merge list is a set of multiple uni-predictors. Each single-prediction merge candidate is a first merge candidate of a corresponding candidate in the first merge list. Contains one motion vector.

[0134] In one example, an apparatus for video coding is provided. The apparatus includes a processor. and a memory 1320 configured to store instructions executable by the processor. 304, where the processor, upon execution of the instructions, performs a method as shown in FIG. The apparatus is configured to execute the following steps:

[0135] In another example, a non-transitory computer readable storage medium 13 having instructions stored thereon. 04. These instructions, when executed by the processor 1320, The processor executes the method shown in FIG.

[0136] The description of the present disclosure has been presented for convenience of illustration and is not intended to be exhaustive or limiting of the present disclosure. Numerous modifications, variations, and alternative implementations of the above are intended to be included in the It will be apparent to one skilled in the art having the benefit of the teachings presented in the description and associated drawings.

[0137] The embodiments explain the principles of the present disclosure, and allow those skilled in the art to understand and base on the disclosure for various implementations. The principles and various modifications are best utilized to suit the particular use anticipated. The present disclosure is therefore intended to be illustrative and not restrictive. The scope is not limited to the specific examples of implementations disclosed, and modifications and other implementations are within the scope of this disclosure. It should be understood that the term "common" refers to a combination of the above.

Claims

1. Partitioning a video image into a plurality of blocks; constructing a first merge list including a plurality of candidates for the current block, each candidate including one or more motion vectors; obtaining a plurality of uni-predictive merge candidates directly from the first merge list, each of the uni-predictive merge candidates including one motion vector of a corresponding candidate in the first merge list, without creating a uni-predictive merge candidate list; obtaining a predictor for the current block using the motion vectors of the plurality of single-prediction merge candidates; forming a video bitstream using the predictors of said blocks; Including, 11. A method for video encoding, wherein each candidate in the first merge list includes at least one of a list 0 motion vector and a list 1 motion vector, and each uni-predictive merge candidate includes only one of the list 0 motion vector and the list 1 motion vector of each candidate in the first merge list.

2. 2. The method of claim 1, wherein each uni-predictive merge candidate is associated with an integer-valued merge index, and the list 0 motion vector and the list 1 motion vector are selected based on pre-defined rules for the uni-predictive merge candidates.

3. 2. The method of claim 1, wherein obtaining a plurality of uni-predictive merge candidates comprises indexing list 0 and / or list 1 motion vectors of each candidate in the first merge list based on a particular order.

4. The method of claim 2 , wherein each uni-predictive merge candidate having a merge index value includes a list 0 or list 1 motion vector of a corresponding candidate in the first merge list having the same merge index.

5. 3. The method of claim 2, wherein each uni-predictive merge candidate having an even merge index value includes a list 0 motion vector of a corresponding candidate in the first merge list if the list 0 motion vector of the corresponding candidate in the first merge list is determined to be available, or includes a list 1 motion vector of a corresponding candidate in the first merge list if the list 0 motion vector of the corresponding candidate is determined to be unavailable.

6. 3. The method of claim 2, wherein each uni-predictive merge candidate having an odd merge index value includes the list 1 motion vector of a corresponding candidate in the first merge list if the list 1 motion vector of the corresponding candidate in the first merge list is determined to be available, or includes the list 0 motion vector of a corresponding candidate in the first merge list if the list 1 motion vector of the corresponding candidate is determined to be unavailable.

7. 3. The method of claim 2, wherein each uni-predictive merge candidate having a merge index value within a first range includes a list 0 motion vector of a corresponding candidate in the first merge list, and each uni-predictive merge candidate having a merge index value within a second range includes a list 1 motion vector of a corresponding candidate in the first merge list.

8. one or more processors; a memory configured to store instructions executable by the one or more processors; Including, An apparatus for video encoding, the one or more processors, when executing the instructions, causing the apparatus to perform a method according to any one of claims 1 to 7.

9. A method for transmitting a bitstream generated by a method according to any one of claims 1 to 7.

10. 1. A computer program having instructions for storing a bitstream, comprising: A computer program product, the bitstream comprising encoded video data generated by the encoding method according to any one of claims 1 to 7.

11. 1. A method for storing a bitstream generated by a video encoding method, comprising: The method of video encoding, Partitioning a video image into a plurality of blocks; constructing a first merge list including a plurality of candidates for the current block, each candidate including one or more motion vectors; obtaining a plurality of uni-predictive merge candidates directly from the first merge list, each of the uni-predictive merge candidates including one motion vector of a corresponding candidate in the first merge list, without creating a uni-predictive merge candidate list; obtaining a predictor for the current block using the motion vectors of the plurality of single-prediction merge candidates; forming a video bitstream using the predictor of the block; Including, A method for storing a bitstream, wherein each candidate in the first merge list includes at least one of a list 0 motion vector and a list 1 motion vector, and each single-prediction merge candidate includes only one of the list 0 motion vector and the list 1 motion vector of each candidate in the first merge list.

Images