Method and apparatus for video encoding / decoding using geometric partition mode

Abstract

A method and apparatus for video encoding / decoding are provided. A video decoding method according to the present disclosure may comprise the steps of: determining a motion vector of a first region of a current block; determining a motion vector of a second region of the current block; determining partition direction candidates by using a template block of the current block, and determining partition direction information among the partition direction candidates; generating a prediction block of the first region and generating a prediction block of the second region; determining a mixing region by using the segmentation direction information, and determining a mixing matrix by using the mixing region; and generating a prediction block of the current block by using the hybrid matrix, the prediction block of the first region, and the prediction block of the second region.

Description

Technical Field

[0001] In some embodiments, the present disclosure relates to methods and devices for encoding and decoding using a geometric partitioning mode. More specifically, the present disclosure relates to video encoding / decoding methods and devices for determining motion vectors of two partitioned regions in a geometric partitioning mode, determining the geometric partitioning mode, and determining a mixed region, thereby generating a prediction block for a current block. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and may not constitute prior art.

[0003] Since video data has a large amount of data compared to audio or still image data, without compression processing, the video data requires a large amount of hardware resources (including memory) to store or transmit the video data.

[0004] Therefore, encoders are typically used to compress and store or transmit video data. Decoders receive the compressed video data, decompress it, and play the decompressed video data. Video compression technologies include H.264 / Advanced Video Coding (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Coding (VVC). VVC offers approximately 30% or greater improvement in decoding efficiency compared to HEVC.

[0005] However, as image size, resolution, and frame rate continue to increase, the amount of data to be encoded will also increase. Therefore, new compression technologies are needed that provide higher decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] The geometric partition mode is a method of generating a prediction block for the current block by weighted averaging two inter-frame prediction signals generated by dividing the coding unit (CU) into two regions and performing inter-frame prediction independently in the two regions. When generating the prediction block of the current block by using the geometric partition mode, it is necessary to improve encoding / decoding efficiency.

[0007] Detailed Description of the Invention

[0008] Technical issues

[0009] In some embodiments, the present disclosure seeks to provide methods and apparatus for determining motion vectors for various regions by performing template matching in a geometric partitioning mode.

[0010] The present disclosure also seeks to provide methods and apparatus for determining segmentation direction information by performing template matching in a geometric segmentation mode.

[0011] The present disclosure also seeks to provide a method and apparatus for generating a final prediction block of a current block by weighted averaging the prediction blocks of various regions using a mixing matrix.

[0012] The present disclosure also seeks to provide methods and apparatus for improving video encoding / decoding efficiency.

[0013] The present disclosure also seeks to provide a recording medium storing a bitstream generated by the video encoding / decoding method or apparatus of the present disclosure.

[0014] The present disclosure also seeks to provide a method and apparatus for transmitting a bitstream generated by the video encoding / decoding method or apparatus of the present disclosure. Summary of the Invention

[0015] The present disclosure provides a video decoding method in at least one embodiment, which may include: determining a motion vector of a first region of a current block by using a merge candidate list; determining a motion vector of a second region of the current block by using the motion vector of the first region and a template block of the current block; determining a partition direction candidate by using the template block of the current block, and determining partition direction information of the current block among the partition direction candidates; generating a prediction block of the first region by using the motion vector of the first region, and generating a prediction block of the second region by using the motion vector of the second region; determining a mixed region by using the partition direction information, and determining a mixed matrix by using the mixed region; and generating a prediction block of the current block by using the mixed matrix, the prediction block of the first region, and the prediction block of the second region.

[0016] The present disclosure provides a video encoding method in another embodiment, which may include: determining a motion vector of a first region of a current block by using a merge candidate list; determining a motion vector of a second region of the current block by using the motion vector of the first region and a template block of the current block; determining a partitioning direction candidate by using the template block of the current block, and determining partitioning direction information of the current block among the partitioning direction candidates; generating a prediction block of the first region by using the motion vector of the first region, and generating a prediction block of the second region by using the motion vector of the second region; determining a mixed region by using the partitioning direction information, and determining a mixed matrix by using the mixed region; and generating a prediction block of the current block by using the mixed matrix, the prediction block of the first region, and the prediction block of the second region.

[0017] Furthermore, the present disclosure provides a method of transmitting a bitstream generated by the video encoding method or apparatus according to the present disclosure.

[0018] Furthermore, the present disclosure may provide a recording medium storing a bit stream generated by the video encoding method or apparatus according to the present disclosure.

[0019] Furthermore, the present disclosure may provide a recording medium storing a bitstream, which is received and decoded by a video decoding apparatus according to the present disclosure and used to reconstruct a video.

[0020] Beneficial effects

[0021] The present disclosure may provide a method and apparatus for determining motion vectors of respective regions by performing template matching in a geometric partitioning mode.

[0022] Furthermore, the present disclosure may provide a method and apparatus for determining segmentation direction information by performing template matching in a geometric segmentation mode.

[0023] Furthermore, the present disclosure may provide a method and apparatus for generating a final prediction block of a current block by performing weighted averaging on prediction blocks of respective regions using a mixing matrix.

[0024] Furthermore, the present disclosure may provide a method and apparatus for improving video encoding / decoding efficiency.

[0025] Effects obtainable with the present disclosure are not limited to the above-mentioned effects, and other unmentioned effects can be clearly understood by those skilled in the art from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a block diagram of a video encoding device that may implement the techniques of this disclosure.

[0027] Figure 2 A method for partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0028] Figure 3a and Figure 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0029] Figure 4 Neighboring blocks of the current block are shown.

[0030] Figure 5 is a block diagram of a video decoding device that may implement the techniques of this disclosure.

[0031] Figure 6 is a diagram illustrating a method of applying a geometric partitioning mode to a 32×32 block according to at least one embodiment of the present disclosure.

[0032] Figure 7a and Figure 7b is a diagram illustrating angle parameters and distance parameters in a geometric partitioning mode according to at least one embodiment of the present disclosure.

[0033] Figure 8 is a diagram illustrating a lookup table for segmentation direction information in a geometric segmentation mode according to at least one embodiment of the present disclosure.

[0034] Figure 9 is a diagram illustrating a geometric partition boundary and two sub-regions within a current block in a geometric partition mode according to at least one embodiment of the present disclosure.

[0035] Figure 10 is a flowchart of a process of generating a prediction block of a current block by using a geometric partitioning mode according to at least one embodiment of the present disclosure.

[0036] Figure 11 is a diagram illustrating a current block and template blocks adjacent to the current block according to at least one embodiment of the present disclosure.

[0037] Figure 12 is a diagram illustrating a method of performing template matching according to at least one embodiment of the present disclosure.

[0038] Figure 13 is a diagram illustrating a method of determining segmentation direction information by performing template matching according to at least one embodiment of the present disclosure.

[0039] Figure 14 is a flow chart of a video decoding process according to at least one embodiment of the present disclosure.

[0040] Figure 15 is a flow chart of a video encoding process according to at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] Hereinafter, some embodiments of the present disclosure are described in detail with reference to the accompanying drawings. In the following description, the same reference numerals represent the same elements, even though the elements are shown in different drawings. In addition, in the following description of some embodiments, for the purpose of clarity and brevity, detailed descriptions of related known components and functions may be omitted when it is considered that they would obscure the subject matter of the present disclosure.

[0042] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present disclosure. Figure 1 , describing a video encoding device and components of the device.

[0043] The encoding apparatus may include a picture divider 110 , a predictor 120 , a subtractor 130 , a transformer 140 , a quantizer 145 , a rearrangement unit 150 , an entropy encoder 155 , an inverse quantizer 160 , an inverse transformer 165 , an adder 170 , a loop filter unit 180 , and a memory 190 .

[0044] Each component of the encoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0045] A video consists of one or more sequences including multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a picture is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (CTUs). In addition, each CTU is divided into one or more coding units (CUs) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to all CUs included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more pictures is encoded as a picture parameter set (PPS) or a picture header. In addition, information commonly referenced by multiple pictures is encoded into a sequence parameter set (SPS). In addition, information commonly referenced by one or more SPSs is encoded into a video parameter set (VPS). In addition, information commonly applied to a tile or tile group can also be encoded as the syntax of the tile or tile group header. The syntax included in an SPS, a PPS, a slice header, a tile, or a tile group header may be referred to as a high-level syntax.

[0046] The picture partitioner 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or PPS and delivered to a video decoding apparatus.

[0047] The picture partitioner 110 partitions each picture constituting a video into a plurality of coding tree units (CTUs) of a predetermined size, and then recursively partitions the CTUs using a tree structure. Leaf nodes in the tree structure become coding units (CUs), which are basic units of encoding.

[0048] The tree structure can be a quadtree (QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures among the QT structure, the BT structure, and the TT structure are mixed. For example, a quadtree plus binary tree (QTBT) structure can be used, or a quadtree plus binary tree ternary tree (QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be referred to as a multi-type tree (MTT).

[0049] Figure 2 is a diagram for describing a method of partitioning a block by using a QTBTTT structure.

[0050] like Figure 2 As shown, first, the CTU can be divided into a QT structure. The quadtree partitioning can be recursive until the size of the partitioned block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is divided into four nodes of the lower layer is encoded by the entropy encoder 155 and transmitted to the video decoding device by the signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can also be divided into at least one of the BT structure or the TT structure. There can be multiple partitioning directions in the BT structure and / or the TT structure. For example, there can be two directions, namely, the direction in which the blocks of the corresponding nodes are partitioned horizontally and the direction in which the blocks of the corresponding nodes are partitioned vertically. As shown in FIG. Figure 2 As shown, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether the node is split and a flag indicating the split direction (vertical or horizontal) and / or a flag indicating the split type (binary or trifurcated) if the node is split are encoded by the entropy encoder 155 and signaled to the video decoding device.

[0051] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes in the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the partition tree structure and becomes a CU as a basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag through the above scheme.

[0052] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type are encoded by the entropy encoder 155 and delivered to the video decoding device. At the same time, a type in which the block of the corresponding node is split into two blocks that are asymmetric to each other may be additionally presented. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0053] A CU can have different sizes depending on the QTBT or QTBTTT partitioning of a CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., a leaf node of a QTBTTT) is referred to as the "current block." Due to the QTBTTT partitioning, the shape of the current block can be rectangular in addition to square.

[0054] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0055] Typically, each current block in a picture can be predictively coded. Typically, prediction of the current block can be performed using intra-prediction techniques (using data from the picture containing the current block) or inter-prediction techniques (using data from pictures coded before the picture containing the current block). Inter-prediction includes both unidirectional and bidirectional prediction.

[0056] The intra-frame predictor 122 predicts pixels in the current block by using pixels (reference pixels) located around the current block in the current picture including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3aAs shown, the plurality of intra prediction modes may include two non-directional modes (including a planar mode and a DC mode) and may include 65 directional modes. Neighboring pixels and an arithmetic equation to be used are defined differently according to each prediction mode.

[0057] In order to perform efficient directional prediction for a current block having a rectangular shape, the following can be additionally used: Figure 3b The directional modes (#67 to #80, intra prediction modes #-1 to #-14) are indicated by dotted arrows in FIG. The directional modes may be referred to as “wide-angle intra prediction modes”. Figure 3b , the arrows indicate the corresponding reference samples used for prediction without indicating the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width and height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height smaller than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle smaller than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees is available.

[0058] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block using multiple intra-frame prediction modes and may further select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by using a rate-distortion analysis for multiple test intra-frame prediction modes and may further select an intra-frame prediction mode having the best rate-distortion characteristics from the test mode.

[0059] The intra-frame predictor 122 selects an intra-frame prediction mode from among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) determined according to the selected intra-frame prediction mode and an arithmetic equation. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0060] The inter-frame predictor 124 generates a prediction block for the current block using motion compensation processing. The inter-frame predictor 124 searches for a block most similar to the current block in a reference picture that was encoded and decoded earlier than the current picture, and generates a prediction block for the current block using the searched block. Furthermore, a motion vector (MV) is generated that corresponds to the displacement between the current block in the current picture and the prediction block in the reference picture. Typically, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma components. Motion information, including information about the reference picture and information about the motion vector used to predict the current block, is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0061] The inter-frame predictor 124 may also interpolate a reference picture or reference block to increase prediction accuracy. In other words, by applying filter coefficients to a plurality of consecutive integer samples including two integer samples, subsamples between the two consecutive integer samples are interpolated. When searching for a block most similar to the current block for the interpolated reference picture, the motion vector may be expressed with decimal unit precision rather than integer sample unit precision. The precision or resolution of the motion vector may be set differently for each target region to be encoded (e.g., a unit such as a slice, tile, CTU, CU, etc.). When this adaptive motion vector resolution (AMVR) is applied, information regarding the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information regarding the motion vector resolution applied to each CU is signaled. The information regarding the motion vector resolution may be information indicating the precision of the motion vector difference, which will be described below.

[0062] Meanwhile, the inter-frame predictor 124 can perform inter-frame prediction using bidirectional prediction. Bidirectional prediction uses two reference pictures and two motion vectors representing the block positions in each reference picture that are most similar to the current block. The inter-frame predictor 124 selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for blocks in the corresponding reference pictures that are most similar to the current block to generate first and second reference blocks. Furthermore, a prediction block for the current block is generated by averaging or weighted averaging the first and second reference blocks. Furthermore, motion information including information about the two reference pictures used to predict the current block and information about the two motion vectors is delivered to the entropy encoder 155. Reference picture list 0 may consist of pictures that precede the current picture in display order among the pre-reconstructed pictures, and reference picture list 1 may consist of pictures that follow the current picture in display order among the pre-reconstructed pictures. However, although not particularly limited thereto, a pre-reconstructed picture after the current picture in display order may be additionally included in reference picture list 0. Conversely, a pre-reconstructed picture before the current picture may also be additionally included in reference picture list 1.

[0063] To minimize the number of bits consumed for encoding motion information, various methods can be used.

[0064] For example, when the reference picture and motion vector of the current block are the same as those of a neighboring block, information that can identify the neighboring block is encoded to deliver the motion information of the current block to the video decoding device. This method is called merge mode.

[0065] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as “merge candidates”) from the neighboring blocks of the current block.

[0066] As neighboring blocks for deriving merge candidates, such as Figure 4 As shown, all or some of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current picture can be used. In addition, in addition to the current picture in which the current block is located, blocks located in a reference picture (which may be the same as or different from the reference picture used to predict the current block) can also be used as merge candidates. For example, a block located at the same position as the current block or a block adjacent to the block located at the same position in the reference picture can also be used as a merge candidate. If the number of merge candidates selected by the method described above is less than the preset number, a zero vector is added to the merge candidate.

[0067] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates using adjacent blocks. A merge candidate to be used as motion information for the current block is selected from the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and delivered to the video decoding device.

[0068] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients used for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, and images with screen content.

[0069] Hereinafter, the merge mode and the merge skip mode are collectively referred to as the merge / skip mode.

[0070] Another method for encoding motion information is the Advanced Motion Vector Prediction (AMVP) mode.

[0071] In the AMVP mode, the inter-frame predictor 124 derives a motion vector predictor candidate for the motion vector of the current block by using the neighboring blocks of the current block. As the neighboring blocks for deriving the motion vector predictor candidate, the neighboring blocks may be used. Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1, and upper left block B2 adjacent to the current block in the current picture shown. Furthermore, in addition to the current picture in which the current block is located, blocks located within a reference picture (which may be the same as or different from the reference picture used to predict the current block) may also be used as adjacent blocks for deriving motion vector predictor candidates. For example, a block co-located with the current block or a block adjacent to the co-located block within the reference picture may be used. If the number of motion vector candidates selected by the above method is less than a preset number, a zero vector is added to the motion vector candidates.

[0072] The inter-frame predictor 124 derives a motion vector predictor candidate by using the motion vector of the adjacent block, and determines a motion vector predictor for the motion vector of the current block by using the motion vector predictor candidate. In addition, a motion vector difference is calculated by subtracting the motion vector predictor from the motion vector of the current block.

[0073] The motion vector predictor can be obtained by applying a predefined function (e.g., center value and average calculation) to the motion vector predictor candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector predictor candidate are blocks that have already been encoded and decoded, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode information for identifying the motion vector predictor candidate. Therefore, in this case, information about the motion vector difference and information about the reference picture used to predict the current block are encoded.

[0074] Meanwhile, the motion vector predictor can also be determined by selecting any one of the motion vector predictor candidates. In this case, information for identifying the selected motion vector predictor candidate is additionally encoded together with information about the motion vector difference and information about the reference picture used to predict the current block.

[0075] The subtractor 130 generates a residual block by subtracting the prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0076] The transformer 140 converts the residual signal in the residual block having pixel values in the spatial domain into transform coefficients in the frequency domain. The transformer 140 can transform the residual signal in the residual block by using the total size of the residual block as a transform unit, or can also divide the residual block into multiple sub-blocks and perform the transform using the sub-blocks as transform units. Alternatively, the residual block is divided into two sub-blocks, a transform region and a non-transform region, to transform the residual signal using only the transform region sub-block as a transform unit. Here, the transform region sub-block can be one of two rectangular blocks with a size ratio of 1:1 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicates that only the sub-block is transformed, and directional (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoder 155 and signaled to the video decoding device. In addition, the size of the transform region sub-block can have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) of dividing the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0077] At the same time, the transformer 140 can perform transformations on the residual block separately in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a multiple transform set (MTS). The transformer 140 can select a transformation function pair with the highest transformation efficiency in the MTS and can transform the residual block in each of the horizontal and vertical directions. Information about the transformation function pair in the MTS (mts_idx) is encoded by the entropy encoder 155 and is signaled to the video decoding device.

[0078] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using a quantization parameter and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 can also immediately quantize the relevant residual block without having to transform any block or frame. The quantizer 145 can also apply different quantization coefficients (scaling values) according to the position of the transform coefficient in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions can be encoded and signaled to the video decoding device.

[0079] The rearrangement unit 150 may perform recalibration of coefficient values with respect to the quantized residual values.

[0080] The rearrangement unit 150 can convert the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can output a 1D coefficient sequence by scanning the DC coefficient into high-frequency domain coefficients using zigzag scanning or diagonal scanning. Depending on the size of the transform unit and the intra-frame prediction mode, vertical scanning that scans the 2D coefficient array in the column direction and horizontal scanning that scans the 2D block type coefficients in the row direction can also be used instead of zigzag scanning. In other words, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined from zigzag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0081] The entropy encoder 155 generates a bitstream by encoding the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 using various encoding schemes including context-based adaptive binary arithmetic coding (CABAC), Exponential Golomb, and the like.

[0082] In addition, the entropy encoder 155 encodes information related to block partitioning (such as CTU size, CTU partition flag, QT partition flag, MTT partition type, MTT partition direction, etc.) to allow the video decoding device to partition the block equally with the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about the intra-frame prediction mode) or inter-frame prediction information (in the case of merge mode, merge index, and in the case of AMVP mode, information about the reference picture index and motion vector difference) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about the quantization parameter and information about the quantization matrix).

[0083] The inverse quantizer 160 dequantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0084] The adder 170 reconstructs the current block by adding the reconstructed residual block and the prediction block generated by the predictor 120. When intra prediction is performed on a next sequential block, pixels in the reconstructed current block may be used as reference pixels.

[0085] The loop filter unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit 180 as a loop filter may include all or some of the deblocking filter 182, the sample adaptive offset (SAO) filter 184, and the adaptive loop filter (ALF) 186.

[0086] The deblocking filter 182 filters the boundaries between reconstructed blocks to remove blocking artifacts caused by block-by-block encoding / decoding, while the SAO filter 184 and ALF 186 additionally filter the deblocked filtered video. The SAO filter 184 and ALF 186 are filters used to compensate for differences between reconstructed and original pixels due to lossy decoding. The SAO filter 184 applies offsets as CTU units to enhance subjective image quality and coding efficiency. Meanwhile, the ALF 186 performs block-by-block filtering, applying different filters based on the corresponding block boundaries and the degree of variation to compensate for distortion. Information regarding the filter coefficients used for the ALF can be encoded and signaled to the video decoding device.

[0087] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0088] The video encoding device may store the bit stream of the encoded video data in a non-transitory storage medium, or transmit the bit stream to the video decoding device through a communication network.

[0089] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present disclosure. Figure 5 , describes a video decoding device and components of the device.

[0090] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filter unit 560 , and a memory 570 .

[0091] and Figure 1 Similar to the video encoding device of the present invention, each component of the video decoding device can be implemented as hardware or software or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and a microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0092] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0093] The entropy decoder 510 extracts information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS) to determine the size of the CTU and partitions the picture into CTUs of the determined size. Furthermore, the CTU is determined as the highest level of the tree structure, i.e., the root node, and partition information of the CTU can be extracted to partition the CTU using the tree structure.

[0094] For example, when a CTU is segmented using a QTBTTT structure, the first flag (QT_split_flag) associated with the QT segmentation is first extracted to segment each node into four nodes in the lower layer. In addition, for the nodes corresponding to the leaf nodes of the QT, the second flag (mtt_split_flag) associated with the MTT segmentation, the segmentation direction (vertical / horizontal), and / or the segmentation type (binary / trifurcated) are extracted to segment the corresponding leaf nodes into the MTT structure. Thus, each node below the leaf node of the QT is recursively segmented into a BT structure or a TT structure.

[0095] As another example, when a CTU is split using the QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, zero or more recursive MTT splits may occur after zero or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or, conversely, multiple QT splits may occur.

[0096] As another example, when a CTU is split using a QTBT structure, a first flag (QT_split_flag) related to the splitting of the QT is extracted to split each node into four nodes of the lower layer. In addition, a split flag (split_flag) indicating whether a node corresponding to a leaf node of the QT is further split into BTs and split direction information are extracted.

[0097] Meanwhile, when the entropy decoder 510 determines the current block to be decoded by partitioning using a tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements for inter-prediction information (i.e., motion vectors and reference pictures referenced by the motion vectors).

[0098] Also, the entropy decoder 510 extracts quantization-related information of the current block and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0099] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding apparatus.

[0100] The inverse quantizer 520 dequantizes the quantized transform coefficients and dequantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 can also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 can perform dequantization by applying a matrix of quantization coefficients (scaling values) from the video encoding device to the 2D array of quantized transform coefficients.

[0101] The inverse transformer 530 reconstructs a residual signal by inversely transforming the dequantized transform coefficient from the frequency domain into the spatial domain to generate a residual block of the current block.

[0102] In addition, when the inverse transformer 530 inversely transforms a partial area (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) indicating that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the area that has not been inversely transformed by using the value "0" as the residual signal to generate the final residual block of the current block.

[0103] In addition, when MTS is applied, the inverse transformer 530 uses the MTS information (mts_idx) transmitted from the video encoding device signal to determine the transformation function or transformation matrix applied in each of the horizontal and vertical directions. The inverse transformer 530 also performs inverse transformation on the transformation coefficients in the transformation block in the horizontal and vertical directions by using the determined transformation function.

[0104] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0105] The intra predictor 542 determines an intra prediction mode of the current block among a plurality of intra prediction modes according to the syntax elements for the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0106] The inter predictor 544 determines a motion vector of a current block and a reference picture to which the motion vector refers by using a syntax element for the inter prediction mode extracted from the entropy decoder 510 .

[0107] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a block to be decoded later, pixels within the reconstructed current block are used as reference pixels.

[0108] The loop filter unit 560, which serves as a loop filter, may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundaries between reconstructed blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering to compensate for differences between reconstructed and original pixels due to lossy decoding. The filter coefficients of the ALF are determined using information about the filter coefficients decoded from the bitstream.

[0109] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one picture are reconstructed, the reconstructed picture can be used as a reference picture for inter-frame prediction of blocks in a picture to be encoded later.

[0110] The following embodiments may be performed by the intra-frame prediction unit 122, the transform unit 140, and the inverse transform unit 165 in the video encoding device. They may also be performed by the inverse transform unit 530 and the intra-frame prediction unit 542 in the video decoding device. Hereinafter, the video decoding device is described as performing the respective processing steps, but the video encoding device may also perform the same steps.

[0111] Figure 6 This figure illustrates a method for applying geometric partitioning mode to a 32×32 block according to at least one embodiment of the present disclosure. In geometric partitioning mode, a coding unit is partitioned into two regions using a straight partitioning boundary. Inter-frame prediction is performed in each of the two partitioned regions using different motion information. An inter-frame prediction block is generated for each of the two partitioned regions. A final prediction block in geometric partitioning mode is generated by taking a weighted average of the two inter-frame prediction blocks. Geometric partitioning mode uses angle and distance parameters to define partitioning boundary regions defined by straight lines.

[0112] refer to Figure 6 , the 32×32 block is divided into two regions. Inter-frame prediction can be performed on each of the two divided regions. It can be an angle parameter. ρ can be a distance parameter. By using the angle parameter and the distance parameter, a straight line that divides the 32×32 block can be established.

[0113] Figure 7a and7b is a diagram illustrating angle parameters and distance parameters in a geometric partitioning mode according to at least one embodiment of the present disclosure.

[0114] refer to Figure 7a , the angle parameter can be defined as a total of 20 quantized angles by symmetrically dividing the 360-degree range within the coding unit.

[0115] refer to Figure 7b , the distance parameter can be defined as four quantized distances. From the total 80 segmentation directions that can be generated by the combination of the angle parameter and the distance parameter, 10 redundant segmentation directions and 6 segmentation directions that are redundant using binary tree segmentation and ternary tree segmentation can be excluded. Therefore, a total of 64 segmentation directions can be used in the geometric segmentation mode.

[0116] Figure 8 is a diagram illustrating a lookup table for segmentation direction information in a geometric segmentation mode according to at least one embodiment of the present disclosure. A combination of an angle parameter and a distance parameter can be represented by a lookup table. The segmentation direction information can be transmitted for each coding unit. The geometric segmentation mode can configure a merge candidate list for the geometric segmentation mode from a conventional merge candidate list, which only includes unidirectional motion information. This simplifies the encoding of motion information and reduces the number of possible combinations. By using the merge candidate list for the geometric segmentation mode, the merge index that has been used for each segmented region can be transmitted.

[0117] refer to Figure 8 , based on information about an angle parameter (e.g., angleIdx) and information about a distance parameter (e.g., distanceIdx), split direction information (e.g., merge_gpm_partition_idx) may be determined. merge_gpm_partition_idx according to a combination of angleIdx and distanceIdx may be represented by a lookup table. The value of merge_gpm_partition_idx may be in the range of 0 to 63. merge_gpm_partition_idx may be transmitted for each coding unit.

[0118] Figure 9 2 is a diagram illustrating a geometric partition boundary and two sub-regions within a current block in a geometric partition mode according to at least one embodiment of the present disclosure. In the following, a video decoding device is described as performing various processes, but a video encoding device may also perform these processes.

[0119] refer to Figure 9When the prediction mode of the current block is the inter prediction mode and the geometric partition mode, the video decoding device partitions the current block into sub-region 0 and sub-region 1 by using the geometric partition boundary. The video decoding device generates a prediction block in sub-region 0 and a prediction block in sub-region 1. The video decoding device generates a final prediction block for the current block by using the prediction block of sub-region 0 and the prediction block of sub-region 1.

[0120] Figure 10 is a flowchart of a process of generating a prediction block of a current block by using a geometric partitioning mode according to at least one embodiment of the present disclosure.

[0121] refer to Figure 10 , information indicating whether to perform the derivation-based geometric partitioning mode (e.g., Derive_GPM_flag) can be parsed. When Derive_GPM_flag is a first value (e.g., 0), the video decoding device can perform the conventional geometric partitioning mode. If Derive_GPM_flag is a second value (e.g., 1), the video decoding device can perform the derivation-based geometric partitioning mode. The derivation-based geometric partitioning mode is a mode in which the motion vector of sub-region 0 is used to determine the motion vector of sub-region 1, and the motion vector of sub-region 0 and the motion vector of sub-region 1 are used to determine the partitioning direction information.

[0122] When Derive_GPM_flag is a second value (e.g., 1), the video decoding device generates a candidate list for the geometric partition mode (S1010). The video decoding device may determine a conventional merge candidate list as a candidate list for the geometric partition mode. The video decoding device may use the conventional merge candidate list to generate a candidate list for the geometric partition mode. The video decoding device may generate the candidate list for the geometric partition mode by performing the following operations: if the index of the conventional merge candidate list is an even number, selecting motion information present in the first reference picture list, and if the index is an odd number, selecting motion information present in the second reference picture list.

[0123] The video decoding apparatus determines motion information of the sub-region 0 (S1020). The video decoding apparatus may determine the motion information of the sub-region 0 by using an index (e.g., PM_idx_0) of the parsed motion information of the sub-region 0. The video decoding apparatus may determine the motion information of the sub-region 0 from a candidate list of geometric partitioning modes using PM_idx_0.

[0124] The video decoding apparatus determines motion information of sub-region 1 (S1030). The video decoding apparatus may determine the motion information of sub-region 1 by using the index of the parsed motion information of sub-region 1 (e.g., PM_idx_1) and the prediction block of sub-region 0. The video decoding apparatus determines partition direction information (S1040). The video decoding apparatus may determine the partition direction information by using the motion information of sub-region 1 and the index for the partition direction information (e.g., GPMmode_idx).

[0125] The video decoding device determines a blending area (S1050). The video decoding device may determine the blending area by using the segmentation direction information, an index for the blending area (e.g., BlendingArea_idx), or the prediction block of sub-area 0 and the prediction block of sub-area 1. The video decoding device generates a final prediction block of the current block by weighted averaging the prediction block of sub-area 0 and the prediction block of sub-area 1 (S1060). The video decoding device may determine a blending matrix using the blending area. By using the blending matrix, the video decoding device may weighted average the prediction block of sub-area 0 and the prediction block of sub-area 1 to generate the final prediction block of the current block.

[0126] Figure 11 is a diagram illustrating a current block and template blocks adjacent to the current block according to at least one embodiment of the present disclosure.

[0127] refer to Figure 11 , the reconstructed template blocks adjacent to the current block can be at the top, left, and upper-left side of the current block. The reconstructed template blocks adjacent to the current block can be blocks A, B, C, D, and E. The position and size of the reconstructed template blocks adjacent to the current block can be changed based on the size and aspect ratio of the current block.

[0128] Figure 12 is a diagram illustrating a method of performing template matching according to at least one embodiment of the present disclosure.

[0129] refer to Figure 12 , the video decoding device may use PM_idx_0 to determine the motion information of sub-region 0 from the candidate list of geometric partitioning modes. The motion information of sub-region 0 includes the motion vector MV0 of sub-region 0. The video decoding device may determine the reconstructed template blocks adjacent to the current block as blocks A, B, C, D, and E. The video decoding device may use the motion vector MV0 to determine the template blocks as blocks A', B', C', D', and E' surrounding the reference block in the reference picture.

[0130] The video decoding device can perform template matching by using block A, block B, block C, block D and block E, and block A', block B', block C', block D and block E' to determine the motion vector of sub-region 1. The video decoding device can perform template matching in the order of block A, block B, block C, block D and block E. However, the order of template matching can be changed without restriction. Template matching can be to find the block with the smallest error value of the loss function. The loss function can be, for example, mean square error (MSE) or sum of absolute difference (SATD). The video decoding device can calculate the error value of the loss function between block A and block A', the error value of the loss function between block B and block B', the error value of the loss function between block C and block C', the error value of the loss function between block D and block D', and the error value of the loss function between block E and block E'.

[0131] Among blocks A, B, C, D, and E, the video decoding device may select n blocks having the smallest error value of the loss function. The n blocks may be any predefined number of blocks. The number n may be determined based on the number of template blocks. The video decoding device may use template blocks other than the selected n blocks to determine the motion vector of sub-region 1. In another embodiment, the video decoding device may perform template matching by selecting template blocks having an error value less than a predefined threshold.

[0132] The video decoding apparatus may determine the motion vector of the sub-region 1 by using a template block other than the matched template block and a motion vector other than MV0 which is the motion vector of the sub-region 0 included in the candidate list of the geometric partitioning mode.

[0133] For example, among blocks A, B, C, D, and E, the blocks selected by using template matching may be blocks A, B, and D, and the candidate list of the geometric partitioning mode may include three motion vectors. Then, the video decoding device may determine the motion vector of sub-region 1 by using blocks C and E, and two motion vectors other than MV0 from the motion vectors included in the candidate list of the geometric partitioning mode. If the two motion vectors are MV' and MV", the video decoding device may use the motion vector MV' to determine reference block 1 in reference picture 1. The video decoding device may determine blocks C" and E" among the template blocks surrounding reference block 1 as template blocks corresponding to blocks C and E. The video decoding device may use the motion vector MV" to determine reference block 2 in reference picture 2. The video decoding device may determine blocks C"' and E"' among the template blocks surrounding reference block 2 as template blocks corresponding to blocks C and E.

[0134] The video decoding device can calculate K1 and K2, where K1 is the sum of the error value of the loss function between the C block and the C” block and the error value of the loss function between the E and E” blocks, and K2 is the sum of the error value of the loss function between the C block and the C”' block and the error value of the loss function between the E and E”' blocks. If K1 is less than K2, the video decoding device can determine the motion vector MV' as the motion vector of sub-region 1. If K2 is less than K1, the video decoding device can determine the motion vector MV” as the motion vector of sub-region 1.

[0135] The video decoding apparatus may determine the motion vector of the sub-region 1 by using the parsed PM_idx_1 and the template block other than the matched template block, and the motion vectors other than MV0, which is the motion vector of the sub-region 0, included in the candidate list of the geometric partition mode. If m is the number of motion vectors included in the candidate list of the geometric partition mode, PM_idx_1 may be an integer value less than m.

[0136] For example, among blocks A, B, C, D, and E, the blocks selected by template matching may be blocks A, B, and D, and the candidate list of the geometric partition mode may include three motion vectors. Then, the video decoding apparatus may determine the motion vector of the sub-region 1 by using the parsed PM_idx_1 and blocks C and E, and two motion vectors other than MV0 from the motion vectors included in the candidate list of the geometric partition mode.

[0137] If the two motion vectors are MV' and MV", the video decoding device can use the motion vector MV' to determine the reference block 1 in the reference picture 1. The video decoding device can determine the C" block and the E" block among the template blocks surrounding the reference block 1 as the template blocks corresponding to the C block and the E block. The video decoding device can use the motion vector MV" to determine the reference block 2 in the reference picture 2. The video decoding device can determine the C"' block and the E"' block among the template blocks surrounding the reference block 2 as the template blocks corresponding to the C block and the E block.

[0138] The video decoding device can calculate K1 and K2, K1 is the sum of the error value of the loss function between the C block and the C” block and the error value of the loss function between the E and E” blocks, and K2 is the sum of the error value of the loss function between the C block and the C”' block and the error value of the loss function between the E and E”' blocks. If K1 is less than K2, the video decoding device can arrange the motion vector candidates in the order of MV' and MV”. The video decoding device can use the parsed PM_idx_1 to determine the motion vector of the sub-region 1 between the arranged motion vector candidates. If K2 is less than K1, the video decoding device can arrange the motion vector candidates in the order of MV” and MV’. The video decoding device can use the parsed PM_idx_1 to determine the motion vector of the sub-region 1 between the arranged motion vector candidates.

[0139] Figure 13 is a diagram illustrating a method of determining segmentation direction information by performing template matching according to at least one embodiment of the present disclosure.

[0140] refer to Figure 13 , the video decoding apparatus may determine the reconstructed template blocks adjacent to the current block as blocks A, B, C, D, and E. The video decoding apparatus may use the motion vector MV0 of the sub-region 0 to determine the template blocks as blocks A', B', C', D', and E' around the reference block in the reference picture L0.

[0141] Having already executed Figure 12 In the case where the template matching described in

[15] is used to match blocks A, B, and D, the video decoding device may use blocks C and E to determine a partition direction candidate for the current block from the 64 types of partition direction information. The video decoding device determines the partition direction candidate by using a geometric partition boundary that passes between blocks B and C and between blocks D and E. For example, if there are three geometric partition boundaries that pass between blocks B and C and between blocks D and E among the 64 geometric partition boundaries, the video decoding device may determine three partition direction candidates from the 64 types of partition direction information.

[0142] The video decoding apparatus determines the partition direction information of the current block among the partition direction candidates by using the weighted sum signal of the template block in the reference picture. The weighted sum signal P of the template block for the partition direction candidate G (i, j) can be expressed as shown in Equation 1.

[0143] [Equation 1]

[0144] P G (i,j)=(W B (i,j)×P0+(2 k -W B (i,j))×P1+(2k-1 ))>>k

[0145] Here, W B Is the geometric matrix of the segmentation direction candidate. B The value of each coefficient can be from 0 to 2 k Integer value in the range. Here, k is an integer. k can be any predefined positive integer value. For each segmentation direction candidate, the predefined W B P0 may be a signal of a region in the reference picture L0 (including the B' block, F' block, C' block, D' block, G' block, and E' block, which are template blocks of the reference block). The F block exists between the B block and the C block, and the G block exists between the D block and the E block. The B' block, F' block, C' block, D' block, G' block, and E' block correspond to the B block, F block, C block, D block, G block, and E block, and are template blocks in the reference picture L0.

[0146] P1 may be a signal of a region in the reference picture L1 (including B' block, F' block, C' block, D' block, G' block, and E' block as template blocks of the reference block). B' block, F' block, C' block, D' block, G' block, and E' block correspond to B block, F block, C block, D block, G block, and E block, and are template blocks in the reference picture L1. The video decoding device may calculate P for each partition direction candidate. G (i, j). For example, if three geometric partition boundaries among the 64 geometric partition boundaries pass from between block B and block C to between block D and block E, the video decoding device can determine three partition direction candidates. The video decoding device can determine three partition direction candidates by using Equation 1, W B , P0 and P1, to calculate the P of each of the three segmentation direction candidates G (i, j).

[0147] The video decoding device may calculate a weighted sum signal P for each of the three partition direction candidates. G The error value of the loss function between (i, j) and the area including the B block, F block, C block, D block, G block and E block in the area of the template block including the current block. The loss function may be MSE, SATD, etc. For example, if the three determined segmentation direction candidates are direction information 1, direction information 2 and direction information 3, the video decoding device may calculate P for direction information 1. G1 (i, j), direction information 2 P G2 (i, j) and direction information 3 P G3 (i, j).

[0148] The video decoding device can calculate P G1The error value E1 of the loss function between (i, j) and the region containing B block, F block, C block, D block, G block and E block. The video decoding device can calculate P G2 The error value E2 of the loss function between (i, j) and the region containing B block, F block, C block, D block, G block and E block. The video decoding device can calculate P G3 The error value E3 of the loss function between (i, j) and the region including the B block, the F block, the C block, the D block, the G block, and the E block. If the value of E1 is the smallest among the calculated error values E1, E2, and E3, the video decoding device may determine that the segmentation direction information of the current block is direction information 1. If the value of E2 is the smallest among the calculated error values E1, E2, and E3, the video decoding device may determine that the segmentation direction information of the current block is direction information 2. If the value of E3 is the smallest among the calculated error values E1, E2, and E3, the video decoding device may determine that the segmentation direction information of the current block is direction information 3.

[0149] The video decoding device can use the P of each segmentation direction candidate G (i, j) and the region including B block, F block, C block, D block, G block and E block and the parsed GPMmode_idx are used to determine the partition direction information of the current block. The value of GPMmode_idx can be an integer value less than the number of partition direction candidates.

[0150] For example, if the three determined segmentation direction candidates are direction information 1, direction information 2, and direction information 3, the video decoding device may calculate P of direction information 1. G1 (i, j), direction information 2 P G2 (i, j) and direction information 3 P G3 (i, j). The video decoding device can calculate P G1 The error value E1 of the loss function between (i, j) and the area containing B block, F block, C block, D block, G block and E block, and calculate P G2 The error value E2 of the loss function between (i, j) and the area containing B block, F block, C block, D block, G block and E block, and calculate P G3 The error value E3 of the loss function between (i, j) and the region including the B block, F block, C block, D block, G block, and E block. The video decoding device may rearrange the partition direction candidates in the order of decreasing error values. If E1, E3, and E2 are in the order of decreasing error values, the video decoding device may reorder the partition direction candidates to be arranged as direction information 1, direction information 3, and direction information 2. The video decoding device may use the parsed GPMmode_idx and the reordered partition direction candidates to determine the partition direction information of the current block.

[0151] Determine the mixing area

[0152] The video decoding device generates a prediction block for sub-region 0 and a prediction block for sub-region 1 using the determined partition direction information of the current block and the motion information of sub-region 0 and sub-region 1. The video decoding device determines a mixed region by using the partition direction information, etc. The video decoding device uses the mixed region to determine a mixing matrix W used in the process of weighted averaging the prediction block for sub-region 0 and the prediction block for sub-region 1. B f (i, j).

[0153] For example, the video decoding device may determine the blending area using the size of the current block and the partition direction information of the current block. Here, the blending area may be a predefined area based on the size of the current block and the partition direction information of the current block. For example, the video decoding device may determine the blending area using the parsed BlendingArea_idx. For example, the video decoding device may determine the blending area using an initial prediction signal around the geometric partition boundary of the current block.

[0154] The video decoding device can use the determined mixing area to determine the mixing matrix W B f (i, j). The mixing matrix can be a predefined matrix based on the mixing area. B f Each coefficient in (i, j) can have a value from 0 to 2 w Here, w can be determined based on the size of the current block and / or the partition direction information of the current block. Here, w can be an integer value.

[0155] Perform weighted averaging

[0156] The video decoding device uses the prediction block of sub-region 0, the prediction block of sub-region 1 and the mixing matrix W B f (i, j), to generate the final prediction block of the current block. The final prediction block P of the current block K (i, j) can be expressed as Equation 2.

[0157] [Equation 2]

[0158]

[0159] Here, P0 may be a prediction block obtained by using the motion vector of sub-region 0, and P1 may be a prediction block obtained by using the motion vector of sub-region 1. B f (i, j) is the mixing matrix determined according to the mixing area. W can be any predefined value.

[0160] Final prediction block of chroma block

[0161] When the video decoding device generates the final prediction block of the chrominance block, the video decoding device may generate the final prediction block of the chrominance block by sampling the mixed area of the luminance block at the corresponding position based on the color format. In another embodiment, the video decoding device may generate the final prediction block of the chrominance block by sampling the motion information, segmentation direction information, or mixed area of the luminance block based on the color format.

[0162] Figure 14 is a flow chart of a video decoding process according to at least one embodiment of the present disclosure.

[0163] refer to Figure 14 , the video decoding apparatus determines a motion vector of a first region of the current block by using a merge candidate list (S1410). The process of determining the motion vector of the first region includes: determining a candidate list of a geometric partition mode by using an index of the merge candidate list, and determining the motion vector of the first region by using the candidate list of the geometric partition mode.

[0164] The video decoding apparatus determines a motion vector of a second region of the current block by using the motion vector of the first region and the template block of the current block (S1420). The process of determining the motion vector of the second region includes: determining a template block of a reference block in a first reference picture by using the motion vector of the first region, calculating a first error value of a loss function between the template block of the current block and the template block of the reference block, and determining a block for which template matching is not performed using the first error value.

[0165] The process of determining the motion vector of the second region also includes: determining a template block located in the reference picture and corresponding to a block that is not template matched by using a motion vector included in the candidate list of the geometric partitioning pattern, excluding the motion vector of the first region, calculating a second error value of a loss function between the block that is not template matched and the template block in the reference picture, and using the second error value to determine the motion vector of the second region.

[0166] The video decoding apparatus determines partition direction candidates by using the template block of the current block, and determines partition direction information among the partition direction candidates (S1430). The process of determining the partition direction information includes: determining a third region including a template block corresponding to a block not template-matched in a first reference picture, and determining a fourth region including a template block corresponding to a block not template-matched in a second reference picture by using a motion vector of the second region.

[0167] The process of determining the segmentation direction information further includes: calculating a weighted sum signal of the segmentation direction candidates by applying the third region, the fourth region, and the geometric matrix to the first equation 1, calculating a third error value of a loss function between the weighted sum signal and a region of the template block containing the current block, and determining the segmentation direction information using the third error value. The geometric matrix may be a matrix predefined based on the segmentation direction candidates.

[0168] The video decoding device generates a prediction block of the first area by using the motion vector of the first area, and generates a prediction block of the second area by using the motion vector of the second area (S1440). The video decoding device determines a mixed area by using the segmentation direction information, and determines a mixing matrix by using the mixed area (S1450). The video decoding device generates a prediction block of the current block by using the mixing matrix, the prediction block of the first area, and the prediction block of the second area (S1460). The process of generating the prediction block of the current block includes: applying the mixing matrix, the prediction block of the first area, and the prediction block of the second area to the second equation 2 to generate the prediction block of the current block. The mixing matrix can be a matrix predefined based on the mixed area.

[0169] Figure 15 is a flow chart of a video encoding process according to at least one embodiment of the present disclosure.

[0170] refer to Figure 15 , the video encoding apparatus determines a motion vector of a first region of a current block by using a merge candidate list (S1510). The process of determining the motion vector of the first region includes: determining a candidate list of a geometric partition mode by using an index of the merge candidate list, and determining the motion vector of the first region by using the candidate list of the geometric partition mode.

[0171] The video encoding apparatus determines a motion vector of a second region of the current block by using the motion vector of the first region and the template block of the current block (S1520). The process of determining the motion vector of the second region includes: determining a template block of a reference block in a first reference picture by using the motion vector of the first region, calculating a first error value of a loss function between the template block of the current block and the template block of the reference block, and determining a block for which template matching is not performed using the first error value.

[0172] The process of determining the motion vector of the second region also includes: determining a template block located in the reference picture and corresponding to a block that is not template matched by using a motion vector included in the candidate list of the geometric partitioning pattern, excluding the motion vector of the first region, calculating a second error value of a loss function between the block that is not template matched and the template block in the reference picture, and using the second error value to determine the motion vector of the second region.

[0173] The video encoding apparatus determines partition direction candidates by using the template block of the current block, and determines partition direction information among the partition direction candidates (S1530). The process of determining the partition direction information includes: determining a third region including a template block corresponding to a block not template-matched in a first reference picture, and determining a fourth region including a template block corresponding to a block not template-matched in a second reference picture by using a motion vector of the second region.

[0174] The process of determining the segmentation direction information further includes: calculating a weighted sum signal of the segmentation direction candidates by applying the third region, the fourth region, and the geometric matrix to the first equation 1, calculating a third error value of a loss function between the weighted sum signal and a region of the template block containing the current block, and determining the segmentation direction information using the third error value. The geometric matrix may be a matrix predefined based on the segmentation direction candidates.

[0175] The video encoding device generates a prediction block of the first area by using the motion vector of the first area, and generates a prediction block of the second area by using the motion vector of the second area (S1540). The video encoding device determines a mixed area by using the segmentation direction information, and determines a mixing matrix by using the mixed area (S1550). The video encoding device generates a prediction block of the current block by using the mixing matrix, the prediction block of the first area, and the prediction block of the second area (S1560). The process of generating the prediction block of the current block includes: applying the mixing matrix, the prediction block of the first area, and the prediction block of the second area to the second equation 2 to generate the prediction block of the current block. The mixing matrix can be a matrix predefined based on the mixed area.

[0176] Although the steps in the various flowcharts are described as being performed sequentially, these steps merely illustrate the technical concepts of some embodiments of the present disclosure. Therefore, a person skilled in the art of the present disclosure may perform the steps by changing the order depicted in the various figures or by performing two or more steps in parallel. Therefore, the steps in the various flowcharts are not limited to the chronological order shown.

[0177] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware, and / or a combination of hardware, software, and firmware. It should also be understood that the functional components described in this disclosure are labeled "... units" to strongly emphasize their independent implementation capabilities.

[0178] Meanwhile, the various methods or functions described in some embodiments may be implemented as instructions stored in a non-transitory recording medium that can be read and executed by one or more processors. For example, the non-transitory recording medium may include various types of recording devices in which data is stored in a form readable by a computer system. For example, the non-transitory recording medium may include a storage medium such as an erasable programmable read-only memory (EPROM), a flash drive, an optical drive, a magnetic hard drive, and a solid-state drive (SSD).

[0179] Although the embodiments of the present disclosure have been described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications, additions, and substitutions are possible without departing from the concept and scope of the present disclosure. Therefore, for the sake of brevity and clarity, the embodiments of the present disclosure have been described. The scope of the technical concept of the embodiments of the present disclosure is not limited by the figures. Therefore, it will be appreciated by those skilled in the art that the scope of the present disclosure should not be limited by the embodiments explicitly described above, but by the claims and their equivalents.

[0180] CROSS-REFERENCE TO RELATED APPLICATIONS

[0181] This application claims priority from Korean Patent Application No. 10-2022-0168738, filed on December 6, 2022, and Korean Patent Application No. 10-2023-0172358, filed on December 1, 2023, which are hereby incorporated by reference in their entirety.

Claims

1. A video decoding method, comprising: determining a motion vector of a first region of a current block by using a merge candidate list; determining a motion vector of a second region of the current block by using the motion vector of the first region and a template block of the current block; determining partition direction candidates by using the template block of the current block, and determining partition direction information of the current block among the partition direction candidates; generating a prediction block of the first region by using the motion vector of the first region, and generating a prediction block of the second region by using the motion vector of the second region; determining a mixing area by using the segmentation direction information, and determining a mixing matrix by using the mixing area; and A prediction block of the current block is generated by using the mixing matrix, the prediction block of the first region, and the prediction block of the second region.

2. The method according to claim 1, wherein Determining the motion vector of the first area includes: Determining a candidate list of geometric partitioning modes by using the index of the merge candidate list; and The motion vector of the first region is determined by using the candidate list of the geometric partitioning mode.

3. The method according to claim 2, wherein: Determining the motion vector of the second area includes: determining a template block of a reference block in a first reference picture by using the motion vector of the first region; Calculating a first error value of a loss function between the template block of the current block and the template block of the reference block; and The first error value is used to determine blocks for which template matching is not performed.

4. The method according to claim 3, wherein: Determining the motion vector of the second area further includes: determining a template block located in a reference picture and corresponding to the block not subjected to template matching, by using motion vectors included in the candidate list of the geometric partitioning mode, excluding the motion vector of the first region; Calculating a second error value of a loss function between the block that has not been template-matched and the template block that is located in the reference picture and corresponds to the block that has not been template-matched; and The motion vector of the second region is determined using the second error value.

5. The method according to claim 4, wherein Determining the segmentation direction information includes: determining a third region including the template block located in the first reference picture and corresponding to the block for which template matching is not performed; and By using the motion vector of the second region, a fourth region including a template block located in a second reference picture and corresponding to the block not subjected to template matching is determined.

6. The method according to claim 5, wherein: Determining the segmentation direction information includes: calculating a weighted sum signal of the segmentation direction candidates by applying the third area, the fourth area, and a geometric matrix predefined based on the segmentation direction candidates to a first equation; calculating a third error value of a loss function between the weighted sum signal and a region of the template block including the current block; and The segmentation direction information is determined using the third error value.

7. The method according to claim 1, wherein Generating the predicted block of the current block includes: generating the prediction block of the current block by applying the mixing matrix, the prediction block of the first region, and the prediction block of the second region to a second equation, The mixing matrix is predefined based on the mixing area.

8. A video encoding method, comprising: determining a motion vector of a first region of a current block by using a merge candidate list; determining a motion vector of a second region of the current block by using the motion vector of the first region and a template block of the current block; determining partition direction candidates by using the template block of the current block, and determining partition direction information of the current block among the partition direction candidates; generating a prediction block of the first region by using the motion vector of the first region, and generating a prediction block of the second region by using the motion vector of the second region; determining a mixing area by using the segmentation direction information, and determining a mixing matrix by using the mixing area; and A prediction block of the current block is generated by using the mixing matrix, the prediction block of the first region, and the prediction block of the second region.

9. The method according to claim 8, wherein Determining the motion vector of the first area includes: Determining a candidate list of geometric partitioning modes by using the index of the merge candidate list; and The motion vector of the first region is determined by using the candidate list of the geometric partitioning mode.

10. The method according to claim 9, wherein: Determining the motion vector of the second area includes: determining a template block of a reference block in a first reference picture by using the motion vector of the first region; Calculating a first error value of a loss function between the template block of the current block and the template block of the reference block; and The first error value is used to determine blocks for which template matching is not performed.

11. The method according to claim 10, wherein: Determining the motion vector of the second area further includes: determining a template block located in a reference picture and corresponding to the block not subjected to template matching, by using motion vectors included in the candidate list of the geometric partitioning mode, excluding the motion vector of the first region; Calculating a second error value of a loss function between the block that has not been template-matched and the template block that is located in the reference picture and corresponds to the block that has not been template-matched; and The motion vector of the second region is determined using the second error value.

12. The method according to claim 11, wherein Determining the segmentation direction information includes: determining a third region including the template block located in the first reference picture and corresponding to the block for which template matching is not performed; and By using the motion vector of the second region, a fourth region including a template block located in a second reference picture and corresponding to the block not subjected to template matching is determined.

13. The method according to claim 12, wherein: Determining the segmentation direction information includes: calculating a weighted sum signal of the segmentation direction candidates by applying the third area, the fourth area, and a geometric matrix predefined based on the segmentation direction candidates to a first equation; calculating a third error value of a loss function between the weighted sum signal and a region of the template block including the current block; and The segmentation direction information is determined using the third error value.

14. The method according to claim 8, wherein Generating the predicted block of the current block includes: generating the prediction block of the current block by applying the mixing matrix, the prediction block of the first region, and the prediction block of the second region to a second equation, The mixing matrix is predefined based on the mixing area.

15. A computer-readable recording medium storing a bitstream generated by a video encoding method, the video encoding method comprising: determining a motion vector of a first region of a current block by using a merge candidate list; determining a motion vector of a second region of the current block by using the motion vector of the first region and a template block of the current block; determining partition direction candidates by using the template block of the current block, and determining partition direction information of the current block among the partition direction candidates; generating a prediction block of the first region by using the motion vector of the first region, and generating a prediction block of the second region by using the motion vector of the second region; determining a mixing area by using the segmentation direction information, and determining a mixing matrix by using the mixing area; and A prediction block of the current block is generated by using the mixing matrix, the prediction block of the first region, and the prediction block of the second region.

Images