Method and apparatus for video coding with geometric partition-based inter / intra prediction

Abstract

Methods and apparatus for video coding and decoding with geometric partition-based inter / intra prediction are disclosed. In this embodiment, an image decoding device decodes geometric partition information of a current block from a bitstream, and partitions the current block into a plurality of sub-regions according to the geometric partition information. An image decoding device parses an index indicating a prediction mode for each of a plurality of sub-regions. The image decoding device generates a predictor for each sub-block by using inter prediction, intra prediction, or template matching on the basis of the value of the index, and generates a final predictor for the current block by weighted summation of the predictors for the plurality of sub-regions.

Description

Technical Field

[0001] The present invention relates to a method and apparatus for using inter-frame / intra-frame prediction based on geometric partitioning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and may not necessarily constitute prior art.

[0003] Since video data has a large amount of data compared to audio data or still image data, video data requires a large amount of hardware resources (including memory) to store or transmit the video data without being compressed.

[0004] Accordingly, the encoder is generally used to compress and store or transmit video data. The decoder receives the compressed video data, decompresses the received compressed video data, and plays the decompressed video data. Video compression technologies include H.264 / Advanced Video Codec (AVC), High Efficiency Video Coding (HEVC), and Versatile Video Codec (VVC), which improves the coding efficiency of HEVC by about 30% or more.

[0005] However, as image size, resolution and frame rate gradually increase, the amount of data to be encoded will also increase. Accordingly, new compression technologies are needed that provide higher encoding and decoding efficiency and improved image enhancement effects than existing compression technologies.

[0006] VVC predicts the current block by using square and rectangular partitions based on quadtrees, binary trees or ternary trees. VVC also uses inter-frame prediction technology, Geometric Partitioning Mode (GPM), for prediction based on more flexible partitions. GPM performs prediction by using a mode index (which indicates which predefined modes partition the current block into two regions) and motion vector information for each region. The encoder sends a mode index (which indicates which predefined modes of GPM partition the current block into two regions) and motion vector information. The decoder partitions the current block into two regions according to the received GPM partition mode. The decoder then generates a prediction block for each region and then uses a weighted summation process to generate the final prediction block. At this point, the weighted summation coefficient used in the weighted summation process can be determined according to the GPM partition mode. On the other hand, in order to improve video encoding and decoding efficiency and enhance video quality, it is necessary to consider improving the GPM technology mentioned above. Summary of the invention

[0007] Technical issues

[0008] The present invention is directed to providing a video coding method and apparatus that, when predicting sub-blocks based on geometric partitions in a prediction of a current block, effectively performs inter-frame and / or intra-frame prediction on an area of ​​each partition.

[0009] Technical Solution

[0010] At least one aspect of the present invention provides a method for predicting a current block performed by a video decoding device. The method includes decoding geometric partition information of the current block from a bitstream. The method also includes partitioning the current block into a plurality of sub-regions according to the geometric partition information. The method also includes parsing an index of each of the plurality of sub-regions, and the index indicates a prediction mode for each sub-region. The method also includes generating a prediction factor for each sub-block by utilizing inter-frame prediction, intra-frame prediction, or template matching based on the value of the index.

[0011] Another aspect of the present invention provides a method for predicting a current block performed by a video encoding device. The method includes determining geometric partition information of the current block. The method also includes partitioning the current block into a plurality of sub-regions according to the geometric partition information. The method also includes determining an index of each of the plurality of sub-regions, and the index indicates a prediction mode for each sub-region. The method also includes generating a prediction factor for each sub-block by using inter-frame prediction, intra-frame prediction, or template matching based on the value of the index.

[0012] Another aspect of the present invention provides a computer-readable recording medium storing a bitstream generated by a video encoding method. The video encoding method includes determining geometric partition information of a current block. The video encoding method also includes partitioning the current block into a plurality of sub-regions according to the geometric partition information. The video encoding method also includes determining an index of each of the plurality of sub-regions, and the index indicates a prediction mode for each sub-region. The video encoding method also includes generating a prediction factor for each sub-block based on the value of the index by using inter-frame prediction, intra-frame prediction, or template matching.

[0013] Beneficial Effects

[0014] As described above, the present invention provides a video coding method and apparatus that effectively performs inter-frame and / or intra-frame prediction on the region of each partition when predicting sub-blocks based on geometric partitions in the prediction of the current block. Therefore, the video coding method and apparatus improves video coding efficiency and enhances video image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention.

[0016] Figure 2A method of partitioning blocks using a quadtree plus binary tree ternary tree (QTBTTT) structure is shown.

[0017] Figure 3a and Figure 3b Multiple intra prediction modes are shown including a wide-angle intra prediction mode.

[0018] Figure 4 Shows the neighboring blocks of the current block.

[0019] Figure 5 is a block diagram of a video decoding device that can implement the technology of the present invention.

[0020] Figure 6 is a block diagram detailing a portion of a video decoding apparatus according to at least one embodiment of the present invention.

[0021] Figure 7 is a flow chart of an application of a geometric prediction mode in accordance with at least one embodiment of the present invention.

[0022] Figure 8 is a schematic diagram showing prediction of a current block according to a geometric prediction mode.

[0023] Figure 9a and Figure 9b is a flowchart of prediction of a current block based on a geometric prediction mode according to at least one embodiment of the present invention.

[0024] Fig.10 is a schematic diagram illustrating blocks of spatially adjacent sub-regions according to at least one embodiment of the present invention.

[0025] Fig.11 is a schematic diagram illustrating blocks of spatially adjacent sub-regions according to another embodiment of the present invention.

[0026] Fig.12 is a schematic diagram illustrating geometric partitioning of a current block according to at least one embodiment of the present invention.

[0027] Fig.13a and Fig.13b is a flowchart of prediction of a current block based on a geometric prediction mode according to another embodiment of the present invention.

[0028] Fig.14 is a schematic diagram showing a case where a sub-region is predicted according to intra prediction.

[0029] Fig.15 is a schematic diagram showing a directional prediction mode rotated 180 degrees based on the aspect ratio of the current block.

[0030] Fig.16is a schematic diagram of angles defining directional prediction modes according to at least one embodiment of the present invention.

[0031] Fig.17 is a diagram showing a directional prediction mode rotated 180 degrees based on the aspect ratio of a region including a sub-region.

[0032] Fig.18 is a schematic diagram illustrating the width and height of a region including sub-regions according to at least one embodiment of the present invention.

[0033] Fig.19 is a schematic diagram showing the width and height of a region including sub-regions according to another embodiment of the present invention.

[0034] Fig. 20 and Fig.21 is a schematic diagram illustrating the use of an initial directional mode based on the aspect ratio of a region including a sub-region. DETAILED DESCRIPTION

[0035] Hereinafter, some embodiments of the present invention will be described in detail with reference to the attached illustrative drawings. In the following description, the same reference numerals represent the same elements, although the elements are shown in different drawings. In addition, in the following description of some embodiments, when it is considered that the detailed description of related known components and functions obscures the subject matter of the present invention, the detailed description of related known components and functions may be omitted for the sake of clarity and brevity.

[0036] Figure 1 is a block diagram of a video encoding device that can implement the technology of the present invention. Figure 1 A video encoding device and components of the device are described in detail.

[0037] The encoding device may include: an image divider 110, a predictor 120, a subtractor 130, a transformer 140, a quantizer 145, a rearrangement unit 150, an entropy encoder 155, an inverse quantizer 160, an inverse transformer 165, an adder 170, a loop filtering unit 180 and a memory 190.

[0038] Each component of the encoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0039] A video consists of one or more sequences including multiple images. Each image is divided into multiple regions, and encoding is performed on each region. For example, an image is divided into one or more tiles or / and slices. Here, one or more tiles can be defined as a tile group. Each tile or / and slice is divided into one or more coding tree units (coding tree unit, CTU). In addition, each CTU is divided into one or more coding units (coding unit, CU) through a tree structure. The information applied to each coding unit (CU) is encoded as the syntax of the CU, and the information commonly applied to the CU included in a CTU is encoded as the syntax of the CTU. In addition, the information commonly applied to all blocks in a slice is encoded as the syntax of the slice header, and the information applied to all blocks constituting one or more images is encoded as a picture parameter set (Picture Parameter Set, PPS) or a picture header. In addition, the information commonly referenced by multiple images is encoded as a sequence parameter set (Sequence Parameter Set, SPS). In addition, the information commonly referenced by one or more SPSs is encoded as a video parameter set (Video Parameter Set, VPS). In addition, information commonly applied to one tile or tile group may also be encoded as syntax of the tile or tile group header.The syntax included in the SPS, PPS, slice header, tile or tile group header may be referred to as a high-level syntax.

[0040] The image divider 110 determines the size of a coding tree unit (CTU). Information on the size of the CTU (CTU size) is encoded as a syntax of an SPS or a PPS and transmitted to a video decoding device.

[0041] The image divider 110 divides each image constituting a video into a plurality of coding tree units (CTUs) having a predetermined size, and then recursively divides the CTUs by using a tree structure. A leaf node in the tree structure becomes a coding unit (CU), which is a basic unit of coding.

[0042] The tree structure can be a quadtree (quadtree, QT), in which a higher node (or parent node) is split into four lower nodes (or child nodes) of the same size. The tree structure can also be a binary tree (binarytree, BT), in which a higher node is split into two lower nodes. The tree structure can also be a ternary tree (ternarytree, TT), in which a higher node is split into three lower nodes at a ratio of 1:2:1. The tree structure can also be a structure in which two or more structures in the QT structure, BT structure and TT structure are mixed. For example, a quadtree plus binary tree (quadtree plus binarytree, QTBT) structure can be used, or a quadtree plus binary tree ternary tree (quadtree plus binarytreeternarytree, QTBTTT) structure can be used. Here, a binary tree ternary tree (BTTT) is added to the tree structure to be called a multiple-type tree (multiple-type tree, MTT).

[0043] Figure 2 is a schematic diagram for describing a method of dividing a block by using a QTBTTT structure.

[0044] like Figure 2 As shown, the CTU can be first split into a QT structure. The quadtree splitting can be recursive until the size of the split block reaches the minimum block size (MinQTSize) of the leaf node allowed in QT. The first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoder 155 and notified to the video decoding device with a signal. When the leaf node of QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in BT, the leaf node can be further split into at least one of a BT structure or a TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions, namely, a direction for horizontally splitting blocks of the corresponding node and a direction for vertically splitting blocks of the corresponding node. As Figure 2 As shown, when MTT splitting starts, the entropy encoder 155 encodes a second flag (mtt_split_flag) indicating whether the node is split, and when the node is split, a flag additionally indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (binary or trifurcated) and notifies the video decoding device of the same by signal.

[0045] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of the lower layer, the CU split flag (split_cu_flag) indicating whether the node is split may also be encoded. When the value of the CU split flag (split_cu_flag) indicates that each node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU, which is the basic unit of encoding. When the value of the CU split flag (split_cu_flag) indicates that each node is split, the video encoding device first starts encoding the first flag in the above scheme.

[0046] When QTBT is used as another example of a tree structure, there may be two types, namely, a type in which the block of the corresponding node is horizontally split into two blocks of the same size (i.e., symmetrical horizontal splitting) and a type in which the block of the corresponding node is vertically split into two blocks of the same size (i.e., symmetrical vertical splitting). The entropy encoder 155 encodes a split flag (split_flag) indicating whether each node of the BT structure is split into blocks of the lower layer and split type information indicating the split type, and transmits them to the video decoding device. On the other hand, there may be another type in which the block of the corresponding node is split into two blocks that are asymmetric to each other. The asymmetric form may include a form in which the block of the corresponding node is split into two rectangular blocks with a size ratio of 1:3, or may also include a form in which the block of the corresponding node is split in a diagonal direction.

[0047] The CU may have various sizes according to the QTBT or QTBTTT partitioned from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block". When QTBTTT partitioning is adopted, the shape of the current block may also be a rectangular shape in addition to a square shape.

[0048] The predictor 120 predicts the current block to generate a predicted block. The predictor 120 includes an intra predictor 122 and an inter predictor 124.

[0049] Typically, each of the current blocks in an image may be predictively encoded. Typically, the prediction of the current block may be performed by utilizing an intra-prediction technique (which utilizes data from an image including the current block) or an inter-prediction technique (which utilizes data from an image encoded prior to the image including the current block). Inter-prediction includes both unidirectional prediction and bidirectional prediction.

[0050] The intra-frame predictor 122 predicts pixels in the current block by using pixels (reference pixels) located adjacent to the current block in the current image including the current block. There are multiple intra-frame prediction modes according to the prediction direction. For example, Figure 3aAs shown, the plurality of intra prediction modes may include two non-directional modes including a planar mode and a DC mode, and may include 65 directional modes. Neighboring pixels and algorithm equations to be used are defined differently according to each prediction mode.

[0051] In order to perform efficient directional prediction for a current block with a rectangular shape, it is possible to additionally use Figure 3b The directional modes are shown in the dashed arrows (#67 to #80, intra-frame prediction modes #-1 to #-14). The directional modes can be called "wide angle intra-frame prediction modes". Figure 3b In the figure, the arrows indicate the corresponding reference samples used for prediction, rather than representing the prediction direction. The prediction direction is opposite to the direction indicated by the arrow. When the current block has a rectangular shape, the wide-angle intra prediction mode is a mode in which prediction is performed in a direction opposite to the specific direction mode without additional bit transmission. In this case, in the wide-angle intra prediction mode, some wide-angle intra prediction modes available for the current block can be determined by the ratio of the width to the height of the current block having a rectangular shape. For example, when the current block has a rectangular shape with a height less than the width, a wide-angle intra prediction mode (intra prediction modes #67 to #80) with an angle less than 45 degrees is available. When the current block has a rectangular shape with a width greater than the height, a wide-angle intra prediction mode with an angle greater than -135 degrees is available.

[0052] The intra-frame predictor 122 may determine an intra-frame prediction to be used for encoding the current block. In some examples, the intra-frame predictor 122 may encode the current block by utilizing multiple intra-frame prediction modes, and may also select an appropriate intra-frame prediction mode to be used from a test mode. For example, the intra-frame predictor 122 may calculate a rate-distortion value by utilizing a rate-distortion analysis of multiple tested intra-frame prediction modes, and may also select an intra-frame prediction mode having the best rate-distortion characteristic in the test mode.

[0053] The intra-frame predictor 122 selects an intra-frame prediction mode among a plurality of intra-frame prediction modes and predicts the current block by using adjacent pixels (reference pixels) and an algorithm equation determined according to the selected intra-frame prediction mode. Information about the selected intra-frame prediction mode is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0054] The inter-frame predictor 124 generates a prediction block for the current block by using a motion compensation process. The inter-frame predictor 124 searches for a block that is most similar to the current block in a reference image that has been encoded and decoded earlier than the current image, and generates a prediction block for the current block by using the searched block. In addition, a motion vector (MV) is generated, which corresponds to the displacement between the current block in the current image and the prediction block in the reference image. Typically, motion estimation is performed on the luminance (luma) component, and the motion vector calculated based on the luminance component is used for both the luminance component and the chrominance component. The motion information including information of the reference image and information about the motion vector used to predict the current block is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0055] The inter-frame predictor 124 may also perform interpolation of a reference image or a reference block to increase the accuracy of the prediction. In other words, a subsample is interpolated between two consecutive integer samples by applying a filter coefficient to a plurality of consecutive integer samples including two integer samples. When performing a process of searching for a block most similar to the current block on the interpolated reference image, a decimal unit precision may be represented for the motion vector instead of an integer sample unit precision. For each target region to be encoded, for example, a unit such as a slice, a tile, a CTU, a CU, etc., the precision or resolution of the motion vector may be set differently. When such an adaptive motion vector resolution (AMVR) is applied, information about the motion vector resolution to be applied to each target region should be signaled for each target region. For example, when the target region is a CU, information about the motion vector resolution applied to each CU is signaled. The information about the motion vector resolution may be information representing the precision of the motion vector difference to be described below.

[0056] On the other hand, the inter-frame predictor 124 may perform inter-frame prediction by utilizing bidirectional prediction. In the case of bidirectional prediction, two reference images and two motion vectors representing the block positions most similar to the current block in each reference image are used. The inter-frame predictor 124 selects a first reference image and a second reference image from a reference image list 0 (RefPicList0) and a reference image list 1 (RefPicList1), respectively. The inter-frame predictor 124 also searches for a block most similar to the current block in the corresponding reference image to generate a first reference block and a second reference block. In addition, a prediction block of the current block is generated by averaging or weighted averaging the first reference block and the second reference block. In addition, motion information including information about two reference images used to predict the current block and including information about two motion vectors is transmitted to the entropy encoder 155. Here, the reference image list 0 may be composed of images that are before the current image in the display order in the pre-reconstructed image, and the reference image list 1 may be composed of images that are after the current image in the display order in the pre-reconstructed image. However, although not particularly limited thereto, the pre-reconstructed image that is after the current image in the display order may be additionally included in the reference image list 0. Conversely, a pre-reconstructed image preceding the current image may also be additionally included in the reference image list 1 .

[0057] In order to minimize the amount of bits consumed for encoding motion information, various methods may be used.

[0058] For example, when the reference image and motion vector of the current block are the same as those of the neighboring block, information capable of identifying the neighboring block is encoded to transmit the motion information of the current block to the video decoding device. This method is called merge mode.

[0059] In the merge mode, the inter predictor 124 selects a predetermined number of merge candidate blocks (hereinafter, referred to as 'merge candidates') from neighboring blocks of the current block.

[0060] As the neighboring blocks for deriving the merge candidate, all or some of the left block A0, the lower left block A1, the upper block B0, the upper right block B1 and the upper left block B2 adjacent to the current block in the current image may be used, such as Figure 4 As shown. In addition, in addition to the current image where the current block is located, a block located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as a merge candidate. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the above method is less than the preset number, a zero vector is added to the merge candidate.

[0061] The inter-frame predictor 124 configures a merge list including a predetermined number of merge candidates by using adjacent blocks. A merge candidate to be used as motion information of the current block is selected from among the merge candidates included in the merge list, and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoder 155 and transmitted to the video decoding device.

[0062] Merge skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy coding are close to zero, only the neighbor block selection information is transmitted without the residual signal. By using merge skip mode, relatively high coding efficiency can be achieved for images with slight motion, still images, screen content images, etc.

[0063] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.

[0064] Another method for encoding motion information is the advanced motion vector prediction (AMVP) mode.

[0065] In the AMVP mode, the inter-frame predictor 124 derives a motion vector prediction candidate for a motion vector of a current block by using neighboring blocks of the current block. As neighboring blocks for deriving motion vector prediction candidates, the neighboring blocks may be used. Figure 4 All or some of the left block A0, lower left block A1, upper block B0, upper right block B1 and upper left block B2 adjacent to the current block in the current image shown. In addition, in addition to the current image where the current block is located, blocks located in a reference image (which may be the same as or different from the reference image used to predict the current block) may also be used as adjacent blocks for deriving motion vector prediction candidates. For example, a co-located block of the current block in the reference image or a block adjacent to the co-located block may be used. If the number of motion vector candidates selected by the above method is less than the preset number, a zero vector is added to the motion vector candidates.

[0066] The inter-frame predictor 124 derives a motion vector prediction candidate by using the motion vectors of the neighboring blocks, and determines a motion vector prediction of the motion vector of the current block by using the motion vector prediction candidate. In addition, a motion vector difference is calculated by subtracting the motion vector prediction from the motion vector of the current block.

[0067] The motion vector prediction can be obtained by applying a predefined function (e.g., median and mean calculations, etc.) to the motion vector prediction candidate. In this case, the video decoding device also knows the predefined function. In addition, since the neighboring blocks used to derive the motion vector prediction candidate are blocks that have completed encoding and decoding, the video decoding device may also already know the motion vectors of the neighboring blocks. Therefore, the video encoding device does not need to encode the information for identifying the motion vector prediction candidate. Accordingly, in this case, information about the motion vector difference and information about the reference image used to predict the current block are encoded.

[0068] On the other hand, the motion vector prediction can also be determined by selecting any one of the motion vector prediction candidates. In this case, information for identifying the selected motion vector prediction candidate is additionally encoded together with information about the motion vector difference used to predict the current block and information about the reference image.

[0069] The subtractor 130 generates a residual block by subtracting a prediction block generated by the intra predictor 122 or the inter predictor 124 from the current block.

[0070] The transformer 140 transforms the residual signal in the residual block having the pixel value of the spatial domain into the transform coefficient of the frequency domain. The transformer 140 may transform the residual signal in the residual block by using the entire size of the residual block as the transform unit, or may also divide the residual block into a plurality of sub-blocks, and may perform the transform by using the sub-block as the transform unit. Alternatively, the residual block is divided into two sub-blocks, namely, a transform region and a non-transform region, so as to transform the residual signal by using only the transform region sub-block as the transform unit. Here, the transform region sub-block may be one of two rectangular blocks having a size ratio of 1:1 based on the horizontal axis (or the vertical axis). In this case, the entropy encoder 155 encodes a flag (cu_sbt_flag) indicating that only the sub-block is transformed, and direction (vertical / horizontal) information (cu_sbt_horizontal_flag) and / or position information (cu_sbt_pos_flag), and notifies the video decoding device with a signal. In addition, the size of the transform region subblock may have a size ratio of 1:3 based on the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_quad_flag) for dividing the corresponding partition is additionally encoded by the entropy encoder 155 and signaled to the video decoding apparatus.

[0071] On the other hand, the transformer 140 may perform transformation of the residual block separately in the horizontal direction and the vertical direction. For the transformation, various types of transformation functions or transformation matrices may be used. For example, paired transformation functions for horizontal transformation and vertical transformation may be defined as a multiple transform set (MTS). The transformer 140 may select a transformation function pair with the highest transformation efficiency in the MTS, and may transform the residual block in each of the horizontal and vertical directions. The information (mts_idx) about the transformation function pair in the MTS is encoded by the entropy encoder 155 and notified to the video decoding device with a signal.

[0072] The quantizer 145 quantizes the transform coefficients output from the transformer 140 using the quantization parameters and outputs the quantized transform coefficients to the entropy encoder 155. The quantizer 145 may also quantize the relevant residual block immediately without transforming any block or frame. The quantizer 145 may also apply different quantization coefficients (scaling values) according to the position of the transform coefficients in the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to the video decoding device.

[0073] The rearrangement unit 150 may perform rearrangement of coefficient values ​​on the quantized residual value.

[0074] The rearrangement unit 150 can change the 2D coefficient array into a 1D coefficient sequence by using coefficient scanning. For example, the rearrangement unit 150 can use a zig-zag scan or a diagonal scan to scan the DC coefficient to the coefficient of the high frequency region to output a 1D coefficient sequence. Depending on the size of the transform unit and the intra-frame prediction mode, the zig-zag scan can also be replaced by a vertical scan that scans the 2D coefficient array in the column direction and a horizontal scan that scans the 2D block type coefficient in the row direction. In other words, depending on the size of the transform unit and the intra-frame prediction mode, the scanning method to be used can be determined among zig-zag scanning, diagonal scanning, vertical scanning, and horizontal scanning.

[0075] The entropy encoder 155 encodes the sequence of 1D quantized transform coefficients output from the rearrangement unit 150 by using various encoding schemes including Context-based Adaptive Binary Arithmetic Code (CABAC), Exponential Golomb, etc. to generate a bitstream.

[0076] In addition, the entropy encoder 155 encodes information related to block segmentation (e.g., CTU size, CTU segmentation flag, QT segmentation flag, MTT segmentation type, and MTT segmentation direction, etc.) so that the video decoding device can segment the block equivalent to the video encoding device. In addition, the entropy encoder 155 encodes information about the prediction type indicating whether the current block is encoded by intra-frame prediction or inter-frame prediction. The entropy encoder 155 encodes intra-frame prediction information (i.e., information about intra-frame prediction mode) or inter-frame prediction information (merge index in the case of merge mode, and information about reference image index and motion vector difference in the case of AMVP mode) according to the prediction type. In addition, the entropy encoder 155 encodes information related to quantization (i.e., information about quantization parameters and information about quantization matrices).

[0077] The inverse quantizer 160 inversely quantizes the quantized transform coefficient output from the quantizer 145 to generate a transform coefficient. The inverse transformer 165 transforms the transform coefficient output from the inverse quantizer 160 from the frequency domain to the spatial domain to reconstruct a residual block.

[0078] The adder 170 adds the reconstructed residual block and the prediction block generated by the predictor 120 to reconstruct the current block. When intra-prediction is performed on the next block, the pixels in the reconstructed current block are used as reference pixels.

[0079] The loop filtering unit 180 performs filtering on the reconstructed pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transform / quantization. The loop filtering unit 180 as an in-loop filter may include all or some of a deblocking filter 182, a sample adaptive offset (SAO) filter 184, and an adaptive loop filter (ALF) 186.

[0080] The deblocking filter 182 filters the boundaries between the reconstructed blocks to remove blocking artifacts that occur due to block unit encoding / decoding, and the SAO filter 184 and the ALF 186 perform additional filtering on the deblocking filtered video. The SAO filter 184 and the ALF 186 are filters for compensating for the difference between the reconstructed pixels and the original pixels that occur due to lossy coding. The SAO filter 184 applies an offset in units of CTU to enhance subjective image quality and encoding efficiency. On the other hand, the ALF 186 performs block unit filtering and applies different filters to compensate for distortion by dividing the boundaries of the corresponding blocks and the degree of the amount of change. Information about the filter coefficients to be used for the ALF can be encoded and notified to the video decoding device with a signal.

[0081] The reconstructed blocks filtered by the deblocking filter 182, the SAO filter 184, and the ALF 186 are stored in the memory 190. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0082] Figure 5 is a functional block diagram of a video decoding device that can implement the technology of the present invention. Figure 5 , describes a video decoding device and components of the device.

[0083] The video decoding apparatus may include an entropy decoder 510 , a rearrangement unit 515 , an inverse quantizer 520 , an inverse transformer 530 , a predictor 540 , an adder 550 , a loop filtering unit 560 , and a memory 570 .

[0084] Similar to Figure 1 Each component of the video encoding device and the video decoding device can be implemented as hardware or software, or as a combination of hardware and software. In addition, the function of each component can be implemented as software, and the microprocessor can also be implemented to execute the function of the software corresponding to each component.

[0085] The entropy decoder 510 extracts information related to block partitioning by decoding a bitstream generated by a video encoding apparatus to determine a current block to be decoded, and extracts prediction information required to reconstruct the current block and information about a residual signal.

[0086] The entropy decoder 510 determines the size of the CTU by extracting information about the CTU size from a sequence parameter set (SPS) or a picture parameter set (PPS), and divides the image into CTUs of the determined size. In addition, the CTU is determined as the highest level (i.e., the root node) of the tree structure, and the partition information of the CTU can be extracted to partition the CTU by using the tree structure.

[0087] For example, when CTU is segmented by using the QTBTTT structure, the first flag (QT_split_flag) related to the segmentation of QT is first extracted to segment each node into four nodes of the lower layer. In addition, the second flag (mtt_split_flag) related to the segmentation of MTT, the segmentation direction (vertical / horizontal) and / or the segmentation type (binary / tripartite) are extracted with respect to the node corresponding to the leaf node of QT to segment the corresponding leaf node into the MTT structure. As a result, each node below the leaf node of QT is recursively segmented into a BT or TT structure.

[0088] As another example, when a CTU is split by using a QTBTTT structure, a CU split flag (split_cu_flag) indicating whether the CU is split is extracted. When the corresponding block is split, a first flag (QT_split_flag) may also be extracted. During the splitting process, for each node, 0 or more recursive MTT splits may occur after 0 or more recursive QT splits. For example, for a CTU, MTT splits may occur immediately, or, conversely, multiple QT splits may occur only.

[0089] As another example, when CTU is segmented by using the QTBT structure, the first flag (QT_split_flag) related to the segmentation of QT is extracted to segment each node into four nodes of the lower layer. In addition, a segmentation flag (split_flag) indicating whether the node corresponding to the leaf node of QT is further segmented into BT and segmentation direction information are extracted.

[0090] On the other hand, when the entropy decoder 510 determines the current block to be decoded by using the partitioning of the tree structure, the entropy decoder 510 extracts information about the prediction type indicating whether the current block is intra-predicted or inter-predicted. When the prediction type information indicates intra-prediction, the entropy decoder 510 extracts syntax elements of intra-prediction information (intra-prediction mode) for the current block. When the prediction type information indicates inter-prediction, the entropy decoder 510 extracts information representing syntax elements of inter-prediction information, that is, a motion vector and a reference image to which the motion vector refers.

[0091] In addition, the entropy decoder 510 extracts quantization-related information and extracts information on a quantized transform coefficient of the current block as information on a residual signal.

[0092] The rearrangement unit 515 may change the sequence of 1D quantized transform coefficients entropy-decoded by the entropy decoder 510 into a 2D coefficient array (ie, block) again in the reverse order of the coefficient scanning order performed by the video encoding device.

[0093] The inverse quantizer 520 inversely quantizes the quantized transform coefficients, and inversely quantizes the quantized transform coefficients by using quantization parameters. The inverse quantizer 520 may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in 2D. The inverse quantizer 520 may perform inverse quantization by applying a matrix of quantization coefficients (scaling values) from a video encoding device to a 2D array of quantized transform coefficients.

[0094] The inverse transformer 530 reconstructs a residual signal by inversely transforming the inversely quantized transform coefficient from the frequency domain to the spatial domain to generate a residual block of the current block.

[0095] In addition, when the inverse transformer 530 inversely transforms a partial region (sub-block) of the transform block, the inverse transformer 530 extracts a flag (cu_sbt_flag) that only the sub-block of the transform block is transformed, direction (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block. The inverse transformer 530 also inversely transforms the transform coefficient of the corresponding sub-block from the frequency domain to the spatial domain to reconstruct the residual signal, and fills the region that is not inversely transformed with a value of "0" as the residual signal to generate a final residual block of the current block.

[0096] In addition, when MTS is applied, the inverse transformer 530 determines the transform index or transform matrix to be applied in each of the horizontal and vertical directions by using the MTS information (mts_idx) signaled from the video encoding device. The inverse transformer 530 also performs inverse transform on the transform coefficients in the transform block in the horizontal and vertical directions by using the determined transform function.

[0097] The predictor 540 may include an intra predictor 542 and an inter predictor 544. The intra predictor 542 is activated when the prediction type of the current block is intra prediction, and the inter predictor 544 is activated when the prediction type of the current block is inter prediction.

[0098] The intra predictor 542 determines an intra prediction mode of a current block among a plurality of intra prediction modes according to syntax elements of the intra prediction mode extracted from the entropy decoder 510. The intra predictor 542 also predicts the current block by using neighboring reference pixels of the current block according to the intra prediction mode.

[0099] The inter predictor 544 determines a motion vector of a current block and a reference image to which the motion vector refers by using a syntax element of the inter prediction mode extracted from the entropy decoder 510 .

[0100] The adder 550 reconstructs the current block by adding the residual block output from the inverse transformer 530 and the prediction block output from the inter predictor 544 or the intra predictor 542. When intra-predicting a subsequent block to be decoded, pixels within the reconstructed current block are used as reference pixels.

[0101] The loop filtering unit 560 as an in-loop filter may include a deblocking filter 562, an SAO filter 564, and an ALF 566. The deblocking filter 562 performs deblocking filtering on the boundary between the reconstructed blocks in order to remove block artifacts that occur due to block unit decoding. The SAO filter 564 and the ALF 566 perform additional filtering on the reconstructed blocks after deblocking filtering in order to compensate for the difference between the reconstructed pixels and the original pixels that occurs due to lossy encoding. The filter coefficients of the ALF are determined by using information about the filter coefficients decoded from the bitstream.

[0102] The reconstructed blocks filtered by the deblocking filter 562, the SAO filter 564, and the ALF 566 are stored in the memory 570. When all blocks in one image are reconstructed, the reconstructed image can be used as a reference image for inter-frame prediction of blocks within an image to be subsequently encoded.

[0103] In some embodiments, the present invention relates to encoding and decoding video images as described above. More specifically, the present invention provides a video encoding and decoding method and apparatus that effectively performs inter-frame and / or intra-frame prediction for each partitioned area when predicting sub-blocks based on geometric partitions in the prediction of the current block.

[0104] The following implementation may be performed by the predictor 120 in the video encoding device. The following implementation may also be performed by the predictor 540 in the video decoding device.

[0105] The video encoding device can generate signaling information associated with the present embodiment from the perspective of optimizing rate distortion when predicting the current block. The video encoding device can encode the signaling information using the entropy encoder 155 and send the encoded signaling information to the video decoding device. The video decoding device can decode the signaling information associated with the prediction of the current block from the bitstream using the entropy decoder 510.

[0106] In the following description, the term 'target block' may be used interchangeably with a current block or a coding unit (CU), or may refer to some regions of a coding unit.

[0107] In addition, a flag value of true indicates a case where the flag is set to 1. In addition, a flag value of false indicates a case where the flag is set to 0.

[0108] The following embodiments are described with respect to a video decoding apparatus, but they may also be performed by a video encoding apparatus as described above.

[0109] Figure 6 is a block diagram detailing a portion of a video decoding apparatus according to at least one embodiment of the present invention.

[0110] The video decoding device according to this embodiment can determine the prediction unit and the transform unit, and perform prediction and inverse transform on the current block corresponding to the determined unit by using the determined prediction technology and prediction mode, thereby finally generating a reconstructed block of the current block. Figure 6 As shown in FIG. 1 , this can be performed in the video decoding apparatus by the inverse transformer 530, the predictor 540, and the adder 550. On the other hand, the same as the above can be performed in the video encoding apparatus by the inverse transformer 165, the image divider 110, the predictor 120, and the adder 170. Figure 6 In this case, the video decoding device uses the encoded information parsed from the bit stream, while the video encoding device can use the encoded information set from a higher level in terms of minimizing rate distortion. Hereinafter, for convenience, the present embodiment will be described with the video decoding device as the center.

[0111] According to the prediction technology, Figure 5 The predictor 540 shown in FIG. 5 includes an intra-frame predictor 542 and an inter-frame predictor 544, but Figure 6 The predictor 540 shown in FIG. 5 may include all or part of a prediction unit determiner 602, a prediction technique determiner 604, a prediction mode determiner 606, and a prediction executor 608. The prediction unit determiner 602 determines a prediction unit (PU). The prediction unit may be a subblock of a current block or a subblock obtained by partitioning the current block. The prediction technique determiner 604 determines a prediction technique for the prediction unit, for example, intra prediction, inter prediction or intra block copy (IBC) prediction, a palette mode, etc. The prediction mode determiner 606 determines a detailed prediction mode for the prediction technique. The prediction executor 608 generates a prediction block of the current block according to the determined prediction mode.

[0112] The inverse transformer 530 includes a transform unit determiner 610 and an inverse transform performer 612. The transform unit determiner 610 determines a transform unit (TU) for an inverse quantized signal of a current block, and the inverse transform performer 612 generates a residual signal by inversely transforming the transform unit represented by the inverse quantized signal.

[0113] The adder 550 sums the predicted block and the residual signal to generate a reconstructed block. The reconstructed block is stored in a memory and can be used for future prediction of other blocks.

[0114] As described above, the prediction technique of the current block may be determined by the prediction technique determiner 604. The prediction technique may be one of techniques such as inter prediction, intra prediction, IBC mode, palette mode, template matching, and the like.

[0115] In one example, if the prediction technique for the current block is not intra prediction, the prediction technique determiner 604 parses a one-bit flag. If the flag is skip mode, the prediction technique determiner 604 may determine the prediction technique for the current block as inter prediction, IBC mode, or merge mode, and may omit the inverse transform process.

[0116] On the other hand, if the current block is not in skip mode, the prediction technique determiner 604 may parse a one-bit flag and may determine one of inter prediction, intra prediction, IBC mode, palette mode, and template matching as the prediction technique for the current block based on the flag value.

[0117] As another example, if the prediction technique of the current block is determined to be inter prediction, the prediction technique determiner 604 may parse a one-bit flag to determine whether to perform prediction of the current block in merge mode or advanced motion vector prediction (AMVP) mode based on the flag value.

[0118] Figure 7 is a flow chart of an application of a geometric prediction mode in accordance with at least one embodiment of the present invention.

[0119] In the following, the geometric prediction mode generally refers to the prediction mode based on geometric partitions.

[0120] On the other hand, if the prediction technique of the current block is inter prediction, the prediction mode determiner 606 parses a one-bit flag indicating whether the geometric prediction mode is applied (hereinafter, "geometric prediction mode flag") (S700), and checks the parsed flag (S702). If the geometric prediction mode flag is true (S702 is yes), the prediction mode determiner 606 determines the prediction mode of the current block as the geometric prediction mode. Then, the video decoding apparatus performs prediction of the current block in the geometric prediction mode (S704), as shown in FIG. Figure 8 as shown in the example.

[0121] On the other hand, if the geometric prediction mode flag is false (No in S702), the prediction mode determiner 606 determines the prediction mode of the current block as one of the inter prediction modes. Then, the video decoding apparatus performs prediction of the current block according to the determined inter prediction mode (S706).

[0122] according to Figure 7 The application of the example geometric prediction mode can be similarly performed by the video encoding device.

[0123] Figure 9a and Figure 9b is a flowchart of prediction of a current block based on a geometric prediction mode according to at least one embodiment of the present invention.

[0124] In the geometric prediction mode, such as Figure 9a and Figure 9b As shown, the video decoding apparatus partitions the current block into a plurality of sub-regions, performs prediction on each sub-region, and then uses the sub-regions to generate a final predicted block. Figure 9a and Figure 9b The example is a flowchart detailing step S704 performed when the geometric prediction mode flag is true.

[0125] The video decoding device parses the geometric partition information of the current block (S900). In one example, the geometric partition information can be predefined by using a common lookup table (LUT) between the video encoding device and the video decoding device. For example, if the current block is partitioned into two sub-areas, the geometric partition information may include the angle of the partition line and the distance from a specific pixel.

[0126] In this embodiment, the current block may be partitioned into P sub-regions based on the geometric partition information (where P is an integer greater than or equal to 2). The video decoding device may parse the index indicating the geometric partition information, and then may obtain the geometric partition information of the current block from the LUT described above by using the parsed index.

[0127] In one example, a prediction technique such as inter-frame prediction, intra-frame prediction, template matching, etc. may be determined for each sub-region.

[0128] The video decoding apparatus parses sub-region specific indexes for P sub-regions (S902). Here, the sub-region specific index k may indicate a prediction mode for each sub-region.

[0129] The video decoding apparatus determines whether the sub-region specific index k is less than a preset L ( S904 ).

[0130] If the sub-region specific index k is less than the preset L (S904: Yes), the video decoding apparatus performs the following steps to perform inter-frame prediction on each sub-region.

[0131] In one example, the video decoding apparatus composes a motion vector candidate list by using information about the reconstructed neighboring blocks of the current block (S906). To compose the motion vector candidate list, the video decoding apparatus may use a spatially reconstructed neighboring region of the current block, a temporal neighboring region at a corresponding position of a reconstructed reference frame, a spatially reconstructed neighboring block according to a coding order, etc. In this case, the number of candidates may be L, and 0≤k≤L-1.

[0132] The video decoding device uses the parsed candidate index k to derive motion information of the sub-region from the motion vector information in the candidate list (S908). The video decoding device compensates for the motion of the sub-region by using the motion information to generate a prediction factor signal of the sub-region (S910). The video decoding device performs weighted summation on the prediction factors of the P sub-regions to generate a final prediction factor of the current block (S912). In the following, the prediction signal and the prediction factor may be used interchangeably.

[0133] If the sub-region specific index k is at least the preset L (No in S904), the video decoding apparatus checks whether the sub-region specific index k is less than the preset M (>L) (S920).

[0134] On the other hand, if the sub-region specific index k is less than the preset M (Yes in S920), the video decoding apparatus performs the following steps of performing intra prediction on each sub-region.

[0135] In one example, the video decoding apparatus composes a Most Probable Mode (MPM) candidate list from the reconstructed neighboring regions of the current block (S922). When composing the list, the video decoding apparatus may compose the candidate list by identifying blocks that are spatially adjacent to the sub-region, such as Fig.10 as shown in the example.

[0136] As Fig.11As another example shown, the following describes the case where the current block is partitioned into part 0 (Part0) and part 1 (Part1) based on the partition information. For Part0, the video decoding apparatus can form an MPM list by using spatially adjacent neighboring blocks (such as A, AR, L, LB, and LT) of the sub-region. For Part1, spatially adjacent neighboring blocks in the sub-region, such as L and LB, can be used.

[0137] In this case, the number of candidates may be ML. That is, in this case, a list including the number of candidates ML may be formed by using an index reflecting +L in each round according to the order agreed upon between the video encoding device and the video decoding device. Accordingly, k may be defined as L≤k≤M-1.

[0138] As another example, the video decoding device may organize the prediction mode in the same direction as the partition direction or the partition angle into a candidate list based on the geometric partition information of the current block. In this case, the candidate list may be further composed of the modes in the +1, +2, -1 and -2 directions of the prediction mode, the mode in the direction perpendicular to the partition direction, and the plane mode.

[0139] As another example, if the reconstructed adjacent block of the current block is predicted in the intra prediction mode, the video decoding device can use the prediction mode to form a candidate list. In addition, the candidate list can be formed by further utilizing the modes in the +1, +2, -1 and -2 directions of the mode and the plane mode.

[0140] As another example, if the reconstructed neighboring block of the current block is predicted in a geometric prediction mode, the video decoding device may form a candidate list with a prediction mode in the same direction as the partition direction based on the geometric partition information of the block. The candidate list may be formed by further using modes in the +1, +2, -1, and -2 directions of the prediction mode, a mode in a direction perpendicular to the partition direction, and a plane mode.

[0141] As another example, the video decoding device may derive the directionality of the reconstructed adjacent region from the reconstructed adjacent region of the current block, and then compose a candidate list of prediction modes corresponding to the derived directionality. At this time, the video decoding device may calculate the gradient value of the pixels in the adjacent region, and derive the directionality by using the calculated gradient value.

[0142] The video decoding device derives an intra prediction mode from the MPM candidate list by using the parsed candidate index k (S924). The video decoding device performs intra prediction according to the intra prediction mode to generate a prediction signal of the sub-region (S926). The video decoding device performs weighted summation on the prediction factors of the P sub-regions to generate a final prediction factor of the current block (S912).

[0143] As an example, the following reconstructed samples of the spatial neighborhood above and to the left of the current block are Fig.10 In this case, the video decoding apparatus may perform intra-frame prediction on the sub-region containing the reconstructed adjacent samples having the largest region. For example, the video decoding apparatus may Fig.10 Part0 shown in FIG. 1 performs intra prediction.

[0144] On the other hand, if the sub-region specific index k is the preset M (No in S920), the video decoding apparatus checks whether template matching is possible (S930).

[0145] As an example, if the size of the neighboring area between the upper and left reference sample lines of the current block and the sub-area is less than a preset threshold, such as Fig.12 In Part 1 shown in , the video decoding device does not perform template matching. In this case, a common value based on the size of the current block can be implicitly used as a threshold in the video encoding device and the video decoding device.

[0146] If template matching is feasible (Yes at S930 ), the video decoding apparatus performs the following steps for template matching on each sub-region.

[0147] The video decoding apparatus defines a template for the sub-region by using the reconstructed neighboring region of the sub-region, and then performs template matching to generate a predicted signal of the sub-region (S932). The video decoding apparatus performs weighted summation on the prediction factors of the P sub-regions to generate a final prediction factor for the current block (S912).

[0148] On the other hand, if template matching is not feasible (No at S930 ), the video decoding apparatus performs the following steps for each sub-region.

[0149] The video decoding apparatus generates a prediction factor signal of the sub-region by compensating the motion of the sub-region using a zero motion vector (S940). The video decoding apparatus performs weighted summation on the prediction factors of the P sub-regions to generate a final prediction factor of the current block (S912).

[0150] For example, if the size of the neighboring area between the upper and left reference sample lines of the current block and the sub-region is less than a predetermined threshold, and the index of the sub-region is M, the video decoding device may generate a predicted signal of the sub-region by using a zero motion vector. Here, the zero motion vector indicates an area at the same position as the sub-region in the reference frame. For example, the video decoding device may generate a predicted signal of the sub-region by using a zero motion vector in the reference frame. Fig.12 The predicted signal of Part 1 is generated in the area at the same position as Part 1 shown.

[0151] according to Figure 9a and Figure 9b The application of the example geometric prediction mode can be similarly performed by the video encoding device.

[0152] Fig.13a and Fig.13b is a flowchart of prediction of a current block based on a geometric prediction mode according to another embodiment of the present invention.

[0153] In the geometric prediction mode, such as Fig.13a and Fig.13b As shown, the video decoding apparatus partitions the current block into a plurality of sub-regions, performs prediction on each sub-region, and then uses the sub-regions to generate a final prediction block. Fig.13a and Fig.13b The example is a flowchart detailing step S704 performed when the geometric prediction mode flag is true.

[0154] The video decoding apparatus parses the geometric partition information of the current block (S1300). As described above, based on the geometric partition information, the current block can be partitioned into P sub-regions.

[0155] The video decoding apparatus parses a common prediction technique flag (S1302). Here, the common prediction technique flag indicates whether the sub-regions are subjected to the same prediction technique.

[0156] The video decoding apparatus checks the common prediction technique flag (S1304).

[0157] If the common prediction technique flag is false (No in S1304), since the prediction techniques used for the sub-regions are different, the video decoding apparatus performs the prediction according to Figure 9a and Figure 9b Steps for the example.

[0158] On the other hand, if the common prediction technique flag is true (S1304 is yes), the video decoding device may predict all sub-regions by using inter-frame prediction, predict all sub-regions by using intra-frame prediction, or predict all sub-regions by using template matching according to the following steps.

[0159] The video decoding apparatus parses the prediction technique index (S1306). Based on the prediction technique index, the video decoding apparatus may determine the prediction technique to be used for the sub-region.

[0160] The video decoding apparatus determines whether the prediction technique index is zero ( S1308 ).

[0161] If the prediction technique index is zero (Yes in S1308 ), indicating inter-frame prediction, the video decoding apparatus may perform the following steps.

[0162] The video decoding apparatus parses sub-region specific indexes k for P sub-regions ( S1310 ).

[0163] The video decoding apparatus composes a motion vector candidate list for the sub-region (S1312). The video decoding apparatus uses the parsed candidate index k to derive motion information of the sub-region from the motion vector information in the candidate list (S1314).

[0164] The video decoding apparatus compensates for the motion of the sub-region by using the motion information to generate a prediction factor signal of the sub-region (S1316). The video decoding apparatus performs weighted summation on the prediction factors of the P sub-regions to generate a final prediction factor of the current block (S1318).

[0165] On the other hand, if the prediction technique index is non-zero (No in S1308 ), the video decoding apparatus determines whether the prediction technique index is 1 ( S1330 ).

[0166] If the prediction technique index is 1 (Yes in S1330 ), indicating intra prediction, the video decoding apparatus performs the following steps.

[0167] The video decoding apparatus parses the sub-region specific index k for the P sub-regions ( S1332 ).

[0168] The video decoding device composes an MPM candidate list for the sub-region (S1334). The video decoding device uses the parsed candidate index k to derive an intra prediction mode from the MPM candidate list (S1336). The video decoding device performs intra prediction according to the intra prediction mode to generate a signal for prediction of the sub-region (S1338). The video decoding device performs weighted summation of the prediction factors of the P sub-regions to generate a final prediction factor for the current block (S1318).

[0169] On the other hand, if the prediction technique index is not 1 (No in S1330 ), for example, the prediction technique index is 2, indicating template matching, the video decoding apparatus performs the following steps.

[0170] The video decoding apparatus defines a template of the sub-region by using the reconstructed adjacent region of the sub-region, and then performs template matching on the sub-region to generate a prediction signal of the sub-region (S1340). The video decoding apparatus performs weighted summation on the prediction factors of the P sub-regions (S1318) to generate a final prediction factor of the current block.

[0171] according to Fig.13a and Fig.13b The application of the example geometric prediction mode can be similarly performed by the video encoding device.

[0172] The following is used to predict the sub-region according to intra prediction and predict according to the directional mode as follows Fig.14 Describe the prediction process for the situation of the Part0 sub-region shown.

[0173] In one example, according to the aspect ratio (W:H) of the current block of the prediction mode for the Part0 sub-region, the video decoding device can perform prediction by using a direction mode rotated 180 degrees. Here, W represents the width of the current block, and H represents the height of the current block.

[0174] In one example, if W > H, the direction prediction mode with an angle greater than 180 degrees and less than 225 degrees can be changed to the direction prediction mode rotated 180 degrees, as shown in the example of Fig.15 If W < H, the direction prediction mode with an angle greater than 45 degrees and less than 90 degrees can be changed to the direction prediction mode rotated 180 degrees, as shown in the example of Fig.15 In this case, the range of the mode for performing 180-degree rotation prediction can vary according to the aspect ratio of the current block.

[0175] The angle of the direction prediction mode can be defined as in the example of Fig.16 B represents the position of the reference sample for predicting the center of the current block according to the direction prediction mode. A represents the point where the partition line passing through the center O and dividing the current block into two horizontal parts intersects the right boundary of the current block. Here, ∠AOB can be defined as the angle of the direction prediction mode.

[0176] As an example, as shown in Fig.17 For the prediction mode of the Part0 sub-region, when using the partition information of the current block, the present invention can use a direction mode rotated 180 degrees according to the aspect ratio (W':H') of the region including the Part0 sub-region for prediction, rather than the aspect ratio of the current block.

[0177] In one example, if W' > H', the direction prediction mode with an angle greater than 180 degrees and less than 225 degrees can be changed to the direction prediction mode rotated 180 degrees, as shown in the example of Fig.15 If W' < H', the direction prediction mode with an angle greater than 45 degrees and less than 90 degrees can be changed to the direction prediction mode rotated 180 degrees, as shown in the example of Fig.15 In this case, the range of the mode for performing 180-degree rotation prediction can vary according to the aspect ratio of the region including the sub-region.

[0178] As an example, after partitioning the current block according to the geometric partition information, W' and H' can be determined as the width and height of the rectangular shape including each sub-region, as shown in the example of Fig.18 In the example of

[0179] As another example, the case where the range of the weighted summation coefficients used in the geometric prediction mode is from 0 to K is described below, as shown on the left side of Fig.19 . In the left side example of Fig.19 , around the Part0 sub-region, the farther the distance from the partition boundary to the inside of the Part0 sub-region, the closer the weighted summation coefficient is to K, and the farther the distance from the partition boundary to the outside of the Part0 sub-region, the closer the weighted summation coefficient is to 0. For any real number "a" satisfying 0 ≤ a ≤ 1, W' and H' can be determined as the size of the region including the weighted summation coefficients from K to round(K × (1 - a)), as shown in the example on the right side of Fig.19 . In the first example on the right side of Fig.19 , where "a" is 1, the range of the weighted summation factor is from 0 to K. In the second example on the right side of Fig.19 , where 0.5 < a < 1, the range of the weighted summation coefficients can be from round(K × (1 - a)) to K. In the third example on the right side of Fig.19 , where "a" is 0.5, the range of the weighted summation coefficients can be from 0.5K to K.

[0180] At this time, the video encoding device can determine the optimal "a" in terms of rate distortion optimization and signal the optimal "a" to the video decoding device. The video decoding device can utilize the parsed value of "a". Alternatively, "a" can be a preset value based on a previous agreement between the video encoding device and the video decoding device.

[0181] As an example, the case where the upper left coordinates of the current block are (a, b) and W > H or W' > H' is described below, as shown in the example of Fig. 20 . If the current sub-region Part0 includes the lower left corner (a, b + H - 1) of the current block, and the angle (θ) of the partition directionality including the partition boundary of the current sub-region is greater than 90 degrees, the 180-degree rotation direction prediction mode is not used, and the initial direction prediction mode can be used. In this case, W' and H' can be determined as in the example of Fig.18 or Fig.19 .

[0182] As an example, the case where the upper left coordinates of the current block are (a, b) and W < H or W' < H' is described below, as shown in the example of Fig.21 . If the current sub-region Part1 includes the upper right corner (a + W - 1, b) of the current block, and the angle (θ) of the partition directionality including the partition boundary of the current sub-region is greater than 90 degrees, the 180-degree rotation direction prediction mode is not used, and the initial direction prediction mode can be used. In this case, W' and H' can be determined as in the example of Fig.18 or Fig.19 Determine W' and H' as in the example of .

[0183] On the other hand, after determining the final directional prediction mode, if the slope of the final directional prediction mode is an integer, a smoothing filter can be used to correct the reference samples on the reconstructed reference sample line in the spatial neighborhood of the current block. Alternatively, if the gradient of the final directional prediction mode is a real number, the reference samples can be corrected by using an interpolation filter (IF). In this case, IF can be implemented as DCT-IF based on Discrete Cosine Transform (DCT). In addition, the directional prediction mode can be an integer slope mode or a fractional slope mode, depending on the ratio between the width and height (or width and height) used to estimate the slope.

[0184] As described above, the video decoding apparatus may generate a prediction signal for each sub-region, and finally perform weighted summation on the prediction signals to generate a final prediction signal for the current block.

[0185] In one example, the weighted sum coefficient may be determined according to a previous agreement between the video encoding apparatus and the video decoding apparatus based on the height, width, and partition information of the current block.

[0186] As another example, the video encoding device may determine the weighted sum coefficient from the perspective of optimizing the rate distortion, and signal the weighted sum coefficient to the video decoding device. The video decoding device may parse the weighted sum coefficient and may use the weighted sum coefficient to generate a weighted sum of the prediction factors of the sub-regions. In this case, when one or more of the sub-regions are predicted by intra-frame prediction, the region including the sub-regions for determining the final intra-frame prediction method may be based on Fig.18 or Fig.19 The area determined by the example.

[0187] In one example, the video decoding device may use the prediction signal of each sub-region to obtain the amount of change in the brightness value of the pixels in each sub-region, and then may use the pixel brightness change to adaptively derive and use the weighted summation coefficient. In this case, when one or more of the sub-regions are predicted by intra-frame prediction, the region including the sub-regions for determining the final intra-frame prediction method may be based on Fig.18 or Fig.19 The area determined by the example.

[0188] Although the steps in the various flowcharts are described as being performed sequentially, these steps merely illustrate the technical ideas of some embodiments of the present invention. Therefore, a person of ordinary skill in the art to which the present invention belongs can perform the steps by changing the order described in the various figures or by performing two or more steps in parallel. Therefore, the steps in the various flowcharts are not limited to the order shown in the order in which they occur.

[0189] It should be understood that the above description presents illustrative embodiments that can be implemented in various other ways. The functions described in some embodiments can be implemented by hardware, software, firmware and / or their combination. It should also be understood that the functional components described in the present invention are marked as "... units" to highlight the possibility of their independent implementation.

[0190] On the other hand, the various methods or functions described in some embodiments can be implemented as instructions stored in a non-volatile recording medium, which can be read and executed by one or more processors. Non-volatile recording media can include various types of recording devices that store data in a form that is readable by a computer system. For example, non-volatile recording media can include storage media, such as erasable programmable read-only memory (EPROM), flash drives, optical disk drives, magnetic hard drives, and solid-state drives (SSDs), etc.

[0191] Although exemplary embodiments of the present invention are described for illustrative purposes, it will be appreciated by those skilled in the art that various modifications, additions and substitutions may be made without departing from the spirit and scope of the present invention. Therefore, embodiments of the present invention are described for simplicity and clarity. The scope of the technical ideas of the embodiments of the present invention are not limited by the examples. Accordingly, it will be appreciated by those skilled in the art that the scope of the present invention should not be limited by the embodiments clearly described above, but by the claims and their equivalents.

[0192] (reference numerals)

[0193] 120: Predictor

[0194] 155: Entropy Encoder

[0195] 510: Entropy Decoder

[0196] 530: Inverter

[0197] 540: Predictor

[0198] 602: Prediction unit determiner

[0199] 604: Prediction Technology Determiner

[0200] 606: Prediction Mode Determiner

[0201] 608: Predictive Executor.

[0202] CROSS-REFERENCE TO RELATED APPLICATIONS

[0203] This application claims priority to and the benefit of Korean Patent Application No. 10-2022-0057158, filed on May 10, 2022, and Korean Patent Application No. 10-2023-0043673, filed on April 3, 2023, the entire contents of each of which are incorporated herein by reference.

Claims

1. A method for predicting a current block, which is performed by a video decoding device, the method comprising: Decoding geometric partition information of the current block from the bitstream; Partition the current block into multiple sub-regions according to the geometric partition information; parsing an index of each of the plurality of sub-regions, the index indicating a prediction mode of each sub-region; as well as Based on the value of the index, a prediction factor for each subblock is generated by utilizing inter prediction, intra prediction, or template matching.

2. The method according to claim 1, further comprising: The final predictor of the current block is generated by weighted summing of the predictors of multiple sub-regions.

3. The method according to claim 1, further comprising: decoding a geometry prediction mode flag from a bitstream, the geometry prediction mode flag indicating whether a geometry prediction mode is applied to a current block based on geometry partition information; as well as Check the geometry prediction mode flag, Wherein, when the geometric prediction mode flag is true, the method further comprises: Decodes geometry partition information.

4. The method according to claim 1, wherein: Decoding geometric partition information includes: Decoding an index indicating geometric partition information from a bitstream; and The geometric partition information is obtained from a predefined lookup table (LUT) by utilizing an index.

5. The method according to claim 1, wherein: When using inter prediction, generating the prediction factors includes: composing a motion vector candidate list for each sub-block from information about reconstructed neighboring blocks of the current block; parsing candidate indices from a bitstream and deriving motion information from a motion vector candidate list by using the candidate indices; and The motion of each sub-block is compensated according to the motion information.

6. The method according to claim 1, wherein: When using intra prediction, generating the prediction factors includes: From the reconstructed neighboring areas of the current block, a most probable mode candidate list (MPM candidate list) is formed for each sub-block; parsing a candidate index from a bitstream and deriving a prediction mode from an MPM candidate list by using the candidate index; and Prediction of each sub-block is performed according to the prediction mode.

7. The method according to claim 6, wherein: The MPM candidate list includes: Based on the geometric partition information, grouping prediction modes having a common direction with the partition direction or the partition angle into an MPM candidate list; and The candidate list is composed by further utilizing modes in the +1, +2, -1, and -2 directions of the prediction mode in the common direction, a mode in a direction perpendicular to the partition direction, and a planar mode.

8. The method according to claim 6, wherein: When predicting the reconstructed neighboring blocks of the current block in intra prediction mode, the MPM candidate list includes: Grouping intra prediction modes into an MPM candidate list; and The candidate list is composed by further utilizing modes in the +1, +2, -1, and -2 directions of the intra prediction mode and the planar mode.

9. The method according to claim 6, wherein: When predicting the reconstructed neighboring blocks of the current block in geometric prediction mode, the MPM candidate list includes: Combining prediction modes having a common direction with a partition direction or a partition angle into an MPM candidate list based on geometric partition information of at least one of the neighboring blocks; and The candidate list is composed by further utilizing modes in the +1, +2, -1, and -2 directions of the prediction mode in the common direction, a mode in a direction perpendicular to the partition direction, and a planar mode.

10. The method according to claim 1, wherein: When template matching is used, generating the prediction factor for each sub-block includes: determining whether template matching is feasible for each sub-block; and When template matching is feasible, a template is defined for each sub-region and then template matching is performed.

11. The method according to claim 10, wherein: Determining whether template matching is feasible includes: Whether template matching is feasible is determined by comparing the upper side and left side reference sample lines of the current block and the neighboring area between each sub-area with a predetermined threshold in terms of size.

12. The method according to claim 10, wherein: When template matching is not feasible, generating predictors for each sub-block involves: A predictor is generated for each sub-block by utilizing a zero motion vector.

13. A method for predicting a current block performed by a video encoding device, the method comprising: Determine the geometric partition information of the current block; Partition the current block into multiple sub-regions according to the geometric partition information; determining an index for each of the plurality of sub-regions, the index indicating a prediction mode for each sub-region; as well as Based on the value of the index, a prediction factor for each subblock is generated by utilizing inter prediction, intra prediction, or template matching.

14. The method according to claim 13, further comprising: The first predictor of the current block is generated by weighted summing the predictors of the multiple sub-regions.

15. The method according to claim 14, further comprising: generating a second predictor of the current block by performing inter-frame prediction on the current block; determining a geometric prediction mode flag based on the first predictor and the second predictor, the geometric prediction mode flag indicating whether a geometric prediction mode is applied to the current block based on the geometric partition information; as well as Encodes the geometry prediction mode flag.

16. The video encoding method according to claim 13, further comprising: Determining an index indicating geometric partition information from a predefined lookup table (LUT); as well as Encoding index.

17. A computer-readable recording medium storing a bit stream generated by a video encoding method, the video encoding method comprising: Determine the geometric partition information of the current block; Partition the current block into multiple sub-regions according to the geometric partition information; determining an index for each of the plurality of sub-regions, the index indicating a prediction mode for each sub-region; as well as Based on the value of the index, a prediction factor for each subblock is generated by utilizing inter prediction, intra prediction, or template matching.

Images