Video encoding / decoding method and device using combined intra / inter prediction mode based on sub-block
By employing a combined intra/inter prediction mode that weights inter and intra prediction blocks on a subblock basis, the method improves video encoding and decoding efficiency, effectively addressing the increasing data size challenges and enhancing picture quality in video compression.
Patent Information
- Application Number
- PCT/KR2024/006290
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2024-05-08
- Filing Date
- 2024-05-09
- Publication Date
- 2024-11-14
AI Technical Summary
Current video encoding and decoding technologies face inefficiencies in compressing and processing video data due to increasing data sizes, necessitating improved coding efficiency and picture quality, particularly through enhanced prediction methods.
The implementation of a combined intra/inter prediction mode based on subblocks, where inter prediction blocks of subblocks and intra prediction blocks are weighted to generate a prediction block, improving encoding and decoding efficiency by performing prediction on a subblock basis.
This approach enhances video encoding and decoding efficiency by optimizing prediction methods, leading to better coding efficiency and improved picture quality, addressing the growing data size challenges in video compression.
Smart Images

Figure KR2024006290_14112024_PF_FP_ABST
Abstract
Description
Video encoding / decoding method and device using combined intra / inter prediction mode based on subblocks
[0001] The present disclosure relates to a video encoding / decoding method and device using a combined intra / inter prediction mode based on subblocks, and more specifically, to a video encoding / decoding method and device that generates a prediction block of a current block by weighting inter prediction blocks of subblocks and intra prediction blocks of a current block.
[0002] The content described below merely provides background information related to the present embodiment and does not constitute prior art.
[0003] Since video data has a large amount of data compared to voice data or still image data, it requires a lot of hardware resources, including memory, to store or transmit it without processing for compression.
[0004] Therefore, when storing or transmitting video data, an encoder is used to compress the video data and store or transmit it, and a decoder receives the compressed video data, decompresses it, and plays it back. Examples of such video compression technologies include H.264 / AVC, HEVC (High Efficiency Video Coding), and VVC (Versatile Video Coding), which improves encoding efficiency by about 30% compared to HEVC.
[0005] However, as the size, resolution, and frame rate of images are gradually increasing, and the amount of data that needs to be encoded is also increasing, a new compression technology that has better encoding efficiency and better image quality improvement than existing compression technologies is required.
[0006] Combined Inter / Intra Prediction (CIIP) mode generates a prediction block for the current block by weighting the inter-prediction and intra-prediction signals. In CIIP mode, prediction is performed on a subblock-by-subblock basis to improve encoding and decoding efficiency.
[0007] The present disclosure aims to provide a method and device for performing prediction on a sub-block basis in a Combined Inter / Intra Prediction (CIIP) mode.
[0008] In addition, the present disclosure aims to provide a method and device for generating a prediction block of a current block by weighting inter-prediction blocks of sub-blocks and intra-prediction blocks of a current block in CIIP mode.
[0009] In addition, the present disclosure aims to provide a method and device for improving video encoding / decoding efficiency.
[0010] In addition, the present disclosure aims to provide a recording medium storing a bitstream generated by the video encoding / decoding method or device of the present disclosure.
[0011] In addition, the present disclosure aims to provide a method and device for transmitting a bitstream generated by the video encoding / decoding method or device of the present disclosure.
[0012] According to the present disclosure, the method may include: determining an inter-prediction mode of a current block based on at least one of motion vectors of first sub-blocks within a reference picture of the current block and motion vectors of control points of the current block; determining an intra-prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block; generating inter-prediction blocks of the sub-blocks using the inter-prediction mode of the current block, generating an intra-prediction block of the current block using the intra-prediction mode of the current block; and generating a prediction block of the current block by weighting the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block.
[0013] According to the present disclosure, a video encoding method may include a step of determining an inter-prediction mode of a current block based on at least one of motion vectors of first sub-blocks within a reference picture of the current block and control point motion vectors of the current block, a step of determining an intra-prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block, a step of generating inter-prediction blocks of the sub-blocks using the inter-prediction mode of the current block and generating an intra-prediction block of the current block using the intra-prediction mode of the current block, and a step of generating a prediction block of the current block by weighting the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block.
[0014] Additionally, according to the present disclosure, a method for transmitting a bitstream generated by a video encoding method or device according to the present disclosure can be provided.
[0015] In addition, according to the present disclosure, a recording medium storing a bitstream generated by a video encoding method or device according to the present disclosure can be provided.
[0016] In addition, according to the present disclosure, a recording medium storing a bitstream received and decoded by a video decoding device according to the present disclosure and used for restoring an image can be provided.
[0017] According to the present disclosure, a method and device for performing prediction on a sub-block basis in a Combined Inter / Intra Prediction (CIIP) mode can be provided.
[0018] In addition, according to the present disclosure, a method and device for generating a prediction block of a current block by weighting inter-prediction blocks of sub-blocks and intra-prediction blocks of a current block in a CIIP mode can be provided.
[0019] Additionally, according to the present disclosure, a method and device for improving video encoding / decoding efficiency can be provided.
[0020] The effects that can be obtained from the present disclosure are not limited to the effects mentioned above, and other effects that are not mentioned will be clearly understood by a person having ordinary skill in the art to which the present disclosure pertains from the description below.
[0021] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure.
[0022] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.
[0023] FIGS. 3A and 3B are diagrams illustrating multiple intra prediction modes, including wide-angle intra prediction modes.
[0024] Figure 4 is an example diagram of the surrounding blocks of the current block.
[0025] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure.
[0026] FIG. 6 is a diagram for explaining a method for generating a prediction block of a current block in a combined inter / intra prediction (CIIP) mode.
[0027] FIG. 7 is a diagram for explaining peripheral blocks referenced to determine weight values in CIIP mode according to one embodiment of the present disclosure.
[0028] FIG. 8 is a diagram illustrating a method for determining a weight value in CIIP mode according to one embodiment of the present disclosure.
[0029] FIG. 9 is a diagram for explaining a method for determining a motion vector in units of subblocks according to one embodiment of the present disclosure.
[0030] FIG. 10 is a diagram for explaining calculation of a pixel-by-pixel affine model according to one embodiment of the present disclosure.
[0031] FIG. 11 is a diagram for explaining the calculation of an affine model using a center vector according to one embodiment of the present disclosure.
[0032] FIG. 12 is a diagram for explaining calculation of an affine model in sub-block units according to one embodiment of the present disclosure.
[0033] FIG. 13 is a flowchart illustrating a process of generating a prediction block of a current block by performing prediction on a sub-block basis in CIIP mode according to one embodiment of the present disclosure.
[0034] FIG. 14 is a diagram illustrating a method for calculating template costs of sub-blocks according to one embodiment of the present disclosure.
[0035] FIG. 15 is a diagram for explaining a method for determining a weight value used to weight the inter-prediction blocks of sub-blocks and the intra-prediction block of the current block in CIIP mode according to one embodiment of the present disclosure.
[0036] FIG. 16a and FIG. 16b are diagrams for explaining a method of dividing a current block into multiple regions and a weight value of each region according to one embodiment of the present disclosure.
[0037] FIG. 17a and FIG. 17b are diagrams for explaining a method of dividing a current block into multiple regions and a weight value of each region according to another embodiment of the present disclosure.
[0038] FIG. 18a and FIG. 18b are diagrams for explaining a weight function of a sub-block according to one embodiment of the present disclosure.
[0039] FIG. 19 is a diagram for explaining a weight value of a subblock when the motion vector of the corresponding subblock in a reference picture is a zero motion vector, according to one embodiment of the present disclosure.
[0040] FIG. 20 is a flowchart illustrating a video decoding process according to one embodiment of the present disclosure.
[0041] FIG. 21 is a flowchart illustrating a video encoding process according to one embodiment of the present disclosure.
[0042] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary drawings. When designating components in each drawing, it should be noted that, where possible, identical components are given the same reference numerals, even if they appear in different drawings. Furthermore, in describing the present embodiments, detailed descriptions of related known configurations or functions will be omitted if they are deemed to obscure the gist of the present embodiments. In this disclosure, the terms "image" and "video" may be used interchangeably.
[0043] FIG. 1 is an exemplary block diagram of an image encoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image encoding device and its subcomponents will be described with reference to the illustration in FIG. 1.
[0044] The video encoding device may be configured to include a picture segmentation unit (110), a prediction unit (120), a subtractor (130), a transformation unit (140), a quantization unit (145), a reordering unit (150), an entropy encoding unit (155), an inverse quantization unit (160), an inverse transformation unit (165), an adder (170), a loop filter unit (180), and a memory (190).
[0045] Each component of the video encoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.
[0046] A single image (video) is composed of one or more sequences containing multiple pictures. Each picture is divided into multiple regions, and encoding is performed for each region. For example, a single picture is divided into one or more tiles and / or slices. Here, one or more tiles can be defined as a tile group. Each tile or / slice is divided into one or more Coding Tree Units (CTUs). Each CTU is then divided into one or more Coding Units (CUs) by a tree structure. Information applied to each CU is encoded as the syntax of the CU, and information commonly applied to CUs included in a CTU is encoded as the syntax of the CTU. In addition, information commonly applied to all blocks within a single slice is encoded as the syntax of the slice header, and information applied to all blocks constituting one or more pictures is encoded in the Picture Parameter Set (PPS) or the picture header. Furthermore, information commonly referenced by multiple pictures is encoded in a Sequence Parameter Set (SPS). And, information commonly referenced by one or more SPS is encoded in a Video Parameter Set (VPS). In addition, information commonly applied to one tile or tile group may be encoded as syntax of a tile or tile group header. Syntaxes included in an SPS, PPS, slice header, tile or tile group header may be referred to as high level syntax.
[0047] The picture segmentation unit (110) determines the size of the CTU. Information about the size of the CTU (CTU size) is encoded as the syntax of SPS or PPS and transmitted to the image decoding device.
[0048] The picture segmentation unit (110) divides each picture constituting an image into a plurality of CTUs having a predetermined size, and then recursively divides the CTUs using a tree structure. A leaf node in the tree structure becomes a CU, which is a basic unit of encoding.
[0049] The tree structure may be a QuadTree (QT) in which an upper node (or parent node) is divided into four lower nodes (or child nodes) of the same size, a BinaryTree (BT) in which an upper node is divided into two lower nodes, or a TernaryTree (TT) in which an upper node is divided into three lower nodes in a 1:2:1 ratio, or a structure that mixes two or more of the QT structures, BT structures, and TT structures. For example, a QTBT (QuadTree plus BinaryTree) structure may be used, or a QTBTTT (QuadTree plus BinaryTree TernaryTree) structure may be used. Here, BTTT may be combined and referred to as a MTT (Multiple-Type Tree).
[0050] Figure 2 is a drawing for explaining a method of dividing a block using the QTBTTT structure.
[0051] As illustrated in FIG. 2, a CTU may first be split into a QT structure. The quadtree splitting may be repeated until the size of the splitting block reaches the minimum block size (MinQTSize) of the leaf node allowed in the QT. A first flag (QT_split_flag) indicating whether each node of the QT structure is split into four nodes of the lower layer is encoded by the entropy encoding unit (155) and signaled to the image decoding device. If the leaf node of the QT is not larger than the maximum block size (MaxBTSize) of the root node allowed in the BT, it may be further split into one or more of the BT structure or the TT structure. There may be multiple splitting directions in the BT structure and / or the TT structure. For example, there may be two directions in which the block of the corresponding node is split horizontally and two directions in which the block is split vertically. As illustrated in FIG. 2, when MTT splitting begins, a second flag (mtt_split_flag) indicating whether nodes have been split, and if splitting has occurred, a flag indicating the splitting direction (vertical or horizontal) and / or a flag indicating the splitting type (Binary or Ternary) are encoded by the entropy encoding unit (155) and signaled to the image decoding device.
[0052] Alternatively, before encoding the first flag (QT_split_flag) indicating whether each node is split into four nodes of a lower layer, a CU split flag (split_cu_flag) indicating whether the node is split may be encoded. If the CU split flag (split_cu_flag) value indicates that the node is not split, the block of the corresponding node becomes a leaf node in the split tree structure and becomes a CU (coding unit), which is a basic unit of encoding. If the CU split flag (split_cu_flag) value indicates that the node is split, the video encoding device starts encoding from the first flag in the above-described manner.
[0053] As another example of a tree structure, when QTBT is used, there may be two types: a type that horizontally splits the block of the corresponding node into two blocks of the same size (i.e., symmetric horizontal splitting) and a type that vertically splits it (i.e., symmetric vertical splitting). A split flag (split_flag) indicating whether each node of the BT structure is split into blocks of a lower layer and split type information indicating the type of split are encoded by the entropy encoding unit (155) and transmitted to the image decoding device. Meanwhile, there may additionally be a type that splits the block of the corresponding node into two blocks of an asymmetrical shape. The asymmetric shape may include a shape that splits the block of the corresponding node into two rectangular blocks with a size ratio of 1:3, or a shape that splits the block of the corresponding node in a diagonal direction.
[0054] A CU can have various sizes depending on the QTBT or QTBTTT partitioning from the CTU. Hereinafter, the block corresponding to the CU to be encoded or decoded (i.e., the leaf node of the QTBTTT) is referred to as the "current block." Depending on the QTBTTT partitioning employed, the current block may be rectangular as well as square.
[0055] The prediction unit (120) predicts the current block and generates a prediction block. The prediction unit (120) includes an intra prediction unit (122) and an inter prediction unit (124).
[0056] In general, each current block within a picture can be predictively coded. Prediction of the current block can typically be performed using either intra-prediction (using data from the picture containing the current block) or inter-prediction (using data from a picture coded before the picture containing the current block). Inter-prediction encompasses both unidirectional and bidirectional prediction.
[0057] The intra prediction unit (122) predicts pixels within the current block using pixels (reference pixels) located around the current block within the current picture including the current block. There are multiple intra prediction modes depending on the prediction direction. For example, as shown in Fig. 3a, the multiple intra prediction modes may include two non-directional modes including the Planar mode and the DC mode, and 65 directional modes. The surrounding pixels to be used and the calculation formula are defined differently depending on each prediction mode.
[0058] For efficient directional prediction for a rectangular current block, directional modes (intra prediction modes 67 to 80 and -1 to -14) indicated by dotted arrows in Fig. 3b may be additionally used. These may be referred to as "wide-angle intra-prediction modes." In Fig. 3b, the arrows point to corresponding reference samples used for prediction, and do not indicate the prediction direction. The prediction direction is opposite to the direction indicated by the arrows. Wide-angle intra-prediction modes are modes that perform prediction in the opposite direction of a specific directional mode without additional bit transmission when the current block is rectangular. At this time, among the wide-angle intra-prediction modes, some wide-angle intra-prediction modes available for the current block may be determined based on the ratio of the width and height of the rectangular current block. For example, wide-angle intra prediction modes (intra prediction modes 67 to 80) having an angle less than 45 degrees are available when the current block is a rectangular shape whose height is smaller than its width, and wide-angle intra prediction modes (intra prediction modes -1 to -14) having an angle greater than -135 degrees are available when the current block is a rectangular shape whose width is larger than its height.
[0059] The intra prediction unit (122) can determine an intra prediction mode to be used to encode the current block. In some examples, the intra prediction unit (122) can encode the current block using multiple intra prediction modes and select an appropriate intra prediction mode to be used from the tested modes. For example, the intra prediction unit (122) can calculate bit-rate distortion values using rate-distortion analysis for multiple tested intra prediction modes and select an intra prediction mode with the best bit-rate distortion characteristics among the tested modes.
[0060] The intra prediction unit (122) selects one intra prediction mode from among multiple intra prediction modes and predicts the current block using surrounding pixels (reference pixels) and an operation formula determined according to the selected intra prediction mode. Information about the selected intra prediction mode is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.
[0061] The inter prediction unit (124) generates a prediction block for the current block using a motion compensation process. The inter prediction unit (124) searches for a block most similar to the current block within reference pictures that were encoded and decoded before the current picture, and generates a prediction block for the current block using the searched block. Then, a motion vector (MV) corresponding to the displacement between the current block within the current picture and the prediction block within the reference picture is generated. Generally, motion estimation is performed on the luma component, and the motion vector calculated based on the luma component is used for both the luma component and the chroma component. The motion information including information on the reference picture used to predict the current block and information on the motion vector is encoded by the entropy encoding unit (155) and transmitted to the image decoding device.
[0062] The inter prediction unit (124) may perform interpolation on a reference picture or a reference block to improve prediction accuracy. That is, subsamples between two consecutive integer samples are interpolated by applying filter coefficients to a plurality of consecutive integer samples including the two integer samples. When a process of searching for a block most similar to the current block is performed on the interpolated reference picture, the motion vector can be expressed up to a precision in decimal units rather than a precision in integer sample units. The precision or resolution of the motion vector can be set differently for each target region to be encoded, such as a slice, tile, CTU, CU, etc. When such adaptive motion vector resolution (AMVR) is applied, information on the motion vector resolution to be applied to each target region must be signaled for each target region. For example, when the target region is a CU, information on the motion vector resolution applied to each CU is signaled. Information on the motion vector resolution may be information indicating the precision of a differential motion vector, which will be described later.
[0063] Meanwhile, the inter prediction unit (124) can perform inter prediction using bi-prediction. In the case of bi-prediction, two reference pictures and two motion vectors indicating the block position most similar to the current block within each reference picture are used. The inter prediction unit (124) selects a first reference picture and a second reference picture from reference picture list 0 (RefPicList0) and reference picture list 1 (RefPicList1), respectively, and searches for a block similar to the current block within each reference picture to generate a first reference block and a second reference block. Then, the first reference block and the second reference block are averaged or weighted averaged to generate a prediction block for the current block. Then, motion information including information on two reference pictures used to predict the current block and information on two motion vectors is transmitted to the entropy encoding unit (155). Here, reference picture list 0 may be composed of pictures that are before the current picture in display order among the restored pictures, and reference picture list 1 may be composed of pictures that are after the current picture in display order among the restored pictures. However, this is not necessarily limited to this, and restored pictures that are after the current picture in display order may be additionally included in reference picture list 0, and conversely, restored pictures that are before the current picture may be additionally included in reference picture list 1.
[0064] Various methods can be used to minimize the number of bits required to encode motion information.
[0065] For example, if the reference picture and motion vector of the current block are identical to those of a neighboring block, the motion information of the current block can be transmitted to the image decoding device by encoding information that can identify the neighboring block. This method is called 'merge mode'.
[0066] In merge mode, the inter prediction unit (124) selects a predetermined number of merge candidate blocks (hereinafter referred to as 'merge candidates') from the surrounding blocks of the current block.
[0067] As the surrounding blocks for deriving merge candidates, all or part of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block within the current picture may be used, as illustrated in FIG. 4. In addition, a block located within a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as a merge candidate. For example, a block co-located with the current block within the reference picture or blocks adjacent to the block at the co-located block may be additionally used as a merge candidate. If the number of merge candidates selected by the method described above is less than a preset number, a 0 vector is added to the merge candidates.
[0068] The inter prediction unit (124) uses these surrounding blocks to construct a merge list containing a predetermined number of merge candidates. Among the merge candidates included in the merge list, the merge candidate to be used as motion information of the current block is selected and merge index information for identifying the selected candidate is generated. The generated merge index information is encoded by the entropy encoding unit (155) and transmitted to the video decoding device.
[0069] Merge Skip mode is a special case of merge mode. After quantization, when all transform coefficients for entropy encoding are close to zero, only neighboring block selection information is transmitted without transmitting residual signals. By utilizing merge skip mode, relatively high encoding efficiency can be achieved for low-motion images, still images, and screen content images.
[0070] Hereinafter, merge mode and merge skip mode are collectively referred to as merge / skip mode.
[0071] Another method for encoding motion information is Advanced Motion Vector Prediction (AMVP) mode.
[0072] In AMVP mode, the inter prediction unit (124) derives predicted motion vector candidates for the motion vector of the current block using neighboring blocks of the current block. As neighboring blocks used to derive predicted motion vector candidates, all or some of the left block (A0), the lower left block (A1), the upper block (B0), the upper right block (B1), and the upper left block (B2) adjacent to the current block in the current picture as shown in FIG. 4 may be used. In addition, a block located in a reference picture (which may or may not be the same as the reference picture used to predict the current block) other than the current picture in which the current block is located may be used as the neighboring block used to derive predicted motion vector candidates. For example, a block co-located with the current block in the reference picture or blocks adjacent to the block in the co-located block may be used. If the number of motion vector candidates is less than a preset number by the method described above, a 0 vector is added to the motion vector candidates.
[0073] The inter prediction unit (124) derives predicted motion vector candidates using the motion vectors of these surrounding blocks, and determines a predicted motion vector for the motion vector of the current block using the predicted motion vector candidates. Then, the predicted motion vector is subtracted from the motion vector of the current block to produce a differential motion vector.
[0074] The predicted motion vector can be obtained by applying a predefined function (e.g., median, mean, etc.) to the predicted motion vector candidates. In this case, the image decoding device also knows the predefined function. In addition, since the surrounding blocks used to derive the predicted motion vector candidates are blocks that have already been encoded and decoded, the image decoding device also already knows the motion vectors of the surrounding blocks. Therefore, the image encoding device does not need to encode information to identify the predicted motion vector candidates. Therefore, in this case, information about the differential motion vector and information about the reference picture used to predict the current block are encoded.
[0075] Alternatively, the predicted motion vector can be determined by selecting one of the predicted motion vector candidates. In this case, information for identifying the selected predicted motion vector candidate is additionally encoded, along with information about the differential motion vector and the reference picture used to predict the current block.
[0076] The subtractor (130) subtracts the prediction block generated by the intra prediction unit (122) or inter prediction unit (124) from the current block to generate a residual block.
[0077] The transformation unit (140) transforms residual signals within a residual block having pixel values in a spatial domain into transform coefficients in a frequency domain. The transformation unit (140) may transform the residual signals within the residual block using the entire size of the residual block as a transformation unit, or may divide the residual block into a plurality of sub-blocks and use the sub-blocks as transformation units to perform the transformation. Alternatively, the residual signals may be transformed using only the transformation domain sub-block as a transformation unit by dividing the sub-blocks into two sub-blocks, that is, a transformation domain and a non-transform domain. Here, the transformation domain sub-block may be one of two rectangular blocks having a size ratio of 1:1 with respect to the horizontal axis (or vertical axis). In this case, a flag (cu_sbt_flag) indicating that only a sub-block has been converted, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag), and / or position information (cu_sbt_pos_flag) are encoded by the entropy encoding unit (155) and signaled to the image decoding device. In addition, the size of the conversion area sub-block may have a size ratio of 1:3 with respect to the horizontal axis (or vertical axis), and in this case, a flag (cu_sbt_quad_flag) distinguishing the corresponding division is additionally encoded by the entropy encoding unit (155) and signaled to the image decoding device.
[0078] Meanwhile, the transformation unit (140) can individually perform transformations on the residual block in the horizontal and vertical directions. For the transformation, various types of transformation functions or transformation matrices can be used. For example, a pair of transformation functions for horizontal transformation and vertical transformation can be defined as a Multiple Transform Set (MTS). The transformation unit (140) can select one transformation function pair with the best transformation efficiency among the MTS and transform the residual block in the horizontal and vertical directions, respectively. Information (mts_idx) on the transformation function pair selected among the MTS is encoded by the entropy encoding unit (155) and signaled to the image decoding device.
[0079] The quantization unit (145) quantizes the transform coefficients output from the transform unit (140) using quantization parameters and outputs the quantized transform coefficients to the entropy encoding unit (155). The quantization unit (145) may directly quantize a related residual block without transformation for a certain block or frame. The quantization unit (145) may also apply different quantization coefficients (scaling values) according to the positions of the transform coefficients within the transform block. The quantization matrix applied to the quantized transform coefficients arranged in two dimensions may be encoded and signaled to an image decoding device.
[0080] The rearrangement unit (150) can perform rearrangement of coefficient values for quantized residual values.
[0081] The reordering unit (150) can change a two-dimensional coefficient array into a one-dimensional coefficient sequence by using coefficient scanning. For example, the reordering unit (150) can output a one-dimensional coefficient sequence by scanning from the DC coefficient to the coefficients of the high-frequency region by using a zig-zag scan or a diagonal scan. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional coefficient array in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficients in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, the scanning method to be used may be determined among the zig-zag scan, the diagonal scan, the vertical scan, and the horizontal scan.
[0082] The entropy encoding unit (155) generates a bitstream by encoding a sequence of one-dimensional quantized transform coefficients output from the rearrangement unit (150) using various encoding methods such as CABAC (Context-based Adaptive Binary Arithmetic Code) and Exponential Golomb.
[0083] In addition, the entropy encoding unit (155) encodes information related to block division, such as CTU size, CU division flag, QT division flag, MTT division type, and MTT division direction, so that the image decoding device can divide the block in the same manner as the image encoding device. In addition, the entropy encoding unit (155) encodes information about a prediction type indicating whether the current block is encoded by intra prediction or inter prediction, and encodes intra prediction information (i.e., information about an intra prediction mode) or inter prediction information (information about an encoding mode of motion information (merge mode or AMVP mode), a merge index in the case of a merge mode, and a reference picture index and a differential motion vector in the case of an AMVP mode) according to the prediction type. In addition, the entropy encoding unit (155) encodes information related to quantization, that is, information about a quantization parameter and information about a quantization matrix.
[0084] The inverse quantization unit (160) inversely quantizes the quantized transform coefficients output from the quantization unit (145) to generate transform coefficients. The inverse transform unit (165) transforms the transform coefficients output from the inverse quantization unit (160) from the frequency domain to the spatial domain to restore the residual block.
[0085] An adder (170) adds the restored residual block and the predicted block generated by the prediction unit (120) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting the next block.
[0086] The loop filter unit (180) performs filtering on restored pixels to reduce blocking artifacts, ringing artifacts, blurring artifacts, etc. that occur due to block-based prediction and transformation / quantization. The loop filter unit (180) may include all or part of a deblocking filter (182), a sample adaptive offset (SAO) filter (184), and an adaptive loop filter (ALF, 186) as an in-loop filter.
[0087] The deblocking filter (182) filters the boundaries between restored blocks to remove blocking artifacts caused by block-based encoding / decoding, and the SAO filter (184) and the ALF (186) perform additional filtering on the deblocking-filtered image. The SAO filter (184) and the ALF (186) are filters used to compensate for the differences between restored pixels and original pixels caused by lossy coding. The SAO filter (184) improves not only subjective image quality but also encoding efficiency by applying an offset in units of CTUs. In contrast, the ALF (186) performs block-based filtering, and compensates for distortion by applying different filters by distinguishing the edges and degrees of variation of the corresponding block. Information on filter coefficients to be used in the ALF can be encoded and signaled to an image decoding device.
[0088] The restored blocks filtered through the deblocking filter (182), SAO filter (184), and ALF (186) are stored in the memory (190). When all blocks within a picture are restored, the restored picture can be used as a reference picture for inter-predicting blocks within a picture to be encoded later.
[0089] The video encoding device can store the bitstream of encoded video data on a non-transitory storage medium or transmit it to the video decoding device using a communication network.
[0090] FIG. 5 is an exemplary block diagram of an image decoding device capable of implementing the techniques of the present disclosure. Hereinafter, the image decoding device and its subcomponents will be described with reference to FIG. 5.
[0091] The video decoding device may be configured to include an entropy decoding unit (510), a rearrangement unit (515), an inverse quantization unit (520), an inverse transformation unit (530), a prediction unit (540), an adder (550), a loop filter unit (560), and a memory (570).
[0092] Similar to the video encoding device of FIG. 1, each component of the video decoding device may be implemented in hardware, software, or a combination of hardware and software. Furthermore, the functions of each component may be implemented in software, with a microprocessor executing the software functions corresponding to each component.
[0093] The entropy decoding unit (510) decodes the bitstream generated by the image encoding device to extract information related to block division, thereby determining the current block to be decoded, and extracts prediction information, information on residual signals, etc. required to restore the current block.
[0094] The entropy decoding unit (510) extracts information about the CTU size from the Sequence Parameter Set (SPS) or the Picture Parameter Set (PPS), determines the size of the CTU, and divides the picture into CTUs of the determined size. Then, the CTU is determined as the top layer of the tree structure, i.e., the root node, and the CTU is divided using the tree structure by extracting division information about the CTU.
[0095] For example, when splitting a CTU using the QTBTTT structure, first, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Then, for the nodes corresponding to the leaf nodes of QT, the second flag (mtt_split_flag) related to the splitting of MTT and the split direction (vertical / horizontal) and / or split type (binary / ternary) information are extracted, and the corresponding leaf nodes are split into the MTT structure. Accordingly, each node below the leaf nodes of QT are split recursively into the BT or TT structure.
[0096] As another example, when splitting a CTU using the QTBTTT structure, the CU split flag (split_cu_flag) indicating whether the CU is split is first extracted, and if the block is split, the first flag (QT_split_flag) may be extracted. During the splitting process, each node may undergo zero or more repeated QT splits followed by zero or more repeated MTT splits. For example, a CTU may undergo an MTT split right away, or conversely, may undergo only multiple QT splits.
[0097] As another example, when splitting a CTU using the QTBT structure, the first flag (QT_split_flag) related to the splitting of QT is extracted, and each node is split into four nodes of the lower layer. Furthermore, for nodes corresponding to leaf nodes of QT, a split flag (split_flag) indicating whether to further split into BTs and splitting direction information are extracted.
[0098] Meanwhile, when the entropy decoding unit (510) determines the current block to be decoded by using the division of the tree structure, it extracts information on the prediction type indicating whether the current block is intra-predicted or inter-predicted. If the prediction type information indicates intra-prediction, the entropy decoding unit (510) extracts syntax elements for intra-prediction information (intra-prediction mode) of the current block. If the prediction type information indicates inter-prediction, the entropy decoding unit (510) extracts syntax elements for inter-prediction information, i.e., information indicating a motion vector and a reference picture referenced by the motion vector.
[0099] Additionally, the entropy decoding unit (510) extracts information about the quantized transform coefficients of the current block as information related to quantization and information about residual signals.
[0100] The rearrangement unit (515) can change the sequence of one-dimensional quantized transform coefficients entropy-decoded in the entropy decoding unit (510) back into a two-dimensional coefficient array (i.e., block) in the reverse order of the coefficient scanning performed by the image encoding device.
[0101] The inverse quantization unit (520) inversely quantizes the quantized transform coefficients and inversely quantizes the quantized transform coefficients using the quantization parameters. The inverse quantization unit (520) may also apply different quantization coefficients (scaling values) to the quantized transform coefficients arranged in two dimensions. The inverse quantization unit (520) may perform inverse quantization by applying a matrix of quantized coefficients (scaling values) from an image encoding device to a two-dimensional array of quantized transform coefficients.
[0102] The inverse transform unit (530) inversely transforms the inverse quantized transform coefficients from the frequency domain to the spatial domain to restore residual signals, thereby generating a residual block for the current block.
[0103] In addition, when the inverse transform unit (530) inversely transforms only a portion of a transform block (sub-block), it extracts a flag (cu_sbt_flag) indicating that only a sub-block of the transform block has been transformed, directionality (vertical / horizontal) information (cu_sbt_horizontal_flag) of the sub-block, and / or position information (cu_sbt_pos_flag) of the sub-block, and inversely transforms the transform coefficients of the corresponding sub-block from the frequency domain to the spatial domain to restore residual signals, and fills “0” values with residual signals for areas that have not been inversely transformed, thereby generating a final residual block for the current block.
[0104] In addition, when MTS is applied, the inverse transform unit (530) determines a transform function or a transform matrix to be applied in the horizontal and vertical directions using MTS information (mts_idx) signaled from the image encoding device, and performs inverse transform on the transform coefficients within the transform block in the horizontal and vertical directions using the determined transform function.
[0105] The prediction unit (540) may include an intra prediction unit (542) and an inter prediction unit (544). The intra prediction unit (542) is activated when the prediction type of the current block is intra prediction, and the inter prediction unit (544) is activated when the prediction type of the current block is inter prediction.
[0106] The intra prediction unit (542) determines the intra prediction mode of the current block among a plurality of intra prediction modes from the syntax elements for the intra prediction mode extracted from the entropy decoding unit (510), and predicts the current block using reference pixels around the current block according to the intra prediction mode.
[0107] The inter prediction unit (544) uses the syntax elements for the inter prediction mode extracted from the entropy decoding unit (510) to determine the motion vector of the current block and the reference picture referenced by the motion vector, and predicts the current block using the motion vector and the reference picture.
[0108] An adder (550) adds the residual block output from the inverse transform unit (530) and the predicted block output from the inter prediction unit (544) or the intra prediction unit (542) to restore the current block. The pixels within the restored current block are used as reference pixels when intra-predicting a block to be decoded later.
[0109] The loop filter unit (560) may include a deblocking filter (562), an SAO filter (564), and an ALF (566) as in-loop filters. The deblocking filter (562) deblocks the boundaries between restored blocks to remove blocking artifacts caused by block-by-block decoding. The SAO filter (564) and the ALF (566) perform additional filtering on restored blocks after deblocking filtering to compensate for differences between restored pixels and original pixels caused by lossy coding. The filter coefficients of the ALF are determined using information about filter coefficients decoded from the non-stream.
[0110] The restored blocks filtered through the deblocking filter (562), SAO filter (564), and ALF (566) are stored in the memory (570). When all blocks within a picture are restored, the restored picture is used as a reference picture for inter-predicting blocks within a picture to be encoded later.
[0111] The following embodiments may be performed by various components within an image encoding device. They may also be performed by various components within an image decoding device.
[0112] Figure 6 is a diagram illustrating a method for generating a prediction block of a current block in a Combined Inter / Intra Prediction (CIIP) mode. In the CIIP mode, the inter-prediction block of the current block can be generated in the same manner as in the general Merge mode. The intra-prediction block can be generated by applying the planar mode to neighboring reference pixels around the current block. A weighted value can be applied to the generated inter-prediction block and intra-prediction block to generate a final prediction block based on the CIIP.
[0113] Referring to Figure 6, a reference block (P) within a reference picture is based on the merge mode. inter ) can be derived. The planar mode is applied to the reference pixels adjacent to the current block to generate an intra prediction block (P Planar ) can be created. The reference block (P inter ) and intra prediction block (P Planar ) to apply weights to the prediction block (P) based on CIIP. CIIP ) can be generated. Prediction blocks (P) based on CIIP CIIP )silver can be generated using the mathematical formula.
[0114] FIG. 7 is a diagram illustrating neighboring blocks referenced for determining weights in CIIP mode according to one embodiment of the present disclosure. In CIIP mode, weights may be determined by considering whether neighboring blocks adjacent to the current block are encoded in intra prediction mode.
[0115] Referring to FIG. 7, in the CIIP mode, the weight value can be determined by considering whether the adjacent block A above the current block is encoded in the intra prediction mode and whether the adjacent block L to the left of the current block is encoded in the intra prediction mode.
[0116] FIG. 8 is a diagram illustrating a method for determining weights in CIIP mode according to one embodiment of the present disclosure. If many neighboring blocks adjacent to the current block are encoded in intra prediction mode, a large weight value may be assigned to the intra prediction block. Conversely, if few neighboring blocks adjacent to the current block are encoded in intra prediction mode, a small weight value may be assigned to the intra prediction block.
[0117] Referring to FIG. 8, when the upper peripheral block A and the left peripheral block L of the current block of FIG. 7 are encoded in the intra prediction mode, the weight value assigned to the intra prediction block may be 3. When the upper peripheral block A is encoded in the intra prediction mode and the left peripheral block L is not encoded in the intra prediction mode, the weight value assigned to the intra prediction block may be 2. When the upper peripheral block A is not encoded in the intra prediction mode and the left peripheral block L is encoded in the intra prediction mode, the weight value assigned to the intra prediction block may be 2. When the upper peripheral block A is not encoded in the intra prediction mode and the left peripheral block L is not encoded in the intra prediction mode, the weight value assigned to the intra prediction block may be 1.
[0118] FIG. 9 is a diagram for explaining a method for determining a motion vector in units of subblocks according to one embodiment of the present disclosure.
[0119] Referring to Fig. 9, Subblock-based Temporal Motion Vector Prediction (SbTMVP) uses motion information in collocated pictures to improve prediction of motion vectors of each subblock in merge mode. Each subblock is one of the blocks generated by dividing the current block, and the collocated picture is a picture that includes the collocated block. If the reference picture of the neighboring block A1 adjacent to the lower left of the current block is identical to the collocated picture, SbTMVP applies motion shift using the motion vector of the neighboring block A1 in the current picture. If the reference picture of the neighboring block A1 adjacent to the lower left of the current block is not identical to the collocated picture, SbTMVP uses a zero motion vector.
[0120] SbTMVP applies motion shift to a corresponding block in a corresponding picture. SbTMVP extracts motion information (motion vector and reference picture indexes) of a corresponding subblock in the corresponding picture for each subblock. Here, motion information of the center position of each corresponding subblock can be extracted. The size of each subblock can be 8x8. SbTMVP generates motion information of each subblock by applying temporal motion scaling to the extracted motion information. SbTMVP extracts a first motion vector and a second motion vector of a corresponding subblock (hereinafter, “corresponding subblock”) in the corresponding picture for each subblock. SbTMVP scales the first motion vector of the corresponding subblock and the second motion vector of the corresponding subblock to generate a motion vector of a subblock of a current block (hereinafter, “current subblock”). The first motion vector of the corresponding sub-block and the second motion vector of the corresponding sub-block are scaled to generate the first motion vector of the current sub-block and the second motion vector of the current sub-block.
[0121] FIG. 10 is a diagram for explaining calculation of a pixel-by-pixel affine model according to one embodiment of the present disclosure.
[0122] Referring to FIG. 10, the image decoding device uses the possibility that the pixel values and motion information of the current block and the surrounding blocks are the same or similar. The image decoding device predicts the control point motion vector (CPMV) using the motion vector of the block adjacent to the position corresponding to the vertex (A, B, C) of the current block. The image decoding device models the geometric transformation relationship between the current block and the predicted block, i.e., the affine model, using the control point motion vectors, and then performs the prediction of the current block based on the modeled transformation relationship. In the example of FIG. 10, a 6-parameter model using the control point motion vectors of three points A, B, and C is presented, but a 4-parameter model using the control point motion vectors of two points A, B or A, C may be used depending on the embodiment. According to the 4-parameter model and the 6-parameter model, the motion vector (mv) at the target block (x, y) pixel x , mv y ) can be expressed as in mathematical expressions 1 and 2, respectively, using the control point motion vector and the position of each pixel.
[0123]
[0124]
[0125] Here, W and H are the width and height of the current block. (cpmv ix , cpmv iy) is the motion vector of the i-th control point. The predicted value for each pixel of the current block can be predicted using the motion vector calculated according to Equation 1 or Equation 2. Although not included in the affine model, CPMVP4 can be defined for position D, as in the example of Fig. 10. Since position D is a pixel that has not yet been restored, the motion vector of the collocated pixel in the reference picture can be used as CPMVP4.
[0126] In the example of Fig. 10, CPMVP represents a control point motion vector predictor in the affine AMVP mode. Since the motion vector difference (MVD) is not transmitted in the affine merge mode, CPMVP is identical to CPMV. Since the motion vector difference is transmitted in the affine AMVP mode, CPMV can be calculated by adding the CPMVP and MVD. The example of Fig. 10 shows a method of using a combination among the methods for generating a control point motion vector predictor in the affine AMVP mode. The affine merge mode and the affine AMVP mode will be described later.
[0127] FIG. 11 is a diagram for explaining the calculation of an affine model using a center vector according to one embodiment of the present disclosure.
[0128] In addition, in order to reduce the computational complexity according to an embodiment, the image decoding device can perform prediction on a block-by-block basis using each control point motion vector as a center vector, as in the example of FIG. 11. At this time, assuming that the current block has four control point motion vectors, the current block can be divided into four blocks. In the method illustrated in FIG. 11, blocks having each control point motion vector as a center vector can be predicted based on the same motion vector. Therefore, compared to the embodiment in which calculation is performed on a pixel-by-pixel basis, the prediction accuracy may be lower, but the computational complexity may be reduced.
[0129] FIG. 12 is an exemplary diagram showing the calculation of an affine model in sub-block units according to one embodiment of the present disclosure.
[0130] As another embodiment, in order to reduce computational complexity, the video decoding device may perform prediction in units of subblocks, as in the example of FIG. 12. If the width or height of the current block is greater than the width or height of the subblock, the video decoding device may divide the current block into subblocks. The video decoding device derives a control point motion vector of each vertex position of the divided subblocks using the control point motion vectors of the current block, and then derives a representative motion vector of one of the control point motion vectors of each subblock. The video decoding device generates a prediction block in units of subblocks using the derived representative motion vector, and then combines the prediction blocks in units of subblocks to generate a first prediction block of the current block.
[0131] In some embodiments, the video decoding device may filter the first prediction block to generate a second prediction block. Thereafter, the video decoding device may generate a final prediction block of the current block using one of the first prediction block and the second prediction block.
[0132] To reduce the amount of bits required for encoding control point motion vectors, the general interprediction (translational motion prediction) methods described above, i.e., affine merge mode and affine AMVP mode, can be applied. Hereinafter, affine merge mode and affine AMVP mode are referred to as control point motion vector prediction methods.
[0133] As an example, in the case of the affine merge mode, the inter prediction unit (124) of the video encoding device constructs a predetermined number (e.g., 5) of affine merge candidate lists. First, the video encoding device derives inherited affine merge candidates from the surrounding blocks of the target block. For example, the merge candidate list is generated by inducing a predetermined number of inherited affine merge candidates from the surrounding samples (A0, A1, B0, B1, B2) of the target block illustrated in FIG. 4. Each of the inherited affine merge candidates included in the candidate list corresponds to a combination of two or three CPMVs.
[0134] An image encoding device derives inherited affine merge candidates from the control point motion vectors of neighboring blocks of a target block predicted in affine mode. In some embodiments, the number of merge candidates derived from neighboring blocks predicted in affine mode may be limited. For example, the image encoding device may derive two inherited affine merge candidates, one from A0 and A1, and one from B0, B1, and B2, from neighboring blocks predicted in affine mode. The priority may be in the order of A0, A1, and then in the order of B0, B1, and B2.
[0135] Meanwhile, when the total number of merge candidates is three or more, the image encoding device can derive the insufficient number of constructed affine merge candidates from the translational motion vectors of the surrounding blocks, as in the example of FIG. 10.
[0136] The video encoding device derives control point motion vectors CPMV1, CPMV2, and CPMV3 from the neighboring block groups {B2, B3, A2}, {B1, B0}, and {A1, A0}, respectively. As an example, the priorities within each neighboring block group may be in the order of B2, B3, and A2, then in the order of B1, B0, and then in the order of A1, A0. In addition, another control point motion vector CPMV4 is derived from the collocated block C0 within the reference picture. The video encoding device generates the missing number of combined affine merge candidates by combining two or three of the four control point motion vectors. The priorities of the combinations are as follows. The elements within each group are listed in the order of upper-left, upper-right, and lower-left control point motion vectors.
[0137] {CPMV1, CPMV2, CPMV3}, {CPMV1, CPMV2, CPMV4}, {CPMV1, CPMV3, CPMV4},
[0138] {CPMV2, CPMV3, CPMV4}, {CPMV1, CPMV2}, {CPMV1, CPMV3}
[0139] If the merge candidate list cannot be filled using the inherited affine merge candidates and the combined affine merge candidates, the video encoding device can add a zero motion vector as a candidate.
[0140] In order to optimize encoding efficiency, a video encoding device selects a merge candidate from a merge candidate list and determines a merge index indicating the selected merge candidate. The video encoding device performs affine motion prediction on a target block using the selected merge candidate. When the merge candidate consists of two control point motion vectors, affine motion prediction is performed using a four-parameter model. On the other hand, when the merge candidate consists of three control point motion vectors, affine motion prediction is performed using a six-parameter model. The video encoding device encodes the merge index and signals it to a video decoding device.
[0141] The video decoding device decodes the merge index. The inter prediction unit (544) of the video decoding device constructs a merge candidate list in the same manner as the video encoding device, and performs affine motion prediction using control point motion vectors corresponding to the merge candidates indicated by the merge index.
[0142] As another example, in the case of the affine AMVP mode, in terms of optimizing encoding efficiency, the inter prediction unit (124) of the video encoding device determines the form of the affine model and the actual control point motion vectors according to the affine model for the target block. The video encoding device calculates the MVD, which is the difference between the actual control point motion vector and the MVP (i.e., the control point motion vector predictor) of each control point, for each control point, and then encodes the MVD of each control point. To derive each control point motion vector predictor, the inter prediction unit (124) constructs a predefined number (e.g., 2) of affine AMVP candidate lists. When the target block is of the 4-parameter type, the candidates included in the list each consist of pairs of two control point motion vectors. On the other hand, when the target block is of the 6-parameter type, the candidates included in the list each consist of pairs of three control point motion vectors.
[0143] Below, we describe how to construct a candidate list in the affine AMVP mode. The affine AMVP candidate list can be derived similarly to the method for constructing the affine merge candidate list described above.
[0144] The video encoding device verifies whether the reference picture of the inherited affine AMVP candidate is identical to the reference picture of the current block. Here, the inherited affine AMVP candidate may be a block predicted in affine mode among the neighboring blocks (A0, A1, B0, B1, B2) of the target block illustrated in FIG. 4, as in the aforementioned affine merge mode.
[0145] If the reference picture of the inherited affine AMVP candidate and the reference picture of the current block are the same, the video encoding device adds the corresponding inherited affine AMVP candidate.
[0146] On the other hand, if the reference picture of the inherited affine AMVP candidate and the reference picture of the current block are not the same, the video encoding device checks whether the reference pictures of all CPMVs of the combined affine AMVP candidate and the reference picture of the current block are the same. Here, all CPMVPs of the combined affine AMVP candidate can be derived from the motion vectors of the surrounding samples shown in Fig. 10, as in the affine merge mode described above. If the reference pictures of all CPMVPs of the combined affine AMVP candidate and the reference picture of the current block are the same, the video encoding device adds the corresponding combined affine AMVP candidate.
[0147] At this time, the affine model type of the target block must be considered. If the affine model type of the target block is a 4-parameter type, the video encoding device derives two control point motion vectors (the upper-left and upper-right control point motion vectors of the target block) using the affine model of the surrounding blocks. If the affine model type of the target block is a 6-parameter type, it derives three control point motion vectors (the upper-left, upper-right, and lower-left control point motion vectors of the target block) using the affine model of the surrounding blocks.
[0148] If the reference pictures of all CPMVPs and the reference picture of the current block are not the same, the video encoding device adds a translational motion vector as an affine AMVP candidate.
[0149] If the candidate list cannot be filled even after performing all of the above processes (i.e., the preset number cannot be filled), the video encoding device adds the zero motion vector as an affine AMVP candidate.
[0150] The video encoding device selects one candidate from the affine AMVP list and determines a candidate index indicating the selected candidate. At this time, each control point motion vector of the selected candidate corresponds to each control point motion vector predictor. In terms of optimizing encoding efficiency, the video encoding device determines the actual control point motion vector for each control point of the target block, and then calculates the MVD between the actual control point motion vector and the control point motion vector predictor. The video encoding device encodes the affine model form of the target block, the candidate index, and the MVD of each control point, and signals them to the video decoding device.
[0151] The video decoding device decodes the affine model form, candidate index, and MVD of each control point. The inter prediction unit (544) of the video decoding device generates an affine AMVP list in the same manner as the video encoding device, and selects a candidate indicated by the candidate index within the affine AMVP list. The video decoding device adds the MVD corresponding to each control point motion vector predictor of the selected candidate to restore the motion vector of each control point. The video decoding device performs affine motion prediction using the restored control point motion vectors.
[0152] FIG. 13 is a flowchart illustrating a process of generating a prediction block of a current block by performing prediction on a sub-block basis in CIIP mode according to one embodiment of the present disclosure.
[0153] Referring to FIG. 13, a video decoding device determines motion vectors of subblocks (S1310). The video decoding device can generate motion vectors of subblocks using SbTMVP. Here, the sizes of the subblocks can each be 8x8. An index indicating whether SbTMVP is used can be signaled. The video decoding device sequentially searches motion vectors of up to N neighboring blocks adjacent to the current block to identify neighboring blocks whose reference pictures of the neighboring blocks are the same as those of the current block. The video decoding device applies motion shift using motion vectors of neighboring blocks whose reference pictures of the neighboring blocks are the same as those of the current block among the N neighboring blocks.
[0154] If there are multiple neighboring blocks among N neighboring blocks whose reference pictures are the same as those of the current block, the video decoding device applies motion shift using the motion vector of one of the multiple neighboring blocks using the parsed index. If there is only one neighboring block among N neighboring blocks whose reference pictures are the same as those of the current block, the index may not be parsed. The video decoding device generates merge candidates for the current block using the motion vectors of the subblocks generated using SbTMVP.
[0155] A video decoding device generates merge candidates for the current block using an affine model. The video decoding device generates a merge candidate for a subblock using at least one of an inherited affine merge candidate, a combined affine merge candidate, a history-based affine merge candidate, and a pair-wise affine merge candidate. Here, a history-based affine merge candidate is a merge candidate generated using neighboring blocks adjacent to the current block and a history parameter table in which affine parameters are stored. A pair-wise affine merge candidate is a merge candidate generated using a new CPMV generated by averaging two CPMVs.
[0156] The video decoding device generates merge candidates for the current block using at least one of SbTMVP and an affine model to generate a merge candidate list for the current block. The video decoding device can generate K merge candidates based on SbTMVP and L merge candidates based on the affine model, and generate a merge candidate list for the current block using the K+L merge candidates.
[0157] A video decoding device can generate a final merge candidate list for the current block using M merge candidates with low template costs from the merge candidate list for the current block. The template cost is calculated using templates adjacent to the subblock and templates of corresponding subblocks in the reference picture. The video decoding device determines the motion vectors of the subblocks using the final merge candidate list for the current block. The video decoding device generates inter-prediction blocks for the subblocks using the motion vectors of the subblocks.
[0158] The video decoding device determines the intra prediction mode of the current block (S1320). The intra prediction mode of the current block may be implicitly determined as a specific mode. For example, the intra prediction mode of the current block may be determined as a planar mode. The video decoding device may determine the intra prediction mode of the current block using a histogram of gradients. The video decoding device calculates the gradients of the reference pixels by applying a Sobel filter to reference pixels around the current block, and generates a histogram of gradients using the calculated gradients. The video decoding device selects N gradients in descending order of gradients from the histogram of gradients, and maps the N gradients to N intra prediction modes, respectively. The video decoding device determines the intra prediction mode of the current block using the N intra prediction modes. The video decoding device generates an intra prediction block of the current block using the intra prediction mode of the current block.
[0159] An image decoding device can determine an intra prediction mode of a current block using a template-based intra prediction mode determination method. The image decoding device applies the directionality of each candidate mode in the MPM (Most Probable Mode) list of the current block to the true pixels of the template of the current block to generate a prediction template. The image decoding device calculates the SATD or SAD between the pixels of the prediction template and the pixels of the restored template. The image decoding device can determine the mode with the smallest SATD or SAD among the candidate modes in the MPM list as the intra prediction mode of the current block. The image decoding device generates an intra prediction block of the current block using the intra prediction mode of the current block.
[0160] In another embodiment, the video decoding device applies the directionality of M predefined intra prediction modes among N intra prediction modes of the current block to the true pixels of the template of the current block to generate a prediction template. For example, the N intra prediction modes may be modes including wide-angle intra prediction modes in 67 intra prediction modes. The video decoding device calculates the SATD or SAD between the pixels of the prediction template and the pixels of the restored template. The video decoding device may determine the mode with the smallest SATD or SAD among the M intra prediction modes as the intra prediction mode of the current block. The video decoding device generates an intra prediction block of the current block using the intra prediction mode of the current block. Here, step S1320 may be performed first, and then step S1310 may be performed.
[0161] The video decoding device calculates weights used for weighting the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block (S1330). The video decoding device calculates the weights using whether intra-prediction is applied to blocks adjacent to the current block and whether inter-prediction is applied to blocks adjacent to the current block. The video decoding device generates a prediction block of the current block (S1340). The video decoding device generates a prediction block of the current block by weighting the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block. The video decoding device determines a weight function using the determined positions of the motion vectors of the sub-blocks in SbTMVP and generates a prediction block of the current block using the weight function.
[0162] FIG. 14 is a diagram illustrating a method for calculating template costs of sub-blocks according to one embodiment of the present disclosure.
[0163] Referring to FIG. 14, the video decoding device generates merge candidates for the current block using at least one of SbTMVP and an affine model to generate a merge candidate list for the current block. The video decoding device can generate a final merge candidate list for the current block using M merge candidates with low template costs in the merge candidate list for the current block. The video decoding device calculates SATD or SAD between templates of the current sub-blocks and templates of corresponding sub-blocks in reference pictures. Here, the reference pictures are pictures determined based on candidate modes in the merge candidate list for the current block. The video decoding device can generate a final merge candidate list for the current block using M merge candidates in descending order of SATD or SAD in the merge candidate list for the sub-block.
[0164] FIG. 15 is a diagram for explaining a method for determining a weight value used to weight the inter-prediction blocks of sub-blocks and the intra-prediction block of the current block in CIIP mode according to one embodiment of the present disclosure.
[0165] Referring to FIG. 15, weights used to weight-sum the inter-prediction blocks of sub-blocks and the intra-prediction blocks of the current block can be determined using whether intra-prediction is applied to blocks within an A region adjacent to the top of the current block and whether inter-prediction is applied to blocks within an A region adjacent to the top of the current block. Weights used to weight-sum the inter-prediction blocks of sub-blocks and the intra-prediction blocks of the current block can be determined using whether intra-prediction is applied to blocks within an L region adjacent to the left of the current block and whether inter-prediction is applied to blocks within an L region adjacent to the left of the current block.
[0166] For example, if the number of blocks encoded by intra prediction within region A and region L is the same as the number of blocks encoded by inter prediction within region A and region L, the image decoding device can perform weighted sum by applying the same weight value to the intra prediction blocks of the current block and the inter prediction blocks of the sub-blocks. For example, if the number of blocks encoded by intra prediction within region A and region L is greater than the number of blocks encoded by inter prediction within region A and region L, the image decoding device can perform weighted sum by applying a higher weight value to the intra prediction blocks of the current block than to the inter prediction blocks of the sub-blocks. For example, if the number of blocks encoded by intra prediction within region A and region L is less than the number of blocks encoded by inter prediction within region A and region L, the image decoding device can perform weighted sum by applying a higher weight value to the inter prediction blocks of the sub-blocks than to the intra prediction blocks of the current block.
[0167] FIG. 16A and FIG. 16B are diagrams illustrating a method for dividing a current block into multiple regions and weighting values for each region, according to one embodiment of the present disclosure. The weighting values for each region are weighting values assigned to the intra-prediction signal of each region within the current block and the inter-prediction signal of each region within the current block.
[0168] Referring to FIG. 16a, if the intra prediction mode of the current block is a directional prediction mode and is close to the vertical direction, the image decoding device can divide the current block horizontally into a first region and a second region. If the intra prediction mode of the current block is a directional prediction mode and is close to the horizontal direction, the image decoding device can divide the current block vertically into a first region and a second region. The number of regions to be divided may be a predefined number or determined based on the size of the current block.
[0169] Referring to FIG. 16b, the image decoding device can determine the weight value of the first region and the weight value of the second region. The weight value applied to the intra prediction signal of the first region is 2 p -q, and the weight value applied to the inter prediction signal of the first region may be q. The weight value applied to the intra prediction signal of the second region may be q, and the weight value applied to the inter prediction signal of the second region may be 2. p -q can be, where q is a predefined value and 2 p -q may be a value greater than q. The video decoding device may generate an intra prediction block of the current block by assigning weight values of each region to the intra prediction signal of the first region and the intra prediction signal of the second region. The video decoding device may generate inter prediction blocks of the sub-blocks by assigning weight values of each region to the inter prediction signal of the first region and the inter prediction signal of the second region.
[0170] FIG. 17a and FIG. 17b are diagrams for explaining a method of dividing a current block into multiple regions and a weight value of each region according to another embodiment of the present disclosure.
[0171] Referring to FIG. 17a, if the intra prediction mode of the current block is a directional prediction mode and is a mode close to the vertical direction, the image decoding device can divide the current block horizontally into a first region, a second region, a third region, and a fourth region. If the intra prediction mode of the current block is a directional prediction mode and is a mode close to the horizontal direction, the image decoding device can divide the current block vertically into a first region, a second region, a third region, and a fourth region. The number of regions to be divided may be a predefined number or determined based on the size of the current block.
[0172] Referring to FIG. 17b, the image decoding device can determine a weight value of the first region, a weight value of the second region, a weight value of the third region, and a weight value of the fourth region. The weight value applied to the intra prediction signal of the first region is 2 p -q0, and the weight applied to the inter prediction signal of the first region may be q0. The weight applied to the intra prediction signal of the second region may be 2. p -q1, and the weight applied to the inter prediction signal of the second region may be q1. The weight applied to the intra prediction signal of the third region may be q1, and the weight applied to the inter prediction signal of the third region may be 2. p -q1 can be. The weight applied to the intra prediction signal in the 4th region is q0, and the weight applied to the inter prediction signal in the 4th region is 2. p -q0 can be. Here, q0 and q1 are predefined values, and q1 can be a value greater than q0. 2 p -q0 is a value greater than q0, and 2 p -q1 can be a value greater than q1.
[0173] The video decoding device can generate an intra prediction block of the current block by assigning weight values to the intra prediction signal of the first region, the intra prediction signal of the second region, the intra prediction signal of the third region, and the intra prediction signal of the fourth region, respectively. The video decoding device can generate inter prediction blocks of the sub-blocks by assigning weight values to the inter prediction signal of the first region, the inter prediction signal of the second region, the inter prediction signal of the third region, and the inter prediction signal of the fourth region, respectively.
[0174] FIG. 18A and FIG. 18B are diagrams for explaining a weighting function of a sub-block according to an embodiment of the present disclosure. When an image decoding device determines a motion vector based on the center position of a corresponding sub-block using SbTMVP, the image decoding device can reduce the proportion of the weighted value of the inter-prediction signal as the distance from the center of the sub-block increases using the weighting function s(i, j). For example, the weighting function can be applied only when the motion vectors of the sub-blocks are determined by the affine mode.
[0175] Referring to Fig. 18a, when the size of a subblock is 4x4, the image decoding device can reduce the proportion of the weight value of the inter-prediction signal as it gets farther from the center of the subblock by using a weight function s(i, j). Here, the weight function may be a distance function based on the distance L-infinity, L1, L2, etc. from the center of the subblock. L1 may be a Chebyshev distance, and L2 may be a Euclidean distance. The weight value applied to the inter-prediction signal of 4 a regions within the subblock may be different from the weight value applied to the inter-prediction signal of 12 b regions within the subblock by the weight function.
[0176] Referring to FIG. 18b, when the size of a subblock is 8x8, the image decoding device can reduce the proportion of the weight value of the inter-prediction signal as it gets farther from the center of the subblock by using a weight function s(i, j). Here, the weight function may be a distance function based on the distance L-infinity, L1, L2, etc. from the center of the subblock. L1 may be a Chebyshev distance, and L2 may be a Euclidean distance. The weight values applied to the inter-prediction signals of 4 a regions within the subblock, the weight values applied to the inter-prediction signals of 12 b regions within the subblock, the weight values applied to the inter-prediction signals of 20 c regions within the subblock, and the weight values of the inter-prediction blocks of 28 d regions within the subblock may be different by the weight function.
[0177] FIG. 19 is a diagram for explaining a weight value of a subblock when the motion vector of the corresponding subblock in a reference picture is a zero motion vector, according to one embodiment of the present disclosure.
[0178] Referring to FIG. 19, when the motion vector of the corresponding subblock is a zero motion vector, when the motion vector used for inter prediction of the corresponding subblock is a zero motion vector, and when inter prediction is not applied to the corresponding subblock, the motion vector of the corresponding subblock is a zero motion vector. When the video decoding device generates motion vectors of subblocks using SbTMVP and the motion vector of the corresponding subblock is a zero motion vector, the video decoding device determines the weight value of the subblock by using whether inter prediction is applied to the corresponding subblock.
[0179] For example, if a video decoding device applies motion shift to generate a motion vector of a subblock using SbTMVP and inter prediction is not applied to a corresponding subblock, so that the motion vector of the corresponding subblock is a zero motion vector, the video decoding device can apply a low weight value to the inter prediction blocks of the subblocks.
[0180] A method for weighting the inter-prediction blocks of sub-blocks and the intra-prediction blocks of the current block.
[0181] The inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block can be weighted using the weight values determined in FIG. 15 and the weight functions determined in FIGS. 18a and 18b. In another embodiment, the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block can be weighted using the weight values determined in FIG. 16b or the weight values determined in FIG. 17b and the weight functions determined in FIGS. 18a and 18b. The prediction block of the current block, P CIIP can be expressed as in mathematical formula 3.
[0182]
[0183] Here, W inter is a weight applied to the inter-prediction blocks of the sub-blocks, s(i, j) is a weight function, and P inter are inter-prediction blocks of sub-blocks, and W intra is the weight applied to the intra prediction block of the current block, and P intra is the intra prediction block of the current block. Here, p is log2(W inter +W intra ) is calculated.
[0184] Restoration of residual blocks
[0185] The residual block is restored by performing an inverse transform on the transform coefficients of the inverse quantized residual block. Here, if the residual block on which the inverse transform is performed has been predicted in units of subblocks in the CIIP, the inverse transform kernel can be determined using the kernel determination process of the inter prediction mode regardless of the intra prediction mode. If the residual block on which the inverse transform is performed has been predicted in units of subblocks in the CIIP, the inverse transform kernel can be determined based on the intra prediction mode determined in the process of performing prediction in units of subblocks in the CIIP. If the residual block on which the inverse transform is performed has been predicted in units of subblocks in the CIIP, a gradient value is calculated for the prediction block of the current block generated in the process of performing prediction in units of subblocks in the CIIP, and the inverse transform kernel can be determined using the intra prediction mode of the current block determined based on the gradient value.
[0186] FIG. 20 is a flowchart illustrating a video decoding process according to one embodiment of the present disclosure.
[0187] Referring to FIG. 20, the image decoding device determines an inter-prediction mode of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block within the reference picture of the current block (S2010). The process of determining the inter-prediction mode of the current block may include a process of generating candidate modes of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block, a process of calculating variable values between templates of the sub-blocks and templates of blocks corresponding to the sub-blocks within the reference pictures, a process of generating a candidate list of the current block by sorting the candidate modes based on the variable values, and a process of determining the inter-prediction mode of the current block using the candidate list of the current block. The reference pictures may be pictures determined based on the candidate modes of the current block.
[0188] The video decoding device determines an intra prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block (S2020). The process of determining the intra prediction mode of the current block may include a process of calculating a gradient of the reference pixels by applying a predetermined filter to the reference pixels of the current block, a process of generating a histogram by using the gradient of the reference pixels, and a process of determining the intra prediction mode of the current block by using the histogram. The process of determining the intra prediction mode of the current block may include a process of generating prediction pixels by applying predefined intra prediction modes of the current block to reference pixels of the template of the current block, a process of calculating first variable values between the prediction pixels and reconstructed pixels of the template of the current block, and a process of determining the intra prediction mode of the current block based on the first variable values.
[0189] An image decoding device generates inter-prediction blocks of sub-blocks using an inter-prediction mode of a current block, and generates an intra-prediction block of the current block using an intra-prediction mode of the current block (S2030). The image decoding device weights the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block to generate a prediction block of the current block (S2040). Weights used to weight-sum the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be determined based on whether intra-prediction is applied to blocks within an area adjacent to the top of the current block and whether inter-prediction is applied to blocks within an area adjacent to the left of the current block. The weights used to weight-sum the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be predefined values. A weight function used to weight-sum the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be determined based on a distance from the center of the sub-blocks.
[0190] FIG. 21 is a flowchart illustrating a video encoding process according to one embodiment of the present disclosure.
[0191] Referring to FIG. 21, the video encoding device determines an inter-prediction mode of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block within the reference picture of the current block (S2110). The process of determining the inter-prediction mode of the current block may include a process of generating candidate modes of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block, a process of calculating variable values between templates of the sub-blocks and templates of blocks corresponding to the sub-blocks within the reference pictures, a process of generating a candidate list of the current block by sorting the candidate modes based on the variable values, and a process of determining the inter-prediction mode of the current block using the candidate list of the current block. The reference pictures may be pictures determined based on the candidate modes of the current block.
[0192] An image encoding device determines an intra prediction mode of a current block based on at least one of reference pixels of the current block and a template of the current block (S2120). The process of determining the intra prediction mode of the current block may include a process of calculating a gradient of the reference pixels by applying a predetermined filter to the reference pixels of the current block, a process of generating a histogram by using the gradient of the reference pixels, and a process of determining the intra prediction mode of the current block by using the histogram. The process of determining the intra prediction mode of the current block may include a process of generating prediction pixels by applying predefined intra prediction modes of the current block to reference pixels of the template of the current block, a process of calculating first variable values between the prediction pixels and reconstructed pixels of the template of the current block, and a process of determining the intra prediction mode of the current block based on the first variable values.
[0193] An image encoding device generates inter-prediction blocks of sub-blocks using an inter-prediction mode of a current block, and generates an intra-prediction block of the current block using an intra-prediction mode of the current block (S2130). The image encoding device weights and adds the inter-prediction blocks of the sub-blocks and the intra-prediction blocks of the current block to generate a prediction block of the current block (S2140). Weight values used to weight-add the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be determined based on whether intra-prediction is applied to blocks in an area adjacent to the top of the current block and whether inter-prediction is applied to blocks in an area adjacent to the left of the current block. The weight values used to weight-add the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be predefined values. A weight function used to weight-add the inter-prediction blocks of the sub-blocks and the intra-prediction block of the current block may be determined based on the distance from the center of the sub-blocks.
[0194] Although the flowchart / timing diagram of this specification describes each process as being executed sequentially, this is merely an illustrative description of the technical idea of one embodiment of the present disclosure. In other words, a person of ordinary skill in the art to which one embodiment of the present disclosure belongs may modify and apply various modifications and variations by changing the order described in the flowchart / timing diagram without departing from the essential characteristics of one embodiment of the present disclosure, or by executing one or more of the processes in parallel. Therefore, the flowchart / timing diagram is not limited to a chronological order.
[0195] It should be understood that the exemplary embodiments described above can be implemented in many different ways. The functions or methods described in one or more examples can be implemented in hardware, software, firmware, or any combination thereof. It should be understood that the functional components described herein are labeled as "units" to further emphasize their implementation independence.
[0196] Meanwhile, the various functions or methods described in this embodiment may also be implemented as instructions stored on a non-transitory storage medium that can be read and executed by one or more processors. Non-transitory storage media include, for example, all types of storage devices that store data in a form readable by a computer system. For example, non-transitory storage media include storage media such as erasable programmable read-only memory (EPROM), flash drives, optical drives, magnetic hard drives, and solid-state drives (SSDs).
[0197] The above description is merely an example of the technical idea of the present embodiment, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present embodiment. Therefore, the present embodiments are not intended to limit the technical idea of the present embodiment, but rather to explain it, and the scope of the technical idea of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment should be interpreted by the claims below, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of rights of the present embodiment.
[0198]
[0199] CROSS-REFERENCE TO RELATED APPLICATION
[0200] This patent application claims priority to Korean patent application No. 10-2023-0060325, filed in Korea on May 10, 2023, and Korean patent application No. 10-2024-0060640, filed in Korea on May 8, 2024, the entire contents of which are incorporated herein by reference.
Claims
1. A process for determining an inter-prediction mode of a current block based on at least one of motion vectors of first sub-blocks in a reference picture of the current block and motion vectors of control points of the current block, wherein the first sub-blocks are blocks corresponding to sub-blocks of the current block; A process of determining an intra prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block; A process of generating inter-prediction blocks of the sub-blocks using the inter-prediction mode of the current block and generating an intra-prediction block of the current block using the intra-prediction mode of the current block; and A video decoding method, comprising a process of generating a prediction block of the current block by weighting the inter-prediction blocks of the above sub-blocks and the intra-prediction block of the current block.
2. In paragraph 1, The process of determining the inter-prediction mode of the current block is as follows: A process of generating candidate modes of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block; A process of calculating variable values between templates of the above sub-blocks and templates of blocks corresponding to the above sub-blocks within reference pictures, the reference pictures being pictures determined based on candidate modes of the current block; A process of generating a candidate list of the current block by sorting the candidate modes based on the above variable values; and A video decoding method, comprising a process of determining an inter-prediction mode of the current block using a candidate list of the current block.
3. In paragraph 1, The process of determining the intra prediction mode of the current block is as follows: A process of calculating the gradient of reference pixels by applying a predetermined filter to the reference pixels of the current block; A process of generating a histogram using the slope of the above reference pixels; and A video decoding method, comprising a process of determining an intra prediction mode of the current block using the histogram.
4. In paragraph 1, The process of determining the intra prediction mode of the current block is as follows: A process of generating prediction pixels by applying the predefined intra prediction modes of the current block to the false pixels of the template of the current block; A process of calculating first variable values between the above predicted pixels and the restored pixels of the template of the current block; and A video decoding method, comprising a process of determining an intra prediction mode of the current block based on the first variable values.
5. In paragraph 1, A video decoding method, wherein the weighting values used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the current block are determined based on whether intra-prediction is applied to blocks in an area adjacent to the top of the current block and whether inter-prediction is applied to blocks in an area adjacent to the left of the current block.
6. In paragraph 1, A video decoding method, wherein the weighting values used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the above current block are predefined values.
7. In paragraph 1, A video decoding method, wherein a weighting function used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the current block is determined based on the distance from the center of the above sub-blocks.
8. A process for determining an inter-prediction mode of the current block based on at least one of motion vectors of first sub-blocks in a reference picture of the current block and motion vectors of control points of the current block, wherein the first sub-blocks are blocks corresponding to sub-blocks of the current block; A process of determining an intra prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block; A process of generating inter-prediction blocks of the sub-blocks using the inter-prediction mode of the current block and generating an intra-prediction block of the current block using the intra-prediction mode of the current block; and A video encoding method, comprising a process of generating a prediction block of the current block by weighting the inter-prediction blocks of the above sub-blocks and the intra-prediction block of the current block.
9. In paragraph 8, The process of determining the inter-prediction mode of the current block is as follows: A process of generating candidate modes of the current block based on at least one of the motion vectors of the first sub-blocks and the control point motion vectors of the current block; A process of calculating variable values between templates of the above sub-blocks and templates of blocks corresponding to the above sub-blocks within reference pictures, the reference pictures being pictures determined based on candidate modes of the current block; A process of generating a candidate list of the current block by sorting the candidate modes based on the above variable values; and A video encoding method, comprising a process of determining an inter-prediction mode of the current block using a candidate list of the current block.
10. In paragraph 8, The process of determining the intra prediction mode of the current block is as follows: A process of calculating the gradient of reference pixels by applying a predetermined filter to the reference pixels of the current block; A process of generating a histogram using the slope of the above reference pixels; and A video encoding method, comprising a process of determining an intra prediction mode of the current block using the histogram.
11. In paragraph 8, The process of determining the intra prediction mode of the current block is as follows: A process of generating prediction pixels by applying the predefined intra prediction modes of the current block to the false pixels of the template of the current block; A process of calculating first variable values between the above predicted pixels and the restored pixels of the template of the current block; and A video encoding method, comprising a process of determining an intra prediction mode of the current block based on the first variable values.
12. In paragraph 8, A video encoding method, wherein the weighting values used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the current block are determined based on whether intra-prediction is applied to blocks in an area adjacent to the top of the current block and whether inter-prediction is applied to blocks in an area adjacent to the left of the current block.
13. In paragraph 8, A video encoding method, wherein the weighting values used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the above current block are predefined values.
14. In paragraph 8, A video encoding method, wherein a weighting function used to weight the inter-prediction blocks of the above sub-blocks and the intra-prediction blocks of the current block is determined based on the distance from the center of the above sub-blocks.
15. A method for transmitting a bitstream related to video data, said method comprising: A process of generating a bitstream by encoding the above video data; and Including a process of transmitting the above bitstream to a video decoding device, The process of generating the above bitstream is: A process of determining an inter-prediction mode of a current block based on at least one of motion vectors of first sub-blocks within a reference picture of the current block and motion vectors of control points of the current block, wherein the first sub-blocks are blocks corresponding to sub-blocks of the current block; A process of determining an intra prediction mode of the current block based on at least one of reference pixels of the current block and a template of the current block; A process of generating inter-prediction blocks of the sub-blocks using the inter-prediction mode of the current block and generating an intra-prediction block of the current block using the intra-prediction mode of the current block; and A method comprising a process of generating a prediction block of the current block by weighting the inter-prediction blocks of the above sub-blocks and the intra-prediction block of the current block.
Citation Information
Patent Citations
Fusion protein comprising il-2 protein and CD80 protein fragment or mutein thereof and use thereof
KR1020210117208A
α-Alumina Flakes
KR102409322B1
Infants and young children development analysis system using an online observation journal that records the development history of infants and young children
KR102459894B1
Method and apparatus for image encoding, and method and apparatus for image decoding
US20220201299A1
Method and apparatus of combined intra-inter prediction using matrix-based intra prediction
US20220256141A1
