Image encoding / decoding method and device

Abstract

The present disclosure relates to an image encoding / decoding method and device, and an image decoding method according to an embodiment of the present disclosure may comprise the steps of: determining prediction mode condition information of the current block; deriving prediction mode type information of the current block on the basis of the prediction mode condition information; generating a prediction block of the current block on the basis of the prediction mode type information; determining the integrity of the prediction block; and outputting a result of determining the integrity, wherein the prediction mode condition information is determined on the basis of a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

Description

Video encoding / decoding method and device

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more specifically, to an image encoding / decoding method and apparatus with an improved prediction mode type information determination algorithm.

[0002] Recently, the demand for multimedia data, such as video, has been increasing rapidly. In particular, the demand for high-resolution, high-quality video, such as HD (High Definition) and UHD (Ultra High Definition), is growing across various application fields. High-resolution, high-quality video data involves a much larger volume compared to conventional video data. Consequently, the transmission and storage costs for storing and / or transmitting such high-resolution, high-quality video data increase compared to conventional video data.

[0003] To solve these problems, high-efficiency video encoding / decoding technology for videos with higher resolution and quality is required.

[0004] To encode images, various techniques are used, such as intra-prediction techniques that predict pixel values ​​within the current picture using pixel information within the current picture, intra-prediction techniques that predict pixel values ​​from previous or subsequent pictures, transformation and quantization techniques to compress the energy of residual signals—the difference between the predicted signal and the original signal—and entropy coding techniques that assign short codes to values ​​with high frequency and long codes to values ​​with low frequency. Furthermore, to improve image encoding efficiency, various tools are being developed to implement each of these techniques. Additionally, to decode the encoded image, the image can be restored and reproduced through image decoding techniques that utilize technologies and tools corresponding to the image encoding techniques.

[0005] By utilizing these video encoding and video decoding technologies, video data can be effectively compressed, transmitted, stored, and played back.

[0006] The present disclosure aims to provide an image encoding / decoding method and apparatus that improves the inefficiency of a prediction mode type information determination algorithm and enhances the compression efficiency of image data.

[0007] In addition, the present disclosure aims to provide an image encoding / decoding method and apparatus that improve the compression efficiency of image data by improving the algorithm of the encoding / decoding process.

[0008] The technical problems to be solved by the present disclosure are not limited to those mentioned above. In addition, other technical problems not mentioned in the present disclosure will be clearly understood by those skilled in the art from the present disclosure.

[0009] A video decoding method according to one embodiment of the present disclosure comprises the steps of determining prediction mode condition information of a current block, deriving prediction mode type information of the current block based on the prediction mode condition information, generating a prediction block of the current block based on the prediction mode type information, determining the integrity of the prediction block, and outputting the integrity determination result, wherein the prediction mode condition information may be determined based on a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

[0010] In the above image decoding method, the integrity of the prediction block can be determined based on the prediction block of the neighbor block of the current block.

[0011] In the above image decoding method, the integrity of the prediction block can be determined based on the movement information of the neighbor blocks of the current block.

[0012] In the above image decoding method, the integrity of the prediction block can be determined based on the prediction block of the call block included in the reference picture corresponding to the position of the current block.

[0013] In the above image decoding method, the integrity of the prediction block can be determined based on the movement information of the call block included in the reference picture corresponding to the position of the current block.

[0014] A video decoding method according to one embodiment of the present disclosure comprises: a step of determining prediction mode condition information of a current coding tree block based on at least one of a first condition set, a second condition set, and a third condition set; a step of deriving prediction mode type information for sub-blocks divided from the current coding tree block based on the prediction mode condition information; and a step of performing a prediction for the sub-blocks of the current coding tree block based on the prediction mode type information. The prediction mode condition information is determined by judging in the order of the first condition set, the second condition set, and the third condition set. When the conditions of the first condition set and the conditions of the second condition set are all false, the conditions of the third condition set are judged. When one or more of the conditions of the third condition set are true, the prediction mode condition information is determined based on the current slice type. When the conditions included in the first condition set, the conditions included in the second condition set, and the conditions included in the third condition set are all false, the prediction mode condition information may be set to a first value.

[0015] In the above image decoding method, if at least one of the conditions of the first condition set is true, the prediction mode condition information can be set to the first value.

[0016] In the above image decoding method, if all conditions of the first condition set are false and at least one condition of the second condition set is true, the prediction mode condition information can be determined as the second value.

[0017] In the above image decoding method, if the conditions of the first condition set and the conditions of the second condition set are both false and one or more of the conditions of the third condition set are true, the prediction mode condition information can be determined based on whether the current slice type is slice I.

[0018] In the above image decoding method, when the current slice type is slice I, the prediction mode condition information is set to the second value, and when the current slice type is not slice I, the prediction mode condition information can be set to the third value.

[0019] In the above image decoding method, the first condition set may include a first condition satisfied when the type of the current slice is an I slice and each coding tree block included in the current slice is implicitly quadrupled into 64x64 luma sample coding blocks, a second condition satisfied when the current coding tree block can use all prediction modes, a third condition satisfied when the chrominance component format of the current coding tree block is monochrome, and a fourth condition satisfied when the chrominance component format of the current coding tree block is a 4:4:4 format.

[0020] In the above image decoding method, the second condition set may include a fifth condition satisfied when the area of ​​the current coding tree block is 64 and the current coding tree block is divided into four parts, a sixth condition satisfied when the area of ​​the current coding tree block is 64 and the current coding tree block is divided into three parts, and a seventh condition satisfied when the area of ​​the current coding tree block is 32 and the current coding tree block is divided into two parts.

[0021] In the above image decoding method, the third condition set may include an eighth condition satisfied when the area of ​​the current coding tree block is 64, the current coding tree block is divided horizontally or vertically, and the chrominance component format of the current coding tree block is a 4:2:0 format; a ninth condition satisfied when the area of ​​the current coding tree block is 128, the current coding tree block is divided horizontally or vertically, and the chrominance component format of the current coding tree block is a 4:2:0 format; a tenth condition satisfied when the width of the current coding tree block is 8, and the current coding tree block is divided horizontally; and an eleventh condition satisfied when the height of the current coding tree block is 16, and the current coding tree block is divided vertically.

[0022] In the above image decoding method, if the value of the prediction mode condition information is the first value (e.g., 0), the prediction mode type information can be set to be the same as the mode type information of the current coding tree.

[0023] In the above image decoding method, if the value of the prediction mode condition information is the second value (e.g., 1), the prediction mode type information can be determined as a first mode type using one of intra prediction, IBC, and palette mode.

[0024] In the above video decoding method, when the value of the prediction mode condition information is the third value (e.g., 2), the prediction mode type information can be determined based on the availability information of the signaling inter-prediction mode.

[0025] In the above image decoding method, if the availability information of the inter prediction mode indicates that inter prediction cannot be used, the prediction mode type information is determined as a first mode type, and if the availability information of the inter prediction mode indicates that inter prediction can be used, the prediction mode type information can be determined as a second mode type that uses the inter prediction mode.

[0026] In the above-described image decoding method, when the current block is in an affine model-based inter-prediction mode, wherein the current block is a sub-block of any one of the sub-blocks divided from the coding tree block, the method further comprises: a step of deriving an affine motion vector difference of the current block; a step of determining the motion vector difference resolution based on an AMVR application flag and an AMVR resolution index; and a step of adjusting the resolution of the affine motion vector difference based on the determined motion vector difference resolution, wherein the affine motion vector difference adjusting the resolution may be a first prediction direction affine motion vector difference and a second prediction direction affine motion vector difference for all motion vector components and all control point indicators.

[0027] In the above image decoding method, when the current block is an inter-prediction mode according to a geometric partitioning mode, where the current block is a sub-block of any one of the sub-blocks partitioned from the coding tree block, the motion vector, reference picture information, prediction direction information, bidirectional prediction index information, and interpolation filter information of the current block can be stored in units of 4 x 4 blocks.

[0028] A video encoding method according to one embodiment of the present disclosure comprises: a step of determining prediction mode condition information of a current coding tree block based on at least one of a first condition set, a second condition set, and a third condition set; a step of deriving prediction mode type information for sub-blocks divided from the current coding tree block based on the prediction mode condition information; and a step of performing a prediction for the sub-blocks of the current coding tree block based on the prediction mode type information. The prediction mode condition information is determined by judging in the order of the first condition set, the second condition set, and the third condition set. When the conditions of the first condition set and the conditions of the second condition set are all false, the conditions of the third condition set are judged. When one or more of the conditions of the third condition set are true, the prediction mode condition information is determined based on the current slice type. When the conditions included in the first condition set, the conditions included in the second condition set, and the conditions included in the third condition set are all false, the prediction mode condition information may be set to a first value.

[0029] A method for transmitting a bitstream generated by an image encoding method according to one embodiment of the present disclosure includes the step of transmitting the bitstream, and the step of determining prediction mode condition information of a current coding tree block based on at least one condition set among a first condition set, a second condition set, and a third condition set; A bitstream generated by a video encoding method can be transmitted, comprising the steps of: deriving prediction mode type information for sub-blocks divided from the current coding tree block based on the prediction mode condition information; and performing a prediction for the sub-blocks of the current coding tree block based on the prediction mode type information, wherein the prediction mode condition information is determined by judging in the order of the first condition set, the second condition set, and the third condition set; when the conditions of the first condition set and the conditions of the second condition set are all false, the conditions of the third condition set are judged; when one or more of the conditions of the third condition set are true, the prediction mode condition information is determined based on the current slice type; and when the conditions included in the first condition set, the conditions included in the second condition set, and the conditions included in the third condition set are all false, the prediction mode condition information is set to a first value.

[0030] According to the present disclosure, an image encoding / decoding method and apparatus can be provided that improve the inefficiency of a prediction mode type information determination algorithm and improve the compression efficiency of image data.

[0031] In addition, according to the present disclosure, an image encoding / decoding method and apparatus can be provided that improve the compression efficiency of image data by improving the algorithm of the encoding / decoding process.

[0032] Additionally, according to the present disclosure, a recording medium storing a bitstream generated by the image encoding method or device of the present disclosure may be provided.

[0033] The effects obtainable from the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which the present disclosure pertains from the description below.

[0034] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present disclosure.

[0035] FIG. 2 is a block diagram showing an image decoding device according to one embodiment of the present disclosure.

[0036] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0037] FIG. 4 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0038] FIG. 5 is a flowchart illustrating an image decoding method related to determining context information of syntax elements for an adaptive loop filter according to one embodiment of the present disclosure.

[0039] FIG. 6 is a flowchart illustrating an image decoding method related to determining prediction mode condition information according to one embodiment of the present disclosure.

[0040] FIG. 7 is a diagram illustrating a signaling mechanism of block division-related syntax elements according to one embodiment of the present disclosure.

[0041] FIG. 8 is a flowchart illustrating an image decoding method related to determining context information of syntax elements regarding multi-type tree partitioning according to one embodiment of the present disclosure.

[0042] FIG. 9 is a parameter model for determining motion vectors in an affine model-based inter prediction according to one embodiment of the present disclosure.

[0043] FIG. 10 is a flowchart illustrating an image decoding method related to resolution adjustment of an affine motion vector difference according to one embodiment of the present disclosure.

[0044] FIG. 11 is a drawing illustrating a partition structure of a current block based on an index indicating a geometric partition mode according to one embodiment of the present disclosure.

[0045] FIG. 12 is a drawing illustrating the current block predicted by a geometric division mode according to one embodiment of the present disclosure.

[0046] FIG. 13 is a flowchart illustrating an image decoding method related to storing motion information of a geometric partition mode block according to one embodiment of the present disclosure.

[0047] FIG. 14 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure.

[0048] The present disclosure is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present disclosure to specific embodiments, and it should be understood that it includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the present disclosure. Similar reference numerals have been used for similar components in the description of each drawing.

[0049] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present disclosure, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0050] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0051] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the disclosure. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0052] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings. Hereinafter, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0053] In the description below, "image" may refer to a video or a picture that constitutes a video. The terms image, picture, and frame may be used interchangeably or have the same meaning.

[0054] An image can be encoded by an image encoding device, and the image encoded by the image encoding device can be decoded by an image decoding device.

[0055] A video encoding device may also be referred to as an encoder or an encoder. And, a video decoding device may also be referred to as a decoder or a decoder.

[0056] In the description below, the target block may refer to the block being encoded and / or the block being decoded. For example, the target block may be the current block, which is the block currently being encoded or decoded. The terms target block representation and current block may be used interchangeably and may have the same meaning.

[0057] In the description below, the terms block and unit may be used interchangeably. Alternatively, the term unit may refer to a unit comprising a luminance component block and a corresponding chrominance component block. For example, a coding tree unit may include a luminance component coding tree block and two corresponding chrominance component coding tree blocks.

[0058] In the description below, a sample may be a unit that constitutes a block. Also, the terms sample, pixel, and pixel may be used interchangeably and have the same meaning.

[0059] FIG. 1 is a block diagram showing an image encoding device according to one embodiment of the present disclosure.

[0060] Referring to FIG. 1, the image encoding device (100) may include an image segmentation unit (101), an intra prediction unit (102), an inter prediction unit (103), a subtraction unit (104), a conversion unit (105), a quantization unit (106), an entropy encoding unit (107), an inverse quantization unit (108), an inverse conversion unit (109), an addition unit (110), a filter unit (111), and a memory (112).

[0061] Each component shown in FIG. 1 is depicted independently to represent different characteristic functions of the image encoding device and does not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for convenience of explanation, but at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform functions, and such integrated and separated embodiments of each component are included within the scope of the present disclosure as long as they do not deviate from the essence of the present disclosure.

[0062] Additionally, some components may not be essential components performing an essential function in the present disclosure, but may be optional components merely for enhancing performance. The present disclosure may be implemented by including only the components essential to embody the essence of the present disclosure, excluding components used merely for enhancing performance, and a structure including only the essential components, excluding optional components used merely for enhancing performance, is also included within the scope of the rights of the present disclosure.

[0063] The image segmentation unit (101) can divide the input image into at least one block. At this time, the input image may have various shapes and sizes, such as a sequence, picture, slice, tile, segment, tile group, or coding tree block. According to another embodiment, the image segmentation unit (101) may divide an input picture into a plurality of sub-pictures defined as a group of rectangular slices, divide each sub-picture into the tile / slice, and divide the tile / slice into coding tree blocks.

[0064] Additionally, the image segmentation unit (101) can recursively segment the segmented coding tree block. The terminal nodes segmented from the coding tree block may be referred to as coding units (CU). A block may refer to a coding unit (CU), or a prediction unit (PU) or a transformation unit (TU) segmented from the coding unit (CU). The segmentation may be performed based on at least one of a quadtree, a binary tree, and a ternary tree. A quadtree is a method of dividing an upper block into lower blocks, each having a width and height that are half that of the upper block. A binary tree is a method of dividing an upper block into lower blocks, each having either a width or a height that is half that of the upper block. A ternary tree is a method of dividing an upper block into three lower blocks. For example, the three lower blocks may be obtained by dividing the width or height of the upper block in a ratio of 1:2:1. Through the aforementioned binary tree-based partitioning, blocks can have shapes that are not only square but also non-square. Blocks can first be partitioned into a quad tree. Blocks corresponding to the leaf nodes of the quad tree may not be partitioned, or may be partitioned into a binary tree or a terminal tree. The leaf nodes of the binary tree or terminal tree may be units of encoding, prediction, and / or transformation.

[0065] The image segmentation unit (101) can recursively segment the CTU into a quad tree (QT) as well as a multi-type tree (MTT). Here, the MTT can be composed of a binary tree (BT) and a triple tree (TT). For example, the MTT structure can be divided into a vertical binary tree segmentation mode (SPLIT_BT_VER), a horizontal binary tree segmentation mode (SPLIT_BT_HOR), a vertical binary tree segmentation mode (SPLIT_TT_VER), and a horizontal triple segmentation mode (SPLIT_TT_HOR).

[0066] Additionally, the image segmentation unit (101) can segment the CTU by applying a dual tree in which the CTU segmentation structures of the luminance and color difference components are used differently, or by applying a single tree in which the luminance and color difference CTBs (Coding Tree Blocks) within the CTU share a coding tree structure.

[0067] The prediction unit (102, 103) may include an intra prediction unit (102) that performs intra prediction and an inter prediction unit (103) that performs inter prediction. The prediction unit (102, 103) may determine whether to use intra prediction or perform inter prediction for a prediction unit. Additionally, the prediction unit (102, 103) may determine specific information (e.g., intra prediction mode, inter prediction mode, motion vector, reference picture, etc.) according to the determined prediction method. At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific details are determined may be different. For example, the prediction unit (102, 103) may determine the prediction method and prediction mode, etc. for each prediction unit and perform prediction according to the transformation unit.

[0068] According to another embodiment, the prediction unit may encode an input image using a third mode (e.g., IBC mode, Palette mode, etc.) that is a mode other than the intra mode and the inter mode. However, if the third mode has functional characteristics similar to the intra mode or the inter mode, the third mode may be classified as the intra mode or the inter mode. In this disclosure, the third mode will be described only when a specific description of the third mode is required.

[0069] The intra prediction unit (102) can generate a prediction block of the current block based on the intra prediction mode of the current block and reference pixel information around the current block, which is pixel information within the current picture. If the surrounding blocks of the current block are predicted by inter prediction, the reference pixels included in the inter-predicted surrounding blocks can be replaced with reference pixels in other surrounding blocks that are intra-predicted. That is, if a reference pixel is not available, the intra prediction unit (102) can perform intra prediction of the current block by replacing the unavailable reference pixel with at least one of the available reference pixels.

[0070] The intra prediction unit (102) may use multiple reference pixel lines for intra prediction of the current block. When multiple reference pixel lines are available, information indicating the reference pixel line used for intra prediction among the multiple reference pixel lines may be signaled.

[0071] The intra prediction mode used for intra prediction may include a directional prediction mode that uses reference pixel information according to the prediction direction and a non-directional mode that does not use directional information. Additionally, the mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra prediction mode information of the luminance component block or the predicted luminance signal information may be utilized to predict chrominance information.

[0072] The intra prediction unit (102) may include a reference sample filter, an interpolation filter, and a DC filter. The reference sample filter is a filter that performs filtering on the reference pixel of the current block and may be adaptively applied depending on the prediction mode, size, shape, and / or whether the reference pixel is included in the reference pixel line immediately adjacent to the current block. If the prediction mode of the current block is a mode that does not perform reference sample filtering, the reference sample filter may not be applied.

[0073] The interpolation filter is a filter that interpolates and filters the prediction samples of the current block, and can be applied adaptively depending on the prediction mode, size, shape, and / or whether the reference pixel is included in the reference pixel line immediately adjacent to the current block.

[0074] If the prediction mode of the current block is DC mode, a prediction block can be generated by applying a DC filter.

[0075] The inter prediction unit (103) generates a prediction block using the previously restored reference image stored in memory (112), the inter prediction mode, and motion information. Here, inter prediction may mean motion prediction or motion compensation.

[0076] The inter prediction unit (103) can set the inter mode of the coding block to one of Skip Mode, Merge Mode, or Advanced Motion Vector Prediction (AMVP) Mode in order to perform motion prediction and / or motion compensation. In addition, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the current block according to the set mode.

[0077] Additionally, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the prediction block by applying the affine mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction based on the inter prediction mode. Additionally, the inter prediction unit (103) can perform motion prediction and / or motion compensation for the prediction block by applying HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), DMVR (Decoder side Motion Vector Refinement), BDOF (Bi-Directional Optical-Flow), PROF (Prediction Refinement With Optical Flow), BCW (Bi-predictive with CU Weights), etc., to improve the performance of each mode.

[0078] Here, AFFINE mode can be used in both AMVP and MERGE modes. It is a technique with high encoding efficiency. AFFINE mode can be a prediction mode using a 4-parameter affine motion model using two control point motion vectors (CPMV) and a 6-parameter affine motion model using three control point motion vectors. Here, CPMV can be a vector representing any one of the affine motion models of the top-left, top-right, or bottom-left of the current block.

[0079] Motion information may include, for example, motion vectors, reference picture indices, List 1 prediction flags, List 0 prediction flags, half-sample interpolation filter indices, bidirectional prediction weight indices, etc.

[0080] A residual block containing residual value information, which is the difference between the prediction unit generated in the prediction unit (102, 103) and the original block of the prediction unit, can be generated. The generated residual block can be input to the conversion unit (130) and converted.

[0081] The subtraction unit (104) subtracts the prediction block generated by the intra prediction unit (102) or the inter prediction unit (103) from the block currently to be encoded to generate a residual block of the current block. The residual value (residual block) between the generated prediction block and the original block can be input to the conversion unit (105).

[0082] In addition, the prediction mode information and motion vector information used for prediction can be encoded in the entropy encoding unit (107) along with the residual value and transmitted to the decoder. When a specific encoding mode is used, it is also possible to encode the original block as is and transmit it to the decoder without generating a prediction block through the prediction unit (102, 103).

[0083] The transformation unit (105) can perform a transformation on a residual block containing residual data to generate and output a transformation coefficient. Here, the transformation coefficient may be a coefficient value generated by performing a transformation on the residual block. When a transform skip mode is applied, the transformation unit (105) may skip the transformation on the residual block.

[0084] The conversion unit (105) can determine a conversion type and a conversion kernel based on at least one of encoding parameters such as the size of the conversion block, color component, and prediction mode, and can perform a conversion on the conversion block using the determined conversion type and conversion kernel.

[0085] According to one embodiment, the transformation unit (105) can perform a transformation on the 4x4 luminance residual block generated from the intra prediction result using a transformation type and transformation kernel according to the Discrete Sine Transform (DST), and for the remaining residual block, perform a transformation using a transformation type and transformation kernel according to the Discrete Cosine Transform (DCT).

[0086] According to another embodiment, the transformation unit (105) may apply MTS (Multiple Transform Selection) technology, which performs transformations by selectively using various transformation types and transformation kernels. That is, the transformation unit (105) may perform transformations on a sub-block basis through SBT (Sub-block Transform) technology. Specifically, SBT may be applied only to inter-predicted blocks, and the current block may be divided into ½ or ¼ sizes in the vertical or horizontal direction, and transformation may be performed on only one of the blocks. For example, the transformation unit (105) may perform transformation on the leftmost or rightmost block among the current blocks divided vertically, and perform transformation on the topmost or bottommost block among the current blocks divided horizontally.

[0087] According to another embodiment, the conversion unit (105) may apply a Low Frequency Non-Separable Transform (LFNST), which is a technique for applying a secondary transform to a residual signal converted into the frequency domain through a DCT or DST. LFNST can additionally perform a transformation on a 4x4 or 8x8 low-frequency region in the upper left to concentrate the residual coefficients to the upper left.

[0088] The quantization unit (106) can quantize the conversion coefficient or residual signal converted into the frequency domain by the conversion unit (105) according to a quantization parameter (QP). The quantization parameter may vary depending on the block or the importance of the image. The value calculated by the quantization unit (106) may be provided to the inverse quantization unit (108) and the entropy encoding unit (107).

[0089] The above-mentioned conversion unit (105) and / or quantization unit (106) may be optionally included in the image encoding device (100). That is, the image encoding device (100) may perform at least one of conversion or quantization on the residual data of the residual block, or may encode the residual block by skipping both conversion and quantization. Even if neither conversion nor quantization is performed in the image encoding device (100), or if neither conversion nor quantization is performed, the block that enters as input to the entropy encoding unit (107) is typically referred to as a conversion block.

[0090] The entropy encoding unit (107) can generate and output a bitstream by performing entropy encoding according to a probability distribution on values ​​output by the quantization unit (106), coding parameter values ​​output during the encoding process, information for decoding an image, etc. Here, the information for decoding an image may include syntax elements, etc.

[0091] Coding parameters may include information (flags, indexes, etc.) that is encoded in the encoding device (100) and signaled to the decoding device (200), such as syntax elements, as well as information derived during the encoding process or decoding process, and may refer to information required when encoding or decoding images.

[0092] The entropy encoding unit (107) can encode various information such as coefficient information of a conversion block, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information. The coefficients of the conversion block can be encoded in units of sub-blocks within the conversion block.

[0093] For encoding the coefficients of a transform block, various syntax elements may be encoded, such as Last_sig, a syntax element indicating the location of the first non-zero coefficient in backscan order; Coded_sub_blk_flag, a flag indicating whether there is at least one non-zero coefficient in the subblock; Sig_coeff_flag, a flag indicating whether it is a non-zero coefficient; Abs_greater1_flag, a flag indicating whether the absolute value of the coefficient is greater than 1; Abs_greater2_flag, a flag indicating whether the absolute value of the coefficient is greater than 2; and Sign_flag, a flag indicating the sign of the coefficient. The remaining values ​​of the coefficients that are not encoded by the above syntax elements alone may be encoded through the syntax element remaining_coeff.

[0094] When entropy encoding is applied, the size of the bit sequence for the symbols to be encoded can be reduced by allocating a small number of bits to symbols with a high probability of occurrence and a large number of bits to symbols with a low probability of occurrence when representing symbols. Input data is entropied. For example, entropy encoding can utilize various encoding methods such as Exponential Golomb and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0095] The inverse quantization unit (108) and the inverse transformation unit (109) can inverse quantize the values ​​quantized in the quantization unit (106) and inverse transform the values ​​transformed in the transformation unit (105). The residual value generated in the inverse quantization unit (108) and the inverse transformation unit (109) can be combined with the prediction unit predicted through the motion estimation unit, motion compensation unit, and intra prediction unit (102) included in the prediction unit (102, 103) to generate a reconstructed block. The addition unit (110) generates a reconstructed block by adding the prediction block generated in the prediction unit (102, 103) and the residual block generated through the inverse transformation unit (109).

[0096] The filter section (111) can apply deblocking filter, Sample Adaptive Offset (SAO), Adaptive Loop Filter (ALF), LMCS (Luma Mapping with Chroma Scaling), etc., to the restored sample, restored block, or restored image as a whole or part of the filtering technique.

[0097] The deblocking filter can remove block distortion caused by boundaries between blocks in the restored picture. To determine whether to perform deblocking, the decision to apply the deblocking filter to the current block can be made based on the pixels contained in a certain number of columns or rows within the block. When applying the deblocking filter to a block, a Strong Filter or a Weak Filter can be applied depending on the required deblocking filtering strength. Additionally, when applying the deblocking filter, horizontal and vertical filtering can be processed in parallel.

[0098] Sample adaptive offset may be a method of correcting the offset from the original image on a sample-by-sample basis for an image that has undergone deblocking. The filter unit (111) may use a method of dividing the samples included in the image into a certain number of regions, determining the region to be offset and applying the offset to that region, or a method of applying the offset by considering the edge information of each sample.

[0099] Adaptive Loop Filtering (ALF) can be performed based on a comparison between the filtered restored image and the original image. After dividing the pixels included in the image into predetermined groups, a single filter to be applied to each group can be determined, allowing for differential filtering for each group. Information regarding whether to apply ALF can be transmitted per coding unit (CU), and the shape and filter coefficients of the ALF filter to be applied may vary depending on each block. Additionally, an ALF filter of the same form (fixed form) can be applied regardless of the characteristics of the block to be applied.

[0100] An adaptive loop filter can perform filtering based on a comparison between the reconstructed image and the original image. After dividing the samples included in the image into predetermined groups, a filter to be applied to each group can be determined, thereby performing differential filtering for each group. Information regarding whether to apply the adaptive loop filter can be signaled for each coding block, and the shape and filter coefficients of the adaptive loop filter to be applied may vary depending on each block.

[0101] The memory (112) can store a restored block or picture calculated through the filter unit (111). The memory (112) may include a reference picture buffer. Additionally, the stored restored block or picture in the memory (112) may be provided to the prediction unit (102, 103) when performing inter-prediction.

[0102] Next, an image decoding device according to one embodiment of the present disclosure will be described with reference to the drawings.

[0103] FIG. 2 is a block diagram showing an image decoding device (200) according to one embodiment of the present disclosure.

[0104] Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (201), an inverse quantization unit (202), an inverse transformation unit (203), a prediction unit (204, 205), an adder unit (206), a filter unit (207), and a memory (208).

[0105] The video decoding device (200) can receive a bitstream output by the video encoding device (100). The video decoding device (200) can receive a bitstream stored in a computer-readable recording medium or receive a bitstream stream streamed through a wired / wireless transmission medium. The video decoding device (200) can decode the bitstream to generate a restored video or a decoded video, and can output the restored video or the decoded video.

[0106] The entropy decoding unit (201) can generate symbols by performing entropy decoding according to the probability distribution of the bitstream. The generated symbols may include symbols in the form of quantized levels. Here, the entropy decoding method may be the inverse process of the entropy encoding method described above.

[0107] The entropy decoding unit (201) can convert a one-dimensional vector-shaped coefficient into a two-dimensional block-shaped coefficient through a conversion coefficient scanning method to decode a conversion coefficient level (quantized level).

[0108] The entropy decoding unit (201) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit (107) of the image encoding device (100) performed. For example, various methods such as Exponential Golomb and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the image encoder.

[0109] When a syntax element is decoded based on a context model, the entropy decoding unit (201) obtains a bin corresponding to the syntax element from the bitstream and can determine a context model using the syntax element and decoding information of the block to be decoded or surrounding blocks, or information of the symbol / bin decoded in the previous step. Then, the entropy decoding unit (201) can derive the value of the syntax element by predicting the occurrence probability of the received bin according to the determined context model and performing arithmetic decoding of the bin. Subsequently, based on the determined context model, the context model of the bin to be decoded thereafter can be updated.

[0110] Additionally, for example, when a syntax element is bypass decoded, the entropy decoding unit (201) can obtain a bin corresponding to the syntax element through a bitstream and decode the input bin by applying a uniform probability distribution.

[0111] Context models can be assigned and updated for each context-coded (normally coded) bean. A context model can be identified based on a context index (ctxIdx) or a context index increment (ctxInc). The context index can be derived based on the context index increment. For example, information indicating the context model for each normally coded bean can be derived based on the sum of context index offsets (ctxIdxOffset). For instance, context index increment information can be derived differently for each bean. Additionally, the context index offset can be derived from the lowest value of the context index. The context index offset is generally a value used to distinguish context models for different syntax elements, and a context model for a single syntax element can be distinguished or derived based on context index increment information.

[0112] The entropy decoding unit (201) can obtain various information such as coefficient information of a conversion block as described above, block type information, prediction mode information, division unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information by decoding.

[0113] The entropy decoding unit (201) can decode information related to the coding tree block. The entropy decoding unit (201) can derive information indicating whether splitting is allowed. Here, the information indicating whether splitting is allowed may include allowSplitQt, which is information indicating whether quadtree splitting is allowed; allowSplitBtVer, which is information indicating whether vertical bipartition is allowed; allowSplitBtHor, which is information indicating whether horizontal bipartition is allowed; allowSplitTtVer, which is information indicating whether vertical tripartition is allowed; and allowSplitTtHor, which is information indicating whether horizontal tripartition is allowed. The information indicating whether splitting is allowed may be determined based on the width, height, position, tree type information (treeType), splitting mode information, and mode type information (modeType) of the current coding tree block.

[0114] The entropy decoding unit (201) can decode the syntax elements related to the division of the coding block based on information indicating whether division is allowed.

[0115] For example, syntax elements related to the splitting of a coding block may include split_cu_flag indicating whether the coding block is split, split_qt_flag indicating whether the coding block is split into a quadtree, mtt_split_cu_vertical_flag indicating whether the coding block being split into a multi-type tree is split vertically, and mtt_split_cu_binary_flag indicating whether the coding block being split into a multi-type tree is bipartitized.

[0116] Here, the coding tree block can be partitioned according to a multi-type tree structure, which is a partitioning structure using a quadruple partition structure and / or one of a bipartite structure and a ternary partition structure.

[0117] First, a coding tree block can be quadrupled based on the value of split_cu_flag. The quadrupled coding block can be further quadrupled according to the value of split_qt_flag. A coding block that is not split in a quadtree can be referred to as a quadtree leaf node.

[0118] Coding blocks, which are quadtree leaf nodes, can be split according to the multi-type tree structure. That is, the split type of a coding block, which is a quadtree leaf node, can be determined as a bipartition structure or a ternary structure depending on the values ​​of mtt_split_cu_vertical_flag, which indicates whether to split the multi-type tree vertically, and mtt_split_cu_binary_flag, which indicates whether to bipartition the multi-type tree.

[0119] The binary splitting structure may include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) may be a structure that splits the current coding block into two equal parts in the vertical direction. On the other hand, horizontal binary splitting (SPLIT_BT_HOR) may be a structure that splits the current coding block into two equal parts in the horizontal direction.

[0120] The ternary splitting structure may include vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). Vertical ternary splitting (SPLIT_TT_VER) may be a structure that splits the current coding block vertically in a 1:2:1 ratio. On the other hand, horizontal ternary splitting (SPLIT_TT_HOR) may be a structure that splits the current coding block horizontally in a 1:2:1 ratio.

[0121] The multi-type tree splitting structure can be determined as shown in Table 1 below based on a combination of values ​​for mtt_split_cu_vertical_flag, which indicates whether to split the coding block in the vertical direction, and mtt_split_cu_binary_flag, which indicates whether to split the coding block that is split in the multi-type tree.

[0122]

[0123] The inverse quantization unit (202) generates a conversion block by performing inverse quantization on the quantized conversion block. It operates substantially the same as the inverse quantization unit (108) of FIG. 1.

[0124] The inverse transformation unit (203) performs an inverse transformation on the transformation block to generate a residual block. At this time, the transformation method may be determined based on information regarding the prediction method (inter or intra prediction), the size and / or shape of the block, the intra prediction mode, etc. It operates substantially the same as the inverse transformation unit (109) of FIG. 1.

[0125] According to one embodiment, the inverse transformation unit (203) can perform an inverse transformation using a transformation type and a transformation kernel according to the Discrete Sine Transform (DST) on the transformation coefficient level of the 4x4 luminance component generated from the intra prediction result, and perform an inverse transformation using a transformation type and a transformation kernel according to the Discrete Cosine Transform (DCT) on the remaining transformation coefficient level.

[0126] According to another embodiment, the inverse transformation unit (203) may apply MTS (Multiple Transform Selection) technology, which performs transformations by selectively using multiple transformation kernels.

[0127] According to another embodiment, the inverse transform unit (203) may apply LFNST (Low Frequency Non-Separable Transform), which is a technique for applying a secondary inverse transform to the inverse transform coefficient level through a DCT or DST-based transform type and a transform kernel.

[0128] The prediction unit (204, 205) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (201) and the previously decoded block or picture information provided by the memory (208).

[0129] The prediction unit (204, 205) may include an intra prediction unit (204) and an inter prediction unit (205). The prediction unit (204, 205) receives various information, such as prediction unit information input from the entropy decoding unit (201), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit from the current encoding unit, and determines the prediction mode of the prediction unit.

[0130] The intra prediction unit (204) can generate a prediction block of the current block based on the intra prediction mode of the current block and reference pixel information around the current block, which is pixel information within the current picture.

[0131] The intra prediction unit (204) can generate a prediction block based on reference pixel information around the current block, which is pixel information within the current picture. The intra prediction unit (204) can use multiple reference pixel lines for intra prediction. If multiple reference pixel lines are available, the intra prediction unit (204) can obtain information indicating a reference pixel line used for intra prediction among the multiple reference pixel lines.

[0132] The intra prediction mode used for intra prediction may be a directional prediction mode or a non-directional mode. Additionally, the mode for predicting luminance information and the mode for predicting chrominance information may be different, and the intra prediction mode information of the luminance component block or the predicted luminance signal information may be utilized to predict chrominance information.

[0133] The intra prediction unit (204) operates substantially the same as the intra prediction unit (102) of FIG. 1.

[0134] The inter prediction unit (205) can perform an inter prediction for the current prediction unit based on information included in at least one of the previous or subsequent pictures of the current picture containing the current prediction unit, using information necessary for inter prediction of the current prediction unit provided by the video encoding device (100). Alternatively, the inter prediction may be performed based on information of a partially restored area within the current picture containing the current prediction unit.

[0135] The inter prediction unit (205) can set the inter mode of the coding block to one of Skip Mode, Merge Mode, or Advanced Motion Vector Prediction (AMVP) Mode in order to perform motion prediction and / or motion compensation. Additionally, the inter prediction unit (205) can perform motion compensation for the prediction block according to the set mode.

[0136] Additionally, the inter prediction unit (205) can perform motion compensation for the prediction block by applying the AFFINE mode of sub-PU-based prediction, the SbTMVP (Subblock-based Temporal Motion Vector Prediction) mode, and the MMVD (Merge with MVD) mode and GPM (Geometric Partitioning Mode) mode of PU-based prediction based on the inter prediction mode. Additionally, the inter prediction unit (205) can perform motion compensation for the prediction block by applying HMVP (History based MVP), PAMVP (Pairwise Average MVP), CIIP (Combined Intra / Inter Prediction), AMVR (Adaptive Motion Vector Resolution), BDOF (Bi-Directional Optical-Flow), BCW (Bi-predictive with CU Weights), etc., to improve the performance of each mode.

[0137] Motion information may include, for example, motion vectors, reference picture indices, List 1 prediction flags, List 0 prediction flags, half-sample interpolation filter indices, bidirectional prediction weight indices, etc.

[0138] The inter prediction unit (205) can operate substantially the same as the inter prediction unit (103) of FIG. 1.

[0139] The adder (206) generates a restoration block by adding the prediction block generated in the intra prediction unit (204) or the inter prediction unit (205) and the residual block generated through the inverse transformation unit (203). It operates substantially the same as the adder (110) of FIG. 1.

[0140] The filter section (207) can reduce various types of noise occurring in the restored blocks. The filter section (207) may include a deblocking filter, a sample adaptive offset, an adaptive loop filter (ALF), and an LMCS.

[0141] The filter unit (207) may receive information regarding whether each filter is applied, information regarding the filter strength, etc. from the image encoding device (100). The filter unit (207) of the image decoding device (200) may receive filter-related information provided by the image encoding device (100) and perform filtering on the corresponding block in the image decoding device (200).

[0142] The filter section (207) can operate substantially the same as the filter section (111) of FIG. 1.

[0143] The memory (208) can store a restoration block generated by the adder (206). For example, the memory (208) may include a reference picture buffer. The memory (208) may operate substantially the same as the memory (112) of FIG. 1.

[0144]

[0145] FIG. 3 is a schematic diagram illustrating a video coding system to which the present disclosure can be applied.

[0146] A video coding system according to one embodiment may include an encoding device (10) and a decoding device (20). The encoding device (10) may transmit encoded video and / or image information or data to the decoding device (20) via a digital storage medium or network in the form of a file or streaming.

[0147] An encoding device (10) according to one embodiment may include an image generation unit (11), an encoding unit (12), and a transmission unit (13). A decoding device (20) according to one embodiment may include a receiving unit (21), a decoding unit (22), and an image playback unit (23). The encoding unit (12) may be called a video / image encoding unit, and the decoding unit (22) may be called a video / image decoding unit. The transmission unit (13) may be included in the encoding unit (12). The receiving unit (21) may be included in the decoding unit (22). The image playback unit (23) may include a display unit, and the display unit may be composed of a separate device or an external component.

[0148] The image generation unit (11) can acquire video / image through a process of capturing, synthesizing, or generating video / image. The image generation unit (11) may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured video / image, etc. The video / image generation device may include, for example, a computer, a tablet, and a smartphone, etc., and can generate video / image (electronically). For example, a virtual video / image may be generated through a computer, etc., in which case the video / image capture process may be replaced by a process of generating related data.

[0149] The encoding unit (12) can encode the input video / image. The encoding unit (12) can perform a series of procedures such as prediction, conversion, and quantization for compression and encoding efficiency. The encoding unit (12) can output the encoded data (encoded video / image information) in the form of a bitstream. The detailed configuration of the encoding unit (12) can be configured in the same way as the encoding device (100) of FIG. 1 described above.

[0150] The transmission unit (13) can transmit encoded video / image information or data output in the form of a bitstream to the receiving unit (21) of the decoding device (20) via a digital storage medium or network in the form of a file or streaming. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmission unit (13) may include elements for creating a media file through a predetermined file format and elements for transmission via a broadcasting / communication network. The receiving unit (21) can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit (22).

[0151] The decoding unit (22) can decode a video / image by performing a series of procedures such as inverse quantization, inverse transformation, and prediction corresponding to the operation of the encoding unit (12). The detailed configuration of the decoding unit (22) can be configured in the same way as the decoding device (200) of FIG. 2 described above.

[0152] The image playback unit (23) can render the decoded video / image. The rendered video / image can be displayed through the display unit.

[0153]

[0154] FIG. 4 is a drawing illustrating an exemplary content streaming system to which an embodiment according to the present disclosure can be applied.

[0155] As illustrated in FIG. 4, a content streaming system to which an embodiment of the present disclosure is applied may largely include a multimedia input device, a media storage, an encoding server, a streaming server, a web server, and a user device.

[0156] The above encoding server plays the role of compressing content input from multimedia input devices, such as smartphones, cameras, and CCTVs, into digital data to generate a bitstream and transmitting it to the above streaming server. Alternatively, the encoding server plays the role of compressing content previously stored in a media storage into digital data to generate a bitstream and transmitting it to the above streaming server.

[0157] As another example, when multimedia input devices such as smartphones, cameras, and CCTVs directly generate bitstreams, the encoding server may be omitted.

[0158] The bitstream may be generated by a video encoding method and / or video encoding device to which an embodiment of the present disclosure is applied, and the streaming server may temporarily or intransitively store the bitstream during the process of transmitting or receiving the bitstream.

[0159] The streaming server transmits multimedia data to a user device based on a user request through a web server, and the web server can act as a medium to inform the user of available services. When a user device requests a desired service from the web server, the web server forwards it to the streaming server, and the streaming server can transmit multimedia data to the user device. At this time, the content streaming system may include a separate control server, and in this case, the control server can perform the role of controlling commands and responses between each device within the content streaming system.

[0160] The streaming server can receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0161] Examples of the above user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, HMDs (head-mounted displays)), digital TVs, desktop computers, digital signage, etc.

[0162] Each server within the above-mentioned content streaming system can be operated as a distributed server, and in this case, data received from each server can be processed in a distributed manner.

[0163]

[0164] According to one embodiment of the present disclosure, context information of syntax for a current block can be derived by utilizing information of blocks adjacent to the current block. Then, a context model is determined based on the derived context information, and syntax for the block can be decoded.

[0165] For example, the syntax for the current block may be the syntax for an adaptive loop filter. Specifically, the syntax for an adaptive loop filter may include alf_ctb_flag indicating whether the adaptive loop filter is applied, alf_ctb_cc_cb_idc[ctbX][ctbY] indicating whether the cross-component filter is applied to the cb component, and alf_ctb_cc_cr_idc[ctbX][ctbY] indicating whether the cross-component filter is applied to the cr component.

[0166] Here, cIdx can indicate the color component of the current block. Also, ctbX can indicate the x-axis position of the coding tree block containing the current block within the frame, and ctbY can indicate the y-axis position of the coding tree block containing the current block within the frame. ctbX and ctbY can be derived as follows.

[0167]

[0168] And, the locations of blocks adjacent to the current block, (ctbAx, ctbAy) and (ctbLx, ctbLy), can be derived as follows.

[0169]

[0170]

[0171] Contextual information regarding syntax elements of an adaptive loop filter can be derived as follows.

[0172]

[0173] To derive context information, context information (ctxInc) can be derived by utilizing condL, a condition regarding the left neighbor block adjacent to the left of the current block; availableL, which indicates the availability of the left neighbor block; condA, a condition regarding the upper neighbor block adjacent to the upper side of the current block; and availableA, which indicates the availability of the upper neighbor block. Here, condL, availableL, condA, and availableA can be derived as shown in Table 2 below.

[0174]

[0175] That is, referring to Equation 3 and Table 2, in order to derive condition information condL regarding the left neighbor block, availability information availableL regarding the left neighbor block, condition information condA regarding the upper neighbor block, and availability information availableA regarding the upper neighbor block, the blocks adjacent to the current block must be determined. Then, based on the color component information (cIdx) and location information (ctbX, ctbY) of the current block, the neighbor blocks are identified, and based on the identified neighbor blocks, the condition and availability information of each neighbor block can be derived.

[0176] Meanwhile, context information of a syntax element according to one embodiment of the present disclosure can be derived without relying on the color component of the current block. Here, the syntax element may be any one of alf_ctb_flag, alf_ctb_cc_cb_idc, or alf_ctb_cc_cb_idc, and the context information may be ctxInc. According to the above embodiment, memory usage can be reduced by using less of a context model during the decoding process. In addition, encoding and decoding speeds can be improved by reducing the number of accesses during the decoding process.

[0177] Specifically, the neighbor blocks of the current block used to derive context information can be determined using values ​​that are not dependent on the color component (e.g., cIdx) value of the current block. For example, the neighbor blocks of the current block used to derive ctxInc of alf_ctb_flag of the current block can be neighbor blocks of the luminance component (i.e., the color component value is 0) regardless of the color component of the current block. That is, condL and condA used to derive alf_ctb_flag in Table 2 can be alf_ctb_flag[0][ctbLx][ctbLy] and alf_ctb_flag[0][ctbAx][ctbAy].

[0178] As another example, the color component value can be a representative value of 0, 1, or 2. And, neighboring blocks can be determined based on the representative value of the color component.

[0179] In addition, condition information regarding neighboring blocks (e.g., condL, condA) and availability information regarding neighboring blocks (e.g., availableL, availableA) can be derived using information about neighboring blocks determined based on color component values ​​that are not dependent on the color component cIdx value of the current block. To this end, a representative value of cIdx determined by the encoder can be implicitly or explicitly transmitted to the decoder.

[0180] Embodiments of the present disclosure may be applied even when the coding tree structure is independent of color components. Additionally, embodiments of the present disclosure may be applied even when the coding tree structure is not independent of color components.

[0181] An image decoding method related to the determination of context information of syntax elements regarding an adaptive loop filter may be as described below.

[0182]

[0183] FIG. 5 is a flowchart illustrating an image decoding method related to determining context information of syntax elements for an adaptive loop filter according to an embodiment of the present disclosure. The image decoding method of FIG. 5 can be performed by an image decoding device.

[0184] Referring to FIG. 5, the image decoder can derive availability information of blocks adjacent to the current block and encoding information related to an adaptive loop filter (S510).

[0185] The image decoder can derive context information of the encoding information related to the adaptive loop filter of the current block based on availability information of blocks adjacent to the current block and encoding information related to the adaptive loop filter (S520). Here, the blocks adjacent to the current block may be blocks of luminance components adjacent to the location of the current block.

[0186] Meanwhile, a block adjacent to the current block may be a block with a color component determined independently of the color component of the current block.

[0187] The video decoder can derive encoding information related to the adaptive loop filter of the current block based on the derived context information (S530).

[0188] Meanwhile, the steps described in FIG. 5 can be performed in the same way in a video encoding method. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 5. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0189]

[0190] According to one embodiment of the present disclosure, the prediction mode condition information of a coding tree block can be derived based on coding parameters including partition structure information of the coding tree block. Here, the prediction mode condition information of the coding tree block can be used to determine the prediction mode type information of the sub-blocks of the coding tree block.

[0191] Meanwhile, the prediction mode type information may indicate any one of the following mode types: a first mode type indicating the use of one of intra prediction, IBC, and palette modes; a second mode type indicating the use of inter prediction mode; and a third mode type indicating the use of all prediction modes.

[0192] For example, when the value of the prediction mode condition information is a first value (e.g., 0), the prediction mode type information can be set to be the same as the mode type information of the current coding tree (e.g., the input mode type information of the current coding tree syntax). That is, depending on the mode type information of the current coding tree, the prediction mode type information may have a value of any one of a first mode type indicating the use of one of intra prediction, IBC, and palette mode, a second mode type indicating the use of inter prediction mode, and a third mode type indicating the use of one of all prediction modes including intra prediction, IBC (intra block copy), palette mode, and inter prediction.

[0193] And, if the value of the prediction mode condition information is a second value (e.g., 1), the prediction mode type information can be determined as a first mode type using one of intra prediction, IBC, and palette mode.

[0194] And, when the value of the prediction mode condition information is a third value (e.g., 2), the prediction mode type information can be determined by considering the availability information of the signaled inter prediction mode. Here, if the availability information of the inter prediction mode indicates that inter prediction cannot be used, the prediction mode type information can be determined as a first mode type. Conversely, if the availability information of the inter prediction mode indicates that inter prediction can be used, the prediction mode type information can be determined as a second mode type that uses the inter prediction mode.

[0195]

[0196] To reiterate, the prediction mode condition information may be an inductive variable used to control the determination of the prediction mode type information at the current node. Depending on the value of the prediction mode condition information, intra-mode fixing, inter / intra selection signaling, or maintenance of the current mode type occurs. Here, intra-mode may refer to intra prediction, IBC, and pallet modes.

[0197] Meanwhile, when the prediction mode type information is determined as the first mode type, the tree type is converted to a lumina-only dual tree according to the coding tree syntax, which may indirectly affect the channel type (chType) and chroma-related constraints (e.g., Smallest Chroma Intra Prediction Unit, SCIPU permission determination) in subsequent processing. Here, the first mode type of the prediction mode type information indicates the use of intra-prediction, the second mode type indicates the use of inter-prediction, and the third mode type indicates that all prediction modes, including both intra-prediction and inter-prediction, can be used.

[0198] Specifically, if the prediction mode type information of the current node is determined to be the first mode type, the tree type is set to DUAL_TREE_LUMA, allowing entry into a lumas-only dual tree (even if treeTypeCurr was a single tree, it switches to a lumas-only dual tree starting from this node).

[0199] Conversely, if the prediction mode type information of the current node is determined to be the second mode type, the tree type (treeType) can maintain treeTypeCurr (simultaneous progression of luminance and chroma as a single tree).

[0200] Since tree type switching changes the call context of the processing channel (chType), the chType = CHROMA context is established at the time of the corresponding chroma subtree call after the luma tree processing, and at this time, the chroma minimum block size / alignment constraints (especially the minimum width, height, and odd / even alignment in 4:2:0) can be checked.

[0201] And, the determination of prediction mode type information based on the value of prediction mode condition information can be explained as follows.

[0202] When the prediction mode condition information is the first value (e.g., 0), the prediction mode type information may retain the mode type of the current coding tree inherited from the parent. In this case, the value already determined in the parent node among the first / second / third mode types is retained, and accordingly, one of the intra-system (including IBC·palette), inter-system, or ALL may be retained.

[0203] When the prediction mode condition information is the second value (e.g., 1), the prediction mode type information is set to the first mode type, and the node is forced to use intra-system prediction (standard intra / IBC / palette, etc.).

[0204] When the prediction mode condition information is a third value (e.g., 2), the prediction mode type information is determined as either the first mode type or the second mode type by considering the availability information of the inter prediction mode. That is, it is determined as either the intra system or the inter system depending on the availability information of the prediction mode.

[0205]

[0206] Meanwhile, prediction mode condition information can be derived by sequentially performing Tables 3 to 5 below.

[0207]

[0208] According to Table 3, if one or more of the sub-conditions are satisfied, the value of the prediction mode condition information (e.g., ModeTypeCondition) may be determined to be the first value (e.g., 0). Here, the first condition among the sub-conditions may mean that the type of the current slice is I and each coding tree block included in the slice is implicitly quadtree split into 64x64 luma sample coding blocks. The second condition may mean that the mode type of the current coding tree block is not MODE_TYPE_ALL. The third condition may mean that the chroma component format of the current coding block is monochrome (sps_chroma_format_idc=0). And, the fourth condition may mean that the chroma component format of the current coding block is 4:4:4 format (sps_chroma_format_idc=3).

[0209]

[0210] According to Table 4, if not all conditions defined in Table 3 are satisfied and one or more of the following sub-conditions are satisfied, the value of the prediction mode condition information may be determined as a second value (e.g., 1). Here, the first sub-condition may mean that the area of ​​the current block is 64 and the current block is divided into four. The second condition may mean that the area of ​​the current block is 64 and the current block is divided into three in the horizontal direction or three in the vertical direction. The third condition may mean that the area of ​​the current block is 32 and the current block is divided into two in the horizontal direction or two in the vertical direction.

[0211]

[0212] According to Table 5, if not all conditions defined in Tables 3 and 4 are satisfied, and one of the following sub-conditions is satisfied, the prediction mode condition information may be determined according to the type of the current slice. Here, the first sub-condition may mean that the area of ​​the current block is 64, the current block is bisected in the horizontal direction or bisected in the vertical direction, and the chroma component format of the current block is 4:2:0 format (sps_chroma_format_idc=1). The second condition may mean that the area of ​​the current block is 128, the current block is trisected in the horizontal direction or trisected in the vertical direction, and the chroma component format of the current block is 4:2:0 format (sps_chroma_format_idc=1). The third condition may mean that the width of the current block is 8, and the current block is bisected in the horizontal direction. And, the fourth condition may mean that the height of the current block is 16 and the current block is divided into three vertical sections.

[0213] And, according to Table 5, if one of the sub-conditions is satisfied and the type of the current slice is I slice, the value of the prediction mode condition information can be determined as a second value (e.g., 1). On the other hand, if one of the sub-conditions is satisfied and the type of the current slice is not I slice, the value of the prediction mode condition information can be determined as a third value (e.g., 2).

[0214] And, if the sub-conditions are not satisfied, or if two or more of the sub-conditions are satisfied, the value of the prediction mode condition information can be determined as a first value (e.g., 0) regardless of the type of the current slice.

[0215] That is, as defined in Table 5, the process of determining the value of the prediction mode condition information based on the type of the current slice may be used at a low frequency. This is because situations in which only one of the sub-conditions is satisfied occur at a low frequency. Therefore, the utility of the process of determining the value of the prediction mode condition information based on the type of the current slice may decrease. Consequently, coding efficiency may be reduced.

[0216] To improve encoding efficiency, if one or more of the sub-conditions are satisfied as in Table 6 instead of Table 5, prediction mode condition information can be determined according to the type of the current slice.

[0217]

[0218] Specifically, if not all conditions defined in Tables 3 and 4 are satisfied, and one or more of the sub-conditions defined in Table 6 are satisfied, the prediction mode condition information can be determined according to the type of the current slice.

[0219] Here, among the sub-conditions, the first condition may mean that the area of ​​the current block is 64, the current block is bisected horizontally or vertically, and the chroma component format of the current block is 4:2:0 format (sps_chroma_format_idc=1). The second condition may mean that the area of ​​the current block is 128, the current block is trisected horizontally or vertically, and the chroma component format of the current block is 4:2:0 format (sps_chroma_format_idc=1). The third condition may mean that the width of the current block is 8 and the current block is bisected horizontally. And, the fourth condition may mean that the height of the current block is 16 and the current block is trisected vertically.

[0220] And, if one or more of the sub-conditions defined in Table 6 are satisfied and the type of the current slice is an I slice, the value of the prediction mode condition information may be determined as a second value (e.g., 1). On the other hand, if one or more of the sub-conditions defined in Table 6 are satisfied and the type of the current slice is not an I slice, the value of the prediction mode condition information may be determined as a third value (e.g., 2).

[0221]

[0222] According to another embodiment of the present disclosure, if the conditions defined in Tables 3 and 4 are not satisfied and more than a predetermined number of conditions among the sub-conditions defined in Table 6 are satisfied, the prediction mode condition information may be determined according to the type of the current slice. Here, the predetermined number may be an integer of 2 or more.

[0223] According to another embodiment of the present disclosure, if the conditions defined in Tables 3 and 4 are not satisfied, and more than a number of conditions among the sub-conditions defined in Table 6 determined by the signaling information are satisfied, the prediction mode condition information may be determined according to the type of the current slice. Here, the signaling information may be signaled in units such as sequences, GOP picture slices, CTUs, etc.

[0224]

[0225] The method for determining prediction mode condition information defined in Table 5 and the method for determining prediction mode condition information defined in Table 6 may differ in the following situations.

[0226] If the current block is an 8x8 block, the multi-type splitting mode of the current block is a vertical bi-splitting mode, and the color difference component format of the current block is a 4:2:0 format, the current block can simultaneously satisfy the first and third conditions defined in Tables 5 and 6. In such a case, according to Table 5, if only one of the multiple conditions is satisfied, the value of the prediction mode condition information can be determined based on the slice type, so the prediction mode condition information of the current block can be determined as 0. On the other hand, according to Table 6, if one or more of the multiple conditions are satisfied, the value of the prediction mode condition information can be determined based on the slice type, so the value of the prediction mode condition information of the current block can be determined according to the slice type.

[0227] In addition, if the current block is a 16x8 block, the multi-type division mode of the current block is a vertical trisection mode, and the color difference component format of the current block is a 4:2:0 format, the current block can simultaneously satisfy the second and fourth conditions defined in Tables 5 and 6. In such a case, according to Table 5, if only one of the multiple conditions is satisfied, the value of the prediction mode condition information can be determined based on the slice type, so the value of the prediction mode condition information of the current block can be determined as 0. On the other hand, according to Table 6, if one or more of the multiple conditions are satisfied, the value of the prediction mode condition information can be determined based on the slice type, so the value of the prediction mode condition information of the current block can be determined according to the slice type.

[0228] That is, according to Table 6, the usability of the process of determining the value of the prediction mode condition information based on the type of the current slice can be increased.

[0229] Meanwhile, the current block in the descriptions of Tables 3 to 6 may be the current coding tree block.

[0230]

[0231] FIG. 6 is a flowchart illustrating an image decoding method related to determining prediction mode condition information according to one embodiment of the present disclosure. The image decoding method of FIG. 6 can be performed by an image decoding device.

[0232] Referring to FIG. 6, the image decoder can determine the prediction mode condition information of the current coding tree block (S610).

[0233] For example, the prediction mode condition information may be determined based on at least one of a first condition set, a second condition set, and a third condition set. When determining the prediction mode condition information, the condition sets may be determined in the order of the first condition set, the second condition set, and the third condition set.

[0234] Specifically, if at least one of the conditions of the first set of conditions is true, the prediction mode condition information can be set to a first value (e.g., 0).

[0235] And, if at least one of the conditions in the first condition set is not true and at least one of the conditions in the second condition set is true, the prediction mode condition information can be set to a second value (e.g., 1).

[0236] And, if at least one of the conditions in the first condition set is not true, at least one of the conditions in the second condition set is not true, and at least one of the conditions in the third condition set is true, the prediction mode condition information can be determined based on the slice type. Here, the prediction mode condition information can be determined based on whether the slice type is an I slice. For example, if the slice type is an I slice, the prediction mode condition information is set to a second value (for example, 1), and if the slice type is not an I slice, the prediction mode condition information can be set to a third value (for example, 2).

[0237] And, if at least one of the conditions in the first condition set is not true, at least one of the conditions in the second condition set is not true, and at least one of the conditions in the third condition set is not true, the prediction mode condition information can be set to the first value (e.g., 0).

[0238] Meanwhile, the above first condition set may include a first condition satisfied when the type of the current slice is I and each coding tree block included in the slice is implicitly quadrupled into 64x64 luma sample coding blocks, a second condition satisfied when the current coding tree block can use all prediction modes, a third condition satisfied when the color difference component format of the current coding tree block is monochrome, and a fourth condition satisfied when the color difference component format of the current coding tree block is a 4:4:4 format.

[0239] And, the second set of conditions may include a fifth condition satisfied when the area of ​​the current coding tree block is 64 and the current coding tree block is divided into four, a sixth condition satisfied when the area of ​​the current coding tree block is 64 and the current coding tree block is divided into three, and a seventh condition satisfied when the area of ​​the current coding tree block is 32 and the current coding tree block is divided into two.

[0240] And, the above third condition set may include an eighth condition satisfied when the area of ​​the current coding tree block is 64, the current coding tree block is divided horizontally or vertically, and the color difference component format of the current coding tree block is a 4:2:0 format; a ninth condition satisfied when the area of ​​the current coding tree block is 128, the current coding tree block is divided horizontally or vertically, and the color difference component format of the current coding tree block is a 4:2:0 format; a tenth condition satisfied when the width of the current coding tree block is 8, and the current coding tree block is divided horizontally; and an eleventh condition satisfied when the height of the current coding tree block is 16, and the current coding tree block is divided vertically.

[0241] And, the image decoder can determine prediction mode type information for blocks divided from the current coding tree block based on prediction mode condition information (S620).

[0242] Here, the prediction mode type information may indicate any one of a first mode type indicating the use of one of intra prediction, IBC, and palette modes, a second mode type indicating the use of an inter prediction mode, and a third mode type indicating the use of all prediction modes.

[0243] Specifically, when the value of the prediction mode condition information is a first value (e.g., 0), the prediction mode type information can be set to be the same as the mode type information of the current coding tree (e.g., the input mode type information of the current coding tree syntax). Here, the prediction mode type information may have a value of any one of the first mode type, the second mode type, and the third mode type that indicates the use of one of all prediction modes including intra prediction, IBC (intra block copy), palette mode, and inter prediction, depending on the mode type information of the current coding tree.

[0244] And, if the value of the prediction mode condition information is a second value (e.g., 1), the prediction mode type information can be determined as a first mode type using one of intra prediction, IBC, and palette mode.

[0245] And, when the value of the prediction mode condition information is a third value (e.g., 2), the prediction mode type information can be determined by considering the availability information of the signaled inter prediction mode. Here, if the availability information of the inter prediction mode indicates that inter prediction cannot be used, the prediction mode type information can be determined as a first mode type. Conversely, if the availability information of the inter prediction mode indicates that inter prediction can be used, the mode type information can be determined as a second mode type that uses the inter prediction mode.

[0246] The image decoding device can perform prediction of the sub-blocks of the current coding tree block based on the prediction mode type information (S630).

[0247] Meanwhile, the above-described image decoding method may further include, when the current block is in an affine model-based inter-prediction mode, where the current block is a sub-block of any one of the sub-blocks split from the coding tree block, a step of deriving an affine motion vector difference of the current block, a step of determining a motion vector difference resolution based on an AMVR application flag and an AMVR resolution index, and a step of adjusting the resolution of the affine motion vector difference based on the determined motion vector difference resolution.

[0248] In addition, the affine motion vector difference for adjusting the above resolution may be a first prediction direction affine motion vector difference and a second prediction direction affine motion vector difference for all motion vector components and all control point indicators. A specific explanation thereof will be provided later with reference to FIGS. 9 and 10.

[0249] Meanwhile, when the current block is in an inter-prediction mode according to a geometric division mode, where the current block is a sub-block of any one of the sub-blocks divided from the coding tree block, the motion vector, reference picture information, prediction direction information, bidirectional prediction index information, and interpolation filter information of the current block can be stored in units of 4 x 4 blocks. A detailed explanation of this will be provided later with reference to FIGS. 11 to 13.

[0250] Meanwhile, the steps described in FIG. 6 can be performed in the same way in a video encoding device. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 6. The bitstream can be stored in a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0251]

[0252] According to one embodiment of the present disclosure, the tree type of a sub-block can be determined based on the prediction mode type information of the sub-blocks being divided from the current coding tree block. Specifically, the tree type can be determined as a single tree or a dual tree depending on the value of the prediction mode condition information of the sub-block.

[0253] If the prediction mode type information of a sub-block indicates a first mode type that indicates that intra prediction, IBC, and palette modes are available, the tree type can be determined as a dual tree. In this case, the tree type of the sub-block of the luminance component may be a dual tree luma (DUAL_TREE_LUMA), and the tree type of the sub-block of the chrominance component may be a dual tree chroma (DUAL_TREE_CHROMA).

[0254] If the tree type is determined to be a dual tree, block division information for the sub-blocks of the luminance component can be derived. Then, based on the derived block division information, the block division structure of the sub-blocks of the luminance component can be determined. Separately, block division information for the sub-blocks of the color component can be derived. Then, based on the derived block division information, the block division structure of the sub-blocks of the color component can be determined.

[0255] In other words, if the tree type is a dual tree, block partitioning information can be derived based on color components. Therefore, the partitioning structure of the luminance component sub-block may differ from the partitioning structure of the chrominance component sub-block.

[0256] Here, block splitting information may include syntax elements related to splitting of coding tree blocks. The syntax elements related to splitting of coding tree blocks may be signaled as follows.

[0257]

[0258] FIG. 7 is a diagram illustrating a signaling mechanism of block division-related syntax elements according to one embodiment of the present disclosure.

[0259] Referring to FIG. 7, the initial coding tree block may be the root node of a quadtree. The coding tree block that is the root node may be split into four. Information (e.g., qt_split_flag) indicating whether to perform a split may be signaled for the current coding block (e.g., a coding tree unit / quadtree node (CTU / QT_node) or a coding unit node (CU_node)). For example, if qt_split_flag is a first value (e.g., "1"), the current coding block may be split into four. On the other hand, if qt_split_flag is a second value (e.g., "0"), the current coding block may not be split into four. The coding block that is not split into four may be referred to as a leaf node (QT_leaf_node) of the quadtree.

[0260] The coding blocks, which are leaf nodes of each quadtree, can be further subdivided into a multi-type tree structure. That is, the coding blocks, which are leaf nodes of the quadtree, can be coding unit nodes (MTT_CU_nodes) of the multi-type tree.

[0261] In a multi-type tree structure, the multi-type tree splitting direction flag of the current coding unit (e.g., mtt_split_cu_vertical_flag) can be signaled. For example, if the value of the multi-type tree splitting direction flag is 1, the current coding unit can be split vertically. On the other hand, if the value of the multi-type tree splitting direction flag is 0, the current coding unit can be split horizontally.

[0262] Afterwards, the multi-type tree split type flag of the current coding unit (e.g., mtt_split_cu_binary_flag) may be signaled. For example, if the value of the multi-type tree split type flag is 1, the current coding unit may be split into two. On the other hand, if the value of the multi-type tree split type flag is 0, the current coding unit may be split into three.

[0263] Here, in order to decode the multi-type tree splitting direction flag, context information can be derived as defined in Table 7 below. And, the multi-type tree splitting direction flag can be entropy decoded based on the derived context information.

[0264]

[0265]

[0266] According to the definition in Table 7, if the sum of the values ​​of the vertical splitting allowance information (e.g., allowSplitBtVer, allowSplitTtVer) is greater than the sum of the horizontal splitting allowance information (e.g., allowSplitBtHor, allowSplitTtHor, etc.), the context information ctxInc of the multi-type tree splitting direction flag (e.g., mtt_split_cu_vertical_flag) can be set to 4.

[0267] On the other hand, if the sum of the values ​​of the vertical splitting allowance information (e.g., allowSplitBtVer, allowSplitTtVer) is smaller than the sum of the horizontal splitting allowance information (e.g., allowSplitBtHor, allowSplitTtHor, etc.), the context information ctxInc of the multi-type tree splitting direction flag (e.g., mtt_split_cu_vertical_flag) can be set to 3.

[0268] In addition, context information of the multi-type tree splitting direction flag may be used to derive context information of the multi-type tree splitting type flag.

[0269] However, as defined in Table 7, the context information of the multi-type tree splitting direction flag can be derived independently of the color components of the current block.

[0270] According to the dual tree partitioning structure, block partitioning-related syntax elements can be obtained for each of the luminance and chrominance components. Nevertheless, the multi-type tree partitioning direction flags for each of the luminance and chrominance components can be decoded using the same context information regardless of color information. Consequently, encoding efficiency may be reduced.

[0271] To improve encoding efficiency, when the tree type is a dual tree, context information of the multi-type tree splitting direction flag can be derived for each of the luminance component and the chrominance component.

[0272] Specifically, when the tree type is a dual tree, block splitting permission information can be derived based on the color component. Furthermore, context information for a multi-type tree splitting direction flag can be derived based on the block splitting permission information. For example, context information for a multi-type tree splitting direction flag for the luminance component can be derived based on the block splitting permission information for the luminance component. Additionally, context information for a multi-type tree splitting direction flag for the chrominance component can be derived based on the block splitting permission information for the chrominance component. Therefore, context information for a multi-type tree splitting direction flag suitable for the color component of the coding block can be derived.

[0273] In addition, if the context information of the multi-type tree splitting direction flag is derived for the luminance component and the chrominance component, respectively, the context information of the multi-type tree splitting type flag can be derived for the luminance component and the chrominance component, respectively.

[0274] That is, when the tree type is a dual tree, the context information of the multi-type tree splitting type flag of the luminance component can be derived using the value and context information of the multi-type tree splitting direction flag of the luminance component. Additionally, the context information of the multi-type tree splitting type flag of the chrominance component can be derived using the value and context information of the multi-type tree splitting direction flag of the chrominance component.

[0275] Alternatively, if information is signaled to induce context information for a multi-type tree splitting direction flag regardless of the color component of the current block, context information for a multi-type tree splitting direction flag may be derived based on block splitting permission information derived according to a predetermined color component. Here, the signaling information may include information indicating a predetermined color component. Furthermore, the signaling information may be signaled in units such as sequences, GOP picture slices, CTUs, etc.

[0276] An image decoding method for determining context information of syntax elements regarding the multi-type tree splitting of the current block may be as described below.

[0277]

[0278] FIG. 8 is a flowchart illustrating an image decoding method related to determining context information of syntax elements regarding multi-type tree partitioning according to an embodiment of the present disclosure. The image decoding method of FIG. 8 can be performed by an image decoding device.

[0279] Referring to FIG. 8, the image decoding device can derive block splitting permission information for the current block (S810). Here, if the tree type of the current block is a dual tree, the block splitting permission information can be derived according to the color component of the current block.

[0280] The image decoding device can derive context information of a multi-type tree splitting flag based on block splitting allowance information (S820). Here, the context information of the multi-type tree splitting flag can be derived based on block splitting allowance information derived according to color components.

[0281] Here, block division permission information may include vertical division permission information and horizontal division permission information. The vertical division permission information may include information on whether vertical bi-division is allowed and information on whether vertical tri-division is allowed. Additionally, the horizontal division permission information may include information on whether horizontal bi-division is allowed and information on whether horizontal tri-division is allowed.

[0282] Here, the context information of the multi-type tree splitting flag can be derived based on the result of comparing the value of the vertical splitting allowance information and the value of the horizontal splitting allowance information.

[0283] The video decoding device can derive the value of the multi-type tree splitting flag of the current block based on the derived context information (S830).

[0284] The video decoder can divide the current block based on a multi-type tree division flag (S840). Here, the multi-type tree division flag may be a flag indicating the direction of the multi-type tree division. Alternatively, the multi-type tree division flag may be a flag indicating the type of the multi-type tree division.

[0285]

[0286] Meanwhile, the video encoding device can perform the following video encoding method.

[0287] Specifically, the video encoding device can determine block splitting permission information for the current block.

[0288] The video encoding device can determine the multi-type tree splitting flag of the current block.

[0289] The video encoding device can determine context information of a multi-type tree splitting flag based on block splitting allowance information.

[0290] The video encoding device can encode the multi-type tree splitting flag based on context information.

[0291] Here, if the tree type of the current block is a dual tree, block splitting permission information can be derived based on the color component of the current block.

[0292] Here, the context information of the multi-type tree splitting flag can be derived based on block splitting allowance information derived according to the color component.

[0293] A bitstream can be generated by the video encoding method described above. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0294]

[0295] Inter prediction can be performed using an affine model.

[0296] FIG. 9 is a diagram illustrating a parameter model for determining motion vectors in an affine model-based inter prediction according to one embodiment of the present disclosure.

[0297] Referring to Fig. 9, the motion vector of the current block predicted based on the affine model can be derived by a motion model using two or more control point motion vectors (CPMV). For example, the motion vector can be derived using either a 4-parameter model or a 6-parameter model.

[0298] Specifically, the parameter model illustrated in FIG. 9(a) may be a 4-parameter model using two control point motion vectors mv0 and mv1. In the 4-parameter model, the motion vector of a pixel within the current block can be derived according to the following mathematical formula.

[0299]

[0300]

[0301] And, the parameter model illustrated in FIG. 9(b) may be a 6-parameter model using three control point motion vectors mv0, mv1, and mv2. In the 6-parameter model, the motion vector of a pixel within the current block can be derived according to the following mathematical formula.

[0302]

[0303]

[0304] Here, mv0 = {mv 0x , mv 0y} is the movement vector of the control point at the position adjacent to the top-left of the current block, and mv1 = {mv 1x , mv 1y} is the movement vector of the control point at the top-right position of the current block, and mv2 = {mv 2x , mv 2y} can be the motion vector of the control point at the bottom-left position of the current block. Also, W and H indicate the width and height of the current block, respectively, and mv = {mv x , mv y} can represent the motion vector of pixel location {x, y}.

[0305]

[0306] Meanwhile, the Adaptive Motion Vector Resolution (AMVR) technique, which adjusts the resolution of the motion vector difference during the inter prediction process, can be applied. That is, when AMVR is applied, the resolution of the motion vector difference of the current block can be determined adaptively.

[0307] For example, when AMVR is applied, the resolution of the motion vector difference can be determined based on at least one of the affine indicator, the prediction mode of the current block, and the motion vector difference resolution index. Here, whether AMVR is applied can be determined based on the AMVR application flag. Table 8 below shows the method for determining the resolution of the motion vector difference.

[0308]

[0309] According to Table 8, when the AMVR application flag (amvr_flag) indicates that AMVR is not applied (amvr_flag == 0), the motion vector difference resolution can be determined as 1 / 4 sample.

[0310] When the AMVR application flag (amvr_flag) indicates that AMVR is applied (amvr_flag == 1), the motion vector difference resolution can be determined in units of 1 / 16 samples, 1 / 2 samples, 1 sample, or 4 samples depending on at least one of the motion vector difference resolution index (amvr_precision_idx) value, the affine indicator (inter_affine_flag), and whether the prediction mode of the current block is IBC mode.

[0311] When affine model-based inter-prediction is applied (i.e., when the inter_affine_flag value is 1), the motion vector difference resolution can be determined in units of 1 / 4 sample, 1 / 16 sample, and 1 sample based on the AMVR application flag (amvr_flag) and the motion vector difference resolution index (amvr_precision_idx).

[0312] And, if the prediction mode of the current block is IBC mode (i.e., CuPredMode[chType][x0][y0] == MODE_IBC), the motion vector difference resolution can be determined as either 1 sample unit or 4 sample units based on the AMVR application flag (amvr_flag) and the motion vector difference resolution index (amvr_precision_idx) values.

[0313] And, when affine model-based inter-prediction is not applied (i.e., when the inter_affine_flag value is 0) and the prediction mode of the current block is a mode other than IBC mode (i.e., CuPredMode[chType][x0][y0] != MODE_IBC), the motion vector difference resolution can be determined as one of 1 / 4 sample unit, 1 / 2 sample unit, 1 sample unit, and 4 sample unit based on the AMVR application flag (amvr_flag) and the motion vector difference resolution index (amvr_precision_idx) values.

[0314] And, a shift value (AmvrShift) corresponding to the resolution value can be determined.

[0315]

[0316] Meanwhile, based on the affine indicator and the determined shift value, the motion vector difference of the current block can be changed as shown in Table 9 below.

[0317]

[0318] If inter-prediction based on an affine model is not applied to the current block, the resolution of the motion vector difference (MvdL0) in the first prediction direction and the motion vector difference (MvdL1) in the second prediction direction may be changed.

[0319] According to mathematical formulas (1) to (4) of Table 9, a shift operation by the AMVRShift value can be applied to MvdL0[x0][y0][compIdx] and MvdL1[x0][y0][compIdx]. Here, compIdx may be a vector component index having a value of 0 to 1.

[0320] On the other hand, when inter-prediction based on an affine model is applied to the current block, the resolution of the affine motion vector difference (MvdCpL0) in the first prediction direction and the affine motion vector difference (MvdCPL1) in the second prediction direction may be changed. Here, the affine motion vector difference may be the control point motion vector difference (MvdCp).

[0321] According to mathematical formulas (5) to (10) of Table 9, among MvdCpL0[x0][y0][cpIdx][compIdx] and MvdCpL1[x0][y0][cpIdx][compIdx], shift operations can be applied by the AMVRShift value to MvdCpL0[x0][y0][0][0], MvdCpL1[x0][y0][0][1][0], MvdCpL1[x0][y0][1][1], MvdCpL0[x0][y0][2][0], and MvdCpL1[x0][y0][2][1]. Here, cpIdx may be an index of a control point having a value of 0 to 2. And, compIdx may be a vector component index having a value from 0 to 1.

[0322] However, as defined in Table 9, when inter-prediction based on an affine model is applied to the current block, the resolution of some components of the affine motion vector difference may not change. That is, shift operations may be applied only to some vector components of the affine motion vector difference.

[0323] In other words, a shift operation can be applied only to the x component of the affine motion vector difference of the first prediction direction (L0), and a shift operation can be applied only to the y component of the affine motion vector difference of the second prediction direction (L1). As a result, different resolutions can be set for each affine motion vector difference vector component.

[0324] To change the resolution of the affine motion vector difference vector components equally, it is necessary to apply a shift operation to the affine motion vector difference of all prediction directions (MvdCpL0[x0][y0][cpIdx][compIdx] and MvdCpL1[x0][y0][cpIdx][compIdx]). (Here, cpIdx is 0~2 and compIdx is 0~1). That is, if the inter prediction of the current block is an inter prediction based on an affine model, the resolution of the motion vector difference of the current block can be changed as shown in Table 10 below.

[0325]

[0326] As defined in Table 10, a shift operation can be applied to all vector components of the affine motion vector difference of the first prediction direction and the second prediction direction. Therefore, the resolution of the affine motion vector difference can be changed equally for all vector components. Errors caused by differences in resolution of the affine motion vector differences can be minimized. Thus, decoding efficiency can be improved.

[0327] FIG. 10 is a flowchart illustrating an image decoding method related to resolution adjustment of an affine motion vector difference according to one embodiment of the present disclosure. The image decoding method of FIG. 10 can be performed by an image decoding device.

[0328] Referring to FIG. 10, the image decoder can derive an affine motion vector difference of the current block (S1010). Here, the affine motion vector difference may include a first predicted direction affine motion vector difference and a second predicted direction affine motion vector difference. The first predicted direction affine motion vector difference and the second predicted direction affine motion vector difference may be defined by a control point indicator and a motion vector component indicator. Here, the control point indicator may indicate any one of the values ​​0, 1, or 2. And, the motion vector component indicator may indicate any one of the values ​​0 representing the x component and 1 representing the y component.

[0329] Meanwhile, the affine motion vector difference of the current block can be derived based on motion vector difference information obtained from the bitstream. Specifically, the motion vector difference information obtained from the signaled bitstream may include a difference presence flag of a specific magnitude, difference absolute value information, and a difference sign flag. A local motion vector difference value can be derived based on the motion vector difference information obtained from the bitstream. Here, the local motion vector difference value can be used to derive a motion vector difference value or an affine motion vector difference value per control point indicator according to the motion model of the inter-prediction.

[0330] And, the video decoder can determine the motion vector difference resolution based on the AMVR application flag and the AMVR resolution index (S1020). Here, the motion vector difference resolution can be any one of 1 / 16 samples, 1 / 4 samples, and 1 sample unit.

[0331] The video decoder can adjust the resolution of the affine motion vector difference based on the determined motion vector difference resolution (S1030).

[0332] Specifically, a shift value corresponding to the motion vector difference resolution is derived, and the shift value is applied to the first prediction direction affine motion vector difference and the second prediction direction affine motion vector difference according to all motion vector component indicators and all control point indicators so that the resolution of the affine motion vector difference can be adjusted.

[0333] Here, the first predicted direction affine motion vector difference and the second predicted direction affine motion vector difference according to all motion vector component indicators and all control point indicators may mean MvdCpL0[x0][y0][cpIdx][compIdx] and MvdCpL1[x0][y0][cpIdx][compIdx] (cpIdx is 0~2, compIdx is 0~1).

[0334] Meanwhile, the steps described in FIG. 10 can be performed in the same way in a video encoding method. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 10. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0335] Meanwhile, the affine motion vector difference in the above embodiment may be the difference of the control point motion vector (CPMV).

[0336]

[0337] Inter prediction can be performed using a geometric partitioning mode (GPM). In the geometric partitioning mode, the current block can be divided into two blocks by a single straight line. The straight line dividing the current block can be derived from the partitioning direction and the partitioning offset. Each block divided from the current block can be inter predicted using motion information derived differently from one another.

[0338] When a geometric partitioning mode is used, indices indicating the geometric partitioning mode and merge indices may be signaled. Here, the index indicating the geometric partitioning mode may indicate a combination of the direction and offset of the straight line dividing the current block. The index indicating the geometric partitioning mode may indicate a combination of the direction and offset of the straight line dividing the current block as shown in Table 11 below.

[0339]

[0340] The current block can be partitioned according to an index indicating the geometric partitioning mode, as described below.

[0341] FIG. 11 is a drawing illustrating a partition structure of a current block based on an index indicating a geometric partition mode according to one embodiment of the present disclosure.

[0342] Referring to FIG. 11, the solid line within the rectangle may be a straight line dividing the current block. The block of the rectangle may be geometrically divided along the solid line.

[0343] Referring to FIG. 11, the lines dividing each of the 20 squares may have different angles (angleIdx). Here, the lines may have different offsets (distanceIdx). Also, the lines indicated by solid lines are defined by an index indicating a geometric division mode and can divide the current block. On the other hand, the lines indicated by dotted lines may be lines not defined by an index indicating a geometric division mode.

[0344] Furthermore, merge indices can indicate merge candidates to derive movement information for each block split from the current block. Accordingly, movement information for each block split from the current block can be derived based on the movement information of merge candidates indicated by different merge indices. Additionally, predicted values ​​for each block split from the current block can be generated based on the derived movement information.

[0345] Furthermore, based on the straight line dividing the current block, a predicted block for the current block can be generated by weighting the predicted values ​​for each block. Additionally, movement information of the predicted current block can be stored using a geometric division mode.

[0346] FIG. 12 is a drawing illustrating the current block predicted by a geometric division mode according to one embodiment of the present disclosure.

[0347] Referring to FIG. 12, a first predicted value, which is a predicted value for a first division block (1210), and a second predicted value, which is a predicted value for a second division block (1220), are generated, and the first predicted value and the second predicted value can be weighted summed.

[0348] And, based on the division direction (φ) and offset (ρ) of the line dividing the current block, the distance (d) between the sample in the current block and the line dividing the current block can be derived. And, based on the derived distance value, a weight value ω0 applied to the sample in the current block can be derived. The predicted value of the sample in the current block can be derived by applying ω0 to the first predicted value and 1-ω0 to the second predicted value.

[0349] That is, when performing inter-prediction for the current block by applying a geometric partitioning mode, stored motion information can be derived based on the motion information of the first partitioned block and the motion information of the second partitioned block. The stored motion information can be used in the decoding process of the block after the decoding of the current block. Here, the stored motion information can be stored in blocks of size 4x4. Here, the stored motion information can be stored as shown in Table 12 below.

[0350]

[0351]

[0352] Referring to Table 12, motion information of the current block in geometric partition mode can be stored in 4x4 block units. Here, the motion information of the current block may include motion vector information, reference picture information, prediction direction usage information, and bidirectional prediction index.

[0353] However, in Table 12, when prediction is performed in geometric partitioning mode, interpolation filter information is not stored. Here, the interpolation filter may be an interpolation filter for inter-predicted luminance samples. For example, the interpolation filter may be a half-sample interpolation filter. And, the interpolation filter information may be an index (e.g., hpelIfIdx) indicating the type of interpolation filter, which is an interpolation filter for fractional sample positions.

[0354] When prediction is performed according to a general inter mode, interpolation filter information is required, but when prediction is performed according to a geometric partitioning mode, interpolation filter information may not be required.

[0355] Therefore, as shown in Table 12, blocks predicted in geometric partitioning mode do not store interpolation filter information, so when using the stored motion information of blocks predicted in geometric partitioning mode during the inter-prediction process of other neighboring blocks, interpolation filter information may not be obtained.

[0356] Movement information of the current block, for which prediction is performed in a geometric division mode according to one embodiment of the present disclosure, can be stored as shown in Table 13 below.

[0357]

[0358] Referring to Table 13, motion information of the current block in geometric partition mode can be stored. Here, the motion information of the current block may further include motion vector information, reference picture information, prediction direction usage information, bidirectional prediction index, and interpolation filter information. Therefore, when the stored motion information of the current block is used in the inter-prediction process of a block that is decoded / encoded after the current block, motion information including the interpolation filter information of the current block can be obtained.

[0359] Specifically, in the inter prediction process of a block referencing the current block, fractional sample interpolation may be applied to the samples of the current block. Fractional sample interpolation can be performed by applying interpolation filter coefficients to the samples of the current block in the vertical and horizontal directions. Here, the interpolation filter coefficients can be defined as shown in Table 14 below.

[0360]

[0361] As defined in Table 14, the interpolation coefficient can be determined based on the interpolation filter information of the current block (e.g., hpelIfIdx).

[0362] That is, interpolation filter coefficients are determined based on the interpolation filter information of the current block, and by applying the interpolation filter coefficients to the prediction samples of the current block, a prediction block of a block referencing the current block can be generated.

[0363]

[0364] FIG. 13 is a flowchart illustrating an image decoding method related to storing motion information of a geometric partition mode block according to one embodiment of the present disclosure. The image decoding method of FIG. 13 can be performed by an image decoding device.

[0365] Referring to FIG. 13, the image decoder can derive a geometric division index of the current block (S1310). Here, the geometric division index can be used to derive the angle of the division line and the distance between the block center and the division line. Here, the division line can asymmetrically divide the current block.

[0366] And, the image decoder can generate a final prediction block by weighting the first prediction block and the second prediction block of the current block based on a geometric division index (S1320). Here, the first prediction block may be generated by first motion information, and the second prediction block may be generated by second motion information. Also, the first motion information may be different from the second motion information.

[0367] Meanwhile, the weights applied to the samples of the first prediction block can be derived based on the distance between the dividing line and the samples.

[0368] And, the video decoding device can store motion information of the current block based on motion information of the first prediction block and the second prediction block (S1330). Here, the motion information of the current block can be stored in blocks of size 4x4.

[0369] In the step of storing motion information of the current block, the motion information of the current block stored may include a motion vector, reference picture information, prediction direction information, bidirectional prediction index information, and interpolation filter information. Here, the value of the bidirectional prediction index information may be fixed to 0.

[0370] Meanwhile, the steps described in FIG. 13 can be performed in the same way in a video encoding method. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 13. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0371]

[0372] FIG. 14 is a flowchart illustrating an image decoding method according to one embodiment of the present disclosure. The image decoding method of FIG. 14 can be performed by an image decoding device.

[0373] Referring to FIG. 14, the image decoder can determine the prediction mode condition information of the current block (S1410).

[0374] The above prediction mode condition information can be determined based on a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

[0375] Meanwhile, the above prediction mode condition information can be determined as described in Fig. 6.

[0376] And, the video decoder can derive the prediction mode type information of the current block based on the prediction mode condition information (S1420).

[0377] The above prediction mode type information can be determined as described in Fig. 6.

[0378] And, the video decoder can generate a prediction block of the current block based on the prediction mode type information (S1430).

[0379] Additionally, the video decoding device can determine the integrity of the prediction block (S1440). Specifically, the integrity of the prediction block can be determined based on the prediction block of the neighbor block of the current block. Alternatively, the integrity of the prediction block can be determined based on the movement information of the neighbor block of the current block.

[0380] Here, the neighbor block of the current block can be the left neighbor block, the top neighbor block, or the top-left neighbor block of the current block.

[0381] According to another embodiment, the integrity of the prediction block may be determined based on the prediction block of the call block included in the reference picture corresponding to the position of the current block. Alternatively, the integrity of the prediction block may be determined based on the motion information of the call block included in the reference picture corresponding to the position of the current block. Here, the motion information may be at least one of a motion vector, a reference picture index, and prediction direction information.

[0382] Meanwhile, the integrity of the prediction block can be used to determine whether it is available as a prediction value during the image decoding / encoding process. The integrity of the prediction block can be determined based on a predefined threshold.

[0383] And, the video decoding device can output the integrity determination result (S1450).

[0384] Meanwhile, the steps described in FIG. 14 can be performed in the same way in a video encoding method. Additionally, a bitstream can be generated by a video encoding method including the steps described in FIG. 14. The bitstream can be stored on a non-transient computer-readable recording medium and can also be transmitted (or streamed).

[0385]

[0386] The exemplary methods of the present disclosure are described as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. To implement the method according to the present disclosure, additional steps may be included in addition to the steps exemplified, steps excluding some steps and including the remaining steps, or steps excluding some steps and including additional steps.

[0387] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0388] Various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0389] Alternatively, various embodiments of the present disclosure may be implemented in the form of program instructions that can be executed through various computer components and recorded on a computer-readable recording medium. And, a bitstream generated by the encoding method according to the embodiment may be stored on a non-transient computer-readable recording medium.

[0390] The above-mentioned computer-readable recording medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the above-mentioned computer-readable recording medium may be those specifically designed and configured for the present disclosure, or they may be those known and available to those skilled in the art of computer software.

[0391] In the foregoing, the present disclosure is described based on specific details, such as specific components, limited embodiments, and drawings. However, the embodiments of the present disclosure are provided merely to aid in the overall understanding of the present disclosure and are not intended to limit the present disclosure to the above embodiments. Accordingly, a person skilled in the art can make various modifications and variations from this description.

[0392] Accordingly, the scope of the present disclosure is not limited to the embodiments described above, and all things equivalent or equivalently modified to the claims set forth below, as well as the claims set forth below, shall be considered to be within the scope of the scope of the present disclosure.

[0393]

[0394] The present disclosure may be used in an image encoding device, an image decoding device, a bitstream transmission device, and a recording medium for storing a bitstream.

Claims

1. In a video decoding method, A step of determining prediction mode condition information of the current block; A step of deriving prediction mode type information of the current block based on the prediction mode condition information; A step of generating a prediction block of the current block based on the prediction mode type information above; A step of determining the integrity of the above prediction block; and It includes a step of outputting the above integrity determination result, and An image decoding method in which the above prediction mode condition information is determined based on a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

2. In Paragraph 1, An image decoding method characterized in that the integrity of the above-mentioned prediction block is determined based on the prediction block of the neighbor block of the above-mentioned current block.

3. In Paragraph 1, A video decoding method characterized in that the integrity of the above-mentioned prediction block is determined based on the movement information of neighbor blocks of the above-mentioned current block.

4. In Paragraph 1, An image decoding method characterized in that, in the above image decoding method, the integrity of the prediction block is determined based on the prediction block of the call block included in the reference picture corresponding to the position of the current block.

5. In a video encoding method, A step of determining prediction mode condition information of the current block; A step of deriving prediction mode type information of the current block based on the prediction mode condition information; A step of generating a prediction block of the current block based on the prediction mode type information above; A step of determining the integrity of the above prediction block; and It includes a step of outputting the above integrity determination result, and An image encoding method characterized in that the above prediction mode condition information is determined based on a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

6. A method for transmitting a bitstream generated by a video encoding method, The above transmission method includes the step of transmitting the bitstream, and The above image encoding method is, A step of determining prediction mode condition information of the current block; A step of deriving prediction mode type information of the current block based on the prediction mode condition information; A step of generating a prediction block of the current block based on the prediction mode type information above; A step of determining the integrity of the above prediction block; and It includes a step of outputting the above integrity determination result, and A transmission method characterized in that the above prediction mode condition information is determined based on a picture type including the current block, a slice type including the current block, and whether the current block is adjacent to a slice boundary or a picture boundary.

Images