Image encoding / decoding method, apparatus, and recording medium

Abstract

An image encoding / decoding method, apparatus, and recording medium of the present disclosure comprise the steps of: for prediction of the current block, configuring a merge list; and generating a prediction block of the current block on the basis of a candidate vector included in the merge list, wherein the merge list is configured by acquiring a motion vector or a block vector from a candidate block around the current block, and the candidate block may comprise an intermediate location block located between corner blocks of the current block.

Description

Video encoding / decoding method, device, and recording medium

[0001] The present disclosure can be utilized in the fields of image encoding / decoding methods, devices, and recording media.

[0002] Recently, the demand for multimedia data, such as video, on the Internet has been increasing rapidly. However, the pace of development in channel bandwidth is struggling to keep up with the rapidly growing volume of multimedia data.

[0003] Furthermore, various video compression technologies have been developed for the efficient transmission and storage of multimedia data. For example, existing video compression standards have sought to reduce data volume by eliminating redundancy in video data using techniques such as prediction, transformation, quantization, and entropy coding. However, with the proliferation of high-resolution and ultra-high-resolution video and various multimedia services, there is a continuously increasing demand for video encoding technologies that offer higher compression efficiency and adaptability to diverse environments.

[0004] The main purpose of this disclosure is to improve image encoding / decoding efficiency.

[0005] The image encoding / decoding method, apparatus, and recording medium of the present disclosure comprise: a step of configuring a merge list for predicting a current block; and a step of generating a predicted block of the current block based on a candidate vector included in the merge list, wherein the merge list is configured by obtaining a motion vector or a block vector from candidate blocks surrounding the current block, and the candidate blocks may include intermediate position blocks located between the corner blocks of the current block.

[0006] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the merge list may be an inter-merged list or an intra-merged list.

[0007] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the inter-merging list includes motion vectors, and the intra-merging list may include block vectors.

[0008] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the intermediate position block may include at least one of a first intermediate position block located between corner blocks on the upper boundary line of the current block or a second intermediate position block located between corner blocks on the left boundary line of the current block.

[0009] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the candidate block does not include a first intermediate location block located between corner blocks on a first boundary line of the current block, and may include a second intermediate location block located between corner blocks on a second boundary line of the current block.

[0010] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the first boundary line is a boundary line adjacent to another block divided from the same upper block as the current block, and the second boundary line may be a boundary line not adjacent to the other block.

[0011] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, whether or not the intermediate position block is used may be determined according to the size of the current block.

[0012] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the search for an intermediate location block for the candidate block may prioritize searching for a block close to the intermediate location of the boundary line of the current block.

[0013] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the search for an intermediate location block for the candidate block may be performed by first searching for a block located in a first direction based on the intermediate location of the boundary line of the current block, and if the block located in the first direction does not have a valid motion vector or block vector, additionally searching for a block located in a second direction.

[0014] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the merging list may further include a new merging candidate vector generated using a plurality of candidate vectors included in the merging list.

[0015] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, the new merged candidate vector may be generated as a weighted average of the plurality of candidate vectors.

[0016] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, if the number of candidate vectors having the same reference index in the merged list is less than a predetermined number, the weighted average can be performed using only the candidate vectors having the same reference index.

[0017] In the image encoding / decoding method, apparatus, and recording medium of the present disclosure, if the number of candidate vectors having the same reference index in the merge list is less than the predetermined number, the vector other than the candidate vectors having the same reference index used for the weighted average may be set as an average vector, an intermediate vector, or a predetermined vector.

[0018] The present disclosure can improve image encoding / decoding efficiency.

[0019] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0020] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0021] FIG. 3 illustrates an example of a surrounding adjacent corner block.

[0022] FIG. 4 illustrates an example of a surrounding adjacent intermediate block.

[0023] FIG. 5 illustrates an example of a merged list configuration.

[0024] FIG. 6 illustrates an example of a surrounding adjacent intermediate block when the number of horizontal or vertical pixels of the current block is greater.

[0025] FIG. 7 illustrates an example of a surrounding adjacent intermediate block of Block B within a Coding Tree Unit (CTU).

[0026] FIG. 8 illustrates an example of a surrounding adjacent intermediate block.

[0027] FIG. 9 illustrates an example of a surrounding adjacent intermediate block based on a dividing line.

[0028] FIG. 10 illustrates an example of the search order of surrounding adjacent intermediate blocks.

[0029] FIG. 11 illustrates an example of the search order of surrounding adjacent intermediate blocks.

[0030] FIG. 12 illustrates one embodiment of a surrounding adjacent intermediate block for all sides.

[0031] FIG. 13 illustrates an example of a video encoding method that predicts by constructing a merged list.

[0032] FIG. 14 illustrates an example of an image decoding method that predicts by constructing a merge list.

[0033] FIG. 15 illustrates an example of in-frame mode prediction based on a reference vector.

[0034] FIG. 16 illustrates an example of an image encoding method based on the selection of an adaptive conversion technique.

[0035] FIG. 17 illustrates an example of an image decoding method based on the selection of an adaptive transformation technique.

[0036] The present invention is susceptible to various modifications and may have various embodiments; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the invention to specific embodiments, and it should be understood that the invention includes all modifications, equivalents, and substitutions that fall within the spirit and scope of the invention. Similar reference numerals have been used for similar components in the description of each drawing.

[0037] Terms such as "first," "second," etc., may be used to describe various components, but said components should not be limited by said terms. These terms are used solely for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of a plurality of related described items or any of a plurality of related described items.

[0038] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0039] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to specify the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0040] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Hereinafter, the same reference numerals are used for identical components in the drawings, and redundant descriptions of identical components are omitted.

[0041] FIG. 1 is a block diagram showing an image encoding device according to an embodiment of the present invention.

[0042] Referring to FIG. 1, the image encoding device (100) may include a picture splitting unit (110), a prediction unit (120, 125), a conversion unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse conversion unit (145), a filter unit (150), and a memory (155).

[0043] Additionally, the image encoding device (100) may further include a neural processing unit or an extended prediction tool processing unit for performing learning-based image processing or extended prediction tool processing.

[0044] Each component shown in FIG. 1 is depicted independently to represent different characteristic functions of the image encoding device and does not imply that each component consists of separate hardware or a single software unit. That is, each component is listed and included as a separate component for the convenience of explanation, but at least two of the components may be combined to form a single component, or a single component may be divided into multiple components to perform functions, and such integrated and separated embodiments of each component are included within the scope of the present invention as long as they do not deviate from the essence of the present invention.

[0045] Furthermore, some components may not be essential components performing an essential function in the present invention, but merely optional components for enhancing performance. The present invention may be implemented by including only the components essential for realizing the essence of the present invention, excluding components used solely for performance enhancement, and a structure including only the essential components, excluding optional components used solely for performance enhancement, is also included within the scope of the rights of the present invention.

[0046] The picture segmentation unit (110) can divide the input picture into at least one block. At this time, the block may refer to a coding unit (CU), a prediction unit (PU), or a transformation unit (TU). The segmentation may be performed based on at least one of a quadtree or a binary tree. A quadtree is a method of dividing an upper block into lower blocks, each having a width and height that are half that of the upper block. A binary tree is a method of dividing an upper block into lower blocks, each having either a width or a height that is half that of the upper block. Through the aforementioned binary tree-based segmentation, the blocks may have a shape that is not only square but also non-square.

[0047] In addition, the block partitioning structure may be performed based on multi-type tree (MTT)-based partitioning, asymmetric partitioning, or learning-based partitioning structures, and may be adaptively determined based on image features or the output of a learned model.

[0048] In the following embodiments of the present invention, the encoding unit may be used to mean a unit that performs encoding, or a unit that performs decoding.

[0049] The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction and an intra prediction unit (125) that performs intra prediction. It may determine whether to use inter prediction or perform intra prediction for a prediction unit, and determine specific information according to each prediction method (e.g., intra prediction mode, motion vector, reference picture, etc.).

[0050] In addition, the prediction unit can additionally perform a fusion prediction mode that combines multiple prediction candidates, a template-based prediction, or a learning-based prediction mode.

[0051] At this time, the processing unit in which the prediction is performed and the processing unit in which the prediction method and specific details are determined may be different. For example, the prediction method and prediction mode, etc., may be determined in the prediction unit, and the prediction may be performed in the transformation unit. The residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, the prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value in the entropy encoding unit (165) and transmitted to the decoder.

[0052] The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous picture or the subsequent picture of the current picture, and in some cases, may predict a prediction unit based on information of a partially encoded area within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

[0053] In addition, the inter-prediction unit can perform multiple reference frame-based prediction, an extended motion model, or learning-based motion compensation (neural motion compensation).

[0054] In the reference picture interpolation unit, reference picture information is provided from memory (155), and pixel information of an integer pixel or less can be generated from the reference picture.

[0055] In addition, the reference pixel interpolation process may be performed using an adaptive filter or a learning-based interpolation filter.

[0056] The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit.

[0057] In addition, motion prediction may be performed using a learning-based motion estimation model or an extended block matching algorithm.

[0058] The intra prediction unit (125) can generate a prediction unit based on reference pixel information around the current block, which is pixel information within the current picture.

[0059] In addition, the intra prediction unit can perform template-based prediction, matrix-based prediction, or neural network-based intra prediction.

[0060] In the conversion unit (130), the residual block containing residual data can be converted using a conversion method such as DCT, DST, etc.

[0061] In addition, the transformation unit may perform adaptive transformation selection or learning-based transformation.

[0062] The quantization unit (135) can quantize the values ​​converted into the frequency domain in the conversion unit (130).

[0063] In addition, quantization parameters may be adaptively determined by image characteristics or a learning-based model.

[0064] The entropy encoding unit (165) can encode residual value coefficient information, block type information, prediction mode information, etc. of the encoding unit from the reordering unit (160) and the prediction unit (120, 125).

[0065] In addition, the entropy encoding unit can additionally encode learning-based prediction mode information or neural network model identification information.

[0066] The filter section (150) may include at least one of a deblocking filter, an offset correction section, and an ALF.

[0067] In addition, the filter section may additionally include a neural network-based post-processing filter or a restoration filter.

[0068] The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter-prediction.

[0069] FIG. 2 is a block diagram showing an image decoding device according to an embodiment of the present invention.

[0070] Referring to FIG. 2, the image decoder (200) may include an entropy decoder (210), a reordering unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

[0071] Additionally, the image decoder (200) may further include a neural network-based processing unit or an extended prediction tool processing unit.

[0072] When a video bitstream is input to a video decoder, the input bitstream can be decoded using the reverse procedure of the video encoder.

[0073] The entropy decoding unit (210) can perform entropy decoding in the opposite procedure to that which the entropy encoding unit of the image encoder performed. For example, various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) can be applied in correspondence with the method performed in the image encoder.

[0074] In addition, the entropy decoder can additionally decode learning-based prediction mode information or neural network model-related information.

[0075] The entropy decoding unit (210) can decode information related to intra-prediction and inter-prediction performed in the encoder.

[0076] The reordering unit (215) can perform reordering based on the method of reordering the entropy-decoded bitstream from the entropy-decoding unit (210) in the encoding unit. It can reorder by restoring the coefficients expressed in the form of a one-dimensional vector back into coefficients in the form of a two-dimensional block. The reordering unit (215) can perform reordering by receiving information related to the coefficient scanning performed in the encoding unit and scanning in reverse based on the scanning order performed in the encoding unit.

[0077] The inverse quantization unit (220) can perform inverse quantization based on the coefficient values ​​of the rearranged block and the quantization parameters provided by the encoder.

[0078] The inverse transformation unit (225) can perform inverse transformation of the inversely quantized transformation coefficients using a predetermined transformation method. At this time, the transformation method can be determined based on information regarding a prediction method (inter / intra prediction), the size / shape of a block, an intra prediction mode, etc.

[0079] The prediction unit (230, 235) can generate a prediction block based on the prediction block generation information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

[0080] In addition, the prediction unit can perform fusion prediction mode, template-based prediction, or neural network-based prediction.

[0081] The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit receives various information, such as prediction unit information input from the entropy decoding unit (210), prediction mode information of the intra prediction method, and motion prediction related information of the inter prediction method, distinguishes the prediction unit in the current encoding unit, and can determine whether the prediction unit performs inter prediction or intra prediction.

[0082] The inter prediction unit (230) can perform an inter prediction for the current prediction unit based on information included in at least one of the previous or subsequent pictures of the current picture containing the current prediction unit, using information required for inter prediction of the current prediction unit provided by the video encoder. Alternatively, the inter prediction may be performed based on information of a partially restored area within the current picture containing the current prediction unit.

[0083] The inter prediction unit (230) may further include a reference picture interpolation unit, a motion compensation unit, and a motion information processing unit. The reference picture interpolation unit can generate pixel information of integer pixels or less using a reference picture stored in memory (245). For example, in the case of a luminance signal, an interpolation filter having multiple filter coefficients may be used to generate interpolated pixels in units of 1 / 4 pixels, and in the case of a chrominance signal, a separate interpolation filter may be used to generate interpolated pixels in units of 1 / 8 pixels.

[0084] The motion compensation unit can generate a prediction block for the current prediction unit from a reference picture using motion vector information decoded in the entropy decoding unit (210). At this time, the motion vector may have a value of an integer pixel or a value of an integer pixel or less, and the prediction block can be generated using the interpolated reference pixel.

[0085] Additionally, the inter prediction unit (230) can generate a prediction block corresponding to the Skip Mode, Merge Mode, or AMVP Mode (Advanced Motion Vector Prediction Mode) used in the encoder, and can restore the same prediction structure using prediction-related information transmitted from the encoder.

[0086] Additionally, the inter prediction unit (230) may perform multiple reference frame-based prediction, an extended motion model, or neural network-based motion compensation.

[0087] The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, it can perform intra prediction based on the intra prediction mode information of the prediction unit provided by the image encoder.

[0088] The intra prediction unit (235) can generate a prediction block using a restored reference pixel located around the current block, and if the reference pixel is not available, it can replace the reference pixel using an available adjacent reference pixel.

[0089] The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a part that performs filtering on the reference pixel of the current block, and can determine whether to apply the filter based on the prediction mode of the current prediction unit.

[0090] The reference pixel interpolation unit can generate a prediction block by interpolating a reference pixel according to the prediction mode, and the DC filter can generate a prediction block through filtering if the prediction mode of the current block is DC mode.

[0091] In addition, the intra prediction unit can perform template-based prediction, matrix-based prediction, or neural intra prediction.

[0092] The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

[0093] In addition, the filter unit can perform neural network-based post-processing filters or restoration filters.

[0094] The memory (245) can store the restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to the output unit.

[0095]

[0096] FIG. 3 illustrates an example of a surrounding adjacent corner block.

[0097] A merge list can be constructed to predict the current block. The merge list is a list for storing candidate vectors to be used for predicting the current block, and it can be generated based on information about blocks located around the current block.

[0098] To construct a merge list, surrounding blocks of the current block can be referenced.

[0099] In one embodiment, the surrounding blocks may include some corner blocks adjacent to the current block and some blocks not directly adjacent to the current block. Taking FIG. 3 as an example, the corner blocks adjacent to the current block may be surrounding blocks that are adjacent to the current block and share the (top-right, bottom-left) corner points of the current block. Additionally, the corner blocks not directly adjacent to the current block may be surrounding blocks that are not adjacent to the current block but share the (top-right, bottom-left) corner points of the current block.

[0100] In one embodiment, the surrounding blocks may include adjacent blocks such as the left block, upper block, upper right block, and lower left block of the current block, and may also include non-adjacent blocks such as reference blocks existing at temporally corresponding positions.

[0101] The merge list may store motion vectors or block vectors obtained from the surrounding blocks, and the stored vectors may be utilized as candidate vectors used in the merge mode. That is, the current block can generate a predicted block by selecting at least one of the candidate vectors included in the merge list.

[0102] In the case of an inter-merged list, candidates can be formed centered on blocks among the surrounding blocks where motion vectors exist, and the motion vectors of those blocks can be stored as candidate vectors for the inter-merged list. On the other hand, in the case of an intra-merged list, candidates can be formed centered on blocks where block vectors exist, and the block vectors of those blocks can be stored as candidate vectors for the intra-merged list.

[0103] Generally, in the case of the inter-merge mode where inter-prediction is performed, an inter-merge list can be configured and used, and in the case of the intra-block copy mode, an intra-merge list can be configured and used. Accordingly, different types of merge lists can be selectively configured and used depending on the prediction mode of the current block.

[0104] FIG. 4 illustrates an example of a surrounding adjacent intermediate block.

[0105] FIG. 5 illustrates an example of a merged list configuration.

[0106] In the process of constructing a merge list, in addition to adjacent corner blocks, intermediate blocks located between corner blocks (= blocks between corners) can also be used as candidate blocks. That is, not only the corner blocks directly adjacent to the current block, but also the blocks located between the corner blocks can be utilized as reference blocks to construct the merge candidates.

[0107] The above intermediate position blocks can be used in the process of constructing an intermerged list, and if the blocks have valid motion vectors, the motion vectors can be added as candidate vectors for the intermerged list. Accordingly, the intermerged list can be configured to include motion vectors obtained from not only adjacent corner blocks but also intermediate position blocks between corners.

[0108] Similarly, in the process of constructing an intra-merge list, intermediate blocks located between corners may be used in addition to adjacent corner blocks. In particular, when the Intra Block Copy mode is performed, block vectors existing in adjacent corner blocks and intermediate blocks can be searched to predict the current block.

[0109] In the process of constructing the intra-merge list in intra-block copy mode, the block vectors of corner blocks adjacent to the current block can be searched first and added to the candidate list. Subsequently, the block vectors of intermediate blocks located between the corner blocks can be additionally searched and included as candidates in the intra-merge list.

[0110] In this way, by configuring the intra-merge list by first searching corner blocks and then intermediate blocks, a wider variety of prediction candidates for the current block can be secured, thereby improving prediction accuracy in intra-block copy mode.

[0111] Additionally, the use of adjacent intermediate blocks can be determined based on the size of the current block, for example, the width or height of the current block. In other words, depending on the size of the current block, it can be selectively decided whether to include intermediate blocks located between corner blocks in the merge list configuration.

[0112] For example, if the number of horizontal pixels of the current block is greater than a predetermined threshold, intermediate blocks between corner blocks located on the upper boundary of the current block may be additionally considered. On the other hand, if the number of horizontal pixels of the current block is smaller than the threshold, intermediate blocks located between corner blocks on the upper boundary may not be considered.

[0113] For example, if the number of vertical pixels of the current block is greater than a predetermined threshold, intermediate blocks located between corner blocks on the left boundary of the current block may be additionally considered. On the other hand, if the number of vertical pixels of the current block is smaller than the threshold, intermediate blocks located between corner blocks on the left boundary may not be considered.

[0114] In this way, by adaptively determining whether to use an intermediate block based on the width or height of the current block, the number of candidate blocks used in merging list construction can be efficiently controlled, thereby improving prediction performance without increasing complexity.

[0115] FIG. 6 illustrates an example of a surrounding adjacent intermediate block when the number of horizontal or vertical pixels of the current block is greater.

[0116] The use of adjacent intermediate blocks can be determined based on the size of the current block, for example, its width or height. In other words, depending on the size of the current block, it can be selectively decided whether to include intermediate blocks located between corner blocks in the merge list configuration.

[0117] For example, if the number of horizontal pixels of the current block is greater than a predetermined threshold, an intermediate block between corner blocks located on the upper boundary of the current block may be considered as a candidate block for merging. On the other hand, if the number of horizontal pixels of the current block is less than or equal to the threshold, an intermediate block located between corner blocks on the upper boundary may not be considered as a candidate block for merging.

[0118] Additionally, if the number of vertical pixels of the current block is greater than a predetermined threshold, the intermediate block between the corner blocks located on the left boundary of the current block may be considered as a merge candidate block. Conversely, if the number of vertical pixels of the current block is less than or equal to the threshold, the intermediate block located between the corner blocks on the left boundary may not be considered as a merge candidate block.

[0119] In one embodiment, the threshold value may be a value signaled from a bitstream.

[0120] In one embodiment, the threshold value may be a pre-defined value.

[0121] For example, if the number of horizontal pixels of the current block is 64 or less, intermediate blocks located between corner blocks on the upper boundary line may not be considered.

[0122] For example, if the number of horizontal pixels of the current block is 32 or less, or 16 or less, the intermediate block on the upper boundary line may not be considered in the same way.

[0123] For example, if the number of vertical pixels of the current block is 64 or less, intermediate blocks located between corner blocks on the left boundary line may not be considered.

[0124] For example, if the number of vertical pixels of the current block is 32 or less, or 16 or less, the middle block on the left boundary line may not be considered in the same way.

[0125] In one embodiment, the threshold value may not be a fixed value but a variable value (X, Y) implicitly set according to the system or encoding conditions. For example, if the number of horizontal pixels of the current block is X or less, intermediate blocks located between corner blocks on the upper boundary line may not be considered, and if the number of vertical pixels of the current block is Y or less, intermediate blocks located between corner blocks on the left boundary line may not be considered.

[0126] Additionally, when constructing a merge list based on spatial or coding structural relationships between specific blocks, intermediate blocks between corners may not be considered. For example, if a first block (Block A) and a second block (Block B) are encoded or decoded simultaneously, or if the first block and the second block are located within the same Coding Tree Unit (CTU), or if the first block and the second block are subdivided and generated from the same compression block, intermediate blocks located between the corner blocks of the upper or left boundary line may not be considered during the process of constructing the merge list of the second block (Block B).

[0127] FIG. 7 illustrates an example of a surrounding adjacent intermediate block of Block B within a Coding Tree Unit (CTU).

[0128] When constructing a merge list based on spatial or coding structural relationships between specific blocks, intermediate blocks located between corner blocks may not be considered.

[0129] In one embodiment, when the first block (Block A) and the second block (Block B) are encoded or decoded simultaneously, or when the first block and the second block are located within the same Coding Tree Unit (CTU), or when the first block and the second block are subdivided and generated from the same compression block, intermediate blocks located between the corner blocks of the upper or left boundary line may not be considered during the process of constructing the merge list of the second block (B Block).

[0130] For example, if the first block (Block A) and the second block (Block B) are subdivided from the same compressed block, when constructing the merge list of the first block (Block A), the intermediate blocks located between the corner blocks of the upper and left boundary lines may be considered as merge candidate blocks. On the other hand, when constructing the merge list of the second block (Block B), depending on the positional relationship of the blocks as illustrated in the drawings, the intermediate blocks located between the corner blocks of the upper boundary line (e.g., in the case of the left drawing) or the left boundary line (e.g., in the case of the right drawing) may not be considered as merge candidate blocks.

[0131] FIG. 8 illustrates an example of a surrounding adjacent intermediate block.

[0132] The number of surrounding adjacent intermediate blocks to be considered can be determined based on the size of the current block, for example, the width or height of the current block. That is, depending on the horizontal or vertical size of the current block, some or all of the intermediate blocks located between the corner blocks can be selectively considered as merge candidate blocks.

[0133] In one embodiment, when the number of horizontal pixels of the current block is large, more intermediate blocks between corner blocks located on the upper boundary line of the current block may be considered. For example, when the number of horizontal pixels of the current block is greater than the number of vertical pixels, more intermediate blocks between corner blocks located on the upper boundary line of the current block may be considered than intermediate blocks between corner blocks located on the left boundary line.

[0134] In one embodiment, when the number of vertical pixels of the current block is large, more intermediate blocks between corner blocks located on the left boundary of the current block may be considered. For example, when the number of vertical pixels of the current block is greater than the number of horizontal pixels, more intermediate blocks between corner blocks located on the left boundary of the current block may be considered than intermediate blocks between corner blocks located on the upper boundary.

[0135] In one embodiment, if the number of horizontal or vertical pixels of the current block is below a predetermined threshold, only some of the intermediate blocks located between the corner blocks may be considered as merge candidate blocks. For example, if the number of horizontal or vertical pixels of the current block is X or less, only Y blocks among the intermediate blocks located between the corner blocks on the upper or left boundary line may be considered as merge candidate blocks. Here, X and Y may be predefined values. For example, X may be 32 and Y may be 1. For example, X may be 16 and Y may be 1.

[0136] Depending on the size of the current block, for example, the width or height of the current block, you can set it to skip some intermediate blocks during the process of searching for surrounding adjacent intermediate blocks.

[0137] In one embodiment, when the horizontal pixel count of the current block is larger, it can be configured to skip more intermediate blocks when searching for intermediate blocks between corner blocks located on the upper boundary line of the current block. For example, when the horizontal pixel count of the current block is larger than the vertical pixel count, it can be configured to skip more intermediate blocks when searching for intermediate blocks between corner blocks located on the upper boundary line of the current block than when searching for intermediate blocks between corner blocks located on the left boundary line.

[0138] In one embodiment, when the vertical pixel count of the current block is large, it can be configured to skip more intermediate blocks when searching for intermediate blocks between corner blocks located on the left boundary of the current block. For example, when the vertical pixel count of the current block is larger than the horizontal pixel count, it can be configured to skip more intermediate blocks when searching for intermediate blocks between corner blocks located on the left boundary of the current block than when searching for intermediate blocks between corner blocks located on the upper boundary.

[0139] In one embodiment, if the number of horizontal or vertical pixels of the current block is below a predetermined threshold, intermediate blocks located between corner blocks can be searched by skipping at regular intervals. For example, if the number of horizontal or vertical pixels of the current block is X or less, intermediate blocks located between corner blocks on the upper or left boundary line can be searched by skipping at intervals of Y. Here, X and Y may be predefined values. For example, X may be 256 and Y may be 4. For example, X may be 64 and Y may be 2.

[0140] FIG. 9 illustrates an example of a surrounding adjacent intermediate block based on a dividing line.

[0141] When constructing the merge list during the Geometric Partitioning Prediction stage, the positions of corner blocks and intermediate blocks can be determined based on the dividing line. That is, if the current block is divided into two or more regions by geometric partitioning, corner blocks and intermediate blocks to be used as merge candidate blocks can be selected by considering the position and direction of the dividing line.

[0142] In one embodiment, when a dividing line touches the left or upper boundary line of the current block, the number of intermediate blocks adjacent to the touching boundary line may be greater than the number of intermediate blocks adjacent to the non-touching boundary line.

[0143] In one embodiment, intermediate blocks adjacent to the boundary line where the dividing line meets may include intermediate blocks adjacent to the contact point or having the contact point as a corner. That is, the position of the intermediate blocks can be determined based on the dividing line.

[0144] In one embodiment, the use of an intermediate block can be determined by considering the number of horizontal and vertical pixels of the current block based on the dividing line. For example, if the dividing line touches the upper boundary line of the current block, the use of an intermediate block on the upper boundary line can be determined based on the number of horizontal pixels, and if the dividing line touches the left boundary line, the use of an intermediate block on the left boundary line can be determined based on the number of vertical pixels. For example, if the dividing line touches the top-left corner, top-right corner, or bottom-left corner of the current block, the use of an intermediate block on the upper boundary line can be determined by considering the number of horizontal pixels, and the use of an intermediate block on the left boundary line can be determined by considering the number of vertical pixels.

[0145] FIG. 10 illustrates an example of the search order of surrounding adjacent intermediate blocks.

[0146] When constructing a merge list using adjacent intermediate blocks, only some of the blocks adjacent to the intermediate position on the boundary line of the current block can be selectively considered. That is, among multiple blocks adjacent to the intermediate position between corner blocks, only blocks in a specific direction can be selected, or merge candidate blocks can be determined by sequentially searching blocks according to a predefined priority.

[0147] In one embodiment, among the blocks adjacent to the middle position on the upper boundary line of the current block, only one of the block located to the right (right block) or the block located to the left (left block) can be considered as a merge candidate block.

[0148] In one embodiment, regarding the intermediate position on the upper boundary line, the right block may be searched first, and if that block does not have a valid motion vector or block vector, the left block may be additionally searched to construct a merge list. Conversely, merge candidate blocks may be determined by searching the left block first, and then searching the right block if that block does not have a valid motion vector or block vector.

[0149] In one embodiment, a similar method may be applied to blocks adjacent to the middle position on the left boundary line of the current block. For example, among the blocks adjacent to the middle position on the left boundary line, only one of the upper block (upper block) or the lower block (lower block) may be considered as a merge candidate block.

[0150] In one embodiment, the upper block may be searched first, and if that block does not have a valid motion vector or block vector, the lower block may be additionally searched to construct the merge list. Conversely, merge candidate blocks may be determined by searching the lower block first, and then searching the upper block if that block does not have a valid motion vector or block vector.

[0151] FIG. 11 illustrates an example of the search order of surrounding adjacent intermediate blocks.

[0152] When considering multiple adjacent intermediate blocks to construct a merge list, the block closest to the intermediate position on the boundary line of the current block can be configured to be searched first. That is, among multiple intermediate blocks located between corner blocks, the block closest to the intermediate position on the boundary line of the current block is searched first, and then the remaining intermediate blocks are additionally searched to construct a merge list.

[0153] In one embodiment, when considering three or more intermediate blocks located between corner blocks on the upper boundary line of the current block, the block closest to the middle position of the upper boundary line is searched first, and then the remaining intermediate blocks located on both sides of the block are searched sequentially to form a merge list.

[0154] In one embodiment, the same method can be applied even when considering three or more intermediate blocks located between corner blocks on the left boundary line of the current block. For example, the block closest to the middle position of the left boundary line can be searched first, and then the remaining intermediate blocks located in the upper or lower direction relative to that block can be additionally searched to form a merge list.

[0155] FIG. 12 illustrates one embodiment of a surrounding adjacent intermediate block for all sides.

[0156] In one embodiment, even when using adjacent blocks based on an already encoded or decoded block rather than the current block during the process of constructing a merge list, blocks adjacent to all boundaries of the said block may be used as merge candidate blocks. For example, an already compressed block may be set as the reference block, and a merge list may be constructed by searching for blocks adjacent to the upper, lower, left, and right boundaries of the reference block as candidate blocks. In this case as well, the aforementioned method may be applied in the same way regarding the selection method, search order, usage, or determination of the number of blocks adjacent to intermediate positions.

[0157] In one embodiment, when constructing a merge list, if one or more of the left, top, right, or bottom boundaries of a block can be used based on an arbitrary block, the list can be constructed by selecting blocks adjacent to the available boundaries as merge candidate blocks. For example, if a specific boundary is determined to be referenceable, the method described above can be applied to adjacent blocks located on that boundary to determine the selection of intermediate position blocks, the search order, or the number of considerations.

[0158] A new merge candidate can be generated using the motion vectors or block vectors of multiple candidate blocks included in an inter-merge list or an intra-merge list. Specifically, the vectors corresponding to N candidate blocks included in the merge list can be weighted averaged to set the vector of the new candidate. Here, N can be 2, 4, or a natural number greater than or equal to 4.

[0159] In one embodiment, when using two candidate blocks included in a merge list, the block vector of candidate 1 and the block vector of candidate 2 can be averaged and used as the block vector of a new candidate. In this case, the block vector of the new candidate can be calculated as follows.

[0160] (Candidate 1's block vector + Candidate 2's block vector) >> 1

[0161] In one embodiment, when using four candidate blocks included in a merge list, the block vectors of candidate 1, candidate 2, candidate 3, and candidate 4 can be averaged and used as the block vector of a new candidate, and in this case, the block vector of the new candidate can be calculated as follows.

[0162] (Candidate 1's block vector + Candidate 2's block vector + Candidate 3's block vector + Candidate 4's block vector) >> 2

[0163] In one embodiment, a new candidate can be generated by performing a weighted average using the motion vectors or block vectors of N or more candidate blocks included in a merge list. In this case, the average calculation can be performed by selecting only the candidate blocks having the same reference index. For example, if the number of candidate blocks having the same reference index among the candidate blocks included in the merge list is N or more, a new candidate vector can be generated by performing a weighted average using the motion vectors or block vectors of said candidate blocks.

[0164] In one embodiment, when the number of candidate blocks having the same reference index is less than N, the average calculation can be performed in various ways as follows.

[0165] For example, a weighted average can be performed using only candidate blocks that have the same reference index.

[0166] As another example, the reference index of the missing candidate can be set to the same value as the reference index of the candidate blocks that have the same reference index, and the motion vector or block vector corresponding to the missing candidate can be set to 0 and then the average calculation can be performed.

[0167] As another example, the reference index of the missing candidate may be set to the same value as the reference index of the candidate blocks having the same reference index, and the motion vector or block vector corresponding to the missing candidate may be set to the average vector, median vector, or arbitrary vector (predefined vector) of the remaining candidate vectors, and then the average calculation may be performed.

[0168] As another example, the reference index of a deficient candidate can be set to the same value as the reference indices of candidate blocks having the same reference index, and the motion vector or block vector corresponding to the deficient candidate can be scaled and used for the average calculation.

[0169] In the present disclosure, a deficient candidate may refer to a candidate that requires additional supplementation because it does not satisfy the number of candidates to be used for the average calculation, when the number of candidate blocks having the same reference index is actually less than the number of candidates (N) set to be used for the average calculation.

[0170] In one embodiment, a new candidate based on a weighted average can be generated using motion vectors or block vectors of N or more candidate blocks included in an inter-merged list or an intra-merged list. In this case, if the number of candidate blocks having the same reference index among the candidate blocks used for the average calculation is less than N, the average calculation can be performed using vectors equal to the number of candidate blocks less than N.

[0171] For example, if the number of candidate blocks having the same reference index is less than N, a new candidate motion vector or block vector can be generated by performing a weighted average using only the candidate blocks.

[0172] As a specific example, when using the motion vectors of candidate 1 and candidate 2, the motion vector of candidate 1 and the motion vector of candidate 2 can be averaged to use as the motion vector of a new candidate, and in this case, the motion vector of the new candidate can be calculated as follows.

[0173] (Candidate 1 motion vector + Candidate 2 motion vector) >> 1

[0174] As another specific example, when using the block vectors of candidate 1, candidate 2, candidate 3, and candidate 4, the block vectors of the candidate blocks can be averaged and used as the block vector of a new candidate, and in this case, the block vector of the new candidate can be calculated as follows.

[0175] (Candidate 1's block vector + Candidate 2's block vector + Candidate 3's block vector + Candidate 4's block vector) >> 2

[0176] FIG. 13 illustrates an example of a video encoding method that predicts by constructing a merged list.

[0177] In one embodiment, the encoding device may construct a merge list using blocks located around the current block (S1310). The surrounding blocks may include corner blocks adjacent to the current block, intermediate location blocks located between corner blocks, or reference blocks that correspond in time. The encoding device may generate a merge candidate vector using the motion vectors or block vectors of the candidate blocks included in the merge list (S1320). Subsequently, the encoding device may generate a prediction block for the current block using the selected candidate vector (S1330). The encoding device may generate a residual based on the difference between the original block and the prediction block, and generate a bitstream by encoding the residual (S1340).

[0178] FIG. 14 illustrates an example of an image decoding method that predicts by constructing a merge list.

[0179] In one embodiment, the decoding device may receive a bitstream (S1410). The decoding device may construct a merge list using blocks located around the current block (S1420). The merge list may include motion vectors or block vectors obtained from surrounding blocks as candidate vectors. The decoding device may generate a merge candidate vector using the motion vectors or block vectors of the candidate blocks included in the merge list (S1430). Subsequently, the decoding device may generate a prediction block for the current block using the selected candidate vector (S1440). The decoding device may recover residual information from the bitstream and recover the current block using the residuals and the prediction block (S1450).

[0180] In the video encoding / decoding process, various transformation techniques may be used to efficiently perform quantization by converting a signal in the spatial domain to another space, such as the frequency domain. To improve this transformation efficiency, a Multiple Transform Set Selection (MTSS) method may be used, which comprises multiple transformation techniques or transformation kernels (matrices) into multiple sets and selectively uses one of these sets.

[0181] In one embodiment, the MTSS method may adaptively select a candidate transformation technique, a set of candidate transformation kernels, or a candidate transformation kernel to be used in the transformation step based on the Intra Prediction Mode (IPM) used for the current block.

[0182] For example, when a Neural Network Based Intra Prediction Mode (IntraNN) is used, a Non-Separable Primary Transform (NSPT) can be applied, and in this case, a subset of a set of multiple candidate transformation kernels can be selected and used. Additionally, the transformation kernel to be applied within the selected set can be determined using an IPM inferred through a Decoder-side Intra Mode Derivation (DIMD) method.

[0183] For example, to identify the set finally selected from a set of multiple candidate transformations, relevant information can be included in the bitstream in the form of a flag and transmitted.

[0184] In order to determine the transformation candidates, the Intra Prediction Mode (IPM) inferred, or the IPM inferred through the Decoder Side Intra Mode Derivation (DIMD), the Intra Template Matching Prediction Mode (IntraTMP), the Template-based Intra Mode Derivation (TIMD), or the Histogram of Gradients (HoG) based method, etc., can be configured as candidate IPMs, and then a small number of IPMs to be finally used among the above candidate IPMs can be determined through an additional process.

[0185] In one embodiment, a prediction block corresponding to each candidate IPM is generated, and a small number of IPMs with small difference values ​​are selected by comparing the difference between the prediction block and the reference block pointed to by a block vector associated with any reference block or the current block, and a transformation candidate can be determined using the selected IPMs.

[0186] In one embodiment, when applying a non-separable primary transform (NSPT) to a neural network-based intra-prediction mode (IntraNN), a small number of IPMs with small difference values ​​are selected by comparing the difference between the prediction block corresponding to each candidate IPM and the prediction block generated by the IntraNN, and the selected IPMs can be used to determine transformation candidates.

[0187] In one embodiment, when applying NSPT to IntraNN, a general IPM (Regular IPM), DIMD, IntraTMP, TIMD, and IPMs inferred by a HoG-based method are made to compete against each other, and a small number of IPMs selected according to the competition result can be used to determine transformation candidates.

[0188] In one embodiment, when applying NSPT to IntraNN, the IPM calculated by applying DIMD or HoG to the predicted pixels predicted by IntraNN and the PLANAR mode can be used as the inferred IPM.

[0189] In one embodiment, when applying NSPT to IntraNN, the gradient direction is calculated for pixels within a reference region or predicted pixels predicted by IntraNN, and the IPMs corresponding to the two largest gradients can be used as the inferred first IPM and second IPM.

[0190] In one embodiment, when NSPT is applied to IntraNN, among a plurality of IPMs calculated by applying DIMD or HoG to the predicted pixels predicted by IntraNN, the first IPM and the second IPM can be used as the inferred IPM.

[0191] In one embodiment, the IPM selected / inferred as above can be utilized as a candidate mode used in the transformation set selection or transformation kernel determination process.

[0192] To determine transformation candidates, IPMs inferred from DIMD, IntraTMP, TIMD, HOG, etc., can be configured as candidate IPMs.

[0193] In one embodiment, when applying NSPT to IntraNN, a candidate set or candidate kernel can be determined using N IPM modes inferred as DIMD, IntraTMP, or TIMD for a template or prediction block.

[0194] In one embodiment, when applying NSPT to IntraNN, a candidate set or candidate kernel can be determined using two IPM modes and a Planar mode inferred by DIMD for a template or prediction block.

[0195] In one embodiment, when applying NSPT to IntraNN, the IPM mode stored in the prediction step, such as DIMD, IntraTMP, TIMD, and HOG, can be reused as is or additional correction can be performed to determine the candidate set or candidate kernel.

[0196] In one embodiment, when the IPM modes inferred by DIMD are similar to each other, the candidate set or candidate kernel can be determined by using IPM modes inferred by other prediction methods such as IntraTMP or TIMD, without using similar IPMs except for one IPM.

[0197] In one embodiment, when applying NSPT to IntraNN, a candidate set or candidate kernel can be determined using the difference between the inferred first IPM and the second IPM.

[0198] In one embodiment, when NSPT is applied to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, the second IPM can be replaced with another mode to determine the candidate set or candidate kernel.

[0199] In one embodiment, when NSPT is applied to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, a candidate set or candidate kernel can be determined using only the first IPM.

[0200] In one embodiment, when NSPT is applied to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, the second mode can be set to PLANAR mode or DC mode to determine the candidate kernel.

[0201] In one embodiment, when NSPT is applied to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, candidate kernels can be determined from two different sets of candidates based on the first IPM.

[0202] In one embodiment, when applying NSPT to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, candidate kernels can be determined from two different sets of candidates based on the second IPM.

[0203] In one embodiment, when applying NSPT to IntraNN, one candidate kernel among three candidate sets can be determined by utilizing the first inferred IPM, and one candidate kernel among another candidate set can be determined by utilizing the second inferred IPM.

[0204] In one embodiment, when applying NSPT to IntraNN, if the difference between the inferred first IPM and the second IPM is 2 or less, one candidate kernel among three candidate sets can be determined by utilizing the inferred first IPM, and one candidate kernel among another candidate set can be determined by reusing the inferred first IPM.

[0205] In one embodiment, when applying NSPT to IntraNN, if the difference between the inferred first IPM and the second IPM is less than or equal to K, one candidate kernel among three candidate sets can be determined by utilizing the inferred first IPM, and a candidate kernel among another candidate set can be determined by utilizing the inferred first IPM or the replaced IPM.

[0206] In one embodiment, the difference between the first IPM and the second IPM can be determined based on the difference in IPM index values.

[0207] In one embodiment, the K value may be set to a different value depending on the block size or prediction mode.

[0208] In one embodiment, the candidate set or candidate kernel may include a transformation set or transformation kernel selected from a plurality of transformation candidates.

[0209] FIG. 15 illustrates an example of in-frame mode prediction based on a reference vector.

[0210] The current block can be predicted using a block-based intra-mode prediction method (BVIMP), which performs intra-frame prediction by comparing a reference block pointed to by a block vector inferred from neighboring blocks with a prediction block generated by IPM. In this case, one of multiple transformation techniques, sets, or kernels can be adaptively selected and used.

[0211] In one embodiment, when BVIMP-based prediction is performed, the same set of transformation candidates or transformation kernels as those used in general IPM may be used.

[0212] In one embodiment, when BVIMP-based prediction is performed, a different set of transformation candidates or transformation kernels from the set of transformation candidates or transformation kernels used in general IPM may be used.

[0213] In one embodiment, when BVIMP-based prediction is performed, the same transformation candidate set or transformation kernel as the candidate set or candidate kernel used in DIMD or IntraNN may be used.

[0214] In one embodiment, when BVIMP-based prediction is performed, a different set of candidate candidates or a different set of candidate kernels from those used in DIMD or IntraNN may be used.

[0215] In one embodiment, the selection of the transformation candidate set or transformation kernel may be determined according to the characteristics of the prediction block generated by BVIMP-based prediction.

[0216] In one embodiment, when multiple transformation techniques, sets, or kernels are adaptively selected and used after prediction with BVIMP, the final candidate can be determined using the IPM mode used in the BVIMP process.

[0217] In one embodiment, a plurality of IPM modes used in the BVIMP process are each set as the first IPM and the second IPM, and a plurality of final candidates can be determined using the said IPM modes.

[0218] In one embodiment, the final candidate can be determined using an IPM mode inferred by DIMD for a template or prediction block generated during the BVIMP process.

[0219]

[0220] In one embodiment, a plurality of IPM modes inferred by DIMD are each set as the first IPM and the second IPM, and a plurality of final candidates can be determined using the IPM modes.

[0221] In one embodiment, one or more final candidates can be determined by using both the IPM mode used in the BVIMP process and the IPM mode inferred by DIMD for the template or prediction block generated in the BVIMP process.

[0222] In one embodiment, the final candidate may include a candidate selected from a set of transformation candidates or a transformation kernel.

[0223] If three transformation candidate techniques, a set of candidate kernels, or candidate kernels exist, the first flag can be used to distinguish between the first and second candidates, and the second flag can be used to distinguish between the third candidate.

[0224] In one embodiment, depending on the type of applied prediction technique (mode), the type or number of inferred candidate modes, prediction and transformation information of surrounding blocks, the existence of a third candidate mode, or the result of the first flag, the second flag may be selectively transmitted, or an index may be transmitted to distinguish the three candidates.

[0225] In one embodiment, if there are two candidates or the final candidate is determined by the first flag, the second flag may not be transmitted.

[0226] In one embodiment, when IntraNN prediction is used, both the first flag and the second flag can be transmitted.

[0227] In one embodiment, the first flag and the second flag may be transmitted using different Contexts, or only the first flag may be transmitted using a Context.

[0228] In one embodiment, the Context may be determined based on the compression information of surrounding blocks.

[0229] In one embodiment, if there are N candidates, the candidates can be distinguished by transmitting (N-1) flags or candidate indices.

[0230] FIG. 16 illustrates an example of an image encoding method based on the selection of an adaptive conversion technique.

[0231] The encoding device can determine a candidate IPM for the current block (S1610). In one embodiment, the candidate IPM may include an IPM inferred using DIMD, IntraTMP, TIMD, or HoG, and may be determined by reusing an IPM used in the prediction step or by selecting some of a plurality of IPMs. Next, the encoding device can determine a transformation candidate set or a candidate kernel using the candidate IPM (S1620). In one embodiment, when applying NSPT to an IntraNN, the candidate set or a candidate kernel may be determined using the inferred first IPM and second IPM, and the candidate set or a candidate kernel may be determined according to the difference between the IPMs. Next, the encoding device can select one candidate from a plurality of transformation candidate sets or candidate kernels (S1630). Next, the encoding device can perform transformation and quantization on the current block using the selected transformation candidate set or candidate kernel (S1640). Finally, the encoding device can transmit information for identifying the selected transformation candidate as a bitstream (S1650). In one embodiment, when there are three transformation candidates, the first flag can be used to distinguish between the first and second candidates, and the second flag can be used to distinguish between the third candidate.

[0232] FIG. 17 illustrates an example of an image decoding method based on the selection of an adaptive transformation technique.

[0233] The decoding device can receive a bitstream (S1710). Next, the decoding device can determine a candidate IPM for the current block (S1720). In one embodiment, the candidate IPM may include an IPM inferred using DIMD, IntraTMP, TIMD, or HoG, and may be determined by selecting or reusing some of a plurality of IPMs. Next, the decoding device can determine a transformation candidate set or a candidate kernel using the candidate IPM (S1730). In one embodiment, when applying NSPT to IntraNN, the candidate set or a candidate kernel may be determined using the inferred first IPM and second IPM, and the candidate set or a candidate kernel may be determined according to the difference between the IPMs. Next, the decoding device can select one of the transformation candidate set or a candidate kernel using transformation candidate selection information received from the bitstream (S1740). Finally, the decoding device can perform an inverse transformation using the selected transformation candidate set or a candidate kernel and restore the current block (S1750).

[0234] The exemplary methods of the present disclosure are described as a series of operations for clarity of description, but this is not intended to limit the order in which the steps are performed, and if necessary, each step may be performed simultaneously or in a different order. To implement the method according to the present disclosure, additional steps may be included in addition to the steps exemplified, steps excluding some steps and including the remaining steps, or steps excluding some steps and including additional steps.

[0235] The various embodiments of the present disclosure are not intended to list all possible combinations but to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0236] In addition, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, it may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0237] The scope of the present disclosure includes software or machine-executable instructions (e.g., operating system, application, firmware, program, etc.) that enable an operation according to a method of various embodiments to be executed on a device or computer, and a non-transitory computer-readable medium on which such software or instructions, etc. are stored and executable on a device or computer.

[0238] The present disclosure can be utilized in industrial fields utilizing image encoding / decoding methods, devices, and recording media.

Claims

In a video decoding method, A step of constructing a merge list for the prediction of the current block; and The method includes the step of generating a predicted block of the current block based on candidate vectors included in the merge list above, wherein The above merge list is constructed by obtaining motion vectors or block vectors from candidate blocks surrounding the current block, and An image decoding method characterized in that the above candidate block includes an intermediate position block located between the corner blocks of the above current block. In paragraph 1, A video decoding method characterized in that the above merge list is an inter-merged list or an intra-merged list. In paragraph 2, The above intermerging list includes motion vectors, and An image decoding method characterized in that the above intra-merging list includes a block vector. In paragraph 1, An image decoding method characterized in that the above intermediate position block includes at least one of a first intermediate position block located between corner blocks on the upper boundary line of the current block or a second intermediate position block located between corner blocks on the left boundary line of the current block. In paragraph 1, An image decoding method characterized in that the above candidate block does not include a first intermediate location block located between corner blocks on a first boundary line of the current block, but includes a second intermediate location block located between corner blocks on a second boundary line of the current block. In paragraph 5, The first boundary line is a boundary line adjacent to another block divided from the same parent block as the current block, and An image decoding method characterized in that the second boundary line is a boundary line not adjacent to the other block. In paragraph 1, An image decoding method characterized by determining whether to use the intermediate position block based on the size of the current block. In paragraph 1, An image decoding method characterized by searching for an intermediate location block for the above candidate block, wherein the search prioritizes searching for a block close to the intermediate location of the boundary line of the above current block. In paragraph 1, The search for an intermediate location block for the above candidate block is, An image decoding method characterized by first searching for a block located in a first direction based on the midpoint of the boundary line of the current block, and if the block located in the first direction does not have a valid motion vector or block vector, additionally searching for a block located in a second direction. In paragraph 1, An image decoding method characterized in that the above merge list further includes a new merge candidate vector generated using a plurality of candidate vectors included in the above merge list. In Paragraph 10, An image decoding method characterized in that the new merged candidate vector is generated as a weighted average of the plurality of candidate vectors. In Paragraph 11, If the number of candidate vectors having the same reference index within the above merge list is less than a predetermined number, An image decoding method characterized in that the above weighted average is performed using only candidate vectors having the same reference index. In Paragraph 11, If the number of candidate vectors having the same reference index within the merged list is less than the predetermined number, A method for decoding images, characterized in that vectors other than the candidate vectors having the same reference index used for the above-mentioned weighted average are set as an average vector, an intermediate vector, or a predefined vector. In a video encoding method, A step of constructing a merge list for the prediction of the current block; and The method includes the step of generating a predicted block of the current block based on candidate vectors included in the merge list above, wherein The above merge list is constructed by obtaining motion vectors or block vectors from candidate blocks surrounding the current block, and An image encoding method characterized in that the above candidate block includes an intermediate position block located between the corner blocks of the above current block. In a non-transient computer-readable recording medium storing a bitstream generated by a video encoding method, The above image encoding method is, A step of constructing a merge list for the prediction of the current block; and The method includes the step of generating a predicted block of the current block based on candidate vectors included in the merge list above, wherein The above merge list is constructed by obtaining motion vectors or block vectors from candidate blocks surrounding the current block, and A non-transient computer-readable recording medium characterized in that the above candidate block includes an intermediate position block located between the corner blocks of the above current block.

Images