Video encoding / decoding method and recording medium storing bit stream

Abstract

A video decoding method according to the present disclosure comprises the steps of: deriving block vector or motion information of a current block; deriving a reference block of the current block based on the motion information or the block vector; obtaining a weight parameter based on the current template around the current block and the reference template around the reference block; and generating a prediction block of the current block based on the weight parameter.

Description

Technical Field

[0001] This disclosure relates to methods and apparatus for processing video signals. Background Technology

[0002] Recently, there has been an increasing demand for high-resolution and high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, across various application areas. As image data becomes higher resolution and higher quality, the data volume increases relative to existing image data, leading to increased transmission and storage costs when transmitting or storing image data using existing wired and wireless broadband circuits or storage media. These problems arising from the increased resolution and quality of image data can be addressed using efficient image compression techniques.

[0003] There are various techniques, such as inter-frame prediction techniques that use image compression technology to predict the pixel values ​​included in the current image based on previous or subsequent images of the current image, intra-frame prediction techniques that use pixel information in the current image to predict the pixel values ​​included in the current image, and entropy coding techniques that assign short symbols to values ​​with high occurrence frequency and long symbols to values ​​with low occurrence frequency. Image data can be effectively compressed, transmitted, or stored by using these image compression techniques.

[0004] On the other hand, with the increasing demand for high-resolution images, the demand for stereoscopic image content as a new image service is also increasing. Video compression techniques for effectively providing high-resolution and ultra-high-resolution stereoscopic image content have been discussed. Summary of the Invention

[0005] Technical issues

[0006] This disclosure provides a method for deriving a predicted block by modifying a reference block derived through inter-frame prediction or intra-frame block copying, and an apparatus for using the method.

[0007] This disclosure provides a method for deriving weight parameters for modifying a reference block based on information from a pre-decoded region, and an apparatus for using the method.

[0008] This disclosure provides a method for deriving a prediction block using multiple reference blocks, and an apparatus for using the method.

[0009] The technical effects of this disclosure are not limited to those mentioned above, and those skilled in the art to which this disclosure pertains can clearly understand other unmentioned technical effects based on the following description.

[0010] Technical solution

[0011] The video decoding method according to this disclosure may include: obtaining the block vector or motion information of the current block; obtaining a reference block of the current block based on the motion information or block vector; obtaining weight parameters based on the current template around the current block and the reference template around the reference block; and generating a predicted block of the current block based on the weight parameters.

[0012] The video coding method according to this disclosure may include: deriving the block vector or motion information of the current block; deriving a reference block of the current block based on the motion information or block vector; deriving weight parameters based on the current template around the current block and the reference template around the reference block; and generating a prediction block of the current block based on the weight parameters.

[0013] In the video encoding / decoding method according to embodiments of the present disclosure, the weight parameters may include filter coefficients, and when the filter is applied to a reference sample within a reference template, the filter coefficients can minimize the difference with the reference sample within the current template.

[0014] In the video encoding / decoding method according to the embodiments of this disclosure, the predicted sample of the current block can be obtained by modifying the sample in the reference block based on the weight parameter.

[0015] In the video encoding / decoding method according to the embodiments of the present disclosure, modified samples within a reference block can be obtained by filtering the samples using adjacent samples adjacent to the samples.

[0016] In the video encoding / decoding method according to the embodiments of the present disclosure, when a neighboring sample adjacent to the sample is unavailable, the value of the neighboring sample can be set to be the same as the value of the sample.

[0017] In the video encoding / decoding method according to the embodiments of the present disclosure, when a neighboring sample adjacent to a sample is located outside a reference block, the value of the neighboring sample is set to be equal to that of the reconstructed sample at the corresponding location.

[0018] In the video encoding / decoding method according to the embodiments of this disclosure, the filter applied to the sample may be a cross-shaped filter with the same width and height.

[0019] In the video encoding / decoding method according to embodiments of the present disclosure, the shape of the filter applied to the sample can be adaptively determined according to the position of the sample.

[0020] According to this disclosure, a computer-readable recording medium may be provided for storing a bitstream generated by an image encoding method.

[0021] The features briefly summarized above regarding this disclosure are merely exemplary aspects of the detailed description of this disclosure described below, and do not limit the scope of this disclosure.

[0022] Technical effect

[0023] According to this disclosure, prediction accuracy can be improved by modifying the reference block obtained through inter-frame prediction or intra-frame block copying to obtain the prediction block.

[0024] According to this disclosure, weight parameters for modifying reference blocks can be derived based on information from pre-decoded regions, thereby improving prediction accuracy without increasing signaling overhead.

[0025] According to this disclosure, prediction accuracy can be improved by using multiple reference blocks to derive prediction blocks.

[0026] The effects achievable through this disclosure are not limited to those described above, and other effects not described herein will be clearly understood by those skilled in the art based on the following description. Attached Figure Description

[0027] Figure 1 This is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

[0028] Figure 2 This is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

[0029] Figure 3 An intra-frame prediction method performed by an image encoding / decoding device according to this disclosure is shown.

[0030] Figure 4 and Figure 5 Examples of several intra-frame prediction modes based on this disclosure are shown.

[0031] Figure 6 An intra-frame prediction method based on planar patterns according to this disclosure is shown.

[0032] Figure 7 An intra-frame prediction method based on DC mode according to this disclosure is shown.

[0033] Figure 8 An intra-frame prediction method based on directional patterns according to this disclosure is shown.

[0034] Figure 9 A method for deriving samples at fractional positions is shown.

[0035] Figure 10 and Figure 11The diagram shows the tangent of the angle scaled by a factor of 32 according to the intra-frame prediction mode.

[0036] Figure 12 This is a diagram showing the intra-frame prediction aspect when the orientation mode is one of mode numbers 34 to 49.

[0037] Figure 13 This is a diagram used to describe an example of generating a top reference sample by interpolating the left reference sample.

[0038] Figure 14 An example is shown in which intra-frame prediction is performed using reference samples arranged in a 1D array.

[0039] Figure 15 This is a diagram that schematically illustrates the process of performing inter-frame prediction in the encoder and decoder.

[0040] Figure 16 An example of performing motion estimation within it is shown.

[0041] Figure 17 and Figure 18 An example is shown in which a predicted block for the current block is generated based on motion information generated through motion estimation.

[0042] Figure 19 The reference location used to derive the predicted motion vector values ​​is shown.

[0043] Figure 20 This is a diagram used to describe template-based motion estimation methods.

[0044] Figure 21 An example of configuring a template is shown.

[0045] Figure 22 This is a graph used to describe motion estimation methods based on bilateral matching.

[0046] Figure 23 This is a graph used to describe motion estimation methods based on one-sided matching.

[0047] Figure 24 and Figure 25 This is a diagram showing an example of how a prediction block is derived based on the accuracy of the motion vector.

[0048] Figure 26 This is a flowchart of the intra-frame block copying method performed by an image encoding / decoding device.

[0049] Figure 27 This is used to describe an example in which a block vector is derived by using template matching.

[0050] Figure 28This represents an example where the predicted block for the current block is derived using a block vector.

[0051] Figure 29 The procedure for deriving the modified reference block is shown.

[0052] Figure 30 The configuration of the current template and the reference template is shown.

[0053] Figure 31 The convolution filter applied to the reference sample is shown.

[0054] Figure 32 This represents an example where padding is performed on unavailable samples.

[0055] Figure 33 This represents an example where values ​​for samples adjacent to a reference block are generated by padding.

[0056] Figure 34 This represents an example in which padding is performed only on sample locations outside the reference block that are not part of the reference template.

[0057] Figure 35 This is a diagram used to describe an example in which the predicted block of the current block is obtained by using multiple reference blocks. Detailed Implementation

[0058] Because this disclosure can be modified and has several embodiments, specific embodiments will be illustrated and described in detail in the accompanying drawings. However, this is not intended to limit this disclosure to the specific embodiments, and it should be understood that this disclosure includes all variations, equivalents, or substitutions that are included within the concept and scope of this disclosure. In describing each drawing, similar reference numerals are used for similar parts.

[0059] Various components may be described using terms such as first, second, etc., but these components should not be limited by the terms. Terms are used only to distinguish one component from others. For example, without departing from the scope of the rights of this disclosure, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component. The term "and / or" includes a combination of multiple related inputs or any single item of multiple related inputs.

[0060] When a component is referred to as "linked" or "connected" to other components, it should be understood that the component may be directly linked or connected to the other component, but there may also be other components in between. On the other hand, when a component is referred to as "directly linked" or "directly connected" to other components, it should be understood that there are no other components in between.

[0061] Because the terminology used in this application is only for describing specific embodiments, it is not intended to limit the scope of this disclosure. Singular expressions include plural expressions unless the singular expression clearly has a different meaning in the context. In this application, it should be understood that terms such as "comprising" or "having" refer to the presence of a characteristic, number, step, movement, component, part, or combination thereof entered in the specification, but do not preclude the possibility of adding or having one or more other characteristics, numbers, steps, movements, components, parts, or combinations thereof.

[0062] In the following, with reference to the accompanying drawings, the desired embodiments of the present disclosure will be described in more detail. In the following drawings, the same reference numerals are used for the same parts, and repeated descriptions of the same parts are omitted.

[0063] Figure 1 This is a block diagram illustrating an image encoding apparatus according to an embodiment of the present disclosure.

[0064] Reference Figure 1 The image encoding device 100 may include an image segmentation unit 110, prediction units 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, a dequantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0065] Although Figure 1 Each building block shown is illustrated independently to represent a different feature function in the image encoding device, but this does not mean that each building block consists of a single hardware or software unit. That is, since each building block is included by listing each building block for ease of description, at least two building blocks of each building block can be combined to form a building block, or a building block can be divided into multiple building blocks to perform functions, and even integrated and individual implementations of the building blocks are included within the scope of the claims of this disclosure as long as they do not depart from the essence of this disclosure.

[0066] Furthermore, some components may be optional components used only to improve performance, rather than essential components for performing the basic functions of this disclosure. This disclosure can be implemented by including only the building blocks necessary to achieve the essence of this disclosure and excluding components used only to improve performance, and structures that include only essential components and exclude optional components used only to improve performance are also included within the scope of the claims of this disclosure.

[0067] Image segmentation unit 110 can segment an input image into at least one processing unit. In this case, the processing unit can be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). In image segmentation unit 110, an image can be segmented into a combination of multiple coding units, prediction units, and transform units, and the image can be encoded by selecting a combination of coding units, prediction units, and transform units according to a predetermined criterion (e.g., a cost function).

[0068] For example, an image can be segmented into multiple coding units. To segment coding units in an image, a recursive tree structure such as a quadtree, ternary tree, or binary tree can be used, and a coding unit that is segmented into other coding units by using an image or the largest coding unit as the root can have as many child nodes as the number of coding units it was segmented into. Coding units that are no longer segmented according to specific constraints are called leaf nodes. As an example, assuming quadtree segmentation is applied to a coding unit, a coding unit can be segmented into at most four other coding units.

[0069] In the following embodiments of this disclosure, the encoding unit may be used as a unit for encoding or as a unit for decoding.

[0070] A prediction unit may be segmented in a coding unit into at least one square or rectangular shape of the same size, or may be segmented such that any prediction unit segmented in a coding unit may have a shape and / or size different from the other prediction units.

[0071] In intra-frame prediction, the transform unit can be set to be the same as the prediction unit. In this case, after the coding unit is divided into multiple transform units, intra-frame prediction can be performed for each transform unit. The coding unit can be divided in the horizontal or vertical direction. Depending on the size of the coding unit, the number of transform units generated by dividing the coding unit can be 2 or 4.

[0072] Prediction units 120 and 125 may include an inter-frame prediction unit 120 performing inter-frame prediction and an intra-frame prediction unit 125 performing intra-frame prediction. It can be determined whether inter-frame or intra-frame prediction is performed for the coding unit, and detailed information based on each prediction method (e.g., intra-frame prediction mode, motion vector, reference image, etc.) can be determined. In this case, the processing unit performing the prediction may be different from the processing unit that determines the prediction method and specific content. For example, the prediction method, prediction mode, etc., may be determined in the coding unit, and the prediction may be performed in the prediction unit or the transform unit. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Furthermore, prediction mode information, motion vector information, etc., used for prediction can be encoded together with the residual value in the entropy coding unit 165 and can be sent to the decoding device. When using a specific coding mode, the original block can be encoded as is and sent to the decoding unit without generating a prediction block through prediction units 120 or 125.

[0073] The inter-frame prediction unit 120 may predict prediction units based on information about at least one of the previous or subsequent images of the current image, or in some cases, based on information about some coded regions in the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0074] The reference image interpolation unit can receive reference image information from memory 155 and generate pixel information equal to or less than integer pixels in the reference image. For luminance pixels, an 8-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information equal to or less than integer pixels in 1 / 4 pixel units. For chrominance signals, a 4-tap DCT-based interpolation filter with different filter coefficients can be used to generate pixel information equal to or less than integer pixels in 1 / 8 pixel units.

[0075] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as FBMA (Bulk Matching Algorithm Based on Full Search), TSS (Three-Step Search), and NTS (New Three-Step Search). Based on the interpolated pixels, the motion vector can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels. The motion prediction unit can predict the current prediction unit by changing the motion prediction method. Various methods can be used as motion prediction methods, such as skipping methods, merging methods, Advanced Motion Vector Prediction (AMVP) methods, and intra-block copying methods.

[0076] Intra-frame prediction unit 125 can generate prediction units based on reference pixel information, which serves as pixel information in the current image. Reference pixel information can be derived from one of a plurality of reference pixel lines. The Nth reference pixel line among the plurality of reference pixel lines may include a left pixel whose x-axis difference from the top-left pixel in the current block is N, and a top pixel whose y-axis difference from the top-left pixel is N. The number of reference pixel lines that can be selected from the current block can be 1, 2, 3, or 4.

[0077] When a neighboring block in the current prediction unit is a block performing inter-frame prediction and a corresponding reference pixel is a pixel performing inter-frame prediction, the reference pixel included in the block performing inter-frame prediction can be used by replacing it with the reference pixel information of the neighboring block performing intra-frame prediction. In other words, when a reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one piece of information from the available reference pixels.

[0078] Intra-frame prediction can have a directional prediction mode that uses reference pixel information based on the prediction direction when performing prediction, and a non-directional mode that does not use directional information. The mode used to predict luminance information can be different from the mode used to predict chrominance information, and chrominance information can be predicted using intra-frame prediction mode information used to predict luminance information or predicted luminance signal information.

[0079] When performing intra-frame prediction, if the size of the prediction unit is the same as the size of the transform unit, the intra-frame prediction of the prediction unit can be performed based on the pixels at the left, top left, and top positions of the prediction unit.

[0080] Intra-frame prediction methods generate prediction blocks after applying a smoothing filter to a reference pixel based on the prediction mode. Whether to apply the smoothing filter depends on the selected reference pixel line.

[0081] To perform intra-prediction, the intra-prediction mode of the current prediction unit can be predicted based on the intra-prediction modes of the surrounding prediction units. When predicting the prediction mode of the current prediction unit using mode information predicted by the surrounding prediction units, if the intra-prediction mode of the current prediction unit is the same as that of the surrounding prediction units, information indicating that the prediction mode of the current prediction unit is the same as that of the surrounding prediction units can be sent using predetermined flag information. If the prediction mode of the current prediction unit is different from that of the surrounding prediction units, the prediction mode information of the current block can be encoded by performing entropy coding.

[0082] Furthermore, residual blocks can be generated that include information about residual values, which are the differences between the original blocks in the prediction units that perform predictions based on the prediction units generated in prediction units 120 and 125. The generated residual blocks can be input to the transformation unit 130.

[0083] Transform unit 130 can transform the original block and the residual block, including residual value information from the prediction units generated by prediction units 120 and 125, using transformation methods such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on at least one of the size of the transform unit, the shape of the transform unit, the prediction mode in the prediction unit, or the intra-frame prediction mode information in the prediction unit.

[0084] The quantization unit 135 can quantize the values ​​transformed to the frequency domain in the transform unit 130. The quantization coefficients can be changed according to the importance or blocks of the image. The values ​​calculated in the quantization unit 135 can be provided to the dequantization unit 140 and the rearrangement unit 160.

[0085] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0086] The rearrangement unit 160 can transform coefficients in the shape of two-dimensional blocks into the shape of one-dimensional vectors using a coefficient scanning method. For example, the rearrangement unit 160 can scan DC coefficients into coefficients in the high-frequency domain using a zigzag scanning method and transform them into the shape of one-dimensional vectors. Depending on the size of the transform unit and the intra-frame prediction mode, instead of zigzag scanning, a vertical scan of coefficients in the shape of two-dimensional blocks along the column direction, a horizontal scan of coefficients in the shape of two-dimensional blocks along the row direction, or a diagonal scan of coefficients in the shape of two-dimensional blocks along the diagonal direction can be used. In other words, the choice of which scanning method—zigzag scanning, vertical scanning, horizontal scanning, or diagonal scanning—to use can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0087] Entropy coding unit 165 can perform entropy coding based on the value calculated by rearrangement unit 160. For example, entropy coding can use various coding methods such as Exponential Golomb, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding).

[0088] Entropy coding unit 165 can encode various information from rearrangement unit 160 and prediction units 120 and 125, such as residual coefficient information and block type information in coding unit, prediction mode information, segmentation unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0089] Entropy coding unit 165 can perform entropy coding on the coefficient values ​​in the coding unit input from rearrangement unit 160.

[0090] Dequantization unit 140 and inverse transform unit 145 dequantize the values ​​quantized in quantization unit 135 and perform inverse transform on the values ​​transformed in transform unit 130. The residual values ​​generated by dequantization unit 140 and inverse transform unit 145 can be combined with prediction units predicted by motion prediction units, motion compensation units, and intra-frame prediction units included in prediction units 120 and 125 to generate reconstructed blocks.

[0091] The filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF).

[0092] Deblocking filters remove block distortion caused by boundaries between blocks in a reconstructed image. To determine whether to perform deblocking, the number of pixels included in a block's rows or columns can be used to decide whether to apply a deblocking filter to the current block. When applying a deblocking filter to a block, a strong or weak filter can be applied depending on the desired deblocking intensity. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be configured to be processed in parallel.

[0093] The offset correction unit can correct the offset from the original image on a pixel-by-pixel basis for the image undergoing deblocking. To perform offset correction on a specific image, the region to be offset can be determined after dividing the pixels in the image into a certain number of regions, and the offset can be applied to the corresponding region or by considering the edge information of each pixel.

[0094] Adaptive Loop Filtering (ALF) can be performed based on values ​​obtained by comparing the filtered reconstructed image with the original image. After dividing the pixels in the image into predetermined groups, filtering can be performed group-by-group by determining a filter to be applied to the corresponding group. Information regarding whether to apply ALF can be transmitted per coding unit (CU) for the luminance signal, and the shape and filter coefficients of the ALF filter to be applied can vary for each block. Furthermore, ALF filters of the same shape (fixed shape) can be applied regardless of the characteristics of the block to which the filter is applied.

[0095] The memory 155 can store the reconstructed blocks or images calculated by the filter unit 150, and the stored reconstructed blocks or images can be provided to the prediction units 120 and 125 when performing inter-frame prediction.

[0096] Figure 2 This is a block diagram illustrating an image decoding apparatus according to an embodiment of the present disclosure.

[0097] Reference Figure 2 The image decoding device 200 may include an entropy decoding unit 210, a rearrangement unit 215, a dequantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filter unit 240, and a memory 245.

[0098] When an image bitstream is input from an image encoding device, the input bitstream can be decoded according to the reverse process of the image encoding device.

[0099] The entropy decoding unit 210 can perform entropy decoding according to a process that is the reverse of the entropy encoding process performed in the entropy encoding unit of the image encoding device. For example, in response to the method performed in the image encoding device, various methods such as Exponential Columbus, CAVLC (Context Adaptive Variable Length Coding), and CABAC (Context Adaptive Binary Arithmetic Coding) can be applied.

[0100] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed in the encoding device.

[0101] The rearrangement unit 215 can perform rearrangement based on the method of rearranging the entropy-decoded bitstream in the entropy decoding unit 210 in the encoding unit. Coefficients expressed in one-dimensional vector form can be rearranged by reconstructing them into two-dimensional blocks. The rearrangement unit 215 can receive information related to the coefficient scan performed in the encoding unit and perform rearrangement by performing the scan in reverse order based on the scan order performed in the corresponding encoding unit.

[0102] The dequantization unit 220 can perform dequantization based on the quantization parameters provided from the encoding device and the coefficient values ​​of the rearranged block.

[0103] The inverse transform unit 225 can perform the transforms performed in the transform unit, namely, the inverse transforms of DCT, DST, and KLT, i.e., performing inverse DCT, inverse DST, and inverse KLT on the result of quantization performed in the image coding device. The inverse transform can be performed based on the transmission unit determined in the image coding device. In the inverse transform unit 225 of the image decoding device, transform techniques (e.g., DCT, DST, KLT) can be selectively performed based on multiple pieces of information (such as prediction method, current block size or shape, prediction mode, intra-frame prediction direction, etc.).

[0104] Prediction units 230 and 235 can generate prediction blocks based on information related to the generation of prediction blocks provided by entropy decoding unit 210 and pre-decoded block or image information provided by memory 245.

[0105] As described above, when intra-prediction is performed in the same manner as in an image coding device, and the size of the prediction unit is the same as the size of the transform unit, intra-prediction of the prediction unit can be performed based on the pixels at the left, upper left, and top positions of the prediction unit. However, when the size of the prediction unit differs from the size of the transform unit during intra-prediction, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using N×N segmentation can be used only for the smallest coding unit.

[0106] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit may receive various information input from the entropy decoding unit 210 (such as prediction unit information, prediction mode information of the intra-frame prediction method, motion prediction related information of the inter-frame prediction method, etc.), divide the prediction units in the current coding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 may perform inter-frame prediction for the current prediction unit by using information provided from the image coding device necessary for inter-frame prediction in the current prediction unit, based on information included in at least one of the previous or subsequent images of the current image including the current prediction unit. Alternatively, inter-frame prediction may be performed based on information about some pre-reconstructed regions in the current image including the current prediction unit.

[0107] To perform inter-frame prediction, the motion prediction method in the prediction units included in the corresponding coding unit can be determined based on the coding unit: skip mode, merge mode, AMVP mode, or intra-block copy mode.

[0108] Intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. When the prediction unit is a prediction unit that performs intra-prediction, intra-prediction can be performed based on intra-prediction mode information from the prediction unit provided by the image coding device. Intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. As part of filtering the reference pixels of the current block, the AIS filter can be applied by determining whether to apply a filter based on the prediction mode in the current prediction unit. By using the prediction mode and AIS filter information from the prediction unit provided by the image coding device, AIS filtering can be performed on the reference pixels of the current block. When the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0109] In the case where the prediction mode in the prediction unit is to perform intra-frame prediction based on pixel values ​​interpolated from reference pixels, the reference pixel interpolation unit can interpolate the reference pixels to generate reference pixels in units equal to or less than integer values. In the case where the prediction mode in the current prediction unit is to generate prediction blocks without interpolating reference pixels, interpolation of reference pixels is not required. When the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks through filtering.

[0110] The reconstructed block or image can be provided to filter unit 240. Filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0111] Information about whether to apply a deblocking filter to a corresponding block or image can be provided from the image encoding device, as well as information about whether a strong or weak filter is applied when applying the deblocking filter. The deblocking filter information provided by the image encoding device can be used in the image decoding device's deblocking filter, and deblocking filtering for the corresponding block can be performed in the image decoding device.

[0112] The offset correction unit can perform offset correction on the reconstructed image based on the type of offset correction applied to the image during encoding, offset value information, etc.

[0113] The ALF can be applied to the coding unit based on information provided by the coding device, such as whether the ALF is applied and ALF coefficient information. Such ALF information can be provided by including it in a specific parameter set.

[0114] The memory 245 can store reconstructed images or blocks for use as reference images or reference blocks, and provide the reconstructed images to the output unit.

[0115] As described above, in the following embodiments of this disclosure, for ease of description, the term "encoding unit" is used, but it can be a unit that performs both decoding and encoding.

[0116] Furthermore, since the current block represents the block to be encoded / decoded, it can represent a coding tree block (or coding tree unit), a coding block (or coding unit), a transform block (or transform unit), a prediction block (or prediction unit), a block to which a loop filter is applied, etc., depending on the encoding / decoding steps. In this specification, "unit" can refer to a basic unit used to perform a specific encoding / decoding process, and "block" can refer to a pixel array of a predetermined size. Unless otherwise categorized, "block" and "unit" are used interchangeably. For example, in the embodiments described later, it will be understood that coding block and coding unit are used interchangeably.

[0117] Furthermore, the image that includes the current block is called the current image.

[0118] Prediction of the current block can be performed based on intra-frame prediction or inter-frame prediction.

[0119] Intra-frame prediction is used to remove redundant data in the current block based on the similarity between the current block and the reference sample.

[0120] Figure 3 An intra-frame prediction method performed by an image encoding / decoding device according to this disclosure is shown.

[0121] Reference Figure 3 This allows us to determine the reference line S300 for intra-frame prediction of the current block.

[0122] The current block can use one or more of a predefined set of reference line candidates from the image encoding / decoding device as reference lines for intra-frame prediction. Here, the predefined set of reference line candidates can include neighboring reference lines adjacent to the current block to be decoded and N non-neighboring reference lines far from the boundary of the current block, one to N samples. N can be an integer of 1, 2, 3, or greater. In the following description, for ease of description, it is assumed that the multiple reference line candidates available for the current block consist of neighboring reference line candidates and three non-neighboring reference line candidates, but this is not a limitation. In other words, of course, the multiple reference line candidates available for the current block can include four or more non-neighboring reference line candidates.

[0123] An image encoding device can determine the optimal reference line candidate from multiple reference line candidates and encode the index used to specify the optimal reference line candidate. An image decoding device can determine the reference line for the current block based on the index, which is signaled via a bitstream. The index can specify any one of the multiple reference line candidates. The reference line candidate specified by the index can be used as the reference line for the current block.

[0124] The number of reference line indices used to determine the current block by signaling can be 1, 2, or more. As an example, when the number of signaled indices is 1, the current block can perform intra-frame prediction using only a single reference line candidate specified by the signaled index among multiple reference line candidates. Alternatively, when the number of signaled indices is at least two, the current block can perform intra-frame prediction using multiple reference line candidates specified by multiple indices among multiple reference line candidates.

[0125] Reference Figure 3 This allows us to determine the intra-prediction mode S310 for the current block.

[0126] The intra-prediction mode for the current block can be determined from multiple predefined intra-prediction modes in the image encoding / decoding device. This can be achieved by referring to... Figure 4 and Figure 5 Several predefined intra-prediction modes are described.

[0127] Figure 4 Examples of several intra-frame prediction modes based on this disclosure are shown.

[0128] Reference Figure 4 The predefined intra-frame prediction modes in the image encoding / decoding device may include non-directional modes and directional modes. Non-directional modes may include at least one of a planar mode or a DC mode. Directional modes may include directional modes 2 to 66.

[0129] Directional patterns can be extended to more than Figure 4 As shown. Figure 5 An example is shown in which the orientation pattern is expanded.

[0130] exist Figure 5The diagram shows the addition of modes -1 to -14 and modes 67 to 80. These orientation modes can be referred to as wide-angle intra-prediction modes. Whether to use a wide-angle intra-prediction mode can be determined based on the shape of the current block. As an example, if the current block is a non-square block with a width greater than its height, some orientation modes (e.g., numbers 2 to 15) can be transformed into wide-angle intra-prediction modes between numbers 67 and 80. On the other hand, if the current block is a non-square block with a height greater than its width, some orientation modes (e.g., numbers 53 to 66) can be transformed into wide-angle intra-prediction modes between numbers -1 and -14.

[0131] The range of available wide-angle intra-frame prediction modes can be adaptively determined based on the aspect ratio of the current block. Table 1 shows the range of available wide-angle intra-frame prediction modes based on the aspect ratio of the current block.

[0132] [Table 1]

[0133] Width / height Available wide-angle intra prediction mode range W / H = 16 67～80 W / H = 8 67～78 W / H = 4 67～76 W / H = 2 67～74 W / H = 1 No W / H = 1 / 2 -1～-8 W / H = 1 / 4 -1～-10 W / H = 1 / 8 -1～-12 W / H = 1 / 16 -1～-14

[0134] Among multiple intra-prediction modes, K candidate modes (most probable modes, MPM) can be selected. A candidate list including the selected candidate modes can be generated. The index of any candidate mode belonging to the candidate list can be signaled. The intra-prediction mode of the current block can be determined based on the candidate mode indicated by the index. As an example, the candidate mode indicated by the index can be configured as the intra-prediction mode of the current block. Alternatively, the intra-prediction mode of the current block can be determined based on the value of the candidate mode indicated by the index and a predetermined difference. The difference can be defined as the difference between the value of the intra-prediction mode of the current block and the value of the candidate mode indicated by the index. The difference can be signaled via the bitstream. Alternatively, the difference can be a predefined value in the image encoding / decoding device. Alternatively, the intra-prediction mode of the current block can be determined based on a flag indicating whether there is a mode in the candidate list that is the same as the intra-prediction mode of the current block. As an example, when the flag is the first value, the intra-prediction mode of the current block can be determined from the candidate list. In this case, the index of any of the multiple candidate modes belonging to the candidate list can be signaled. The candidate mode indicated by the index can be configured as the intra-prediction mode for the current block. Conversely, when the flag is the second value, any of the remaining intra-prediction modes can be configured as the intra-prediction mode for the current block. The remaining intra-prediction modes can refer to modes other than those belonging to the candidate list among a plurality of predefined intra-prediction modes. When the flag is the second value, the index indicating any of the remaining intra-prediction modes can be signaled. The intra-prediction mode indicated by the signaled index can be configured as the intra-prediction mode for the current block. The intra-prediction mode for a chroma block can be selected from multiple intra-prediction mode candidates. For this purpose, the index information indicating one of the intra-prediction mode candidates for the chroma block can be explicitly encoded and signaled via the bitstream. Table 2 shows the intra-prediction mode candidates for the chroma block.

[0135] [Table 2]

[0136]

[0137] In the examples in Table 2, Direct Mode (DM) refers to setting the intra-prediction mode of the luma block located at the same position as the chroma block to the intra-prediction mode of the chroma block. Simultaneously, the luma block located at the same position as the chroma block can be determined based on the position of the top-left sample or the center sample of the chroma block. For example, if the intra-prediction mode (luma mode) of the luma block is number 0 (planar mode) and the index indicates number 2, then the intra-prediction mode of the chroma block can be determined as horizontal mode (number 18). For example, if the intra-prediction mode (luma mode) of the luma block is number 1 (DC mode) and the index indicates number 0, then the intra-prediction mode of the chroma block can be determined as planar mode (number 0).

[0138] Therefore, the intra-prediction mode of the chroma block can also be set to Figure 4 or Figure 5 One of the intra-prediction modes shown. The intra-prediction mode of the current block can be used to determine the reference line of the current block, and in this case, S310 can be executed before S300.

[0139] Reference Figure 3 Intra-prediction S320 can be performed on the current block based on the reference line and the intra-prediction mode of the current block.

[0140] In the following text, by reference Figures 6 to 8 The intra-prediction method by intra-prediction mode is described in detail. However, for ease of description, it is assumed that a single reference line is used for intra-prediction of the current block, but even when multiple reference lines are used, the intra-prediction method described later can be applied in the same / similar manner.

[0141] Figure 6 An intra-frame prediction method based on planar patterns according to this disclosure is shown.

[0142] Reference Figure 6 Let T represent the reference sample located at the top right corner of the current block, and L represent the reference sample located at the bottom left corner of the current block. P1 can be generated by interpolation in the horizontal direction. As an example, P1 can be generated by interpolating T with reference samples located on the same horizontal line as P1. P2 can be generated by interpolation in the vertical direction. As an example, P2 can be generated by interpolating L with reference samples located on the same vertical line as P2. As shown in Equation 1 below, the current sample in the current block can be predicted by the weighted sum of P1 and P2.

[0143] [Equation 1]

[0144] (α×P1+β×P2) / (α+β)

[0145] In Equation 1, weights α and β can be determined by considering the width and height of the current block. Depending on the width and height of the current block, weights α and β can have the same or different values. If the width and height of the current block are the same, weights α and β can be configured to be the same, and the predicted sample for the current sample can be configured as the average of P1 and P2. When the width and height of the current block are not the same, weights α and β can have different values. As an example, when the width is greater than the height, the smaller value can be configured as the weight corresponding to the width of the current block, and the larger value can be configured as the weight corresponding to the height of the current block. Conversely, when the width is greater than the height, the larger value can be configured as the weight corresponding to the width of the current block, and the smaller value can be configured as the weight corresponding to the height of the current block. Here, the weight corresponding to the width of the current block can be β, and the weight corresponding to the height of the current block can be α.

[0146] Figure 7 An intra-frame prediction method based on DC mode according to this disclosure is shown.

[0147] Reference Figure 7 The algorithm can calculate the average value of surrounding samples adjacent to the current block, and the calculated average value can be configured as the predicted value of all samples in the current block. Here, surrounding samples can include the top reference sample and the left reference sample of the current block. However, depending on the shape of the current block, the average value can be calculated using only the top reference sample or the left reference sample. As an example, when the width of the current block is greater than its height, the average value can be calculated using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the top reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the top reference sample of the current block. On the other hand, when the width of the current block is less than its height, the average value can be calculated using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is less than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block. Alternatively, when the ratio of the width to the height of the current block is greater than or equal to a predetermined threshold, the average value can be calculated using only the left reference sample of the current block.

[0148] Figure 8 An intra-frame prediction method based on directional patterns according to this disclosure is shown.

[0149] When the intra-prediction mode of the current block is directional mode, projection onto the reference line can be performed based on the angle of the corresponding directional mode. When a reference sample exists at the projection location, the corresponding reference sample can be configured as the prediction sample for the current sample. If the reference sample does not exist at the projection location, a sample corresponding to the projection location can be generated using one or more surrounding samples adjacent to the projection location. For example, interpolation can be performed based on two or more surrounding samples that are bidirectionally adjacent to the projection location to generate a sample corresponding to the projection location. Alternatively, a surrounding sample adjacent to the projection location can be configured as the sample corresponding to the projection location. In this case, among the multiple surrounding samples adjacent to the projection location, the surrounding sample closest to the projection location can be used. The sample corresponding to the projection location can be configured as the prediction sample for the current sample.

[0150] Reference Figure 8 For the current sample B, when projection onto the reference line is performed based on the angle of the intra-prediction mode at the corresponding position, the reference sample exists at the projection position (i.e., the reference sample R3 at the integer position). In this case, the reference sample at the projection position can be configured as the predicted sample for the current sample B. For the current sample A, when projection onto the reference line is performed based on the angle of the intra-prediction mode at the corresponding position, the reference sample (i.e., the reference sample at the integer position) does not exist at the projection position. In this case, interpolation can be performed based on surrounding samples (e.g., R2 and R3) near the projection position to generate a sample (r) at the fractional position. The sample (r) generated at the fractional position can be configured as the predicted sample for the current sample A.

[0151] Figure 9 A method for deriving samples at fractional positions is shown.

[0152] exist Figure 9 In the example, variable h refers to the vertical distance from the position of predicted sample A to the reference sample line (i.e., vertical distance), and variable w refers to the horizontal distance from the position of predicted sample A to the fractional position sample (i.e., horizontal distance). Additionally, variable θ refers to a predefined angle based on the directionality of the intra-frame prediction pattern, and variable x refers to the fractional position.

[0153] The variable w can be derived from Equation 2 below.

[0154] [Equation 2]

[0155] w = htanθ

[0156] Then, if the integer positions are removed from the variable w, the fractional positions can be obtained.

[0157] Fractional position samples can be generated by interpolating adjacent integer position reference samples. As an example, a fractional position reference sample at position x can be generated by interpolating integer position reference samples R2 and R3.

[0158] In deriving the fractional position samples, a scaling factor can be used to avoid real number operations. As an example, when the scaling factor f is set to 32, as in... Figure 8 In the example shown in (b), the distance between neighboring integer reference samples can be set to 32 instead of 1.

[0159] Additionally, the tangent of the angle θ determined based on the directionality of the intra-frame prediction mode can also be amplified using the same scaling factor (e.g., 32).

[0160] Figure 10 and Figure 11 The diagram shows the tangent of the angle scaled by a factor of 32 according to the intra-frame prediction mode.

[0161] Figure 10 The scaling results of the tangent value for the non-wide-angle intra-prediction mode are shown, and Figure 11 The scaling results of the tangent value for the wide-angle intra-frame prediction mode are shown.

[0162] If the tangent (tanθ) of the angle value of the intra-prediction mode is positive, intra-prediction can be performed using only one of the reference samples belonging to the top line of the current block (i.e., the top reference sample) or the reference sample belonging to the left line of the current block (i.e., the left reference sample). Conversely, when the tangent of the angle value of the intra-prediction mode is negative, both the reference sample at the top position and the reference sample at the left position are used.

[0163] In this case, to simplify the implementation, the left reference sample can be projected to the top or to the left to arrange the reference samples in the form of a 1D array, and the 1D array of reference samples can be used to perform intra-frame prediction.

[0164] Figure 12 This is a diagram showing the intra-frame prediction aspect when the orientation mode is one of mode numbers 34 to 49.

[0165] When the intra-prediction mode of the current block is one of mode numbers 34 to 49, intra-prediction is performed by using not only the top reference sample of the current block but also the left reference sample. In this case, as... Figure 12 In the example shown, the reference sample for the top line can be generated by either copying the reference sample at the left position of the current block to the position of the top line or by interpolating the reference sample at the left position.

[0166] As an example, when a reference sample at position A at the top of the current block is obtained, considering the directionality of the intra-prediction mode of the current block, a projection can be performed on the left-side line of the current block at position A on the top line. If the projected position is called 'a', the value corresponding to position a can be copied, or a fractional position value corresponding to 'a' can be generated and set as the value of position A. As an example, if position a is an integer position, the value of position A can be generated by copying the integer position reference sample. On the other hand, when position a is a fractional position, the reference sample at the top position of position a and the reference sample at the bottom position of position a can be interpolated, and the interpolated value can be set as the value of position A. Meanwhile, at position A at the top of the current block, the direction projected onto the left-side line of the current block can be parallel to and opposite to the direction of the intra-prediction mode of the current block.

[0167] Figure 13 This is a diagram used to describe an example of generating a top reference sample by interpolating the left reference sample.

[0168] exist Figure 13 In this context, variable h represents the horizontal distance between position A on the top line and position a on the left line. Variable w represents the vertical distance between position A on the top line and position a on the left line. Additionally, variable θ refers to a predefined angle based on the directionality of the intra-frame prediction pattern, and variable x refers to the fractional position.

[0169] The variable h can be derived from Equation 3 below.

[0170] [Equation 3]

[0171] h = w / tanθ

[0172] Then, if the integer positions are removed from the variable h, the fractional positions can be obtained.

[0173] In deriving fractional position samples, a scaling factor can be used to avoid real-number operations. As an example, the tangent value of the variable θ can be scaled using a scaling factor f1. Here, since the direction projected onto the left line is parallel to and opposite to the orientation prediction pattern, a scaling factor f1 can be used. Figure 10 and Figure 11 The scaled tangent value is shown.

[0174] When the scaling factor f1 is applied, Equation 3 can be modified and used as Equation 4 below.

[0175] [Equation 4]

[0176] h = (w * f1 * f2) / (f1 * tanθ)

[0177] In the same manner as described above, the 1D reference sample array can be configured using only the reference samples belonging to the top line. Therefore, intra-frame prediction for the current block can be performed using only the top reference samples, which include the 1D array.

[0178] Figure 14 An example is shown in which intra-frame prediction is performed using reference samples arranged in a 1D array.

[0179] like Figure 14 In the example shown, the left reference sample can be projected to generate the top reference sample, and therefore, the predicted sample for the current block can be obtained by using only the reference sample belonging to the top line.

[0180] and Figure 12 and Figure 14 In contrast, the top reference sample can be projected onto the left line to configure the 1D reference sample array using only the reference samples belonging to the left line. Specifically, for orientation patterns 19 to 33 in which the tangent (tanθ) of the orientation pattern's angle is negative, the reference samples belonging to the top line can be projected onto the left line to generate the left reference sample.

[0181] Inter-frame prediction is used to generate a predicted block for the current block based on the similarity between the current block and a reference block in a reference image. In other words, when encoding the current image, inter-frame prediction can remove redundant data between images. Inter-frame prediction can be performed on a block-by-block basis. Specifically, the motion information of the current block can be used to generate a predicted block for the current block based on the reference image. Here, the motion information can include at least one of motion vectors, reference image index, and prediction direction.

[0182] Figure 15 This is a diagram that schematically illustrates the process of performing inter-frame prediction in the encoder and decoder.

[0183] like Figure 15 In the example shown, motion information for the current block can be obtained to perform inter-frame prediction S1510. Here, the motion information may include at least one of a motion vector, a reference image index, or weights applied to the predicted block. For the current block, motion information for at least one of the L0 or L1 directions can be obtained.

[0184] The encoder can derive motion information for the current block through motion estimation and encode this motion information to signal the decoder. Furthermore, the encoding / decoding of motion information can be based on motion information merging patterns, motion vector prediction patterns, template-based motion estimation methods, or bilateral matching methods described later.

[0185] The decoder can derive the motion information of the current block based on the information transmitted from the encoder.

[0186] Alternatively, the decoder can derive motion information for the current block in the same way as in the encoder. This method can be called decoder-side motion estimation.

[0187] When the motion information of the current block is obtained, the predicted block S1520 of the current block can be obtained based on the obtained motion information. As an example, a reference block in the reference image whose position is separated from the current block by a motion vector can be set as the predicted block of the current block.

[0188] The process of performing inter-frame prediction will be described in more detail below.

[0189] Motion information for the current block can be generated through motion estimation.

[0190] Figure 16 An example of performing motion estimation within it is shown.

[0191] exist Figure 16 In this context, assume that the Picture Order Count (POC) of the current image is T, and the POC of the reference image is (T-1).

[0192] The search range for motion estimation can be set from the same location in the reference image as the reference point of the current block. Here, the reference point can be the location of the top-left sample of the current block.

[0193] As an example, in Figure 16 The example shows squares of size (w0+w01) and (h0+h1) set within a search area centered on a reference point. In the example above, w0, w1, h0, and h1 can have the same value. Alternatively, at least one of w0, w1, h0, and h1 can be set to have a different value than the others. Alternatively, the sizes of w0, w1, h0, and h1 can be determined to not exceed the boundaries of a coding tree unit (CTU), slice boundary, tile boundary, or picture boundary.

[0194] Within the search scope, after setting a reference block of the same size as the current block, the cost relative to the current block can be measured for each reference block. The cost can be calculated using the similarity between the two blocks.

[0195] As an example, the cost can be calculated based on the sum of absolute differences (SAD) between the original samples in the current block and the original samples (or reconstructed samples) in the reference block. As the SAD decreases, the cost can be reduced.

[0196] Then, by comparing the cost of each reference block, the reference block with the optimal cost can be set as the prediction block for the current block.

[0197] Then, the distance between the current block and the reference block can be set as the motion vector. Specifically, the difference in x-coordinates and the difference in y-coordinates between the current block and the reference block can be set as the motion vector.

[0198] In addition, the index of the image, including the reference block specified by motion estimation, is set as the reference image index.

[0199] Additionally, the prediction direction can be set based on whether the reference image belongs to the L0 reference image list or the L1 reference image list.

[0200] Furthermore, motion estimation can be performed for each of the L0 and L1 directions. When prediction is performed for both the L0 and L1 directions, motion information for the L0 direction and motion information for the L1 direction can be generated separately.

[0201] Figure 17 and Figure 18 An example is shown of generating a predicted block for the current block based on motion information generated through motion estimation.

[0202] Figure 17 An example of generating prediction blocks through unidirectional (i.e., L0 direction) prediction is shown, and Figure 18 An example of generating prediction blocks through bidirectional (i.e., L0 and L1 directions) prediction is shown.

[0203] For unidirectional prediction, a predicted block for the current block is generated using motion information. As an example, the motion information may include the L0 motion vector, the L0 reference image index, and prediction direction information indicating the L0 direction.

[0204] For bidirectional prediction, a prediction block is generated using two sets of motion information. As an example, a reference block in the L0 direction, specified by motion information in the L0 direction (L0 motion information), can be set as the L0 prediction block, and a reference block in the L1 direction, specified by motion information in the L1 direction (L1 motion information), can be set as the L1 prediction block. The L0 and L1 prediction blocks can then be weighted to generate the prediction block for the current block.

[0205] exist Figures 16 to 18 In the example shown, the L0 reference image exists in the previous direction of the current image (i.e., the POC value is less than the POC value of the current image), and the L1 reference image exists in the subsequent direction of the current image (i.e., the POC value is greater than the POC value of the current image).

[0206] However, unlike the example shown, the L0 reference image can exist in a subsequent direction of the current image, or the L1 reference image can exist in a previous direction of the current image. As an example, both the L0 and L1 reference images can exist in a previous direction of the current image, or both can exist in a subsequent direction of the current image. Alternatively, bidirectional prediction can be performed using both an L0 reference image existing in a subsequent direction of the current image and an L1 reference image existing in a previous direction of the current image.

[0207] Motion information for the blocks performing inter-frame prediction can be stored in memory. In this case, motion information can be stored on a sample-by-sample basis. Specifically, the motion information of the block to which a particular sample belongs can be stored as the motion information of that particular sample. The stored motion information can be used to derive the motion information of neighboring blocks to be encoded / decoded later.

[0208] In the encoder, information obtained by encoding the residual samples corresponding to the difference between the current block's samples (i.e., the original samples) and the predicted samples, along with the motion information necessary to generate the prediction block, can be signaled to the decoder. In the decoder, the information about the signaled difference can be decoded to obtain the residual samples, and reconstructed samples can be generated by adding the predicted samples in the prediction block generated using the motion information to the residual samples.

[0209] In this scenario, to effectively compress the motion information communicated to the decoder via a signal, one of several inter-frame prediction modes can be selected. These multiple inter-frame prediction modes may include a motion information merging mode and a motion vector prediction mode.

[0210] A motion vector prediction mode is a mode that encodes the difference between a motion vector and its predicted value and signals it. Here, the predicted motion vector value can be derived based on the motion information of neighboring blocks or samples adjacent to the current block.

[0211] Figure 19 The reference location used to derive the predicted motion vector values ​​is shown.

[0212] For ease of description, assume the current block has a size of 4×4.

[0213] In the example shown, "LB" represents the sample included in the leftmost column and bottommost row of the current block. "RT" represents the sample included in the rightmost column and topmost row of the current block. A0 to A4 represent samples adjacent to the left side of the current block, and B0 to B5 represent samples adjacent to the top side of the current block. As an example, A1 represents the sample adjacent to the left side of LB, and B1 represents the sample adjacent to the top side of RT.

[0214] Col represents the position of the sample in the co-occurrence picture that is adjacent to the bottom right corner of the current block. The co-occurrence picture is a different picture from the current picture, and information used to specify the co-occurrence picture (e.g., co-occurrence picture index) can be explicitly encoded in the bitstream and signaled. Alternatively, a reference picture with a predefined reference picture index can be set as the co-occurrence picture.

[0215] A block that covers the Col position within a co-located image can be called a co-located block. When the position of the sample adjacent to the lower right of the current block in the reference image is unavailable, the Col position can be set to the position of the center sample or the upper left sample of the current block.

[0216] The motion vector prediction value for the current block can be derived from at least one motion vector prediction candidate included in the motion vector prediction list.

[0217] The number of motion vector prediction candidates that can be inserted into the motion vector prediction list (i.e., the size of the list) can be predefined in the encoder and decoder. As an example, the maximum number of motion vector prediction candidates can be 2.

[0218] Motion vectors stored at the locations of neighboring samples adjacent to the current block, or scaled motion vectors obtained by scaling motion vectors, can be inserted as motion vector prediction candidates into the motion vector prediction list. In this case, neighboring samples adjacent to the current block can be scanned in a predefined order to obtain motion vector prediction candidates.

[0219] As an example, it can be checked whether the motion vectors are stored at each location in the order of A0 to A4. Furthermore, based on the scan order, the first available motion vector found can be inserted as a motion vector prediction candidate into the motion vector prediction list.

[0220] As another example, it is checked whether the motion vectors are stored at each location in the order A0 to A4, and the motion vector at the location with the same reference image as the current block is first found can be inserted into the motion vector prediction list as a motion vector prediction candidate. If no neighboring samples with the same reference image as the current block exist, motion vector prediction candidates can be derived based on the first available vectors found. Specifically, after scaling the first available motion vectors found, the scaled motion vectors can be inserted into the motion vector prediction list as motion vector prediction candidates. In this case, scaling can be performed based on the output order difference (i.e., POC difference) between the current image and the reference image, and the output order difference (i.e., POC difference) between the current image and the reference images of neighboring samples.

[0221] Furthermore, following the order from B0 to B5, it is possible to check whether motion vectors are stored at each location. And, based on the scan order, the first available motion vector found can be inserted as a motion vector prediction candidate into the motion vector prediction list.

[0222] As another example, it is checked whether the motion vectors are stored at each location in the order of B0 to B5, and the motion vector at the location with the same reference image as the current block can be inserted into the motion vector prediction list as a motion vector prediction candidate. If there are no neighboring samples with the same reference image as the current block, motion vector prediction candidates can be derived based on the first available vectors found. Specifically, after scaling the first available motion vectors found, the scaled motion vectors can be inserted into the motion vector prediction list as motion vector prediction candidates. In this case, scaling can be performed based on the output order difference (i.e., POC difference) between the current image and the reference image, and the output order difference (i.e., POC difference) between the current image and the reference images of neighboring samples.

[0223] In the example above, motion vector prediction candidates can be derived from the samples adjacent to the left of the current block, and motion vector prediction candidates can be derived from the samples adjacent to the top of the current block.

[0224] In this scenario, motion vector prediction candidates derived from the left-hand samples can be inserted into the motion vector prediction list before those derived from the top-hand samples. In this case, the index assigned to the motion vector prediction candidates derived from the left-hand samples can have a smaller value compared to those derived from the top-hand samples.

[0225] Conversely, motion vector prediction candidates derived from the top sample can be inserted into the motion vector prediction list before motion vector prediction candidates derived from the left sample.

[0226] Among the motion vector prediction candidates included in the motion vector prediction list, the motion vector prediction candidate with the highest coding efficiency can be set as the motion vector predictor (MVP) for the current block. Furthermore, index information indicating which motion vector prediction candidate among the multiple candidates is set as the motion vector prediction value for the current block can be encoded and signaled to the decoder. When the number of motion vector prediction candidates is 2, the index information can be a 1-bit flag (e.g., the MVP flag). Additionally, the motion vector difference (MVD) (which is the difference between the motion vector of the current block and the motion vector predictor) can be encoded and signaled to the decoder.

[0227] The decoder can construct a list of motion vector predictions in the same way as the encoder. Additionally, index information can be decoded from the bitstream, and a motion vector prediction candidate can be selected from multiple candidates based on the decoded index information. The selected motion vector prediction candidate can then be set as the motion vector prediction value for the current block.

[0228] Alternatively, motion vector differences can be decoded from the bitstream. Then, the motion vector of the current block can be obtained by combining the predicted motion vector values ​​and the motion vector differences.

[0229] When bidirectional prediction is applied to the current block, a list of motion vector predictions can be generated for each of the L0 and L1 directions. In other words, the list of motion vector predictions can include motion vectors in the same direction. Therefore, the motion vectors of the current block and the motion vector prediction candidates included in the list of motion vector predictions have the same direction.

[0230] When the motion vector prediction mode is selected, the reference image index and prediction direction information can be explicitly encoded and signaled to the decoder. As an example, if there are multiple reference images in the reference image list and motion estimation is performed for each of the multiple reference images, the reference image index used to specify the reference image from which the motion information of the current block is derived can be explicitly encoded and signaled to the decoder.

[0231] In this case, when the list of reference images includes only one reference image, the encoding / decoding of the reference image index can be omitted.

[0232] The prediction direction information can be an index indicating one of L0 unidirectional prediction, L1 unidirectional prediction, or bidirectional prediction. Alternatively, an L0 flag indicating whether to perform L0 directional prediction and an L1 flag indicating whether to perform L1 directional prediction can be encoded and signaled respectively.

[0233] Motion information merging mode sets the motion information of the current block to the same mode as that of its neighboring blocks. In motion information merging mode, motion information can be decoded / encoded using a motion information merging list.

[0234] Motion information merging candidates can be derived based on motion information from neighboring blocks or samples adjacent to the current block. For example, after predefining reference positions around the current block, it's possible to check if motion information exists at those predefined reference positions. When motion information exists at a predefined reference position, the motion information at that position can be inserted as a motion information merging candidate into the motion information merging list.

[0235] existFigure 19 In the example, the predefined reference position may include at least one of A0, A1, B0, B1, B5, and Col. Furthermore, motion information merging candidates can be derived in the order of A1, B1, B0, A0, B5, and Col.

[0236] Among the motion information merging candidates included in the motion information merging list, the motion information of the motion information merging candidate with the optimal cost can be set as the motion information of the current block. Furthermore, index information (e.g., a merging index) indicating the selected motion information merging candidate among multiple motion information merging candidates can be encoded and signaled to the decoder.

[0237] In the decoder, the motion information merging list can be constructed in the same way as in the encoder. Furthermore, motion information merging candidates can be selected based on the merging index decoded from the bitstream. The motion information of the selected motion information merging candidate can then be set as the motion information for the current block.

[0238] Unlike the motion vector prediction list, the motion information merging list is constructed as a single list, independent of the prediction direction. In other words, the motion information merging candidates included in the motion information merging list can have only L0 motion information or L1 motion information, or they can have bidirectional motion information (i.e., L0 motion information and L1 motion information).

[0239] The reconstructed sample region surrounding the current block can be used to derive the motion information of the current block. Here, the reconstructed sample region used to derive the motion information of the current block can also be referred to as a template.

[0240] Figure 20 This is a diagram used to describe template-based motion estimation methods.

[0241] exist Figure 16 The text describes how to determine the predicted block for the current block based on the cost between the reference block and the current block within the search range. According to this implementation, [the text continues with further details about the implementation and prediction]. Figure 16 Unlike other methods, motion estimation of the current block can be performed based on the cost between a template adjacent to the current block (hereinafter referred to as the current template) and a reference template having the same size and shape as the current template.

[0242] As an example, the cost can be calculated based on the SAD (Sum of Aspects) between the reconstructed sample in the current template and the reconstructed sample in the reference block. A smaller SAD results in a lower cost.

[0243] When the current template and the reference template with the optimal cost are determined within the search range, the reference block of the neighboring reference template can be set as the prediction block of the current block.

[0244] Furthermore, the motion information of the current block can be set based on the distance between the current block and the reference block, the index of the image to which the reference block belongs, and whether the reference image is included in the L0 reference image list or the L1 reference image list.

[0245] Since the pre-reconstructed region around the current block is defined as a template, the decoder itself can perform motion estimation in the same way as the encoder. Therefore, when motion information is derived using a template, there is no need to encode the motion information and signal it, except for information indicating whether the template was used.

[0246] The current template may include at least one of the regions adjacent to the top of the current block or the region adjacent to the left of the current block. In this case, the region adjacent to the top may include at least one row, and the region adjacent to the left may include at least one column.

[0247] Figure 21 An example of configuring a template is shown.

[0248] According to Figure 21 The example shown is one of the examples used to configure the current template.

[0249] Alternative locations, and Figure 21 Unlike the examples shown, the template can be configured only with the area adjacent to the left of the current block, or the template can be configured only with the area adjacent to the top of the current block.

[0250] The size and / or shape of the current template can be predefined in the encoder and decoder.

[0251] Alternatively, after predefining multiple template candidates with different sizes and / or shapes, the index information of one of the template candidates can be encoded and signaled to the decoder.

[0252] Alternatively, a template candidate can be adaptively selected from a plurality of template candidates based on at least one of the size, shape, or position of the current block. As an example, when the current block is adjacent to the top boundary of the CTU, the current template can be configured only with the region adjacent to the left side of the current block.

[0253] Template-based motion estimation can be performed on each reference image stored in the reference image list. Alternatively, motion estimation can be performed on only some of the reference images. As an example, motion estimation can be performed only on the reference image with reference image index 0, or only on reference images with reference image indices less than a threshold or reference images whose POC difference with the current image is less than a threshold.

[0254] Alternatively, after explicitly encoding and signaling the reference image index, motion estimation can be performed only on the reference image indicated by the reference image index.

[0255] Alternatively, motion estimation can be performed on reference images of neighboring blocks corresponding to the current template. As an example, if the template includes a left-side neighboring region and a top-side neighboring region, at least one reference image can be selected using at least one of the reference image indices of the left-side neighboring block or the top-side neighboring block. Motion estimation can then be performed on at least one selected reference image.

[0256] Information indicating whether template-based motion estimation is applied can be encoded and signaled to the decoder. This information can be a 1-bit flag. As an example, when the flag is true (1), it indicates that template-based motion estimation is applied to the L0 and L1 directions of the current block. On the other hand, when the flag is false (0), it indicates that template-based motion estimation is not applied. In this case, the motion information of the current block can be derived based on the motion information merging mode or the motion vector prediction mode.

[0257] Conversely, template-based motion estimation can be applied only when it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block. As an example, template-based motion estimation can be performed when both the first flag indicating whether the motion information merging mode is applied and the second flag indicating whether the motion vector prediction mode is applied are 0.

[0258] Information regarding whether template-based motion estimation is applied can be signaled for each of the L0 and L1 directions. In other words, it can be determined independently whether template-based motion estimation is applied to the L0 direction and whether it is applied to the L1 direction. Therefore, template-based motion estimation can be applied to either the L0 or L1 direction, while different modes (e.g., motion information merging mode or motion vector prediction mode) can be applied to the other.

[0259] When template-based motion estimation is applied to both the L0 and L1 directions, the prediction block for the current block can be generated based on a weighted sum of the L0 and L1 prediction blocks. Alternatively, even if template-based motion estimation is applied to one of the L0 and L1 directions, but a different mode is applied to the other, the prediction block for the current block can still be generated based on a weighted sum of the L0 and L1 prediction blocks.

[0260] Alternatively, template-based motion estimation methods can be inserted as motion information merging candidates in a motion information merging mode or as motion vector prediction candidates in a motion vector prediction mode. In this case, the application of a template-based motion estimation method can be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate indicates a template-based motion estimation method.

[0261] Motion information for the current block can also be generated using the bilateral matching method.

[0262] Figure 22 This is a graph used to describe motion estimation methods based on bilateral matching.

[0263] The bilateral matching method can only be performed if the chronological order of the current image (i.e., POC) exists between the chronological order of the L0 reference image and the chronological order of the L1 reference image.

[0264] When applying the bilateral matching method, the search scope can be set for each of the L0 and L1 reference images. In this case, the L0 reference image index used to identify the L0 reference image and the L1 reference image index used to identify the L1 reference image can be encoded separately and signaled accordingly.

[0265] As another example, only the L0 reference image index can be encoded and signaled, and the L1 reference image can be selected based on the distance between the current image and the L0 reference image (hereinafter referred to as the L0 POC difference). As an example, among the L1 reference images included in the L1 reference image list, an L1 reference image whose absolute distance to the current image (hereinafter referred to as the L1 POC difference) is the same as the absolute distance between the current image and the L0 reference image can be selected. When no L1 reference image with the same L1 POC difference as the L0 POC difference does not exist, the L1 reference image whose L1 POC difference is most similar to the L0 POC difference can be selected from the L1 reference images.

[0266] In this scenario, an L1 reference image whose temporal direction differs from that of the L0 reference image can be used for bilateral matching. For example, when the POC of the L0 reference image is less than the POC of the current image, an L1 reference image whose POC is greater than that of the current image can be selected.

[0267] Conversely, only the L1 reference image index can be encoded and signaled, and the L0 reference image can be selected based on the distance between the current image and the L1 reference image.

[0268] Alternatively, a bilateral matching method can be performed using the L0 reference image that is closest to the current image among the L0 reference images and the L1 reference image that is closest to the current image among the L1 reference images.

[0269] Alternatively, a bilateral matching method can be performed using an L0 reference image (e.g., index 0) assigned a predefined index to it from the L0 reference image list and an L1 reference image (e.g., index 0) assigned a predefined index to it from the L1 reference image list.

[0270] Alternatively, an LX (X is 0 or 1) reference image can be selected based on a reference image index explicitly indicated by a signal, and an L|X-1| reference image can be selected as the reference image closest to the current image in the L|X-1| reference image list or as a reference image with a predefined index in the L|X-1| reference image list.

[0271] As another example, L0 and / or L1 reference images can be selected based on the motion information of the current block's neighboring blocks. For instance, the L0 and / or L1 reference images to be used for bilateral matching can be selected using the reference image index of the current block's left or top neighboring block.

[0272] The search area can be set within a predetermined range from the co-located blocks in the reference image.

[0273] As another example, the search range can be set based on initial motion information. Initial motion information can be derived from the current block's neighboring blocks. For example, the motion information of the current block's left-hand neighbor or top-hand neighbor can be set as the current block's initial motion information.

[0274] When applying the bilateral matching method, the L0 motion vector and the motion vector in the L1 direction are set in opposite directions. This means that the sign of the L0 motion vector is opposite to the sign of the motion vector in the L1 direction. Furthermore, the magnitude of the LX motion vector can be proportional to the distance between the current image and the LX reference image (i.e., the POC difference).

[0275] Motion estimation can then be performed using the cost between the reference block belonging to the search range of the L0 reference image (hereinafter referred to as the L0 reference block) and the reference block belonging to the search range of the L1 reference image (hereinafter referred to as the L1 reference block).

[0276] When selecting an L0 reference block with vector (x, y) relative to the current block, an L1 reference block can be selected at a distance of (-Dx, -Dy) from the current block. Here, D can be determined by the ratio of the distance between the current image and the L0 reference image to the distance between the L1 reference image and the current image.

[0277] As an example, in Figure 22 In the example shown, the absolute values ​​of the distances between the current image (T) and the L0 reference image (T-1) and between the current image (T) and the L1 reference image (T+1) are the same. Therefore, in the example shown, the L0 motion vector (x0, y0) and the L1 motion vector (x1, y1) have the same magnitude but opposite directions. If an L1 reference image with a POC of (T+2) is used, the L1 motion vector (x1, y1) would be set to (-2*x0, -2*y0).

[0278] When selecting the L0 and L1 reference blocks with optimal costs, each of the L0 and L1 reference blocks can be set as the L0 and L1 prediction blocks for the current block, respectively. The final prediction block for the current block can then be generated through a weighted summation of the L0 and L1 reference blocks.

[0279] When applying the bilateral matching method, the decoder can perform motion estimation in the same way as the encoder. Therefore, information indicating whether the bilateral motion matching method is applied can be explicitly encoded / decoded, while decoding / encoding of motion information such as motion vectors can be omitted. As described above, at least one of the L0 reference image index or the L1 reference image index can be explicitly encoded / decoded.

[0280] As another example, while explicitly encoding / decoding information indicating whether a bilateral matching method is applied, when the bilateral matching method is applied, the L0 motion vector or L1 motion vector can be explicitly encoded and signaled. When signaling the L0 motion vector, the L1 motion vector can be derived based on the POC difference between the current image and the L0 reference image, and the POC difference between the current image and the L1 reference image. Similarly, when signaling the L1 motion vector, the L0 motion vector can be derived based on the POC difference between the current image and the L0 reference image, and the POC difference between the current image and the L1 reference image. In this case, the encoder can explicitly encode the smaller of the L0 and L1 motion vectors.

[0281] The information indicating whether the bilateral matching method is applied can be a 1-bit flag. For example, when the flag is true (e.g., 1), it can indicate that the bilateral matching method is applied to the current block. When the flag is false (e.g., 0), it can indicate that the bilateral matching method is not applied to the current block. In this case, the motion information merging mode or the motion vector prediction mode can be applied to the current block.

[0282] Conversely, the bilateral matching method can be applied only when it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block. As an example, the bilateral matching method can be applied when both the first flag indicating whether the motion information merging mode is applied and the second flag indicating whether the motion vector prediction mode is applied are 0.

[0283] Alternatively, the bilateral matching method can be inserted as a motion information merging candidate in the motion information merging mode or as a motion vector prediction candidate in the motion vector prediction mode. In this case, the application of the bilateral matching method can be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate indicates the bilateral matching method.

[0284] In the bilateral matching method, it is specified that the temporal order of the current image must exist between the temporal order of the L0 reference image and the temporal order of the L1 reference image. A one-way matching method, without the constraint of the bilateral matching method, can be applied to generate the predicted block for the current block. Specifically, in the one-way matching method, two reference images with a temporal order less than the current block's temporal order (i.e., POC) or two reference images with a temporal order greater than the current block's temporal order can be used. In this case, two reference images can be derived from either the L0 reference image list or the L1 reference image list. Alternatively, one of the two reference images can be derived from the L0 reference image list, while the other can be derived from the L1 reference image list.

[0285] Figure 23 This is a graph used to describe motion estimation methods based on one-sided matching.

[0286] A one-sided matching method can be performed based on two reference images with a POC smaller than the current image (i.e., forward reference images) or two reference images with a POC larger than the current image (i.e., backward reference images). Figure 23 The diagram illustrates motion estimation based on a one-sided matching method, using a first reference image (T-1) and a second reference image (T-2) of a POC with a smaller POC than the current image (T).

[0287] In this scenario, the first reference image index for identifying the first reference image and the second reference image index for identifying the second reference image can be encoded and signaled separately. In this case, among the two reference images used in the one-sided matching method, the reference image with a smaller POC difference compared to the current image can be set as the first reference image. Therefore, when the first reference image is selected, only the reference image in the reference image list that has a larger POC difference compared to the first reference image and is in the current image can be set as the second reference image. After reordering the reference images that have the same time direction as the first reference image and have a larger POC difference compared to the first reference image and are in the current image, the second reference image index can be set to indicate the index of one of the reordered reference images.

[0288] Conversely, the reference image with a larger POC difference from the current image among the two reference images can be set as the first reference image. In this case, after reordering the reference images that have the same time direction as the first reference image and have a smaller POC difference from the current image compared to the first reference image, a second reference image index can be set to indicate the index of one of the reference images in the reordered reference images.

[0289] Alternatively, a one-sided matching method can be performed using reference images with predefined indices assigned to them from a list of reference images, and reference images with the same time orientation. As an example, the reference image with index 0 in the list of reference images can be set as the first reference image, and the reference image with the smallest index among those with the same time orientation as the first reference image can be selected as the second reference image.

[0290] You can select either the first reference image or the second reference image from either the L0 reference image list or the L1 reference image list. Figure 23 The diagram illustrates two L0 reference images used in a one-sided matching method. Alternatively, a first reference image can be selected from the L0 reference image list, and a second reference image can be selected from the L1 reference image list.

[0291] Information indicating whether the first reference image and / or the second reference image belong to the L0 reference image list or the L1 reference image list can be encoded / decoded separately.

[0292] Alternatively, one-sided matching can be performed by using either the L0 reference image list or the L1 reference image list as the default. Alternatively, two reference images can be selected from the L0 and L1 reference image lists, regardless of which has the larger number of reference images.

[0293] Then, you can set the search range within the first and second reference images.

[0294] The search area can be set within a predetermined range from the co-located blocks in the reference image.

[0295] As another example, the search range can be set based on initial motion information. Initial motion information can be derived from the current block's neighboring blocks. For example, the motion information of the current block's left-hand neighbor or top-hand neighbor can be set as the current block's initial motion information.

[0296] Motion estimation can then be performed using the cost between a first reference block belonging to the search range of the first reference image and a second reference block belonging to the search range of the second reference image.

[0297] In this case, under the one-sided matching method, the magnitude of the motion vector must be set to increase proportionally to the distance between the current image and the reference image. Specifically, when a first reference block with a vector of (x, y) relative to the current image is selected, the second reference block must be separated from the current block by (Dx, Dy). Here, D can be determined by the ratio of the distance between the current image and the first reference image to the distance between the current image and the second reference image.

[0298] As an example, in Figure 23 In the example, the distance between the current image and the first reference image (i.e., the POC difference) is 1, and the distance between the current image and the second reference image (i.e., the POC difference) is 2. Therefore, when the first motion vector of the first reference block in the first reference image is (x0, y0), the second motion vector (x1, y1) of the second reference block in the second reference image can be set to (2x0, 2y0).

[0299] When selecting the first and second reference blocks with optimal costs, the first and second reference blocks can be respectively set as the first and second prediction blocks for the current block. Then, the final prediction block for the current block can be generated through a weighted summation of the first and second prediction blocks.

[0300] When applying a one-sided matching method, the decoder can perform motion estimation in the same way as the encoder. Therefore, information indicating whether a one-sided motion matching method is applied is explicitly encoded / decoded, while encoding / decoding motion information such as motion vectors can be omitted. As described above, at least one of the first or second reference image indices can be explicitly encoded / decoded.

[0301] As another example, information indicating whether a one-sided matching method is applied is explicitly encoded / decoded, and when a one-sided matching method is applied, either a first motion vector or a second motion vector can be explicitly encoded and signaled. When signaling the first motion vector, the second motion vector can be derived based on the POC difference between the current image and the first reference image, and the POC difference between the current image and the second reference image. When signaling the second motion vector, the first motion vector can be derived based on the POC difference between the current image and the first reference image, and the POC difference between the current image and the second reference image. In this case, the encoder can explicitly encode the smaller of the first and second motion vectors.

[0302] The information indicating whether a one-sided matching method is applied can be a 1-bit flag. For example, when the flag is true (e.g., 1), it can indicate that the one-sided matching method is applied to the current block. When the flag is false (e.g., 0), it can indicate that the one-sided matching method is not applied to the current block. In this case, either the motion information merging mode or the motion vector prediction mode can be applied to the current block.

[0303] Conversely, the one-sided matching method can be applied only when it is determined that the motion information merging mode and the motion vector prediction mode are not applied to the current block. As an example, the one-sided matching method can be applied when both the first flag indicating whether the motion information merging mode is applied and the second flag indicating whether the motion vector prediction mode is applied are 0.

[0304] Alternatively, the one-sided matching method can be inserted as a motion information merging candidate in the motion information merging mode or as a motion vector prediction candidate in the motion vector prediction mode. In this case, the application of the one-sided matching method can be determined based on whether the selected motion information merging candidate or the selected motion vector prediction candidate indicates the one-sided matching method.

[0305] When searching for motion between frames, the precision of the motion vectors can be adjusted, specifically the precision of the motion vectors when searching for motion between images. Specifically, the position of each sample within an image is defined as an integer position. However, the position reflecting motion can be a fractional position, rather than an integer position.

[0306] With this in mind, motion vectors can be searched more accurately by interpolating from reference images.

[0307] Figure 24 and Figure 25 This is a diagram showing an example of how a prediction block is derived based on the accuracy of the motion vector.

[0308] Figure 24 This indicates the position of the current block within the current image, andFigure 25 This indicates the position of the reference block based on the accuracy of the motion vector.

[0309] As in Figure 24 and Figure 25 In the example shown, the motion vector of the current block can be defined as the distance from the sample in the reference image corresponding to the top left position of the current block to the sample corresponding to the top left position of the reference block in the reference image.

[0310] Figure 25 (a) shows the case where the motion vector of the current block has an integer pixel precision, and Figure 25 (b) indicates the case where the motion vector precision of the current block is 1 / 2 pixel. Additionally, Figure 25 (c) shows the case where the motion vector of the current block has a precision of 1 / 4 pixel.

[0311] exist Figure 25 In this context, vector precision is expressed as high as 1 / 4, but motion vectors can also be expressed more precisely, such as 1 / 8, 1 / 16, or 1 / 32.

[0312] Simultaneously, information representing the motion vector precision of the current block can be encoded and signaled. As an example, this information could be an index identifying one of the motion vector precision candidates. Specifically, different indices can be assigned to each of the motion vector precision candidates, and the information can indicate the index applied to the motion vector precision candidate for the current block.

[0313] More precise motion vector search is possible by adjusting the precision of the motion vectors used for inter-frame prediction. When the reference block indicated by the motion vector is located at a real-valued position, samples at real-valued positions can be generated using samples at integer-valued positions and interpolation filters. Furthermore, motion vectors expressed as real-valued numbers can be amplified to integers for encoding / decoding.

[0314] Similar to inter-frame prediction, a block vector for the current block can be derived, and the predicted block for the current block can be obtained using this block vector. Specifically, after deriving the block vector for the current block, the block vector can be used to specify a reference block within the current image. The reference block specified by the block vector can be set as the predicted block for the current block. In this way, the method for predicting the current block using block vectors can be called intra-frame block copying mode.

[0315] Figure 26 This is a flowchart of the intra-frame block copying method performed by an image encoding / decoding device.

[0316] Reference Figure 26 This allows us to obtain the block vector S2610 for the current block.

[0317] The encoder can search for a reference block that has the highest similarity to the current block within the current image. The distance between the searched reference block and the current block can be defined as a block vector.

[0318] In the generated block vector, all reconstructed regions within the current image can be set as the search range.

[0319] Alternatively, for simplicity, the search area can be limited to a predefined size / shape. Specifically, a predefined search area of ​​size / shape can be set within the current image, and reference blocks can be searched only within the set search area.

[0320] As another example, the size of the search range can be set based on the size of the coding tree unit. For example, a reference block can be searched only within a search range that is N times the size of the coding tree unit.

[0321] Meanwhile, at least one of the template matching mode, motion vector prediction mode, or motion information merging mode used for inter-frame prediction can be applied to encode / decode block vectors.

[0322] As an example, when a block vector merging pattern is applied to the current block, merging candidates can be derived from spatially neighboring blocks adjacent to the current block. Here, a spatially neighboring block can represent samples covering locations adjacent to the current block (e.g., ...). Figure 14 The spatial neighboring block (any one of A0 to A4 or B0 to B5 in the model). As an example, the spatial neighboring block may include at least one of the top neighboring block, left neighboring block, top right neighboring block, top left neighboring block, or bottom left neighboring block.

[0323] When a block vector is stored at the location of a spatial neighboring block of the current block (i.e., when a neighboring block is encoded / decoded in intra-block copy mode), the block vector stored at the corresponding location can be set as a block vector for merging candidates.

[0324] On the other hand, when the block vector is not stored at the location of the spatial neighboring block, the corresponding spatial neighboring block may not be usable as a merge candidate.

[0325] A block vector merge list can be generated by inserting at least one merge candidate derived from at least one spatially neighboring block. If the number of merge candidates included in the block vector merge list is less than a threshold, at least one of the candidates included in the IBC HMVP list, paired block vector candidates, or zero block vectors can be inserted into the block vector merge list as a merge candidate.

[0326] When a block vector merge mode is applied to the current block, the block vector of the current block can be the same as the block vector of a merge candidate. Therefore, an index indicating one of the merge candidates included in the block vector merge list can be encoded and signaled.

[0327] As another example, when a block prediction mode is applied to the current block, block vector prediction candidates can be derived from the spatially neighboring blocks adjacent to the current block. The spatially neighboring blocks adjacent to the current block can include at least one of the top neighboring block, the left neighboring block, the top-right neighboring block, the top-left neighboring block, or the bottom-left neighboring block.

[0328] When a block vector is stored at the location of a spatial neighboring block of the current block (i.e., when a neighboring block is encoded / decoded in intra-block copy mode), the block vector stored at the corresponding location can be set as a block vector prediction candidate.

[0329] On the other hand, when a block vector is not stored at the location of a spatially neighboring block, the corresponding spatially neighboring block may not be usable as a block vector prediction candidate.

[0330] A block vector prediction list can be generated by inserting at least one block vector prediction candidate derived from at least one spatially neighboring block. If the number of block vector prediction candidates included in the block vector prediction list is less than a threshold, at least one of the candidates included in the IBC HMVP list, paired block vector candidates, or zero block vectors can be inserted as block vector prediction candidates into the block vector merging list.

[0331] When a block vector prediction mode is applied to the current block, a block vector prediction candidate can be set to the block vector prediction value of the current block. When multiple block vector prediction candidates are included in the block vector merging list, an index indicating one of the block vector prediction candidates can be encoded and signaled.

[0332] The block vector of the current block can be obtained by adding the predicted block vector value and the block vector difference. The encoder can encode the block vector difference and notify it with a signal.

[0333] Simultaneously, it can be determined whether to generate merge candidates or block vector prediction candidates by using spatially neighboring blocks, based on at least one of the size / shape or color components of the current block.

[0334] As an example, when the current block size is 4×4, the process of deriving merge candidates or block vector prediction candidates by using spatially neighboring blocks can be omitted. On the other hand, when the current block size is greater than 4×4, the process of generating merge candidates or block vector prediction candidates by using spatially neighboring blocks can be performed. In other words, merge candidates or block vector prediction candidates derived from spatially neighboring blocks can only be inserted into the list when the current block size is greater than 4×4.

[0335] Temporally neighboring blocks (i.e., co-located blocks within the same image) may not be used as merge candidates or block vector prediction candidates. In other words, block vectors can be derived based on data included in the same image as the current block (i.e., spatial data).

[0336] As another example, the availability of temporal neighbor blocks can be determined based on the type of the slice or image that includes the current block. For instance, when the slice or image including the current block is of type P or B, temporal neighbor blocks can be used to derive merge candidates or block vector prediction candidates. On the other hand, when the slice or image including the current block is of type I, temporal neighbor blocks may not be used to derive merge candidates or block vector prediction candidates.

[0337] As another example, template matching can be used to derive the block vector of the current block.

[0338] Figure 27 This is used to describe an example in which a block vector is derived by using template matching.

[0339] like Figure 27 In the example shown, a reference template similar to the current template, including the pre-reconstructed area around the current block, is searched within the pre-reconstructed area of ​​the current image.

[0340] When the reference template and the current template with the lowest template matching cost are determined, the distance between the current template and the reference template can be set as a block vector.

[0341] At the same time, a search for reference templates can be performed across the entire pre-reconstruction area within the current image.

[0342] Alternatively, reference templates can be searched only within the search scope. As in the example described above, the search scope can be set to have a predefined size / shape, or it can be set to have a size N times that of a coding tree unit.

[0343] When the block vector of the current block is determined, the predicted block S2620 of the current block can be derived based on the block vector.

[0344] Figure 28 This represents an example where the predicted block for the current block is derived using a block vector.

[0345] like Figure 28 In the example shown, a reference block within the current block can be specified using a block vector. The reference block indicated by the block vector can be set as the prediction block of the current block.

[0346] When applying inter-frame prediction, a reference block within a reference image can be specified based on the motion vector of the current block, and this reference block can be set as the predicted block of the current block. When applying intra-frame block copying, a reference block within the current image can be specified based on the block vector of the current block, and this reference block can be set as the predicted block of the current block.

[0347] Inter-frame prediction and intra-frame block copying have something in common: the reference block specified based on the vector components of the current block is set as the prediction block of the current block.

[0348] In this case, the reference block can be modified based on the weight parameters, and the modified reference block can be set as the prediction block of the current block.

[0349] Weight parameters can be derived based on the pre-reconstructed regions surrounding the current block and the pre-reconstructed regions surrounding the reference block. The weight parameters may include filter coefficients used to filter samples within the reference block. For ease of description, in the embodiments and figures described below, it is assumed that the reference block is included in the current image (i.e., intra-frame block copy mode is applied). However, even when the reference block is included in the reference image (i.e., when inter-frame prediction is applied), the method for obtaining the prediction block according to the embodiments described below can still be applied.

[0350] Figure 29 The procedure for deriving the modified reference block is shown.

[0351] Reference Figure 29 The weight parameter S2910 can be obtained through the filter coefficient learning step.

[0352] The weight parameters can be derived by using the current template around the current block and the reference template around the reference block. As an example, the current template may include a pre-reconstructed region located at the top of the current block and a pre-reconstructed region located to the left of the current block, and the reference template may include a pre-reconstructed region located at the top of the reference block and a pre-reconstructed region located to the left of the reference block.

[0353] Figure 30 The configuration of the current template and the reference template is shown.

[0354] like Figure 30 In the example shown, the current template and the reference template can have the same shape and size. For ease of description, it is assumed that the coordinates of the reconstructed samples included in the current template are determined by assuming the position of the top-left sample of the current block is (0, 0). Similarly, it is assumed that the coordinates of the reconstructed samples included in the reference template are determined by assuming the position of the top-left sample of the reference block is (0, 0). In the example shown, the current template and the reference template respectively include left-side reconstructed samples with x-axis coordinates of -1 to -4 and top-side reconstructed samples with y-axis coordinates of -1 to -4.

[0355] The reconstructed samples included in the current template and the reference template are referred to as reference samples. Unless otherwise described, in the embodiments described below, the term "template" is understood to refer to at least one of the current template or the reference template. Furthermore, the term "template" is used even when describing embodiments that can be generally applied to the current template and the reference template.

[0356] Filter coefficients that minimize the difference between the reference samples included in the reference template and the reference samples in the current template can be derived. Specifically, when a filter is applied to reference samples within the reference template, filter coefficients that minimize the difference with reference samples at the same position in the current template can be derived. In this case, the filter applied to the reference samples can be a convolutional filter.

[0357] Figure 31 The convolution filter applied to the reference sample is shown.

[0358] like Figure 31 In the example shown, the output value of the convolution filter can be determined by using a reference sample (C) and four neighboring samples (N, W, S, E).

[0359] Equation 5 shows an example used to derive the filter coefficients.

[0360] [Equation 5]

[0361] E = ∑ (i,j))∈T {Y-(w0·C+w1·N+w2·S+w3·E+w4·W+w5·B)} 2

[0362] In Equation 5, [i, j] represents the coordinates of the reference sample. T represents the template region. In other words, [i, j] ∈ T represents the coordinates of the reference sample within the current template or the reference template.

[0363] C represents the reference sample to which the filter within the reference template is applied. In other words, C can represent the reference sample at position (i, j) within the reference template. Y represents the reference sample within the current template at the same position as C, i.e., the reference sample at position (i, j) within the current template.

[0364] N, S, E, and W represent samples within the reference template that are adjacent to the target reference sample C. For example, N can represent the sample adjacent to the top of the target reference sample C, i.e., the sample at position [i, j-1]. S can represent the sample adjacent to the bottom of the target reference sample C, i.e., the sample at position [i, j+1]. W can represent the sample adjacent to the left of the target reference sample C, i.e., the sample at position [i-1, j]. E can represent the sample adjacent to the right of the target reference sample C, i.e., the sample at position [i+1, j].

[0365] B can be a value derived based on the bit depth of the image. As an example, Equation 6 shows an example in which the variable B is derived.

[0366] [Equation 6]

[0367] B = 1 << (D-1)

[0368] In Equation 6, D represents the bit depth. As an example, when the bit depth is 10 bits, B can be set to 512, an intermediate value within the range that can be expressed in 10 bits. Alternatively, when the bit depth is 8 bits, B can be set to 128, an intermediate value within the range that can be expressed in 8 bits.

[0369] As another example, when a filter is applied to a reference sample within the current template, filter coefficients that minimize the difference with the reference sample at the same location within the reference template can be derived. In this case, in Equation 5, Y can represent the reference sample within the reference template, and C, N, S, W, and E can represent the reference sample within the current template.

[0370] In Equation 5, the filter coefficients w0 to w5 that minimize the value of E can be obtained. Regression analysis can be used to derive the filter coefficients.

[0371] In this scenario, after applying a smoothing filter to the pixels within both the reference template and the current template, filter coefficients can be obtained. The reference block S2920 can be modified by applying these filter coefficients to the reference block. Specifically, samples within the reference block can be modified by performing convolution on the samples using the filter coefficients. The modified samples can then be set as the predicted samples for the current block. Simultaneously, the filter applied to the samples within the reference block can be... Figure 31 The same as shown.

[0372] Equation 7 represents an example of modifying the reconstructed sample within the reference block.

[0373] [Equation 7]

[0374] Y'=w0·C'+w1·N′+w2·S′+w3·E′+w4·W′+w5·B,[i,j]∈M

[0375] In Equation 7 above, M represents the reference block. In other words, [i, j] ∈ M represents the coordinates of a sample within the reference block.

[0376] Y' represents a refined (modified) sample within the reference block. In other words, Y' can represent a refined sample at position (i, j) within the reference block, and the refined sample can be set as the predicted sample at position (i, j) within the current block.

[0377] C' represents a sample within the reference block. In other words, C' can represent a sample at position (i, j) within the reference block.

[0378] N', S', E', and W' represent samples adjacent to sample C' within the reference block. For example, N' could represent the sample at the top of the neighboring sample C, i.e., the sample at position [i, j-1]. S' could represent the sample at the bottom of the neighboring sample C', i.e., the sample at position [i, j+1]. W' could represent the sample to the left of the neighboring sample C', i.e., the sample at position [i-1, j]. E' could represent the sample to the right of the neighboring target reference sample C', i.e., the sample at position [i+1, j].

[0379] Meanwhile, the filters applied to the reference template or reference block are not limited to Figure 31 The example shown illustrates this. As an example, a 1D filter (1×N or N×1 filter) or a square filter (e.g., N×N) can be used to derive filter coefficients or to derive refined samples. Here, N is a natural number and can be 1, 2, 3, 4, or 5.

[0380] An index indicating one of a plurality of filter candidates can be encoded and signaled. Alternatively, the filter type can be adaptively determined based on at least one of the size / shape of the current block or the encoding pattern of a reference block.

[0381] Depending on the filter type, the number of filter coefficients included in the weighting parameters can vary.

[0382] As an example, the number of filter coefficients can be set differently depending on the size of the current block. For instance, when the size of the current block is greater than a threshold, a filter type that includes a larger number of filter coefficients can be used compared to the case where the size of the current block is equal to or less than the threshold.

[0383] Alternatively, multiple thresholds can be defined to determine the filter type for the current block. Different filter types can be mapped to each interval distinguished by multiple thresholds.

[0384] The intervals in which the current block size exists can be determined by comparing the current block size with multiple thresholds. After determining the interval to which the current block size belongs, the predicted sample for the current block can be obtained by using a filter type mapped to the corresponding interval.

[0385] As another example, the number of filter coefficients can be adaptively determined based on the size of the current image, including the current block.

[0386] Alternative sites, such as... Figure 31 The multi-tap filter shown can derive a single weight that minimizes the difference between the sum of reference samples included in the reference template and the sum of reference samples included in the current template. In this case, the predicted block of the current block can be obtained by multiplying the reference block by the weight.

[0387] In the example described above, it is assumed that the filter applied to the reference template and the filter applied to the reference block are of the same type, but the filter applied to the reference template and the filter applied to the reference block can be of different types.

[0388] As described above, the filter coefficient learning step is performed using a pre-reconstructed region within the current image. Therefore, the encoder and decoder can derive the filter coefficients in the same way.

[0389] As another example, information about the filter coefficient set can be encoded and signaled to the decoder. The filter coefficient set can include w0 to w5 as described above. As an example, an index indicating one of a plurality of filter coefficient set candidates can be encoded and signaled.

[0390] Convolution is used in filter learning and filter application processes. In this case, it involves considering the shape of the filter and the locations where samples should be used beyond the template.

[0391] For example, when the application Figure 31 When using the cross-shaped filter shown, the reference sample located at position (-4, -4) in the upper left of the template should be filtered by using the sample at position (-5, -4) and the sample at position (-4, -5) which are outside the reference template.

[0392] For buffer optimization, samples that exceed the template can be made unusable, or filtering can be performed using only reference samples within the template.

[0393] As an example, the filter may not be applied to reference samples that are edges within the reference template. In other words, reference samples that are edges within the reference template are not used as samples at position C in Equation 5, and only reference samples at other positions can be used as samples at position C in Equation 5.

[0394] As another example, the type of filter applied to the reference sample located at the edge of the template can be different from the type of filter applied to the reference sample that is not located at the edge of the template.

[0395] For example, a horizontal 1D filter, such as an N×1 shape filter, can be applied to a reference sample adjacent to the top or bottom boundary of the template. A vertical 1D filter, such as a 1×N shape filter, can be applied to a reference sample adjacent to the left or right boundary of the template.

[0396] As another example, filtering can be performed by replacing the values ​​of samples that are outside the reference template with the values ​​of available reference samples.

[0397] Additionally, at least a portion of the reference template may extend beyond the boundaries of a slice, tile, or image, and unavailable samples may exist within the reference template.

[0398] As mentioned above, when unavailable samples exist, padding can be performed to replace the values ​​of unavailable samples with the values ​​of available samples.

[0399] Figure 32 This represents an example where padding is performed on unavailable samples.

[0400] For ease of description, assume that the samples taken from the template are set to be unavailable.

[0401] Filling can be performed by copying the sample closest to the unavailable sample. As an example, the values ​​of pixels located to the left, right, top, or bottom of the unavailable sample can be copied to the unavailable sample location.

[0402] Meanwhile, when there are multiple available neighboring samples, the unavailable samples can be filled by averaging, weighting, or interpolating the available neighboring samples.

[0403] As an example, for position A outside the reference template, the left neighbor sample and the top neighbor sample are available. In this case, the average of the left neighbor sample and the top neighbor sample can be set as the sample value at position A.

[0404] Similarly, for position A' outside the current template, the left neighbor sample and the top neighbor sample are available. In this case, the average of the left neighbor sample and the top neighbor sample can be set as the sample value at position A'.

[0405] As another example, the values ​​of unavailable samples can be derived by copying the values ​​of samples located in a predefined direction. Here, the predefined direction can be left, right, top, or bottom.

[0406] Furthermore, the predefined orientation can vary depending on the relative position of the unavailable sample to the template. For example, when the unavailable sample is at the top of the template, the value of the bottom neighboring sample can be set to the value of the unavailable sample. Alternatively, when the unavailable sample is at the top left of the template, the value of the right neighboring sample can be set to the value of the unavailable sample.

[0407] As another example, padding can be performed by taking into account the encoding mode of the reference block. For instance, when the reference block is encoded in directional intra-prediction mode, the values ​​of samples located in the forward or backward direction according to the directional intra-prediction mode can be set to the values ​​of unavailable samples.

[0408] As an example, when the upper right diagonal intra-frame prediction mode is applied to a reference block, the values ​​of samples located at unavailable sample locations in the upper right or upper left diagonal direction can be set to the values ​​of unavailable samples.

[0409] The same problem can occur even when applying a filter to a reference block. As an example, when samples located at the edges of the reference block (i.e., at the boundaries of the reference block) are filtered, samples located outside the reference block should be used. For example, to filter... Figure 31 The cross-shaped filter shown is applied to a 4×4 reference block and requires samples within a 6×6 region.

[0410] When samples located outside the reference block are available, the samples at the corresponding locations can be used to filter samples located at the edges inside the reference block.

[0411] As another example, padding can be performed to generate values ​​for samples located outside the reference block. In other words, samples located outside the reference block (e.g., samples included in the reference template) can be set to unavailable, and padding can be performed to generate values ​​for unavailable samples.

[0412] Figure 33 This represents an example where values ​​for samples adjacent to a reference block are generated by padding.

[0413] Padded can be performed by copying samples from adjacent positions within a reference block. For example, the value of a sample adjacent to the top of the reference block can be generated by copying the value of a sample adjacent to the top boundary of the reference block, and the value of a sample adjacent to the bottom of the reference block can be generated by copying the value of a sample adjacent to the bottom boundary of the reference block. Similarly, the value of a sample adjacent to the left of the reference block can be generated by copying the value of a sample adjacent to the left boundary of the reference block, and the value of a sample adjacent to the right of the reference block can be generated by copying the value of a sample adjacent to the right boundary of the reference block.

[0414] As another example, padding can be performed only on samples located outside the reference block and not belonging to the reference template. In other words, samples located outside the reference block but belonging to the reference template can be set up to be used for filtering the reference samples.

[0415] Figure 34 This represents an example in which padding is performed only on sample locations outside the reference block that are not part of the reference template.

[0416] When the reference template includes the top and left pre-reconstructed regions of the reference block, filtering can be performed on samples adjacent to the top or left boundary within the reference block by using reference samples belonging to the reference template. On the other hand, to filter samples adjacent to the right or bottom boundary within the reference block, padding can be performed to generate samples adjacent to the right or bottom boundary of the reference block.

[0417] As another example, the type of filter to be applied to the samples within the reference block can be adaptively determined based on whether neighboring blocks outside the reference block are available. For example, when the left or right neighboring samples of the target sample to which the filter is to be applied are unavailable, a vertical 1D filter (e.g., a 1×N-shaped filter) can be applied to filter the target sample.

[0418] On the other hand, when the top or bottom neighboring samples of the target sample to which the filter is to be applied within the reference block are unavailable, a horizontal 1D filter (e.g., a 1×N-shaped filter) can be applied to filter the target sample. Here, N can be a natural number, such as 1, 3, or 5.

[0419] In the example described above, a reference block modified based on the weight parameters was set as the prediction block for the current block. Unlike the described example, after setting the reference block as the prediction block for the current block, the prediction block can also be modified using the weight parameters. The modified prediction block can then be set as the final prediction block for the current block. In other words, the current block can be reconstructed by adding a residual block to the modified prediction block.

[0420] Simultaneously, modified prediction samples can be obtained by applying a filter to the prediction samples of the current block. In this case, such as... Figure 33 As described, the values ​​of samples outside the current block can be derived by copying the values ​​of the outermost predicted samples within the current block. For example, the values ​​of samples adjacent to the top boundary of the current block can be generated by copying the values ​​of predicted samples adjacent to the top boundary of the current block, and the values ​​of samples adjacent to the bottom boundary of the current block can be generated by copying the values ​​of predicted samples adjacent to the bottom boundary of the current block. Similarly, the values ​​of samples adjacent to the left boundary of the current block can be generated by copying the values ​​of predicted samples adjacent to the left boundary of the current block, and the values ​​of samples adjacent to the right boundary of the current block can be generated by copying the values ​​of predicted samples adjacent to the right boundary of the current block.

[0421] As another example, since the top and left neighboring regions of the current block are pre-reconstructed, the top reconstructed samples or the left reconstructed samples can be used to filter the predicted samples located at the top or left boundary of the current block.

[0422] On the other hand, the right and bottom adjacent regions of the current block have not yet been decoded. Therefore, as Figure 34 As described, samples adjacent to the right side of the current block or adjacent to the bottom of the current block can be generated by padding.

[0423] In other words, such as Figure 34 As described, samples adjacent to the top or left boundary of the current block use the reference samples belonging to the current template as is, while samples adjacent to the right or bottom boundary of the current block can be generated by padding.

[0424] As another example, the values ​​of unavailable samples around the current block can be derived based on samples around a reference block. For instance, since the top and left neighboring regions of the current block are pre-reconstructed, filtering of the predicted samples within the current block can be performed using the top and left reconstructed samples of the current block.

[0425] On the other hand, the right and bottom neighbor regions of the current block have not yet been decoded. Meanwhile, the right and bottom neighbor regions of the reference block, indicated by the motion vector of the current block, can be pre-decoded. With this in mind, the values ​​of the right or bottom neighbor samples of the reference block can be copied and set as the values ​​of the right or bottom neighbor samples of the current block.

[0426] As another example, the predicted block for the current block can be obtained using multiple reference blocks and multiple reference templates. Specifically, for each of the multiple reference blocks, weight parameters are derived based on each reference template. The reference blocks can be modified based on the derived weight parameters. The predicted block for the current block can then be obtained by averaging or weighting the modified reference blocks. For ease of description, assume the number of reference blocks is 2.

[0427] Figure 35 This is a diagram used to describe an example in which the predicted block of the current block is obtained by using multiple reference blocks.

[0428] The first weight parameter of the first reference block can be obtained by using the first reference template adjacent to the first reference block, and the second weight parameter of the second reference block can be obtained by using the second reference template adjacent to the second reference block.

[0429] The first reference block can be modified based on the first weight parameter, and the second reference block can be modified based on the second weight parameter.

[0430] The predicted block for the current block can then be obtained based on the average or weighted sum of the modified first reference block and the modified second reference block. Equation 8 expresses this process as a formula.

[0431] [Equation 8]

[0432]

[0433] In Equation 8 above, P'[i,j] represents the predicted sample at position [i,j]. P[k,i,j] represents the sample at position [i,j] within the k-th reference block. For example, when k is 0, it can represent the sample within the first reference block, and when k is 1, it can represent the sample within the second reference block. Additionally, w[k] represents the weight parameter of reference block k, and w[n] represents the parameter applied to B. B is a variable derived from the bit depth and can be derived, for example, according to Equation 6.

[0434] M represents the reference block. In other words, [i, j] ∈ M can represent the coordinates of a sample within the reference block.

[0435] The weight parameter w[k] of the reference block can include multiple filter coefficients. For example, multiple filter coefficients w[0] to w[n] can be derived based on Equation 9 below.

[0436] [Equation 9]

[0437]

[0438] In Equation 9 described above, N represents the template. In other words, [i, j] ∈ N represents the position of a sample within the template. Additionally, Y[i, j] represents the sample at position [i, j] within the current template, and X[k, i, j] represents the sample at position [i, j] within the k-th reference template. Equation 9 yields the parameters w[0] to w[n] that minimize E. In this case, regression analysis can be used.

[0439] Multiple reference blocks can be derived based on at least one of a merge list (e.g., a motion information merge list or a block vector merge list), a vector prediction list (e.g., a motion vector prediction list or a block vector prediction list), or template matching.

[0440] As an example, among multiple reference blocks, a first reference block can be derived based on any one of the merge list, the vector prediction list, and the template match, and a second reference block can be derived based on another of the merge list, the vector prediction list, and the template match.

[0441] Alternatively, multiple reference blocks can be determined by selecting multiple merge candidates from a merge list. As an example, a first reference block can be determined using a first merge candidate within the merge list, and a second reference block can be determined using a second merge candidate within the merge list. To specify the first and second merge candidates, the two index information can be encoded and signaled.

[0442] Alternatively, among multiple reference blocks, a first reference block can be determined through inter-frame prediction, and a second reference block can be determined through intra-frame block copying. In this case, the first weight parameter of the first reference block can be obtained based on the current template in the current image and the reference template in the reference image, and the second weight parameter of the second reference block can be obtained based on the current template in the current image and the reference template in the current image.

[0443] In the implementation described above, a reference block can be derived based on at least one of a merge list (e.g., a motion information merge list or a block vector merge list), a vector prediction list (e.g., a motion vector prediction list or a block vector prediction list), or template matching.

[0444] Alternatively, the implementation method for modifying the reference block may be applied only when the reference block (or block vector or motion vector) is obtained through template matching.

[0445] Alternatively, in the resulting prediction block, information indicating whether to modify the reference block can be encoded and signaled. This information can be a 1-bit flag.

[0446] Simultaneously, it can be determined whether to encode / decode the information based on at least one of the size / shape of the current block and the pattern used to derive the reference block. As an example, the information can be encoded / decoded when the size of the current block (e.g., at least one of the width, height, or number of included samples) is less than a threshold or when the pattern used to derive the reference block is a template matching pattern.

[0447] Furthermore, in the embodiment described above, the template is shown to include both the top region of the block and the left side region of the block. In this case, the configuration of the template can be adaptively determined based on the current size or shape of the block.

[0448] As an example, when the current block is a square shape with the same width and height, the template can include both the top and left sides of the block. Conversely, when the current block is a rectangle with different widths and heights, the template can include only the top or left side of the block. For example, when the current block is a rectangle with a width greater than its height, the template can include only the top side of the block. Conversely, when the current block is a rectangle with a height greater than its width, the template can include only the left side of the block.

[0449] exist Figure 31 The diagram shows a template with a line count (i.e., thickness) of 4. The template thickness can be predefined in the encoder and decoder. Alternatively, the template thickness can be adaptively determined based on the size of the current block. As an example, when the current block size is equal to or greater than a threshold, the template thickness can be set to N. On the other hand, when the current block size is less than the threshold, the template thickness can be set to a value less than N (e.g., N-1, N-2, or N-3, etc.). N can be a natural number, such as 2, 3, or 4.

[0450] The thickness of the top region and the thickness of the left region can have different values.

[0451] Multiple template candidates can be predefined in the encoder and decoder, and at least one of the template candidates, such as shape or thickness, is different. In this case, an index indicating one of the multiple template candidates can be encoded and signaled. Based on the template candidate selected by the index, at least one of the template's shape or size can be determined.

[0452] Implementations described focusing on the decoding or encoding process are included within the scope of this disclosure when applied to the encoding or decoding process. Implementations described in a different order than those described in the predetermined order are also included within the scope of this disclosure.

[0453] The disclosure described above is based on a series of steps or flowcharts, but it does not limit the temporal order of the invention, and may be performed simultaneously or in different orders if necessary. Furthermore, each component (e.g., unit, module, etc.) configuring the block diagrams in the disclosure described above can be implemented as a hardware device or software, and multiple components can be combined and implemented as a single hardware device or software. As an example, a hardware device may include at least one of a processor for performing operations, a memory for storing data, a transmitter for transmitting data, and a receiver for receiving data.

[0454] The above disclosure can be recorded on a computer-readable recording medium by implementing it in the form of program instructions executable by various computer components. The computer-readable recording medium may include program instructions, data files, data structures, etc., individually or in combination.

[0455] Furthermore, according to this disclosure, a computer-readable recording medium can be provided for storing a bitstream generated by the encoding method described above. The bitstream can be transmitted by an encoding device, and a decoding device can receive the bitstream and decode the image.

[0456] Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as optical floppy disks; and hardware devices specifically configured to store and execute program instructions, such as ROMs, RAMs, and flash memory. The hardware device may be configured to operate as at least one software module to perform processing according to this disclosure, and vice versa.

[0457] Industrial applicability

[0458] This disclosure can be applied to computing or electronic devices that can encode / decode video signals.

Claims

1. A method for decoding video, comprising: Obtain the block vector or motion information of the current block; The reference block for the current block is derived based on the motion information or the block vector; The weight parameters are derived based on the current templates surrounding the current block and the reference templates surrounding the reference block; as well as The prediction block for the current block is generated based on the weight parameters.

2. The method according to claim 1, wherein, The weighting parameters include filter coefficients, and Wherein, when the filter is applied to a reference sample within the reference template, the filter coefficients minimize the difference with the reference sample within the current template.

3. The method according to claim 1, wherein, The predicted samples for the current block are obtained by modifying the samples within the reference block based on the weight parameters.

4. The method according to claim 3, wherein, The modified sample within the reference block is obtained by filtering the sample using neighboring samples adjacent to the sample.

5. The method according to claim 4, wherein, When a neighboring sample adjacent to the sample is unavailable, the value of the neighboring sample is set to be equal to the value of the sample.

6. The method according to claim 4, wherein, When a neighboring sample adjacent to the sample is located outside the reference block, the value of the neighboring sample is set to be equal to that of the reconstructed sample at the corresponding location.

7. The method according to claim 4, wherein, The filter applied to the sample is a cross-shaped filter with the same width and height.

8. The method according to claim 4, wherein, The filter shape applied to the sample is adaptively determined based on the sample's location.

9. A method for encoding video, comprising: Obtain the block vector or motion information of the current block; The reference block for the current block is derived based on the motion information or the block vector; The weight parameters are derived based on the current templates surrounding the current block and the reference templates surrounding the reference block; as well as The prediction block for the current block is generated based on the weight parameters.

10. The method according to claim 9, wherein, The weighting parameters include filter coefficients, and Wherein, when the filter is applied to a reference sample within the reference template, the filter coefficients minimize the difference with the reference sample within the current template.

11. The method according to claim 9, wherein, The predicted samples for the current block are obtained by modifying the samples within the reference block based on the weight parameters.

12. The method according to claim 11, wherein, The modified sample within the reference block is obtained by filtering the sample using neighboring samples adjacent to the sample.

13. The method according to claim 12, wherein, When a neighboring sample adjacent to the sample is unavailable, the value of the neighboring sample is set to be equal to the value of the sample.

14. The method according to claim 12, wherein, When a neighboring sample adjacent to the sample is located outside the reference block, the value of the neighboring sample is set to be equal to that of the reconstructed sample at the corresponding location.

15. A computer-readable recording medium storing a bitstream generated by a video coding method, the video coding method comprising: Obtain the block vector or motion information of the current block; The reference block for the current block is derived based on the motion information or the block vector; The weight parameters are derived based on the current templates surrounding the current block and the reference templates surrounding the reference block; as well as The prediction block for the current block is generated based on the weight parameters.

Images