Image decoding method, image encoding method, and recording medium storing bitstream

Abstract

Disclosed are an image decoding method, an image encoding method, and a recording medium storing a bitstream. The decoding method includes determining a search area for updating motion information of a current block; generating an SAD list including a plurality of SAD candidates for the search area; deriving a delta motion information of the current block based on the SAD candidates in the SAD list; updating the motion information of the current block based on the delta motion information; performing inter prediction on the current block based on the updated motion information; and reconstructing the current block based on a prediction block obtained through the inter prediction, wherein the deriving of the delta motion information is performed based on a comparison result between a reference SAD candidate and a predetermined threshold, the reference SAD candidate means an SAD candidate corresponding to a non-directional offset, and when the reference SAD candidate is greater than or equal to the predetermined threshold, the deriving of the delta motion information includes identifying an SAD candidate having a minimum value among the SAD candidates belonging to the SAD list, and determining the delta motion information based on an offset corresponding to the identified SAD candidate.

Description

[0001] This application is a divisional application of Chinese invention patent application No. 201980019969.4, filed on March 19, 2019, entitled "Image Decoding Method / Apparatus, Image Encoding Method / Apparatus and Recording Medium for Storing Bit Streams". Technical Field

[0002] This invention relates to image encoding / decoding methods and apparatus. More specifically, this invention relates to image decoding methods / apparatus using motion derived from the decoder side, and corresponding image encoding methods / apparatus. Background Technology

[0003] The market demand for high-resolution video is increasing, thus requiring technologies capable of efficiently compressing high-resolution video. In response to these market demands, the ISO / IEC Moving Picture Experts Group (MPEG) and the ITU-T Video Coding Experts Group (VCEG) jointly established the JCT-VC (Joint Collaboration on Video Coding), completing the development of the HEVC (High Efficiency Video Compression Coding) video compression standard in January 2013, and are actively researching and developing next-generation compression standards.

[0004] Video compression mainly consists of intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, and in-loop filtering. Meanwhile, with the increasing demand for high-resolution video, the demand for stereoscopic video content as a new video service is also increasing. Video compression techniques for effectively delivering high-resolution and ultra-high-resolution stereoscopic image content are being discussed.

[0005] Regarding this point, for inter-frame prediction, generating motion information for each current block and sending all generated motion information as a signal can lead to problems such as degradation of coding or encoder efficiency. Summary of the Invention

[0006] Technical issues

[0007] The object of this invention is to provide an image encoding / decoding method and apparatus with improved efficiency. In particular, the object of this invention is to provide an image decoding method and apparatus that utilizes motion information derived from the decoder side. Additionally, the object of this invention is to provide an image encoding method and apparatus that utilizes motion information derived from the decoder side.

[0008] In addition, the purpose of this invention is to provide an image decoding method and apparatus that, when using one or more motion information extraction methods to extract motion information on the decoder side, uses different motion information correction methods according to the target motion information extraction method.

[0009] In addition, the present invention aims to provide an image decoding method and apparatus, which includes a method for prematurely terminating the motion information export process when certain conditions are met while exporting motion information using one or more motion information export methods on the decoder side.

[0010] In addition, the present invention aims to provide an image decoding method and apparatus that uses subsample block matching when deriving motion information using one or more motion information deriving methods on the decoder side.

[0011] In addition, the present invention aims to provide an image encoding method and apparatus that uses signal transmission and provides relevant information to derive motion information on the decoder side using one or more motion information derivation methods.

[0012] Another object of the present invention is to provide a computer-readable recording medium for storing bitstreams generated by a video encoding method / device.

[0013] Technical solutions

[0014] The video encoding / decoding method and apparatus according to the present invention can determine a search region for improving the motion information of the current block, generate a SAD list for the search region, derive the change motion information of the current block based on the SAD candidates in the SAD list, and update the motion information of the current block using the motion information and change motion information of the current block.

[0015] In the video encoding / decoding method and apparatus according to the present invention, the SAD list may include multiple SAD candidates.

[0016] In the image encoding / decoding method and apparatus according to the present invention, the search region is determined to be a region that includes at least one of a reference block or an adjacent region of the reference block, and the adjacent region may be a region extending N sample lines from the boundary of the reference block.

[0017] In the video encoding / decoding method and apparatus according to the present invention, SAD candidates can be determined by the SAD value between the L0 block and the L1 block. The L0 block can be determined based on the position of the L0 reference block of the current block and a predetermined offset. The L1 block can be determined based on the position of the L1 reference block of the current block and a predetermined offset.

[0018] In the image encoding / decoding method and apparatus according to the present invention, the offset may include at least one of non-directional offset or directional offset. Directional offset may include offset in at least one direction: left, right, top, bottom, upper left, upper right, lower left, or lower right.

[0019] In the image encoding / decoding method and apparatus according to the present invention, the step of deriving change motion information can be performed based on the comparison result between a reference SAD candidate and a predetermined threshold, and the reference SAD candidate can refer to the SAD candidate corresponding to the non-directional offset.

[0020] In the image encoding / decoding method and apparatus according to the present invention, when the reference SAD candidate is greater than or equal to a threshold, the step of deriving the change amount motion information may include: identifying the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and determining the change amount motion information based on the offset corresponding to the identified SAD candidate.

[0021] In the image encoding / decoding method and apparatus according to the present invention, when the reference SAD candidate is less than a threshold, the step of deriving the change motion information may include: calculating a predetermined parameter using all or a portion of the SAD candidates belonging to the SAD list; and determining the change motion information based on the calculated parameter.

[0022] In the image encoding / decoding method and apparatus according to the present invention, the motion information of the current block can be updated in a limited manner by considering at least one of the block size, the distance between the current image and the reference image, the inter-frame prediction mode, the prediction direction, or the unit / resolution of the motion information.

[0023] Beneficial effects

[0024] According to the present invention, an image encoding / decoding method and apparatus with improved efficiency can be provided as described below.

[0025] First, according to the present invention, since the decoder uses one or more motion information derivation methods to derive motion information, it can provide an image decoding method and apparatus for improving coding efficiency.

[0026] Secondly, according to the present invention, when one or more motion information extraction methods are used on the decoder side, by using different motion information correction methods according to the motion information extraction methods, an image decoding method and apparatus for improving decoding efficiency can be provided.

[0027] Third, according to the present invention, when one or more motion information derivation methods are used on the decoder side, a method for improving the decoding efficiency of the image decoding method and device is provided by using a method to terminate the motion information derivation process in advance when certain conditions are met.

[0028] Furthermore, according to the present invention, a computer-readable recording medium storing a bitstream generated by the video encoding method / device according to the present invention can be provided. Attached Figure Description

[0029] Figure 1 This is a block diagram illustrating an image encoding device according to the present invention.

[0030] Figure 2 This is a block diagram illustrating an image decoding device according to the present invention.

[0031] Figure 3 The invention illustrates a concept for a motion information extraction method using template matching, as one of the methods for extracting motion information on the decoder side.

[0032] Figure 4 This is a diagram illustrating a concept of a motion information deriving method using bidirectional matching, according to the present invention, as one of the methods for deriving motion information on the decoder side.

[0033] Figure 5 This is an example of a flowchart of a decoding method according to an embodiment of the present invention.

[0034] Figure 6 and Figure 7 Another flowchart of a decoding method according to an embodiment of the present invention is shown.

[0035] Figure 8 This is a diagram illustrating another flowchart of a decoding method according to an embodiment of the present invention.

[0036] Figure 9 This is a diagram illustrating the diamond search and cross search methods.

[0037] Figure 10 and Figure 11 This is a diagram illustrating another flowchart of a decoding method according to an embodiment of the present invention.

[0038] Figure 12 An example of the target block and subsampled pixels within the block for performing block matching is shown.

[0039] Figure 13 The above is shown Figure 12 The subsampled pixels are applied to the method of deriving motion information using template matching.

[0040] Figure 14 The above is shown Figure 12 The subsampled pixels are applied to a method that uses bidirectional matching to derive motion information.

[0041] Figure 15 An example is shown of constructing a list of motion candidates for the initial search location selection to derive motion information on the decoder side.

[0042] Figure 16 The concept of parallel processing of sub-blocks according to the present invention is shown.

[0043] Figure 17 A modified list of motion candidates, for example, is shown for use in parallel processing of sub-blocks. Detailed Implementation

[0044] Best practice

[0045] The video encoding / decoding method and apparatus according to the present invention can determine a search region for improving the motion information of the current block, generate a SAD list for the search region, derive the change motion information of the current block based on the SAD candidates in the SAD list, and update the motion information of the current block using the motion information and change motion information of the current block.

[0046] In the video encoding / decoding method and apparatus according to the present invention, the SAD list may include multiple SAD candidates.

[0047] In the image encoding / decoding method and apparatus according to the present invention, the search region is determined to be a region that includes at least one of a reference block or an adjacent region of the reference block, and the adjacent region may be a region extending N sample lines from the boundary of the reference block.

[0048] In the video encoding / decoding method and apparatus according to the present invention, SAD candidates can be determined by the SAD value between the L0 block and the L1 block. The L0 block can be determined based on the position of the L0 reference block of the current block and a predetermined offset. The L1 block can be determined based on the position of the L1 reference block of the current block and a predetermined offset.

[0049] In the image encoding / decoding method and apparatus according to the present invention, the offset may include at least one of non-directional offset or directional offset. Directional offset may include offset in at least one direction: left, right, top, bottom, upper left, upper right, lower left, or lower right.

[0050] In the image encoding / decoding method and apparatus according to the present invention, the step of deriving change motion information can be performed based on the comparison result between a reference SAD candidate and a predetermined threshold, and the reference SAD candidate can refer to the SAD candidate corresponding to the non-directional offset.

[0051] In the image encoding / decoding method and apparatus according to the present invention, when the reference SAD candidate is greater than or equal to a threshold, the step of deriving the change amount motion information may include: identifying the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and determining the change amount motion information based on the offset corresponding to the identified SAD candidate.

[0052] In the image encoding / decoding method and apparatus according to the present invention, when the reference SAD candidate is less than a threshold, the step of deriving the change motion information may include: calculating a predetermined parameter using all or a portion of the SAD candidates belonging to the SAD list; and determining the change motion information based on the calculated parameter.

[0053] In the image encoding / decoding method and apparatus according to the present invention, the motion information of the current block can be updated in a limited manner by considering at least one of the block size, the distance between the current image and the reference image, the inter-frame prediction mode, the prediction direction, or the unit / resolution of the motion information.

[0054] Embodiments of the present invention

[0055] In the following, embodiments of the invention will be described in detail with reference to the accompanying drawings, enabling those skilled in the art to readily implement the invention. However, the invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. For clarity, components unrelated to the description are omitted throughout the specification, and similar components are indicated by similar reference numerals.

[0056] Throughout the specification, when a claim is made that one part is “connected” to another part, examples are included not only of direct connections between the two parts, but also of electrical connections between the part and another component therebetween.

[0057] Furthermore, throughout this specification, when a component is referred to as "including" an element, it should be understood that, unless otherwise specified, the element may include other elements rather than be separate from other elements.

[0058] Additionally, terms such as "first," "second," etc., may be used to describe various components, but components should not be limited by these terms. These terms are used only for the purpose of distinguishing one component from another.

[0059] Furthermore, in embodiments of the apparatus and method described herein, some components of the apparatus or some steps of the method may be omitted. Additionally, the order of some components of the apparatus or the order of some steps of the method may be changed. Moreover, other components or steps may be inserted into some components of the apparatus or some steps of the method.

[0060] Furthermore, some components or steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or may replace some components or steps of the second embodiment.

[0061] Furthermore, the components shown in the embodiments of the present invention are illustrated independently to indicate different functional features, and do not imply that each component consists of a separate hardware or software component unit. That is, for ease of description, each component is listed as a single component, and at least two components of each component can be combined to form a single component, or a single component can be divided into multiple components to perform functions. Embodiments of integration and separation of each of these components are also included within the scope of the present invention without departing from its spirit.

[0062] First, the terminology used in this application will be briefly described below.

[0063] The decoding device (video decoding device) described later can be a civilian security camera, a civilian security system, a military security camera, a military security system, a personal computer (PC), a laptop computer, a portable multimedia player (PMP), a wireless communication terminal, a smartphone, a device included in a server terminal such as a TV application server and a service server, and can represent a user terminal such as various devices, a communication device such as a modem for performing communication with wired / wireless communication networks, a memory for storing various programs and data for decoding images or performing inter-frame prediction or intra-frame prediction for decoding, and various devices equipped with a microprocessor for executing programs and calculating and controlling programs, etc.

[0064] Additionally, the image encoded into a bitstream by the encoder can be transmitted to an image decoding device via real-time or non-real-time wired / wireless communication networks (e.g., the Internet, local area wireless communication networks, wireless LAN networks, Wi-Fi networks, mobile communication networks), or via various communication interfaces such as cables or Universal Serial Bus (USB), where it is decoded, reconstructed into an image, and reproduced. Alternatively, the bitstream generated by the encoder can be stored in memory. Memory may include both volatile and non-volatile memory. In this specification, memory may be referred to as a recording medium that stores the bitstream.

[0065] Typically, a video can consist of a series of images, and each image can be divided into coding units such as blocks. Furthermore, those skilled in the art to which this embodiment pertains will understand that the term "image" described below can be used by replacing it with another term having an equivalent meaning, such as "picture" or "frame." Additionally, those skilled in the art will understand that the term "coding unit" can be replaced and used with other terms having the same meaning, such as "unit block" and "block."

[0066] In the following description, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Redundant descriptions of identical components will be omitted in the description of the present invention.

[0067] Figure 1 This is a block diagram illustrating an image encoding apparatus according to the present invention. (Refer to...) Figure 1 The conventional image encoding device 100 includes an image segmentation unit 110, a prediction unit 120 and 125, a transformation unit 130, a quantization unit 135, a rearrangement unit 160, an entropy encoding unit 165, an inverse quantization unit 140, an inverse transformation unit 145, a filter unit 150, and a memory 155.

[0068] Image segmentation unit 110 can divide the input image into at least one processing unit. In this case, the processing unit may be a prediction unit (PU), a transform unit (TU), or an encoding unit (CU). In the following, in embodiments of the present invention, the encoding unit may be used to mean either a unit that performs encoding or a unit that performs decoding.

[0069] A prediction unit can be obtained by dividing a coding unit into at least one square or non-square shape of the same size. A coding unit can be divided such that one prediction unit in the prediction unit has a different shape and / or size than another prediction unit. When the prediction unit for intra-frame prediction based on a coding unit is not the smallest coding unit, intra-frame prediction can be performed without dividing it into multiple N×N prediction units.

[0070] Prediction units 120 and 125 may include an inter-frame prediction unit 120 performing inter-frame prediction and an intra-frame prediction unit 125 performing intra-frame prediction. It can be determined whether inter-frame prediction or intra-frame prediction is used for the prediction unit, and specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) based on each prediction method can be determined. The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 130. Additionally, the prediction mode information, motion vector information, etc., used for prediction, along with the residual value, can be encoded by the entropy coding unit 165 and sent to the decoder. However, when the decoder-side motion information derivation method according to the present invention is applied, since the prediction mode information, motion vector information, etc., are not generated in the encoder, the corresponding information is not sent to the decoder. On the other hand, information indicating that motion information is derived and used on the decoder side, as well as information about the method used to derive the motion information, can be transmitted from the encoder via signals.

[0071] The inter-frame prediction unit 120 can predict prediction units based on information from at least one of the previous or subsequent images of the current image, and in some cases, it can predict prediction units based on information from some regions in the current image that have already been encoded. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0072] The reference image interpolation unit can receive reference image information from the memory 155 and generate pixel information for integer pixels or smaller pixels in the reference image. In the case of luminance pixels, an 8-tap DCT-based interpolation filter (DCT-based interpolation filter) with different filter coefficients can be used to generate pixel information for integer pixels or smaller pixels in units of 1 / 4 pixels. In the case of chrominance signals, a 4-tap DCT-based interpolation filter (DCT-based interpolation filter) with different filter coefficients can be used to generate pixel information for integer pixels or smaller pixels in units of 1 / 8 pixels.

[0073] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as the Full Search-Based Block Matching (FBMA), Three-Step Search (TSS), and New Three-Step Search (NTS). Motion vectors can have motion vector values ​​in units of 1 / 2 or 1 / 4 pixels based on the interpolated pixels. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as skip methods, merging methods, AMVP (Advanced Motion Vector Prediction) methods, and intra-frame block copying methods can be used as motion prediction methods. Furthermore, when applying the motion information derivation method on the decoder side according to the present invention, template matching methods and bidirectional matching (bilateral matching) methods using motion trajectories can be applied as methods performed by the motion prediction unit. Relatedly, this will be discussed later. Figure 3 The text describes template matching methods and bidirectional matching methods in detail.

[0074] Intra-prediction unit 125 can generate prediction units based on reference pixel information surrounding the current block, where the reference pixel information is pixel information in the current image. When the neighboring block of the current prediction unit is a block performing inter-frame prediction and the reference pixel is a pixel performing inter-frame prediction, the reference pixel included in the block performing inter-frame prediction can be used by replacing the reference pixel information included in the block performing inter-frame prediction with the reference pixel information of the blocks performing intra-frame prediction surrounding the block performing inter-frame prediction. That is, when the reference pixel is unavailable, the unavailable reference pixel information can be used by replacing it with at least one of the available reference pixels.

[0075] Additionally, residual blocks can be generated that include residual information based on the difference between the original block of the prediction unit and the prediction unit used to perform predictions by the prediction units generated by prediction units 120 and 125. The generated residual blocks can be input to the transformation unit 130.

[0076] Transform unit 130 can transform the residual block, which includes residual information between the prediction units generated by prediction units 120 and 125 and the original block, using transformation methods such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT. The choice between applying DCT, DST, or KLT to transform the residual block can be determined based on the intra-frame prediction mode information of the prediction unit used to generate the residual block.

[0077] The quantization unit 135 can quantize the values ​​transformed to the frequency domain by the transformation unit 130. The quantization coefficients can vary according to the blocks or importance of the image. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the rearrangement unit 160.

[0078] The rearrangement unit 160 can rearrange the coefficient values ​​of the quantized residual values.

[0079] The rearrangement unit 160 can transform coefficients in two-dimensional block form into one-dimensional vector form using a coefficient scanning method. For example, the rearrangement unit 160 can transform coefficients into one-dimensional vector form by scanning from DC coefficients to coefficients in the high-frequency region according to a zig-zag scanning method. In addition to zig-zag scanning, depending on the size of the transform unit and the intra-frame prediction mode, vertical scanning (scanning coefficients in two-dimensional block form in the column direction) and horizontal scanning (scanning coefficients in two-dimensional block form in the row direction) can be used. That is, the choice between zig-zag scanning, vertical scanning, and horizontal scanning can be determined based on the size of the transform unit and the intra-frame prediction mode.

[0080] Entropy coding unit 165 can perform entropy coding based on the values ​​calculated by rearrangement unit 160. Various coding methods can be used for entropy coding, such as exponential Golomb, CAVLC (context-adaptive variable-length coding), and CABAC (context-adaptive binary arithmetic coding). In this regard, entropy coding unit 165 can encode the residual coefficient information from the coding units of rearrangement unit 160 and prediction units 120 and 125. Furthermore, according to the present invention, information indicating that motion information is derived and used on the decoder side, as well as information about the method used to derive the motion information, can be transmitted from the encoder via signals.

[0081] Inverse quantization unit 140 and inverse transform unit 145 perform inverse quantization on the value quantized by quantization unit 135 and inverse transform on the value transformed by transform unit 130. A reconstructed block can be generated by combining the residual value generated by inverse quantization unit 140 and inverse transform unit 145 with the prediction unit predicted by the motion estimation unit, motion compensation unit, and intra-frame prediction unit included in prediction units 120 and 125.

[0082] Filter unit 150 may include at least one of a deblocking filter, an offset correction unit, and an adaptive loop filter (ALF). The deblocking filter removes block distortion caused by boundaries between blocks in the reconstructed image. The offset correction unit corrects the offset relative to the original image pixel-by-pixel for the deblocked image. To perform offset correction on a specific image, after classifying the pixels included in the image into a certain number of regions and determining the regions to which the offset is applied, an offset application method can be used, either by applying the offset to the regions or by applying the offset by considering the edge information of each pixel. The ALF (Adaptive Loop Filter) can be performed based on values ​​obtained by comparing the filtered reconstructed image with the original image. After classifying the pixels included in the image into predetermined groups, a filter to be applied to each group can be determined to perform filtering differently for each group.

[0083] The memory 155 can store the reconstructed blocks or images output from the filter unit 150, and the stored reconstructed blocks or images can be provided to the prediction units 120 and 125 when performing inter-frame prediction.

[0084] Figure 2 This is a block diagram illustrating an image decoding device according to the present invention. (Refer to...) Figure 2 The image decoder 200 may include an entropy decoding unit 210, a rearrangement unit 215, an inverse quantization unit 220, an inverse transform unit 225, prediction units 230 and 235, a filter unit 240, and a memory 245.

[0085] When an image bitstream is input from an image encoder, the input bitstream can be decoded in the reverse process of the image encoder.

[0086] The entropy decoding unit 210 can perform entropy decoding in a process that is the reverse of the entropy encoding process performed by the entropy encoding unit of the image encoder. For example, various methods corresponding to the methods performed in the image encoder can be applied, such as exponential Golomb (CAVLC), context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC).

[0087] The entropy decoding unit 210 can decode information related to intra-frame prediction and inter-frame prediction performed by the encoder.

[0088] Rearrangement unit 215 can rearrange the bitstream entropy decoded by entropy decoding unit 210 based on the rearrangement method of the encoding unit. The coefficients in one-dimensional vector form can be rearranged again into coefficients in two-dimensional block form.

[0089] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficients of the rearrangement block.

[0090] The inverse transform unit 225 can perform inverse transforms on the quantization result performed by the image encoder, corresponding to the transforms (i.e., DCT, DST, and KLT) performed by the transform unit, namely, inverse DCT, inverse DST, and inverse KLT. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit 225 of the image decoder, transform methods (e.g., DCT, DST, KLT) can be selectively performed based on various information such as the prediction method, the size of the current block, and the prediction direction.

[0091] Prediction units 230 and 235 can generate prediction blocks based on information related to prediction block generation provided by entropy decoding unit 210 and previously decoded block or image information provided by memory 245.

[0092] As described above, when the size of the prediction unit and the size of the transform unit are the same when intra-prediction is performed in the same manner as in the image encoder, intra-prediction of the prediction unit can be performed based on pixels located to the left, upper left, and top of the prediction unit. However, when the size of the prediction unit and the size of the transform unit are different when performing intra-prediction, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using an N×N partition can be used only for the smallest coding unit.

[0093] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit can receive various information from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method, classify the prediction units in the current coding unit, and determine whether the prediction unit performs inter-frame prediction or intra-frame prediction. On the other hand, if no motion prediction-related information for inter-frame prediction is sent, but instead information indicating that motion information is derived and used on the decoder side, as well as information about the method used to derive the motion information, is sent from the encoder 100, the prediction unit determination unit determines the prediction performance of the inter-frame prediction unit 23 based on the information sent from the encoder 100.

[0094] Inter-frame prediction unit 230 can perform inter-frame prediction on the current prediction unit by using information required for inter-frame prediction provided by the image encoder, based on information included in at least one of the previous or subsequent images containing the current image. To perform inter-frame prediction, a motion prediction method for the prediction units included in the coding unit can be determined from among skip mode, merge mode, AMVP mode, and intra-block copy mode. Alternatively, inter-frame prediction unit 230 can perform inter-frame prediction by deriving motion information based on information provided by the image encoder indicating that motion information is derived and used on the decoder side, and information about the method used to derive the motion information.

[0095] Intra-prediction unit 235 can generate prediction blocks based on pixel information in the current image. If the prediction unit is one that has already performed intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the image encoder. Intra-prediction unit 235 may include an adaptive intra-smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is the part that performs filtering on the reference pixels of the current block, and can be applied by determining whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the AIS filter information and prediction mode of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode in which AIS filtering is not performed, the AIS filter may not be applied.

[0096] In the case where the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​obtained by interpolating reference pixels, the reference pixel interpolation unit can interpolate the reference pixels to generate reference pixels of integer pixels or smaller. If the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, then interpolation of reference pixels is not required. If the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks through filtering.

[0097] The reconstructed blocks or images can be provided to filter unit 240. Filter unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0098] The video encoder can provide information about whether to apply a deblocking filter to a corresponding block or image, and whether to apply a strong or weak filter when doing so. The video decoder's deblocking filter can provide the same information as that provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

[0099] The offset correction unit can perform offset correction on the reconstructed image based on the type of offset correction and the offset value information applied to the image during encoding. ALF can be applied to the encoding unit based on information provided by the encoder regarding whether ALF is applied, ALF coefficient information, etc. This ALF information can be provided by a specific set of parameters.

[0100] The memory 245 can store reconstructed images or blocks, so that the reconstructed images or blocks can be used as reference images or reference blocks, and can also provide the reconstructed images to the output unit.

[0101] Figure 3 This paper illustrates a motion information extraction method using template matching, as one of the methods for deriving motion information on the decoder side according to the present invention. Specifically, the inter-frame prediction unit 120 in encoder 100 and the inter-frame prediction unit 230 in decoder 200 perform the motion information extraction method using template matching in the same manner, as will be described later. Therefore, encoder 100 does not transmit the residual signal generated during encoding, nor the motion information associated with prediction, or even only transmits the residual signal; decoder 200 itself can derive the motion information using the template matching method and then reconstruct the motion information. The method for deriving motion information using template matching will be described in detail below.

[0102] Reference Figure 3 Derivation of motion information using template matching (hereinafter also referred to as "TM") refers to a method of deriving motion information based on pixel similarity between multiple pixels spatially adjacent to the current target block 310 in the current encoded and decoded target image 300 and pixels of the same shape located in reference images 301 and 302. Here, the multiple pixels spatially adjacent to the current target block 310 and the pixels of the same shape located in reference images 301 and 302 are referred to as templates. Therefore, the shape of the template can be a predetermined form or can be sent through high-level syntax.

[0103] One or more templates can be configured for the current target block 310, and as follows: Figure 3 As shown, the template 320 adjacent to the top and the template 330 adjacent to the left can be used respectively. Additionally, each template can be configured with a template of the same shape and size as shown in the reference image. The templates configured in the reference image can consist of upper templates 321 and 322 corresponding to the upper adjacent template 320 of the current target block 310, and left templates (331, 332) corresponding to the left adjacent template 330 of the current target block 310.

[0104] Here, it checks whether the top and left adjacent pixels are configurable, and configures the template only if the top and left adjacent pixels can be used to configure the template. Template matching can be performed by using some of the predefined templates, excluding the non-configurable ones.

[0105] Relatedly, the template cannot be configured when the current target block is located at the boundary of an image, tile, or slice.

[0106] Figure 4 This diagram illustrates a concept of a motion information derivation method using bidirectional matching, according to the present invention, as one of the methods for deriving motion information on the decoder side. Specifically, the inter-frame prediction unit 120 in encoder 100 and the inter-frame prediction unit 230 in decoder 200 perform the same motion information derivation method using bidirectional matching. Therefore, encoder 100 does not transmit the residual signal generated during encoding, nor the motion information associated with prediction, or even only transmits the residual signal; decoder 200 itself derives the motion information using the bidirectional matching method and then reconstructs the motion information. The method for deriving motion information using bidirectional matching will be described in detail below.

[0107] Reference Figure 4 Using bidirectional matching (hereinafter also referred to as "BM") to derive motion information refers to a method of deriving motion information based on the pixel similarity between two different reference blocks (401, 402) in reference images (301, 302) in two directions of the current target block 310 in the current image 300 to be encoded and decoded.

[0108] Here, a motion trajectory (400) is used to select two different reference blocks 401 and 402 in reference images 301 and 302 located in two directions of the current target block 310.

[0109] Reference Figure 4 According to an embodiment of the present invention, two different reference images 301 and 302 centered on the current target block 310 can be connected using a motion trajectory 400. Additionally, reference blocks 401 and 402 of the same size and shape as the current target block 310 can be configured at the connection point.

[0110] The bidirectional matching method according to an embodiment of the present invention includes the following processes: configuring reference blocks 401 and 402 of the same size and shape as the current target block 310, and obtaining a motion trajectory (400) that minimizes the error between pixels in the two reference blocks 401 and 402.

[0111] Figure 5 This is an example of a flowchart of a decoding method according to an embodiment of the present invention. Specifically, Figure 5 The flowchart involves decoding when the mode for exporting motion information on the decoder side is selected.

[0112] In this invention, an embodiment will be described using one or more motion information extraction methods to extract motion information on the decoder side. Examples of the one or more motion information extraction methods include template matching (TM) and bidirectional matching (BM).

[0113] Specifically, when the mode of using the decoder-side motion vector derivation (DMVD) method is selected for the current target block, the motion information of the current target block and the sub-blocks obtained by dividing the current target block are directly derived on the decoder side by deriving and modifying motion information based on one or more coding units, without the need to send information with signals.

[0114] Reference Figure 5 Assuming an embodiment of the present invention, a mode for selecting the decoder-side motion vector derivation (DMVD) method is chosen for the current target block (S300). Here, this includes determining whether to use template matching based on information indicating one of a variety of decoder-side motion information derivation methods (S310). In step S310, if the current target block uses template matching (TM), motion information is derived and modified using the template matching method on a CU basis (S320); and this also includes additionally modifying the motion information using the template matching method on a sub-block basis (S330). Additionally, if the current target block uses bidirectional matching (BM) in step S310, motion information is derived and modified using the bidirectional matching method on a CU basis (S321), and this also includes additionally modifying the motion information using the bidirectional matching method on a sub-block basis (S331).

[0115] Here, CU stands for Coding Unit, and refers to the coding unit in the encoding and decoding steps. Additionally, a sub-block is one of several sub-blocks obtained by dividing a CU.

[0116] The result of deriving motion information in units of CU obtained in steps (S320, S321) is the optimal motion information based on CU units for the current encoded target block, and the optimal motion information based on CU units can be used as the initial value for modification in the step (S330, S331) of additionally modifying motion information in units of sub-blocks.

[0117] Figure 6 Another flowchart of a decoding method according to an embodiment of the present invention is shown. In particular, a method for performing decoding when a mode for deriving motion information on the decoder side is selected for the current block is shown in more detail.

[0118] When the mode for exporting motion information on the decoder side, as proposed in this invention, is selected, a motion information export process consisting of approximately two steps is performed. Since the motion information export process is performed in the same manner in both the encoder and decoder, motion can be exported without requiring information to be sent from the encoder to the decoder.

[0119] like Figure 5 and Figure 6 As shown, the process of deriving motion information on the decoder side proposed by the present invention can be classified into CU unit improvement step (S400) and sub-block unit improvement step (S420).

[0120] The CU unit improvement step (S400) and the sub-block unit improvement step (S420) can be executed sequentially, but the CU unit improvement step (S400) can also be executed only based on the result of determining whether to execute the sub-block unit improvement step (S420) (S410).

[0121] The processing included in the CU unit improvement step (S400) and the sub-block unit improvement step (S420) is applied equally to the template matching method and the bidirectional matching method described above. However, in the search and determination steps, the matching method for determining pixel similarity is applied differently.

[0122] Specifically, the CU unit improvement step (S400) includes the CU unit motion candidate list generation step (S401), the CU unit optimal motion determination step (S402), and the CU unit improvement search step (S403).

[0123] The CU unit motion candidate list generation step (S401) is a step that generates a motion candidate list required for initial search position selection to perform decoder-side motion derivation of the CU unit. Here, motion candidates to be added to the motion candidate list can be pre-defined, and these motion candidates are added in the same order in both the encoder and decoder. This will be discussed later. Figure 15 Describe in detail the method for constructing the candidate list of motion for CU units.

[0124] The optimal motion determination step (S402) for the CU unit is as follows: by sequentially searching the motion candidates added to the motion candidate list generated in the CU unit motion candidate list generation step (S401), the optimal motion candidate with the lowest cost is selected from the multiple motion candidates.

[0125] At this point, the sum of absolute differences (SAD), used to determine the similarity between pixels included in a block, can be used as the cost for determining the best motion candidate. However, in addition to SAD, SATD (sum of absolute transform differences), SSE (sum of squared errors), etc., can be used in various ways. Rate-distortion (RD) costs can be used by considering both the bit rate value and the similarity between pixels.

[0126] The CU unit improvement search step (S403) is the following step: the optimal motion of the CU unit is finally determined by performing an additional search using the optimal motion information determined in the CU unit optimal motion determination step (S402) as the starting position.

[0127] In the CU unit improvement search step (S403), various search methods can be used as motion search methods to determine the optimal motion of the CU unit. As examples of motion search methods, square search, hexagonal search, rhombus search, cross search, and adaptive cross search can be used.

[0128] According to an embodiment of the present invention, one or more different motion search methods can be sequentially executed in the CU unit improved search step (S403). If the search position remains unchanged during the first search process using the initial diamond search, a second search process can be performed by applying a cross search method, which searches for the top, bottom, left, and right pixel positions corresponding to the given location. This will be discussed later. Figure 9 This is described in detail in the text.

[0129] Regarding this, "the search position remains unchanged" in the search process refers to the case where the search value at the initial position of each search step represents the minimum value among multiple search positions. The search method and its order used in the CU unit improvement search step (S403) can follow a predetermined order, or information about which search method to use can be sent via advanced syntax.

[0130] In the process of deriving motion information on the decoder side according to the present invention, after executing the CU unit improvement step (S400), it is determined whether to execute the sub-block unit improvement step (S410) through the sub-block unit improvement step (S420). As a result of step S410, the sub-block unit improvement step (S420) is executed only if it is determined that the sub-block unit improvement step (S420) will be executed (S410, "Yes" branch).

[0131] In step S410, when determining whether to execute the sub-block unit improvement step, if the optimal motion information of the CU unit determined in the current CU unit motion information derivation step (S400) exists, whether to execute the sub-block unit improvement step (S420) is determined to be true (yes). Here, the existence of optimal motion information of the CU unit means the following situation: a motion candidate list for CU unit motion derivation has been constructed and motion derivation can be performed using at least one motion information of the motion candidate included in the corresponding constructed list.

[0132] The sub-block unit improvement step (S420) includes: a sub-block unit motion candidate list generation step (S421); a sub-block unit optimal motion determination step (S422); and a sub-block unit improvement search step (S423). Furthermore, the sub-block unit improvement step (S420) is performed sequentially for all multiple sub-blocks in the current target block.

[0133] The sub-block unit motion candidate list generation step (S421) is a step to generate a motion candidate list required for the initial search position selection in order to derive the motion of the sub-block unit on the decoder side.

[0134] Here, motion candidates to be added to the motion candidate list can be predefined and added in the same order in both the encoder and decoder. However, when the CU unit improvement step is performed before generating the sub-block unit motion candidate list, the optimal motion information determined in the CU unit is included in the sub-block unit motion candidate list. This will be discussed later. Figure 15 The method for constructing a candidate list of sub-block unit motions is described in detail.

[0135] The sub-block unit optimal motion determination step (S422) is as follows: by sequentially searching the motion candidates added to the motion candidate list determined in the sub-block unit motion candidate list generation step (S421), the motion candidate with the minimum value (cost) among the multiple motion candidates is selected.

[0136] The sub-block element improvement search step (S423) is the following step: the optimal motion of the sub-block element is finally determined by performing an additional search using the motion information determined in the sub-block element optimal motion determination step (S422) as the starting position.

[0137] When determining a motion candidate in a sub-block unit, different optimal motion information can be used to determine the value based on the motion derivation method on the decoder side. According to an embodiment of the present invention, when using a template matching motion derivation method, the SAD used to determine the similarity between pixels included in the block can be used to determine the cost of a motion candidate. On the other hand, when using a bidirectional matching (BM) motion derivation method, the rate-distortion value (RD cost) derived from the weighted sum of the SAD used to determine the similarity between pixels included in the block and the magnitude information of the motion vector can be used to determine the cost of a motion candidate. The rate-distortion value can be expressed as Equation (1) below.

[0138] [Formula 1]

[0139] RDcost = SAD + W (|mvx-mv'y|+|mvy-mv'y|)

[0140] w represents the weight of the magnitude information of the motion vector, mv represents the optimal motion vector of the CU unit, and mv' represents the target motion vector searched on a sub-block basis. However, it is not limited to using rate-distortion values; SAD, SATD, and SSE can be used individually, and the weights can be changed as needed.

[0141] The method for improving motion information in the decoder will be described in detail below. This implementation can be applied to the CU unit improvement steps and / or sub-block unit improvement steps described above.

[0142] The pre-reconstruction motion information of the current block can be improved based on the predetermined change motion information, and this improvement can be expressed as Equation 2 below.

[0143] [Equation 2]

[0144] MV ref =MV rec + deltaMV

[0145] In Equation 2, MV ref This refers to improved motion information, and MV rec`deltaMV` and `pre-reconstructed motion information` can refer to pre-reconstructed motion information and variable motion information, respectively. The decoder can set the initial value of the variable motion information to 0. The decoder can determine the optimal reference block by performing a search based on the position of the reference block of the current block. Here, the reference block of the current block can refer to the region indicated by the pre-reconstructed motion information. The decoder can update the variable motion information based on the position of the optimal reference block. Here, the reference block can include L0 / L1 reference blocks. The L0 reference block can refer to the reference block belonging to the reference image in reference image list 0 (L0), and the L1 reference block can refer to the reference block belonging to the reference image in reference image list 1 (L1).

[0146] Specifically, the decoder can determine a search region (S1) for improving motion information. The search region can be defined as a region that includes at least one of the reference block or its adjacent regions. Here, the position of the upper-left sample of the reference block can be the reference position for the search. The search region can be determined separately for the L0 and L1 directions. An adjacent region can refer to a region extending N sample lines from the boundary of the reference block. Here, N can be an integer of 1, 2, 3, or greater.

[0147] Adjacent regions can be located at least one of the following directions: left, top, right, bottom, top-left, bottom-left, top-right, or bottom-right of the reference block. Here, if the current block is W×H, the search area can be expressed as (W + 2N)×(H + 2N). However, to reduce the complexity of the improved processing, adjacent regions can be located only in some of the above directions. For example, they can be limited to regions adjacent to the left and top of the reference block, or regions adjacent to the right and bottom of the reference block.

[0148] The number of sample lines N can be a fixed value predefined in the decoder, or it can be variably determined by considering block attributes. Here, block attributes can refer to block size / shape, block location, inter-frame prediction mode, part type, etc. Block location can refer to whether a reference block is adjacent to an image or the boundary of a predetermined segment region. Segment region can refer to a picture tile, a column / row of a coding tree block, or a coding tree block. For example, depending on the block attributes, any of 0, 1, or 2 sample lines can be selectively used.

[0149] The decoder can determine a list of specified sums of absolute differences (SAD) for each search location in the search region (S2) (hereinafter referred to as the SAD list).

[0150] The SAD list can include multiple SAD candidates. The number of SAD candidates constituting the SAD list is M, and M can be an integer greater than or equal to 2. M can be limited to a maximum value of 9.

[0151] Candidates for SAD can be determined using the SAD value between blocks L0 and L1. Here, the SAD value can be calculated based on all samples belonging to blocks L0 / L1, or it can be calculated based on some samples within blocks L0 / L1. Here, some samples represent sub-blocks of blocks L0 / L1, and the width or height of the sub-block can be at least half the width or height of block L0 / L1. That is, the size of blocks L0 and L1 is W×H, and some samples can be sub-blocks of size W×H / 2, W / 2×H, or W / 2×H / 2. In this case, when some samples are W×H / 2, some samples can be the upper (or lower) sub-block of block L0 / L1. When some samples are W / 2×H, some samples can be the left (or right) sub-block of block L0 / L1. When some samples are W / 2×H / 2, some samples can be the upper-left sub-block of block L0 / L1, but are not limited to these. Alternatively, some samples can be defined as a set of even or odd sample lines (vertical or horizontal) of the L0 / L1 block. Similarly, some samples can be blocks subsampled along the width or height of the L0 / L1 block. That is, the L0 and L1 blocks have a size of W x H, and the block corresponding to some samples has a size of W x H, but can be one of an odd-numbered set of rows, an even-numbered set of rows, an odd-numbered set of columns, or an even-numbered set of columns among the samples included in the block.

[0152] The position of the L0 block can be determined based on the position of the L0 reference block of the current block and a predetermined offset. This offset can refer to a shift vector between the position of the L0 reference block and the search position. That is, the search position can be a position shifted by p in the x-axis direction and q in the y-axis direction from the position (x0, y0) of the L0 reference block. Here, p and q can be at least one of -1, 0, or 1. The shift vector generated by the combination of p and q can represent the offset. The position of the L0 block can be determined as a position shifted (p, q) from the position (x0, y0) of the L0 reference block. The magnitude (or absolute value) of p and q is 0 or 1, but is not limited to this. For example, the magnitudes of p and q can be 2, 3, or greater.

[0153] Offsets can include at least one of a non-directed offset (0, 0) or a directed offset. Directed offsets can include offsets in at least one of the following directions: left, right, up, down, upper left, upper right, lower left, or lower right. For example, a directed offset is (-1, 0). (0, 1) (0, -1) (0, 1) (-1, -1) (-1, 1) At least one of (1, -1) or (1, 1).

[0154] Similarly, the position of the L1 block can be determined based on the position of the L1 reference block of the current block and a predetermined offset. Here, the offset of the L1 block can be determined based on the offset of the L0 block. For example, if the offset of the L0 block is (p, q), then the offset of the L1 block can be determined as (-p, -q).

[0155] Information regarding the magnitude and / or direction of the aforementioned offset can be predefined in the decoder, or it can be encoded in the encoder and sent to the decoder as a signal. This information can be variably determined by taking into account the aforementioned block properties.

[0156] As an example, the offset can be defined as shown in Table 1 below.

[0157] Table 1

[0158]

[0159] Table 1 defines the offsets used to determine the search position for each index i. However, Table 1 does not restrict the position of the offset corresponding to index i, and the position of the offset for each index can be different from that in Table 1. The offsets according to Table 1 can include the aforementioned undirected offset (0, 0) and eight directed offsets.

[0160] In this scenario, the 0th SAD candidate can be determined based on the position (x, y) and offset (-1, -1) of the reference block. Specifically, the search position is set by shifting the L0 reference block position (x0, y0) by an offset (-1, -1), and the WxH block whose upper left sample is at this search position can be determined as the L0 block.

[0161] Similarly, the search position can be set by shifting the L1 reference block position (x1, y1) by an offset of (1, 1), and the WxH block whose upper left sample is at this search position can be identified as the L1 block. The 0th SAD candidate can be determined by calculating the SAD between the L0 block and the L1 block.

[0162] Through the above processing, the first to eighth SAD candidates can be identified, and a list of SAD candidates including nine SAD candidates can be determined.

[0163] Table 1 does not limit the number of offsets used to improve motion information. Only k of the nine offsets can be used. Here, k can be any value in the range of 2 to 8. For example, in Table 1, three offsets such as [0, 4, 8], [1, 4, 7], [2, 4, 6], and [3, 4, 5] can be used; four offsets such as [0, 1, 3, 4] and [4, 5, 7, 8] can be used; and six offsets such as [0, 1, 3, 4, 6, 7] and [0, 1, 2, 3, 4, 5] can be used.

[0164] In one example, the offset can be defined as shown in Table 2 below. That is, the offset is only defined by the non-directed offset (0,0). Horizontal offset ((-1, 0)) (1, 0)) and vertical offset ((0, -1)) Composed of (0, 1).

[0165] Table 2

[0166]

[0167] Thus, the size and / or shape of the search region can be variably determined based on the magnitude and / or number of offsets. The search region can have geometries other than squares, referencing... Figure 9 This will be described.

[0168] The decoder can select either the first method or the second method (S3) to improve the motion information of the current block based on the SAD candidates in the SAD list mentioned above.

[0169] Selection can be performed based on the comparison between the reference SAD candidate and a predetermined threshold (Implementation Method 1).

[0170] A reference SAD candidate may refer to the SAD candidate corresponding to offset (0, 0). Alternatively, a reference SAD candidate may refer to the SAD candidate corresponding to the position of a reference block or reference location that has been changed by the first method described later.

[0171] The threshold can be determined based on at least one of the width (W) or height (H) of the current block (or reference block). For example, the threshold can be determined as W. H, W (H / 2), (W / 2) H, 2 W H, 4 W H, etc.

[0172] If the reference SAD candidate is greater than or equal to the threshold, the change motion information can be updated based on the first method; otherwise, the change motion information can be updated based on the second method.

[0173] (1) In the case of the first method

[0174] Among the SAD candidates belonging to the SAD list, the SAD candidate with the minimum value can be identified. The identification method will be described later. The change motion information can be updated based on the offset corresponding to the identified SAD candidate.

[0175] Then, the search position corresponding to the identified SAD candidate can be changed to a reference position for the search. Based on the changed reference position, the above process of determining the SAD list and / or identifying the SAD candidate with the minimum value can be performed again, omitting redundant descriptions. Based on the result of this re-execution, the change motion information can be updated.

[0176] (2) In the case of the second method

[0177] The change motion information can be updated based on parameters calculated using all or some of the SAD candidates that belong to the SAD list.

[0178] The parameter may include at least one of a parameter dX for the x-axis direction or a parameter dY for the y-axis direction. For example, the parameter can be calculated as follows, and for ease of description, it will be based on Table 1 above.

[0179] dX can be set to 0 when the sum of sadList[3] and sadList[5] equals (sadList[4] << 1). Here, sadList[3] can be a SAD candidate corresponding to offset (-1, 0), sadList[5] can be a SAD candidate corresponding to offset (1, 0), and sadList[4] can be a SAD candidate corresponding to offset (0, 0).

[0180] When the sum of sadList[3] and sadList[5] is greater than (sadList[4]<<1), dX can be determined according to Equation 3 below.

[0181] [Formula 3]

[0182] dX=((sadList[3]-sadList[5])<<3) / (sadList[3]+sadList[5]-(sadList[4]<<1))

[0183] When the sum of sadList[1] and sadList[7] equals (sadList[4]<<1), dY can be set to 0. Here, sadList[1] can be a SAD candidate corresponding to offset (0, -1), sadList[7] can be a SAD candidate corresponding to offset (0, 1), and sadList[4] can be a SAD candidate corresponding to offset (0, 0).

[0184] When the sum of sadList[1] and sadList[7] is greater than (sadList[4]<<1), dY can be determined according to Equation 4 below.

[0185] [Formula 4]

[0186] dY=((sadList[1]-sadList[7])<<3) / (sadList[1]+sadList[7]-(sadList[4]<<1))

[0187] Based on the calculated parameters, the motion information of the changing quantities can be updated.

[0188] Alternatively, the selection in S3 above can be performed based on whether the reference SAD candidate is the minimum value among the SAD candidates belonging to the SAD list (Implementation Method 2).

[0189] For example, if the minimum value among the SAD candidates in the SAD list is not a reference SAD candidate, the change motion information is updated based on the first method; otherwise, the change motion information can be updated based on the second method. The first and second methods are described above, and detailed descriptions will be omitted.

[0190] Alternatively, the selection in S3 described above can be performed based on a combination of embodiments 1 and 2 described above (embodiment 3).

[0191] For example, when the reference SAD candidate is less than the threshold, the change motion information can be updated using the parameters according to the second method described above.

[0192] However, when the reference SAD candidate is greater than or equal to the threshold, the SAD candidate with the minimum value among the SAD candidates in the SAD list can be identified. When the identified SAD candidate is the reference SAD candidate, the change motion information can be updated using the parameters according to the second method described above. On the other hand, when the identified SAD candidate is not the reference SAD candidate, the change motion information can be updated according to the first method described above.

[0193] This section describes a method for identifying SAD candidates with the minimum value. SAD candidates belonging to the SAD list can be grouped into two, three, or more groups. For ease of description, the case where SAD candidates are divided into two groups will be described below.

[0194] Multiple SAD candidates can be grouped into a first group and a second group. Each group may include at least two SAD candidates from the SAD list. However, the group may be limited to excluding the reference SAD candidate. By performing a minimum operation on each group, the SAD candidate with the minimum value for each group can be extracted.

[0195] The SAD candidate with the minimum value among the SAD candidates extracted from the first group and the SAD candidates extracted from the second group (hereinafter referred to as the temporary SAD candidate) can be extracted again.

[0196] Based on the comparison between the temporary SAD candidate and the reference SAD candidate, the SAD candidate with the minimum value can be identified. For example, when the temporary SAD candidate is less than the reference SAD candidate, the temporary SAD candidate can be identified as the minimum SAD candidate in the SAD list. On the other hand, when the temporary SAD candidate is greater than or equal to the reference SAD candidate, the reference SAD candidate can be identified as the minimum SAD candidate in the SAD list.

[0197] The method described above for improving motion information on the decoder side can be performed on a sub-block basis based on the block size. For example, if the current block size is greater than or equal to a predetermined threshold size, the method for improving motion information on a threshold size basis can be performed. Conversely, if the current block size is less than the predetermined threshold size, the method for improving motion information on a current block size basis can be performed. Here, the threshold size can be determined as at least one of the block width or height being a T value. The T value can be an integer of 16, 32, 64, or larger.

[0198] The above methods for improving motion information on the decoder side can be implemented to a limited extent by considering factors such as block size, distance between the current image and the reference image, inter-frame prediction mode, prediction direction, and the unit or resolution of motion information.

[0199] For example, the improved method can be executed only if either the width or height of the current block is greater than or equal to 8, 16, or 32. Alternatively, the improved method can be applied only if both the width and height of the current block are greater than or equal to 8, 16, or 32. Alternatively, the improved method can be applied only if the area or number of samples of the current block is greater than or equal to 64, 128, or 256.

[0200] The improved method can be applied only when the POC difference between the current image and the L0 reference image and the POC difference between the current image and the L1 reference image are the same.

[0201] The improved method can only be applied when the inter-frame prediction mode of the current block is merge mode, and cannot be applied in other modes (e.g., affine mode, AMVP mode, etc.).

[0202] The improved method can be applied only when bidirectional prediction is performed on the current block. Alternatively, it can be applied only when the unit of motion information is an integer pixel, or only when the unit of motion information is less than a quarter pixel or half a pixel.

[0203] Figure 7 This is a diagram illustrating another flowchart of a decoding method according to an embodiment of the present invention. Specifically, this is an example of a standard procedure for determining whether to perform the sub-block unit improvement step (S420) when a mode for deriving motion information on the decoder side is selected for the current block. Figure 7 When calculating the block matching cost using the motion selected in the CU unit improvement step (S400), the block matching cost value is compared with a predetermined threshold (THR) (S411). When the block matching cost value is less than or equal to the threshold (THR), it can be determined that the sub-block unit improvement step S420 will not be executed. That is, it can be determined that the sub-block unit improvement step S420 will only be executed when the block matching cost value is greater than the threshold (THR).

[0204] According to an embodiment of the invention, the predetermined threshold (THR) can be set to "0". This means that when performing template matching (TM) or bidirectional matching (BM) using the optimal motion derived in the CU unit improvement step (S400), there is no error between the template or reference block used for block matching. However, the threshold is not limited to "0", but can be a predetermined value or sent via advanced syntax. Therefore, complexity can be effectively reduced by using the block matching value to determine whether to perform additional sub-block unit improvements.

[0205] pass Figure 6 Step S410 and Figure 7 Step S411 can terminate the motion information export process in advance without performing the sub-block unit improvement step (S420) under certain conditions, which can greatly help reduce the decoding complexity.

[0206] However, the early termination determination step (S411) using block matching costs is not limited to only Figure 7The CU unit improvement step (S400) and sub-block unit improvement step (S420) shown are executed, but the decoder-side motion prediction method proposed in this invention can be applied to multiple steps in calculating the block matching cost. That is, in the process of calculating the block matching cost, since the calculation of the block matching cost is terminated when the same condition as in step S411 is met, it can help reduce the decoding complexity.

[0207] Figure 8 This is a diagram illustrating another flowchart of a decoding method according to an embodiment of the present invention. Specifically, Figure 8 A flowchart illustrating motion derivation by performing only sub-block unit improvement search and not CU unit improvement search is shown. That is, according to an embodiment of the present invention, in order to perform motion derivation on the decoder side, CU unit improvement search is not performed, and only sub-block unit improvement search can be performed.

[0208] Reference Figure 8 , with the above Figure 6 The motion derivation process shown differs from the one that only performs sub-block element improvement search. Instead of the CU element improvement search (S403) included in the CU element improvement step, the motion derivation process additionally includes an improved sub-block element improvement search (S501). However, since only the sub-block element improvement search (S501) is performed without performing the CU element improvement search, it is more efficient than the process described above. Figure 6 The sub-block cell improvement search (S423) is a more detailed search method to perform the search. On the other hand, since the CU cell improvement search step is not performed and only the sub-block cell improvement search is performed, the complexity generated in the motion derivation process can be effectively reduced.

[0209] Reference Figure 8 The motion derivation process on the decoder side includes the following steps: CU unit motion candidate list generation (S401), CU unit optimal motion determination (S402), sub-block unit motion candidate list generation (S421), and sub-block unit optimal motion determination (S422). The operation of each step is the same as described above. Figure 6 The steps (S401, S402, S421, and S422) are the same. The difference is that the embodiment according to the present invention is characterized by the addition of a sub-block unit improved search step 501.

[0210] Specifically, in the above Figure 6 In the sub-block element improvement search step S423 shown, multiple search steps, such as the CU element improvement search and the sub-block element optimal motion determination step, are performed to search for the optimal motion. Therefore, the motion improvement search can be performed using cross search and adaptive cross search, which have relatively low search complexity.

[0211] On the other hand, in only executing Figure 8 In the case of the sub-block element improvement search shown, step 501 of the sub-block element improvement search does not perform a CU element improvement search, and only uses multiple predefined motion information to search for the optimal motion. Therefore, a diamond search with relatively high search complexity is used to perform the first search, and then a cross search is used to perform the second search. However, it is not limited to using only the diamond search as a search method with relatively high search complexity. Square search, hexagonal search, etc., can be used additionally, and the motion improvement search can be performed by using multiple different steps.

[0212] The method for improving motion compensation on the decoder side will be described in detail below. This implementation can be applied to the sub-block unit improvement step. However, the invention is not limited thereto, and can be applied to the CU unit improvement step described above in the same / similar manner.

[0213] The decoder can improve motion compensation by taking into account the gradients of samples belonging to a predetermined reference region, which is referred to below as the GMC method.

[0214] The decoder can determine a predetermined reference region for the GMC method. The reference region can be defined as a region that includes either a reference block of the current block or at least a neighboring region of the reference block. The reference region can be determined separately for the L0 and L1 directions. A neighboring region can refer to a region extending K sample lines from the boundary of the reference block, where K can be an integer of 1, 2, 3, or greater.

[0215] Adjacent regions can be located in at least one of the following directions of the reference block: left, top, right, bottom, top-left, bottom-left, top-right, or bottom-right. Here, when the current block is W×H, the reference region can be expressed as (W + 2K)×(H + 2K). However, to reduce the complexity of the improved processing, adjacent regions can be located only in some of the above directions. For example, adjacent regions can be limited to those adjacent to the left and top sides of the reference block, or they can be limited to those adjacent to the right and bottom sides of the reference block.

[0216] The number of sample lines (K) can be a fixed value predefined in the decoder, or it can be variably determined by considering block attributes. Here, block attributes can refer to block size / shape, block location, inter-frame prediction mode, part type, etc. Block location can refer to whether a reference block is adjacent to the boundary of a picture or a predetermined segment region. A segment region can refer to a picture tile, a column / row of a coding tree block, or a coding tree block. For example, depending on the block attributes, any of 0, 1, or 2 sample lines can be selectively used.

[0217] The decoder can compute the gradient of samples belonging to the reference region. The gradient of the current sample can be computed as the difference between multiple neighboring samples. These neighboring samples can lie on the same horizontal and / or vertical line as the current sample. Alternatively, they can lie on the same diagonal line as the current sample. Alternatively, one of the neighboring samples can lie on a different line relative to another sample. The number of neighboring samples can be 2, 3, 4, or more.

[0218] The gradient can be calculated for at least one of the L0 or L1 reference regions. Here, the gradient of the L0 / L1 reference region can be calculated for at least one of the horizontal (H) or vertical (V) directions.

[0219] For example, the gradient (gradientHL0) for the L0 reference region can be calculated based on the difference between two adjacent samples to the left and right of the current sample. The gradient (gradientVL0) for the L0 reference region can be calculated based on the difference between two adjacent samples to the top and bottom of the current sample. The gradients (gradientHL1, gradientVL1) for the L1 reference region in the horizontal / vertical directions can be calculated in the same way as above.

[0220] Using the method described above, the gradients of samples within the reference region can be calculated separately. However, the gradients of samples located at the boundaries of the reference region (hereinafter referred to as boundary samples) can share the gradients of their neighboring samples.

[0221] For example, the gradient of the upper boundary sample of the reference region can be set to the gradient of the neighboring sample below the upper boundary sample. The gradient of the lower boundary sample of the reference region can be set to the gradient of the neighboring sample above the lower boundary sample. The gradient of the left boundary sample of the reference region can be set to the gradient of the neighboring sample to the left of the left boundary sample. The gradient of the right boundary sample of the reference region can be set to the gradient of the neighboring sample to the right of the right boundary sample. The corner boundary sample of the reference region can be set to the gradient of the neighboring sample diagonally adjacent to the corner boundary sample. Samples located in a 2x2 region at each corner of the reference region can share the same gradient.

[0222] The decoder can determine one or more parameters for the GMC method. Parameters can be derived based on at least one of the sample values ​​from the L0 / L1 reference regions or the gradients of the samples. Specifically, a first parameter (diff) can be derived based on the difference in sample values ​​between the L0 and L1 reference regions. A second parameter (tempH) can be derived based on at least one of gradientHL0 or gradientHL1, and a third parameter (tempV) can be derived based on at least one of gradientVL0 or gradientVL1. For example, parameters can be derived according to Equation 5 below.

[0223] [Formula 5]

[0224] diff[x][y]=(predSamplesL0[h x ][v y ]>>shift2)-(predSamplesL1[h x ][v y >>shift2)

[0225] tempH[x][y]=(gradientHL0[x][y]+gradientHL1[x][y])>>shift3

[0226] tempV[x][y]=(gradientVL0[x][y]+gradientVL1[x][y])>>shift3

[0227] In Equation 5, it can be based on (h) in the L0 / L1 reference region. x , v y The sample corresponding to position (x, y) is used to derive the first parameter (diff) of the sample at position (x, y), where h x v y It can be determined by Equation 6 below. In Equation 6, nCbW and nCbH refer to the width and height of the current block (or reference block), respectively.

[0228] [Formula 6]

[0229] h x = Clip3( 1, nCbW, x )

[0230] v y = Clip3( 1, nCbH, y )

[0231] The decoder can derive compensation values ​​for improving motion compensation based on at least one of the previously determined gradients or parameters.

[0232] Specifically, compensation values ​​can be derived based on motion offset and gradient. Motion offset can be defined as horizontal motion offset (V... x ) or vertical motion offset (V y At least one of gradients. Here, gradient may include at least one of gradientHL0, gradientHL1, gradientVL0, or gradientVL1.

[0233] For example, the compensation value (mcOffset) can be derived using Equation 7 below.

[0234] [Formula 7]

[0235] mcOffset = Round( ( V x ( gradientHL1[ x + 1 ][ y + 1 ]- gradientHL0[ x+ 1 ][ y + 1 ]) )>>1 ) + Round( ( V y (gradientVL1[ x + 1 ][ y + 1 ]-gradientVL0[ x + 1 ][ y + 1 ]) )>>1 )

[0236] In Equation 7, V x This can represent horizontal motion offset. This can be determined based on at least one of the first parameter (diff) or the second parameter (tempH) mentioned above. V y This can represent vertical motion offset. This can be based on horizontal motion offset (V). x It can be derived from at least one of the following parameters: first parameter (diff), second parameter (tempH), or third parameter (tempW).

[0237] The decoder can generate predicted samples for the current block using the reference block and the compensation value (mcOffset). Here, the reference block can include at least one of the L0 reference block or the L1 reference block.

[0238] For example, the decoder can generate a predicted sample for the current block by adding a compensation value to a weighted average of the samples from the L0 reference block and the L1 reference block.

[0239] The GMC method described above can be selectively used based on predetermined encoding parameters. Here, encoding parameters can represent block size / shape, reference image position, inter-frame prediction mode, prediction direction, part type, differences between reference blocks, etc. Encoding parameters can include information derived from the decoder and information encoded by the encoder and transmitted via signals.

[0240] For example, it can be applied only when the current block is smaller than or greater than a predetermined threshold size. The threshold size can be limited to the minimum or maximum size of the block allowed by the GMC method. At least one of the width or height of the threshold size can be 8, 16, 32, or 64. Alternatively, it can be applied only when the current block has a square shape. Alternatively, it can be applied only when one of the L0 / L1 reference images has a smaller POC compared to the current image and the other reference image has a larger POC compared to the current image. Alternatively, it can be applied only when the sample value difference between the L0 reference block and the L1 reference block is greater than or equal to a predetermined threshold. Here, the threshold can be determined based on the bit depth or at least one of the width and height of the reference block (or the current block).

[0241] The GMC method can be selectively executed based on any one of the above constraints, or based on a combination of at least two of the above constraints.

[0242] Figure 9 This diagram illustrates the diamond search and cross search methods applied to the sub-block cell improvement search step (S501). Specifically, Figure 9 (a) shows a diamond search pattern as an example. Figure 9 (b) shows a cross search pattern as an example.

[0243] Reference Figure 9 (a) For example, since a diamond search is performed in multiple steps or rounds, the search begins, for example, in the diamond direction from the pixels surrounding pixel 601 (round 1). Here, pixel 601 corresponds to the initial starting position (e.g., the optimal motion candidate position). Based on the result of the first round of search, pixel 602 with the lowest cost value is selected, and the same diamond search is performed for pixel 602 (round 2). Additionally, based on the result of the second round of search, pixel 603 with the lowest cost is selected, and the same diamond search is performed for pixel 603 (round 3). The diamond search rounds can be repeated until the target cost is reached, or they can be limited to a predetermined number of times (e.g., 3 rounds).

[0244] Figure 9 (b) illustrates an example of a cross search, where, for instance, a cross search is performed on pixels 611, 612, 613, and 614, vertically and horizontally surrounding the final selected pixel 603, when the diamond search round terminates. After the cross search, the pixel position with the lowest cost value is determined as the final motion information position.

[0245] Figure 10 This is a diagram illustrating another flowchart of a decoding method according to an embodiment of the present invention. Specifically, Figure 10This is a flowchart illustrating the use of different motion information correction methods depending on the target motion information extraction method when one or more motion information extraction methods are used to extract motion information on the decoder side.

[0246] Reference Figure 10 When using template matching (TM) and bidirectional matching (BM) as motion information extraction methods on the decoder side, when performing motion extraction using template matching, only CU unit motion extraction and correction are performed. On the other hand, when performing motion extraction using bidirectional matching, initialization processing for motion search is performed on a CU-by-CU basis, and motion extraction and correction are performed on a sub-block basis.

[0247] Therefore, when using one or more motion information derivation methods to export motion information on the decoder side, this means that block units are used to export different motion information based on the target motion information derivation method. For example, when performing motion export using template matching, motion export can be performed on a CU (cube) basis. When performing motion export using bidirectional matching, motion export can be performed on a sub-block basis.

[0248] Specifically, through steps S300 and S310, if the motion information export mode on the decoder side is selected for the current target block, and the mode of exporting motion using the template matching method is selected (S310 "Yes" branch), then the steps include the CU unit motion candidate list generation step (S401), the CU unit optimal motion determination step (S402), and the CU unit improved search step (S403).

[0249] On the other hand, through steps S300 and S310, if the motion export mode on the decoder side is selected for the current target block, and the mode of exporting motion using the bidirectional matching method is selected (S310 "No" branch), then the steps include CU unit motion candidate list generation step S401, CU best motion determination step S402, sub-block unit motion candidate list generation step S421, sub-block unit best motion determination step S422, and sub-block unit improved search step S501.

[0250] Alternative locations, such as Figure 11 As shown, through steps S300 and S310, when the motion information export mode on the decoder side is selected for the current target block and the mode of exporting motion using the bidirectional matching method is selected (S310 "No" branch), the above-mentioned CU unit motion candidate list generation step (S401) and CU unit optimal motion determination step (S402) can be omitted, and only the sub-block unit motion candidate list generation step (S421), sub-block unit optimal motion determination step (S422), and sub-block unit improved search step (S501) can be included.

[0251] Figure 12 The target block for block matching and the target pixels within the block are shown. Figure 12 The target block for performing block matching and the target pixels within the block are shown. Specifically, Figure 12 (a) illustrates a method for calculating the error between the target block and the reference block using all pixels in block 700 in general block matching, while Figure 12 (b) shows an example of a method that reduces the computational complexity required for block matching by using only some pixels (e.g., 711 or 712) in block 710 to calculate the error between the target block and the reference block.

[0252] For example, according to an embodiment of the present invention, by subsampling the 710 blocks, the 710 blocks are classified into pixels belonging to even-numbered rows 711 and pixels belonging to odd-numbered rows 712. When calculating block matching costs, errors between blocks, etc., the computational complexity can be reduced by performing calculations only based on pixels belonging to even-numbered rows or pixels belonging to odd-numbered rows.

[0253] In this case, the criteria for classifying the target pixels used for matching to reduce computational complexity are not limited to even-numbered and odd-numbered rows. Instead, methods can be used to determine the target rows in units of integer rows that are multiples of 2, or methods can be used to determine them using only the pixels adjacent to the current block. The following section describes the application of the above-described template matching (TM) and bidirectional matching (BM) using subsampled pixels within a block.

[0254] Figure 13 The method for deriving motion information on the decoder side is shown, which will use... Figure 12 The matching of subsampled pixels is applied to the concept of a motion information derivation method using template matching.

[0255] As described above, the method for deriving motion information using template matching is as follows: motion information is derived based on the pixel similarity between multiple pixels spatially adjacent to the current target block 310 in the current encoded and decoded target image 300 and multiple pixels with the same shape located in reference images 301 and 302. The pixels spatially adjacent to the current target block 310 and the pixels with the same shape located in reference images 301 and 302 are referred to as templates. In this case, when configuring the templates, as described above... Figure 12 As shown, instead of using all pixels included in a specific template range, some subsampled pixels can be used to reduce computational complexity.

[0256] Specifically, one or more templates can be configured for the current target block 310, and as follows: Figure 13As shown, template 800 adjacent to the top and template 810 adjacent to the left can be used to configure each template to have the same shape and size as the reference image. However, instead of using templates (e.g. Figure 3 Instead of performing matching on all pixels in templates 320 and 330 shown, only specific pixels from templates 800 and 810 are used for template matching.

[0257] The template configured in the reference image can consist of upper templates 801 and 802 corresponding to the upper template 800 adjacent to the current target block 310, and left templates 811 and 812 corresponding to the left template 810 adjacent to the left of the current target block 310. In this case, it is checked whether the upper and left adjacent pixels are configurable, and the template is configured only if it is possible to configure the template using the upper and left adjacent pixels. Template matching can be performed by using some templates other than the non-configurable templates in one or more predefined templates. The case where the template is not configurable can correspond to the case where the current target block is located at the boundary of the image, tile, or slice.

[0258] Figure 14 The method for deriving motion information on the decoder side is shown, which will use... Figure 12 The matching of subsampled pixels is applied to the concept of a motion information derivation method using bidirectional matching.

[0259] As described above, the method for deriving motion information using bidirectional matching is as follows: motion information is derived based on the pixel similarity of two different reference blocks 901 and 902 in reference images 301 and 302 located in both directions around the current target block 310 in the current encoded and decoded target image 300. In this case, a motion trajectory (400) is used to select two different reference blocks 901 and 902 in reference images 301 and 302 located in both directions around the current target block 310.

[0260] Furthermore, in the block matching process using two different reference blocks 901 and 902, not all pixels included in the block are used; instead, only some subsampled pixels from blocks 901 and 902 are used. By doing so, computational complexity can be reduced.

[0261] Specifically, according to an embodiment of the present invention, two different reference images 301 and 302 can be connected using a motion trajectory 400 centered on the current target block 310. Reference blocks 901 and 902, having the same size and shape as the current target block 310, can be configured at the connection point. However, without using reference blocks (e.g., Figure 4Instead of using all pixels in the reference blocks 201 and 202 shown to perform matching, only specific pixels among the pixels in the reference blocks 901 and 902 are used for bidirectional matching.

[0262] That is, the bidirectional matching method according to an embodiment of the present invention includes the following processes: configuring reference blocks 901 and 902 of the same size and shape as the current target block 310, and obtaining a motion trajectory (400) that minimizes the error between some pixels in the two reference blocks 901 and 902.

[0263] Figure 15 This is a diagram illustrating a method for constructing a list of motion candidates for initial search position selection in order to derive motion information on the decoder side.

[0264] Figure 15 (a) is an implementation method for constructing the CU unit motion candidate list 1000 in the above-described CU unit motion candidate list generation step (S401). See also... Figure 15 (a) According to an embodiment of the present invention, when the current CU is in AMVP mode, AMVP candidate motion information 1501 can be added to the CU unit motion candidate list 1000, and CU unit motion candidates other than sub-block unit motion candidates in the current target block's merge candidates can be added to the CU unit motion candidate list 1000. Additionally, up to four motion candidates 1503 generated by interpolating the unidirectional prediction candidates of the current target sub-block can be added, and spatially adjacent left and top motion candidates 1504 can be added to construct the CU unit motion candidate list 1000.

[0265] Figure 15 (b) illustrates an embodiment of the construction of the sub-block unit motion candidate list 1010 in step S421 of generating the sub-block unit motion candidate list. (Refer to...) Figure 15 (b) According to an embodiment of the present invention, the following motion candidates may be added to the sub-block unit motion candidate list 1010: the best motion candidate 1511 of the target CU to which the current sub-block belongs, the motion candidates 1512 of the left and upper sub-blocks among the spatially adjacent sub-blocks of the current sub-block, the motion candidates 1513 of the upper left and upper right sub-blocks among the spatially adjacent sub-blocks of the current sub-block, the scaling motion candidate 1514 of the co-position sub-block of the reference image, and the motion candidates corresponding to the current sub-block in ATMVP 1515 and STMVP 1516.

[0266] However, when referencing spatially adjacent left, top, upper-left, and upper-right movements for parallel processing of the current target sub-block, such as Figure 16The specific region 1120 configured for parallel processing will be described later. It can also be changed to reference the movement of sub-blocks 1130 located outside this region.

[0267] Alternatively, for the parallel processing of the current target sub-block, in addition to spatially adjacent motions 1020, a candidate list of sub-block unit motions can be constructed.

[0268] Figure 16 The concept of parallel processing of sub-blocks according to the present invention is illustrated. In particular, the concept of referencing motions located outside a specific region while considering parallel processing is illustrated when spatially adjacent motions of reference sub-blocks are performed.

[0269] The unit obtained by dividing a block to be encoded and decoded into multiple smaller blocks is called a sub-block, and the size information of the sub-block used in the current encoding and decoding can be predefined or sent according to the high-level syntax.

[0270] When referencing the spatial adjacent motion of the current target sub-block 1100 using the method of referencing the spatial adjacent motion of existing sub-blocks, one or more of the motions of the left sub-block 1110, the top sub-block 1112, the top left sub-block 1111, and the top right sub-block 1113 are referenced.

[0271] However, when using existing methods to reference the movements of the left sub-block 1110, the top sub-block 1112, the upper left sub-block 1111, and the upper right sub-block 1113, a dependency (redundancy) occurs between sub-blocks because the motion determination process for the current target sub-block 1100 should be executed after the motion determination process for the corresponding sub-block is completed. To resolve the dependency problem between sub-blocks, in this invention, a specific region 1120 for parallel processing is set up considering the parallel processing of sub-blocks, and the movement of the sub-block 1130 located outside the corresponding region is referenced.

[0272] The specific region 1120 used for parallel processing can be a single CU, a region defined by the number of multiple sub-blocks, or a region of a certain size, and the corresponding information can be predefined or sent according to high-level syntax.

[0273] Additionally, when deriving motion information from the decoder side, the motion candidates for sub-blocks can be determined using... Figure 16 The concept of parallel processing of sub-blocks is also applicable to situations where motion information is not derived on the decoder side. In particular, this concept can be applied equivalently to sub-block unit motion information generation processing and sub-block unit motion reference processing.

[0274] Figure 17 Examples of applications are shown. Figure 16 A modified list of motion candidates for parallel processing. (As mentioned above) Figure 15 Compared to the motion candidate list, the CU unit motion candidate list 1000a is similar to the above. Figure 15 The CU unit motion candidate list 1000 is the same. However, since parallel processing is performed on a sub-block basis, it is the same as the above. Figure 15 Compared to the sub-block unit motion candidate list 1010, a sub-block unit motion candidate list 1010a is constructed by excluding motion candidates 1512 of the left and top sub-blocks in the spatially adjacent sub-blocks of the current sub-block, and motion candidates 1513 of the upper left and upper right sub-blocks in the spatially adjacent sub-blocks of the current sub-block.

[0275] The various embodiments of this disclosure are not intended to list all possible combinations, but are intended to describe representative aspects of this disclosure, and the contents described in the various embodiments may be applied independently or in combination of two or more.

[0276] Furthermore, various embodiments of this disclosure can be implemented using hardware, firmware, software, or a combination thereof. In the case of hardware implementation, this disclosure can be implemented using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0277] The scope of this disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable operation of methods according to various embodiments to be performed on an apparatus or computer, and such software or instructions include non-transitory computer-readable media stored and executable on an apparatus or computer.

[0278] Industrial applicability

[0279] This invention can be used to encode / decode video signals.

[0280] In addition, this disclosure also includes the following implementation methods.

[0281] (1). A method for decoding an image, comprising:

[0282] Determine the search area to improve the motion information of the current block;

[0283] Generate a SAD list for the search region, the SAD list including multiple SAD candidates;

[0284] Based on the SAD candidates in the SAD list, the change motion information of the current block is derived; and

[0285] The motion information of the current block is updated using the motion information of the current block and the change motion information.

[0286] (2). According to the method of (1), wherein the search area is determined to be a region including at least one of the reference block or a region adjacent to the reference block, and

[0287] The adjacent region is the region where the boundary of the reference block is extended by N sample lines.

[0288] (3). According to the method in (1), wherein the SAD candidate is determined by the SAD value between the L0 block and the L1 block, and

[0289] The L0 block is determined based on the position of the L0 reference block of the current block and a predetermined offset, and the L1 block is determined based on the position of the L1 reference block of the current block and the predetermined offset.

[0290] (4). The method according to (3), wherein the offset includes at least one of a non-directed offset or a directed offset, and

[0291] The directional offset includes offsets in at least one of the following directions: left, right, up, down, upper left, upper right, lower left, or lower right.

[0292] (5). According to the method of (1), wherein the change motion information is derived based on the comparison result between the reference SAD candidate and the predetermined threshold, and

[0293] The reference SAD candidate refers to the SAD candidate corresponding to the non-directed offset.

[0294] (6). According to the method in (5), when the reference SAD candidate is greater than or equal to the threshold, deriving the change motion information includes:

[0295] Identify the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and

[0296] The change motion information is determined based on the offset corresponding to the identified SAD candidate.

[0297] (7). According to the method described in (5), when the reference SAD candidate is less than the threshold, deriving the change motion information includes:

[0298] The predetermined parameters are calculated using all or a portion of the SAD candidates belonging to the SAD list; and

[0299] The motion information of the change is determined based on the calculated parameters.

[0300] (8). According to the method of (7), the motion information of the current block is updated in a limited manner by taking into account at least one of the block size, the distance between the current image and the reference image, the inter-frame prediction mode, the prediction direction, or the unit / resolution of the motion information.

Claims

1. A method for decoding an image, comprising: Determine the search area to update the motion information of the current block; Generate a SAD list for the search region, the SAD list including multiple SAD candidates; The change motion information of the current block is derived based on the SAD candidates in the SAD list; The motion information of the current block is updated based on the change in motion information. Perform inter-frame prediction on the current block based on the updated motion information; as well as The current block is reconstructed based on the predicted block obtained through inter-frame prediction of the current block. Specifically, the change in motion information is derived based on the comparison result between the reference SAD candidate and a predetermined threshold. Wherein, the reference SAD candidate refers to the SAD candidate corresponding to the non-directed offset, and Wherein, when the reference SAD candidate is greater than or equal to the predetermined threshold, deriving the change motion information includes: Identify the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and The change motion information is determined based on the offset corresponding to the identified SAD candidate.

2. The method according to claim 1, wherein, The search area is defined as a region that includes at least one of the reference block or a region adjacent to the reference block, and The adjacent region is the region extending N sample lines from the boundary of the reference block.

3. The method according to claim 1, wherein, The SAD candidate is determined by the SAD value between block L0 and block L1, and The L0 block is determined based on the position of the L0 reference block of the current block and a predetermined offset, and the L1 block is determined based on the position of the L1 reference block of the current block and the predetermined offset.

4. The method according to claim 3, wherein, The predetermined offset includes at least one of the non-directed offset or the directed offset, and The directional offset includes an offset in at least one of the following directions: left, right, up, down, upper left, upper right, lower left, or lower right.

5. The method according to claim 1, wherein deriving the change motion information when the reference SAD candidate is less than the predetermined threshold comprises: The predetermined parameters are calculated using all or part of the SAD candidates that belong to the SAD list; as well as The motion information of the change is determined based on the calculated parameters.

6. The method according to claim 5, wherein, The motion information of the current block is updated in a limited manner by considering at least one of the following: block size, distance between the current image and the reference image, inter-frame prediction mode, prediction direction, or unit / resolution of the motion information.

7. A method for encoding an image, comprising: Determine the search area to update the motion information of the current block; Generate a SAD list for the search region, the SAD list including multiple SAD candidates; The change motion information of the current block is derived based on the SAD candidates in the SAD list; The motion information of the current block is updated based on the change in motion information. Perform inter-frame prediction on the current block based on the updated motion information; as well as The current block is encoded based on the predicted block obtained by inter-frame prediction of the current block. Specifically, the change in motion information is derived based on the comparison result between the reference SAD candidate and a predetermined threshold. Wherein, the reference SAD candidate refers to the SAD candidate corresponding to the non-directed offset, and Wherein, when the reference SAD candidate is greater than or equal to the predetermined threshold, deriving the change motion information includes: Identify the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and The change motion information is determined based on the offset corresponding to the identified SAD candidate.

8. A non-transitory computer-readable storage medium storing instructions and a bit stream, wherein, When the processor executes the instructions, the instructions cause the processor to generate the bitstream according to a method including the following operations: Determine the search area to update the motion information of the current block; Generate a SAD list for the search region, the SAD list including multiple SAD candidates; The change motion information of the current block is derived based on the SAD candidates in the SAD list; The motion information of the current block is updated based on the change in motion information. Perform inter-frame prediction on the current block based on the updated motion information; as well as The current block is encoded based on the predicted block obtained by inter-frame prediction of the current block. Specifically, the change in motion information is derived based on the comparison result between the reference SAD candidate and a predetermined threshold. Wherein, the reference SAD candidate refers to the SAD candidate corresponding to the non-directed offset, and Wherein, when the reference SAD candidate is greater than or equal to the predetermined threshold, deriving the change motion information includes: Identify the SAD candidate with the minimum value among the SAD candidates belonging to the SAD list; and The change motion information is determined based on the offset corresponding to the identified SAD candidate.

9. A method for transmitting a bit stream, comprising: Perform the method of claim 7 to generate a bit stream; as well as Send the bit stream.

Images