Image encoding and decoding method and device

Abstract

To provide an image encoding and decoding method and device that correct a motion vector prediction value by using an adjustment offset.SOLUTION: An image encoding method includes the following steps of: constituting a motion information candidate list for an object block; selecting a prediction candidate index from the motion information candidate list; guiding an offset for motion vector adjustment; and restoring the motion vector of the object block through the prediction motion vector restored based on the offset.SELECTED DRAWING: Figure 19

Description

[Technical field]

[0001] The present invention relates to an image encoding / decoding method and device. [Background technology]

[0002] With the spread of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is rapidly increasing. Therefore, in order to perform various services and operations through image prediction in various systems, the need for improving the performance and efficiency of image processing systems is greatly increasing, but the results of research and development that can respond to this trend are insufficient.

[0003] Thus, in the conventional image encoding / decoding methods and apparatus, there is a need for improved performance for image processing, particularly image encoding or image decoding. Summary of the Invention [Problem to be solved by the invention]

[0004] The present invention has been made to solve the above problems, and has as its object to provide an image encoding / decoding device that corrects a motion vector prediction value using an adjustment offset. [Means for solving the problem]

[0005] To achieve the above object, a method for decoding an image according to one embodiment of the present invention includes the steps of constructing a motion information prediction candidate list for a target block, selecting a prediction candidate index, deriving a predicted motion vector adjustment offset, and restoring motion information of the target block.

[0006] Here, the step of setting the motion information prediction candidate list may further include a step of including in the candidate group, if there is no overlap between the already included candidates and the candidates obtained based on the offset information and the new candidates and the candidates obtained based on the offset information.

[0007] Here, the step of deriving the predicted motion vector adjustment offset may further include deriving the offset based on an offset application flag and / or offset selection information. Effect of the Invention

[0008] When the above-described inter-prediction according to the present invention is used, the coding performance can be improved by efficiently obtaining a predicted motion vector. [Brief description of the drawings]

[0009] [Figure 1] 1 is a conceptual diagram illustrating an image encoding and decoding system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing the configuration of an image encoding device according to an embodiment of the present invention; [Diagram 3] 1 is a block diagram showing the configuration of an image decoding device according to an embodiment of the present invention; [Figure 4] 4A to 4C are exemplary diagrams showing various division forms that can be obtained by the block division unit of the present invention; [Diagram 5] 4A and 4B are diagrams illustrating various cases in which a predicted block is obtained through inter-picture prediction according to the present invention; [Figure 6] FIG. 2 is an exemplary diagram of constructing a reference picture list according to an embodiment of the present invention. [Figure 7] FIG. 2 is a conceptual diagram illustrating a non-moving motion model according to an embodiment of the present invention. [Figure 8] 4 is an example diagram illustrating sub-block-based motion estimation according to an embodiment of the present invention; [Figure 9] 4 is a flowchart illustrating encoding of motion information according to an embodiment of the present invention. [Figure 10] 1 is a layout diagram of a target block and adjacent blocks according to an embodiment of the present invention; [Figure 11] FIG. 2 is an exemplary diagram of a statistical candidate according to an embodiment of the present invention; [Figure 12]FIG. 2 is a conceptual diagram of a statistical candidate according to a non-moving motion model in accordance with an embodiment of the present invention. [Figure 13] 11 is a diagram illustrating an example of motion information for each control point position stored as a statistical candidate according to an embodiment of the present invention; FIG. [Figure 14] 4 is a flowchart for encoding motion information according to an embodiment of the present invention. [Figure 15] 4 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention; [Figure 16] 4 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention; [Figure 17] 4 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention; [Figure 18] 4 is an exemplary diagram of an arrangement of a plurality of motion vector predictors according to an embodiment of the present invention; [Figure 19] 4 is a flow chart for motion information encoding in merge mode according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The image encoding / decoding method and apparatus according to the present invention can construct a predictive motion candidate list for a target block, derive a predictive motion vector from the motion candidate list based on a predictive candidate index, restore predictive motion vector adjustment offset information, and restore a motion vector for the target block based on the predictive motion vector and the predictive motion vector adjustment offset information.

[0011] In the image encoding / decoding method and apparatus according to the present invention, the motion candidate list may include at least one of spatial candidates, temporal candidates, statistical candidates or combined candidates.

[0012] In the image encoding / decoding method and apparatus according to the present invention, the predicted motion vector adjustment offset may be determined based on at least one of an offset application flag or offset selection information.

[0013] In the image encoding / decoding method and apparatus according to the present invention, the information on whether the predicted motion vector adjustment offset information is supported may be included in at least one of a sequence, a picture, a sub-picture, a slice, a tile, or a brick.

[0014] In the image encoding / decoding method and apparatus according to the present invention, if the target block is encoded in a merge mode, the motion vector of the target block is restored using a zero vector, and if the target block is encoded in a competitive mode, the motion vector of the target block can be restored using a motion vector differential value.

[0015] [Mode for carrying out the invention] Since the present invention can be modified in various ways and has various embodiments, specific embodiments are illustrated in the drawings and described in detail in the detailed description, but it should be understood that the present invention is not limited to the specific embodiments, and includes all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention.

[0016] The terms "first," "second," etc. may be used to describe various elements, but these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element can be named a second element, and similarly, the second element can be named a first element, without departing from the scope of the present invention. The term "and / or" includes any combination of two or more associated listed items or two or more associated listed items.

[0017] When an element is said to be "coupled" or "connected" to another element, it should be understood that the element may be directly coupled or connected to the other element, but there may be other elements between them. In contrast, when an element is said to be "directly coupled" or "directly connected" to another element, it should be understood that there are no other elements between them.

[0018] The terms used in the present invention are merely used to describe certain embodiments and are not intended to limit the present invention. A singular expression includes a plural expression unless the context clearly indicates otherwise. In the present invention, the terms "include" or "have" specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0019] Unless otherwise defined, all terms used herein, including technical or scientific terms, are meant to be the same as commonly understood by a person having ordinary skill in the art to which the present invention belongs. Terms defined in commonly used dictionaries should be interpreted as being consistent with the meaning they have in the context of the relevant art, and should not be interpreted in an ideal or overly formal sense unless expressly defined in the present invention.

[0020] Typically, a picture may be configured with one or more color spaces depending on the color format of the image. Depending on the color format, a picture may be configured with one or more pictures having a certain size or one or more pictures having different sizes. For example, in a YCbCr color configuration, color formats such as 4:4:4, 4:2:2, 4:2:0, and monochrome (configured with only Y) may be supported. For example, in the case of YCbCr4:2:0, a picture may be configured with one luminance component (Y in this example) and two chrominance components (Cb / Cr in this example). In this case, the composition ratio of the chrominance component to the luminance component may be 1:2 horizontally. For example, in the case of 4:4:4, the horizontal and vertical composition ratios may be the same. When a picture is configured with one or more color spaces as in the above example, the picture may be divided into each color space.

[0021] Images can be classified into I, P, B, etc. according to image type (e.g., picture type, subpicture type, slice type, tile type, brick type, etc.), where I image type may mean an image that is encoded by itself without using a reference picture, P image type may mean an image that is encoded using a reference picture but allows only forward prediction, and B image type may mean an image that is encoded using a reference picture and allows forward / backward prediction, but depending on the encoding settings, some of the above types may be combined (P and B combined) or image types with other configurations may be supported.

[0022] Various encoding / decoding information generated in the present invention can be processed explicitly or implicitly. Here, the explicit processing can be understood as generating the encoding / decoding information as a sequence, picture, sub-picture, slice, tile, brick, block, sub-block, etc., and recording it in a bitstream, and parsing related information in the same level unit as the encoder in the decoder to restore the decoded information. Here, the implicit processing can be understood as processing the encoding / decoding information in the same process or rule in the encoder and decoder.

[0023] FIG. 1 is a conceptual diagram showing an image encoding and decoding system according to an embodiment of the present invention.

[0024] Referring to FIG. 1, the image encoding device 105 and the decoding device 100 may be a user terminal such as a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a PlayStation Portable (PSP), a wireless communication terminal, a smart phone, or a TV, or a server terminal such as an application server or a service server, and may include various devices including a communication device such as a communication modem for communicating with various devices or wired and wireless communication networks, a memory (120, 125) for storing various programs and data for inter or intra prediction to encode or decode an image, or a processor (110, 115) for executing a program to perform calculations and control.

[0025] In addition, the image coded into a bit stream by the image coding device 105 can be transmitted to the image decoding device 100 in real time or non-real time via a wired or wireless communication network (Network) such as the Internet, a short-range wireless communication system, a wireless LAN network, a WiBro network, or a mobile communication network, or via various communication interfaces such as a cable or a Universal Serial Bus (USB), and can be restored and played back by being decoded by the image decoding device 100. In addition, the image coded into a bit stream by the image coding device 105 can be transmitted from the image coding device 105 to the image decoding device 100 via a computer-readable recording medium.

[0026] The image encoding device and the image decoding device described above may be separate devices, but may be implemented as a single image encoding / decoding device. In this case, a part of the configuration of the image encoding device is substantially the same technical element as a part of the configuration of the image decoding device, and can be implemented to include at least the same structure or perform at least the same function.

[0027] Therefore, in the following detailed description of technical elements and their operating principles, duplicated descriptions of corresponding technical elements will be omitted. Also, since the image decoding device corresponds to a computing device that applies the image coding method performed in the image coding device to decoding, the following description will focus on the image coding device.

[0028] The computing device may include a memory for storing a program or software module for implementing the image encoding method and / or the image decoding method, and a processor coupled to the memory for executing the program, where the image encoding device may be referred to as an encoder and the image decoding device may be referred to as a decoder.

[0029] FIG. 2 is a block diagram showing the configuration of an image encoding device according to an embodiment of the present invention.

[0030] Referring to FIG. 2, the image encoding device 20 may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, an encoded picture buffer 240, and an entropy encoding unit 245.

[0031] The prediction unit 200 may be realized using a prediction module, which is a software module, and may generate a prediction block for a block to be coded using an intra prediction method or an inter prediction method. The prediction unit 200 may generate a prediction block by predicting a target block to be coded in an image. In other words, the prediction unit 200 may predict a pixel value of each pixel of a target block to be coded in an image using intra prediction or inter prediction, and generate a prediction block having a predicated pixel value of each pixel. In addition, the prediction unit 200 may transmit information required for generating a prediction block, such as information on a prediction mode such as an intra prediction mode or an inter prediction mode, to the coding unit, so that the coding unit may code information on the prediction mode. At this time, a processing unit in which prediction is performed and a processing unit in which a prediction method and specific contents are determined may be determined according to coding settings. For example, a prediction method, a prediction mode, etc. may be determined in a prediction unit, and prediction may be performed in a conversion unit. Also, when a specific encoding mode is used, it is possible to directly encode the original block and transmit it to the decoding unit without generating a predicted block through the predictor.

[0032] The intra prediction unit may have directional prediction modes such as horizontal and vertical modes used according to the prediction direction, and non-directional prediction modes such as DC and Planar that use methods such as averaging and interpolation of reference pixels. A group of intra prediction mode candidates may be configured through the directional and non-directional modes, and any of various candidates such as 35 prediction modes (33 directional + 2 non-directional), 67 prediction modes (65 directional + 2 non-directional), and 131 prediction modes (129 directional + 2 non-directional) may be used as the candidate group.

[0033] The intra prediction unit may include a reference pixel configuration unit, a reference pixel filter unit, a reference pixel interpolation unit, a prediction mode determination unit, a prediction block generation unit, and a prediction mode coding unit. The reference pixel configuration unit may configure pixels that belong to adjacent blocks around the current block and are adjacent to the current block as reference pixels for intra prediction. Depending on the coding setting, the reference pixel configuration unit may configure one most adjacent reference pixel line as reference pixels, or another adjacent reference pixel line as reference pixels, or may configure multiple reference pixel lines as reference pixels. If some of the reference pixels are unavailable, the reference pixels may be generated using available reference pixels, and if all of the reference pixels are unavailable, the reference pixels may be generated using a predetermined value (e.g., a median value of a pixel value range represented by a bit depth).

[0034] The reference pixel filter unit of the intra prediction unit may filter the reference pixels in order to reduce degradation remaining after the encoding process. The filter used may be a low-pass filter such as a 3-tap filter [1 / 4, 1 / 2, 1 / 4] or a 5-tap filter [2 / 16, 3 / 16, 6 / 16, 3 / 16, 2 / 16]. The application of filtering and the type of filtering may be determined based on the encoding information (e.g., block size, shape, prediction mode, etc.).

[0035] The reference pixel interpolation unit of the intra prediction unit may generate a pixel in decimal units through a linear interpolation process of the reference pixel according to the prediction mode, and may determine an interpolation filter to be applied based on the coding information. The interpolation filter to be used may include a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, an 8-tap Kalman filter, etc. Generally, the interpolation is performed separately from the process of performing the low-pass filter, but the filtering process may be performed by integrating the filters applied in the two processes into one.

[0036] A prediction mode determination unit of the intra prediction unit can select at least one optimal prediction mode from a group of prediction mode candidates in consideration of a coding cost, and a prediction block generation unit can generate a prediction block using the selected prediction mode. A prediction mode coding unit can code the optimal prediction mode based on a prediction value. In this case, prediction information can be adaptively coded depending on whether the prediction value applies or does not apply.

[0037] The intra prediction unit may set the predicted value as an MPM (Most Probable Mode), and configure some of the overall modes belonging to the prediction mode candidate group as the MPM candidate group. The MPM candidate group may include a predetermined prediction mode (e.g., DC, planar, vertical, horizontal, diagonal mode, etc.) or a prediction mode of a spatially adjacent block (e.g., left, top, top left, top right, bottom left block, etc.). In addition, a mode induced from a mode already included in the MPM candidate group (in the case of a directional mode, a difference such as 1 or -1) may be configured as the MPM candidate group.

[0038] There may be a priority order of prediction modes for the configuration of MPM candidates. The order of inclusion in the MPM candidates may be determined according to the priority order, and the configuration of the MPM candidates may be completed when the number of MPM candidates (determined according to the number of prediction mode candidates) is filled according to the priority order. In this case, the priority order may be determined in the order of a prediction mode of a spatially adjacent block, a predetermined prediction mode, and a mode derived from a prediction mode first included in the MPM candidates, but other variations are also possible.

[0039] For example, spatially adjacent blocks may be included in the candidate group in an order such as left-top-bottom-left-top-top-top left block, and a predetermined prediction mode may be included in the candidate group in an order such as DC-Planar-vertical-horizontal mode, and a total of six modes may be configured as the candidate group by including a mode obtained by adding +1, -1, etc. to an already included mode in the candidate group. Alternatively, a total of seven modes may be configured as the candidate group by including a mode in a single priority order such as left-top-DC-Planar-bottom-left-top-top-top-(left+1)-(left-1)-(top+1).

[0040] The subtraction unit 205 can generate a residual block by subtracting a prediction block from a target block. That is, the subtraction unit 205 can calculate the difference between the pixel value of each pixel of a target block to be coded and the predicted pixel value of each pixel of a prediction block generated via a prediction unit to generate a residual block that is a block-shaped residual signal. The subtraction unit 205 can also generate a residual block based on a unit other than a block unit acquired via a block division unit described later.

[0041] The transform unit 210 can transform a signal belonging to the spatial domain into a signal belonging to the frequency domain. A signal obtained through the transform process is called a transformed coefficient. For example, a transform block having a transform coefficient can be obtained by transforming a residual block having a residual signal transmitted from the subtraction unit, and the input signal is determined according to the coding setting and is not limited to a residual signal.

[0042] The transform unit may transform the residual block using a transform technique such as a Hadamard transform, a discrete sine transform (DST Based-Transform), or a discrete cosine transform (DCT Based-Transform), but is not limited thereto, and various transform techniques that are improvements or modifications of these may be used.

[0043] At least one of the transformation techniques may be supported, and at least one detailed transformation technique may be supported for each of the transformation techniques, where the detailed transformation techniques may be configured such that a portion of the basis vectors is different for each transformation technique.

[0044] For example, in the case of DCT, one or more detailed conversion techniques from DCT-1 to DCT-8 can be supported, and in the case of DST, one or more detailed conversion techniques from DST-1 to DST-8 can be supported. A group of candidate conversion techniques can be configured by configuring a part of the detailed conversion techniques. For example, DCT-2, DCT-8, and DST-7 can be configured as a group of candidate conversion techniques to perform conversion.

[0045] The transformation can be horizontal / vertical, for example, a spatial domain pixel value can be transformed to a frequency domain using a one-dimensional transform in the horizontal direction, such as a DCT-2 transform technique, and a one-dimensional transform in the vertical direction, such as a DST-7 transform technique, for a total of two-dimensional transformation.

[0046] The conversion can be performed using one fixed conversion technique, or the conversion technique can be adaptively selected according to the encoding setting. In this case, in the adaptive case, the conversion technique can be selected using an explicit or implicit method. In the explicit case, each conversion technique selection information or conversion technique set selection information applied in the horizontal and vertical directions can be generated in units such as blocks. In the implicit case, the encoding setting can be defined according to the image type (I / P / B), color components, block size / shape / position, intra-screen prediction mode, etc., and a predetermined conversion technique can be selected according to the encoding setting.

[0047] Also, depending on the encoding settings, some of the transformations may be omitted, meaning that one or more of the horizontal / vertical units may be omitted, either explicitly or implicitly.

[0048] The transform unit can transmit the information necessary to generate a transform block to the encoder so that it can be encoded, and then record the information into a bitstream and transmit it to the decoder, and the decoding unit of the decoder can parse the information and use it in the inverse transform process.

[0049] The quantization unit 215 may quantize an input signal. At this time, a signal obtained through the quantization process is called a quantized coefficient. For example, a residual block having a residual transform coefficient transmitted from a transform unit may be quantized to obtain a quantized block having a quantized coefficient, but the input signal is determined according to a coding setting, and is not limited to a residual transform coefficient.

[0050] The quantization unit may quantize the transformed residual block using a quantization technique such as Dead Zone Uniform Threshold Quantization, Quantization Weighted Matrix, etc., but is not limited thereto, and various improved and modified quantization techniques may be used.

[0051] The quantization process can be omitted depending on the encoding settings. For example, the quantization process (including the inverse process) can be omitted depending on the encoding settings (e.g., the quantization parameter is 0, i.e., lossless compression environment). As another example, the quantization process can be omitted when the compression performance by quantization is not exhibited depending on the characteristics of the image. In this case, the area in the quantization block (M×N) where the quantization process is omitted is the entire area or a partial area (M / 2×N / 2, M×N / 2, M / 2×N, etc.), and the quantization omission selection information can be determined implicitly or explicitly.

[0052] The quantization unit can transmit the information necessary to generate a quantization block to the encoding unit so that it can be encoded, and the resulting information can be recorded in a bitstream and transmitted to the decoder, and the decoding unit of the decoder can parse the corresponding information and use it in the inverse quantization process.

[0053] In the above example, the explanation is based on the assumption that the residual block is transformed and quantized through a transform unit and a quantization unit, but the residual signal may be transformed to generate a residual block having transform coefficients without performing the quantization process, the residual signal of the residual block may be only subjected to the quantization process without transforming it into transform coefficients, and both the transform and quantization processes may not be performed. This can be determined according to the settings of the encoder.

[0054] The inverse quantization unit 220 inverse quantizes the residual block quantized by the quantization unit 215. That is, the inverse quantization unit 220 inverse quantizes the quantized frequency coefficient sequence to generate a residual block having frequency coefficients.

[0055] The inverse transform unit 225 inversely transforms the residual block dequantized by the inverse quantization unit 220. That is, the inverse transform unit 225 inversely transforms the frequency coefficients of the dequantized residual block to generate a residual block having pixel values, i.e., a reconstructed residual block. Here, the inverse transform unit 225 can perform inverse transform by inversely using the transform method used by the transform unit 210.

[0056] The adder 230 reconstructs the current block by adding the prediction block predicted by the prediction unit 200 and the residual block reconstructed by the inverse transform unit 225. The reconstructed current block is stored in the coding picture buffer 240 as a reference picture (or reference block), and can be used as a reference picture when coding the next block of the current block or another block or another picture following the current block.

[0057] The filter unit 235 may include one or more post-processing filter processes such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF). The deblocking filter may remove block noise occurring at a boundary between blocks from a restored picture. The ALF may perform filtering based on a value obtained by comparing an image restored after a block is filtered through a deblocking filter with an original image. The SAO may restore an offset difference between a residual block to which the deblocking filter is applied and an original image in pixel units. Such post-processing filters may be applied to a restored picture or block.

[0058] The coded picture buffer 240 can store the blocks or pictures reconstructed through the filter unit 235. The reconstructed blocks or pictures stored in the coded picture buffer 240 can be provided to the prediction unit 200 that performs intra prediction or inter prediction.

[0059] The entropy coding unit 245 may generate a quantization coefficient sequence, a transform coefficient sequence, or a signal sequence by scanning the quantization coefficients, transform coefficients, or residual signals of the generated residual block according to at least one scanning order (e.g., zigzag scan, vertical scan, horizontal scan, etc.), and may code the quantization coefficient sequence, transform coefficient sequence, or signal sequence using at least one entropy coding technique. At this time, information on the scan order may be determined according to a coding setting (e.g., image type, coding mode, prediction mode, transformation type, etc.), and may be determined implicitly or related information may be explicitly generated.

[0060] Also, the coding data including the coding information transmitted from each component can be generated and output as a bit stream, which can be realized by a multiplexer (MUX). At this time, the coding can be performed using methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), CABAC (Context-Adaptive Binary Arithmetic Coding), etc. However, the coding is not limited thereto, and various coding techniques that are improvements or modifications of the above can be used.

[0061] When performing entropy coding (assumed to be CABAC in this example) on syntax elements such as the residual block data and information generated during the encoding / decoding process, the entropy coding device may include a binarizer, a context modeler, and a binary arithmetic coder. In this case, the binary arithmetic coder may include a regular coding engine and a bypass coding engine.

[0062] Since the syntax elements input to the entropy coding device may not be binary values, if the syntax elements are not binary values, a binarization unit can binarize the syntax elements and output a bin string consisting of 0 or 1. At this time, a bin indicates a bit consisting of 0 or 1, and can be coded by a binary arithmetic coding unit. At this time, either a regular coding unit or a bypass coding unit can be selected based on the occurrence probability of 0 and 1, which can be determined based on the coding / decoding setting. If the syntax elements are data with the same frequency of 0 and 1, the bypass coding unit can be used, and if not, the regular coding unit can be used.

[0063] Various methods can be used to binarize the syntax elements. For example, fixed length binarization, unary binarization, truncated rice binarization, K-th Exp-Golomb binarization, etc. can be used. Also, signed or unsigned binarization can be performed depending on the range of values ​​of the syntax elements. The binarization process for the syntax elements generated in the present invention can include not only the binarization mentioned in the above examples, but also other additional binarization methods.

[0064] FIG. 3 is a block diagram showing the configuration of an image decoding device according to an embodiment of the present invention.

[0065] Referring to FIG. 3, the image decoding device 30 may include an entropy decoding unit 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, an adder / subtractor 325, a filter 330, and a decoded picture buffer 335.

[0066] Moreover, the prediction unit 310 can further include an intra-frame prediction module and an inter-frame prediction module.

[0067] First, when an image bitstream transmitted from the image encoding device 20 is received, it can be transmitted to the entropy decoding unit 305 .

[0068] The entropy decoding unit 305 can decode the bitstream to generate decoded data including the quantized coefficients and the decoding information to be transmitted to each component.

[0069] The prediction unit 310 may generate a prediction block based on data transferred from the entropy decoding unit 305. In this case, the prediction unit 310 may construct a reference picture list using a default construction technique based on reference images stored in the decoded picture buffer 335.

[0070] The intra-screen prediction unit may include a reference pixel construction unit, a reference pixel filter unit, a reference pixel interpolation unit, a prediction block generation unit, and a prediction mode decoding unit, some of which may perform the same process as the encoder, and some of which may perform the reverse induction process.

[0071] The inverse quantization unit 315 may inverse quantize the quantized transform coefficients provided as a bitstream and decoded by the entropy decoding unit 305 .

[0072] The inverse transform unit 320 may apply an inverse transform technique, such as an inverse DCT, an inverse integer transform, or a similar concept, to the transform coefficients to generate residual blocks.

[0073] At this time, the inverse quantization unit 315 and the inverse transform unit 320 can be realized in various ways by reversing the processes performed by the transform unit 210 and the quantization unit 215 of the image encoding device 20 described above. For example, the inverse quantization unit 315 and the inverse transform unit 320 can use the same processes and inverse transforms shared with the transform unit 210 and the quantization unit 215, or can reverse the transform and quantization processes using information on the transform and quantization processes from the image encoding device 20 (e.g., transform size, transform shape, quantization type, etc.).

[0074] The residual block that has undergone the inverse quantization and inverse transformation processes may be added to the prediction block derived by the prediction unit 310 to generate a reconstructed image block.

[0075] The filter 330 may also apply a deblocking filter to the reconstructed image blocks to remove blocking artifacts if necessary, and another loop filter may also be used before or after the decoding process to further improve the video quality.

[0076] The reconstructed and filtered image blocks may be stored in a decoded picture buffer 335 .

[0077] Although not shown, the image encoding / decoding device may further include a picture dividing unit and a block dividing unit.

[0078] The picture division unit may divide or partition a picture into at least one region based on a predetermined division unit, where the division unit may include a sub-picture, a slice, a tile, a brick, a block (e.g., a maximum coding unit), etc.

[0079] A picture may be divided into one or more rows of tiles or one or more columns of tiles, where a tile may be a block-based unit including a predetermined rectangular area of ​​the picture, where a tile may be divided into one or more bricks, where a brick may be composed of blocks of rows or columns of tiles.

[0080] A slice can have one or more configurations, one of which can be a scan-ordered bundle (e.g., block, brick, tile, etc.), one of which can be a shape containing rectangular regions, and other additional definitions are possible.

[0081] The definition of the slice configuration may be explicitly related information or may be implicitly determined. The definition of the configuration of each division unit as well as the slice may be set in a plurality of ways, and selection information related thereto may be generated.

[0082] A slice can be configured in a rectangular unit such as a block, brick, or tile, and slice position and size information can be expressed based on position information (eg, upper left position, lower right position, etc.) of the division unit.

[0083] In the present invention, it is assumed, but not limited to, that a picture can be composed of one or more subpictures, a subpicture can be composed of one or more slices or tiles or bricks, a slice can be composed of one or more tiles or bricks, and a tile can be composed of one or more bricks.

[0084] The division unit may be an integer number of blocks, but is not limited thereto, and may be a decimal number instead of an integer number. In other words, if the division unit is not an integer number of blocks, at least one division unit may be a sub-block.

[0085] There may be division units such as sub-pictures and tiles other than rectangular slices, and information on the position and size of the units may be expressed based on various methods.

[0086] For example, the position and size information of rectangular units can be expressed based on information on the number of rectangular units, information on the number of columns or rows of rectangular units, information on whether the columns or rows of rectangular units are evenly divided, information on the width or height of the columns or rows of rectangular units, index information of rectangular units, etc.

[0087] In the case of subpictures and tiles, the position and size information of each unit can be expressed based on all or part of the above information, and the image can be divided or partitioned into one or more units based on the position and size information.

[0088] Meanwhile, the image may be divided into blocks of various units and sizes through the block division unit. A basic coding unit (or maximum coding unit, Coding Tree Unit, CTU) may mean a basic (or starting) unit for prediction, transformation, quantization, etc. in an image coding process. At this time, the basic coding unit may be composed of one luminance basic coding block (or maximum coding block, Coding Tree Block, CTB) and two basic chrominance coding blocks according to a color format (YCbCr in this example), and the size of each block may be determined according to a color format. A coding block (CB) may be obtained according to the division process. A coding block may be understood as a unit that is not divided into more coding blocks according to a certain restriction, and may be set as a starting unit for division into lower units. In the present invention, a block may be understood as a broad concept including various shapes such as a triangle and a circle, without being limited to a rectangular shape.

[0089] It should be understood that the following description is for one color component, but can be changed and applied to other color components in proportion to the ratio according to the color format (e.g., in the case of YCbCr4:2:0, the horizontal and vertical length ratio of the luminance component and the chrominance component is 2:1). It should also be understood that although block division that depends on other color components (e.g., in the case of Cb / Cr, depending on the block division result of Y) is possible, independent block division is possible for each color component. It should also be understood that while one common block division setting (taking into consideration the proportionality to the length ratio) can be used, individual block division settings can be used according to the color components.

[0090] In the block division section, blocks can be expressed as M×N, and the maximum and minimum of each block are A value can be obtained within a range. For example, if the maximum value of a block is set to 256×256 and the minimum value is set to 4×4, a block of size 2m×2n (in this example, m and n are integers from 2 to 8) or a block of size 2m×2m (in this example, m and n are integers from 2 to 128) or a block of size m×m (in this example, m and n are integers from 4 to 256) can be obtained. Here, m and n may or may not be the same, and the range in which the blocks such as the maximum and minimum values ​​are supported can occur more than once.

[0091] For example, information on the maximum and minimum sizes of blocks may be generated, and information on the maximum and minimum sizes of blocks in a partial partition setting may be generated. Here, the former may be information on the range of the maximum and minimum sizes that can be generated in an image, and the latter may be information on the maximum and minimum sizes that can be generated based on a partial partition setting. Here, the partition setting may be defined by an image type (I / P / B), color components (YCbCr, etc.), block type (encoding / prediction / transformation / quantization, etc.), partition type (Index or Type), partition method (QT, BT, TT, etc. in the tree method, SI2, SI3, SI4, etc. in the index method), etc.

[0092] In addition, there may be restrictions on the width / length ratio (block shape) that a block can have, and a boundary value condition for this may be set. In this case, only blocks that are equal to or less than an arbitrary boundary value (k) may be supported, and k may be defined based on the width / length ratio such as A / B (A is the longer or same value of the width and length, and B is the remaining value), and may be a real number equal to or greater than 1, such as 1.5, 2, 3, 4, etc. As in the above example, a restriction condition for the shape of one block in an image may be supported, or one or more restriction conditions may be supported depending on the partition setting.

[0093] In summary, whether or not block division is supported can be determined based on the range and conditions described above and the division settings described below, etc. For example, if a candidate (child block) resulting from division of a block (parent block) satisfies the supported block conditions, the division can be supported, and if not, the division cannot be supported.

[0094] The block division unit can be set in relation to each component of the image encoding device and the decoding device, and the size and shape of the block can be determined through this process. At this time, the set block can be defined differently depending on the component, and can be a prediction block in the case of a prediction unit, a transform block in the case of a transformation unit, a quantization block in the case of a quantization unit, etc. However, without being limited thereto, a block unit can be further defined according to other components. In the present invention, the case where the input and output of each component are rectangular in shape will be mainly described, but some components can have input / output of other shapes (e.g., right-angled triangle, etc.).

[0095] The size and shape of the initial (or starting) block of the block division unit can be determined from the upper unit. The initial block can be divided into blocks of smaller size, and when the optimal size and shape by dividing the block is determined, the block can be determined as the initial block of the lower unit. Here, the upper unit can be a coding block, and the lower unit can be a prediction block or a transformation block, but is not limited thereto, and various variations are possible. When the initial block of the lower unit is determined as in the above example, a division process can be performed to find a block of the optimal size and shape like the upper unit.

[0096] In summary, the block division unit can divide a basic coding block (or a maximum coding block) into at least one coding block, and divide the coding block into at least one prediction block / transform block / quantization block. In addition, the prediction block can be divided into at least one transform block / quantization block, and the transform block can be divided into at least one quantization block. Here, some blocks may have a subordinate relationship (i.e., defined by a higher unit and a lower unit) with other blocks, or may have an independent relationship. As an example, the prediction block may be a higher unit of the transform block, or may be a unit independent of the transform block, and various relationship settings are possible depending on the type of block.

[0097] Depending on the encoding setting, it may be determined whether or not to combine the upper unit and the lower unit. Here, combining between units means that the upper unit is not divided into the lower unit, and the encoding process (e.g., prediction unit, transform unit, inverse transform unit, etc.) of the lower unit is performed on the block (size and shape) of the upper unit. In other words, it may mean that the division process in a plurality of units is shared, and the division information is generated in one of the units (e.g., the upper unit).

[0098] For example, prediction, transformation, and inverse transformation processes may be performed on a coding block (when the coding block is combined with a prediction block and a transformation block).

[0099] For example, a prediction process may be performed on a coding block (when the coding block is combined with a prediction block), and a transform and inverse transform process may be performed on a transform block that is the same as or smaller than the coding block.

[0100] For example, a prediction process may be performed on a prediction block that is the same size as or smaller than the coding block (when the coding block is combined with a transformation block), and transformation and inverse transformation processes may be performed on the coding block.

[0101] For example, a prediction process may be performed on a prediction block that is the same size as or smaller than the coding block (when the prediction block is combined with a transformation block), and transformation and inverse transformation processes may be performed on the prediction block.

[0102] As an example, the prediction process may be performed using a prediction block that is the same as or smaller than the coding block (if neither block is combined), and the transformation and inverse transformation processes may be performed using a transformation block that is the same as or smaller than the coding block.

[0103] Although various cases regarding coding, prediction, and transformation blocks are described in the above examples, the present invention is not limited to these.

[0104] The combination between the units may be supported as a fixed setting in an image, or may be supported as an adaptive setting in consideration of various coding factors, which may include image type, color components, coding mode (Intra / Inter), partition setting, block size / shape / position, horizontal / vertical length ratio, prediction related information (e.g., intra prediction mode, inter prediction mode, etc.), transform related information (e.g., transform technique selection information, etc.), quantization related information (e.g., quantization region selection information, quantized transform coefficient coding information, etc.), etc.

[0105] As described above, when a block having an optimal size and shape is found, mode information (e.g., partition information, etc.) for the block can be generated. The mode information can be recorded in a bitstream together with information generated in a component to which the block belongs (e.g., prediction-related information and transformation-related information, etc.) and transmitted to a decoder, where it can be parsed into units of the same level and used in the image decoding process.

[0106] In the following, the division method is described, and for convenience of explanation, it is assumed that the initial block is square in shape, but this is not limited to this, as it can be applied in the same or similar manner when the initial block is rectangular in shape.

[0107] The block division unit may support various types of division. For example, it may support tree-based division or index-based division, or other methods. In the tree-based division, the division type may be determined based on various types of information (e.g., whether or not to divide, tree type, division direction, etc.), and in the index-based division, the division type may be determined based on predetermined index information.

[0108] FIG. 4 is an exemplary diagram showing various division forms that can be obtained by the block division unit of the present invention.

[0109] In this example, it is assumed that the division pattern shown in Fig. 4 is obtained by one division execution (or process), but this is not limited to this, and it may be obtained by multiple division operations. Also, additional division patterns not shown in Fig. 4 are possible.

[0110] (Tree-based partitioning) In the tree-based partitioning of the present invention, a quad tree (QT), a binary tree (BT), a ternary tree (TT), etc. can be supported. When one tree type is supported, it is called single tree partitioning, and when two or more tree types are supported, it is called multi-tree partitioning.

[0111] QT means a method (n) in which a block is divided into two in both the horizontal and vertical directions (i.e., into four), BT means a method (b to g) in which a block is divided into two in either the horizontal or vertical direction, and TT means a method (h to m) in which a block is divided into thirds in either the horizontal or vertical direction.

[0112] Here, in the case of QT, a method (o, p) of dividing the image into four parts by limiting the dividing direction to either horizontal or vertical can be supported. In addition, in the case of BT, only the method (b, c) of dividing the image into equal parts, or only the method (d to g) of dividing the image into non-uniform parts, or both methods can be mixed. In addition, in the case of TT, only the method (h, j, k, m) of dividing the image into a specific direction (1:1:2, 2:1:1, etc. in the left-right or top-bottom direction) can be supported, or only the method (i, l) of dividing the image into the center (1:2:1, etc.) can be supported, or both methods can be mixed. In addition, a method (q) of dividing the image into four parts (i.e., 16 parts) in the horizontal and vertical directions can be supported.

[0113] In addition, the tree method may support z-division methods limited to the horizontal division direction (b, d, e, h, i, j, o), z-division methods limited to the vertical division direction (c, f, g, k, l, m, p), or a combination of both methods, where z can be an integer equal to or greater than 2, such as 2, 3, or 4.

[0114] In the present invention, it is assumed that QT supports n, BT supports b and c, and TT supports i and l.

[0115] Depending on the coding settings, one or more of the tree splitting methods may be supported, for example, QT may be supported, or QT / BT may be supported, or QT / BT / TT may be supported.

[0116] The above example is an example of a case where the basic tree division is QT, and BT and TT are included in the additional division method depending on the presence or absence of other tree support, but various modifications are possible. In this case, information on the presence or absence of other tree support (bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, etc., which can have a value of 0 or 1, where 0 means not supported and 1 means supported) can be implicitly determined according to the coding settings, or can be explicitly determined in units of sequences, pictures, subpictures, slices, tiles, bricks, etc.

[0117] The division information may include information on whether or not to divide (tree_part_flag, or qt_part_flag, bt_part_flag, tt_part_flag, or bt_tt_part_flag, which may have a value of 0 or 1, where 0 indicates no division and 1 indicates division). In addition, depending on the division method (BT and TT), information on the division direction (dir_part_flag, or bt_dir_part_flag, tt_dir_part_flag, or bt_tt_dir_part_flag, which may have a value of 0 or 1, where 0 indicates <horizontal> and 1 indicates <vertical>) may be added, which may be information that can occur when division is performed.

[0118] When multiple tree partitions are supported, various partition information configurations are possible. The following is an example of how partition information is configured at one depth level (i.e., the supported partition depth may be set to one or more, allowing recursive partitioning, but for the sake of convenience, the following description will be given).

[0119] As an example (1), information on whether or not division is to be performed is confirmed. At this time, if division is not to be performed, the division is terminated.

[0120] If a split is to be performed, the selection information regarding the type of split (for example, tree_idx. If it is 0, it is QT, if it is 1, it is BT, if it is 2, it is TT) is checked. At this time, the split direction information is further checked according to the type of split selected, and the next step is proceeded to (if additional splitting is possible because the split depth has not reached the maximum, etc., it starts from the beginning again, if splitting is not possible, the splitting ends).

[0121] As an example (2), information on whether or not a split is to be performed for a partial tree method (QT) is confirmed, and the process moves to the next step. At this time, if splitting is not to be performed, information on whether or not a split is to be performed for a partial tree method (BT) is confirmed. At this time, if splitting is not to be performed, information on whether or not a split is to be performed for a partial tree method (TT) is confirmed. At this time, if splitting is not to be performed, the splitting ends.

[0122] If partial tree-based (QT) splitting is to be performed, proceed to the next step. Also, if partial tree-based (BT) splitting is to be performed, check the split direction information and proceed to the next step. Also, if partial tree-based (TT) splitting is to be performed, check the split direction information and proceed to the next step.

[0123] As an example (3), check information about whether or not a split is to be performed for some tree methods (QT). If the split is not to be performed, check information about whether or not a split is to be performed for some tree methods (BT and TT). If the split is not to be performed, the split ends.

[0124] If some tree-based (QT) splitting is performed, proceed to the next step. Also, if some tree-based (BT and TT) splitting is performed, check the split direction information and proceed to the next step.

[0125] The above examples may have tree splitting priority (examples 2 and 3) or not (example 1), but various variations are possible. Also, the above examples are examples that explain the case where the splitting of the current step is unrelated to the splitting result of the previous step, but it is also possible to set the splitting of the current step to be dependent on the splitting result of the previous step.

[0126] For example, in the cases of 1 to 3, if some tree-based splitting (QT) was performed in the previous step and was transferred to the current step, the same tree-based splitting (QT) can be supported in the current step.

[0127] On the other hand, if some tree-based splits (QT) were not performed in the previous step, and other tree-based splits (BT or TT) were performed and moved to the current step, it is possible to set the partial tree-based splits (BT and TT) except for the partial tree-based split (QT) to be supported in subsequent steps including the current step.

[0128] In the above cases, this means that the tree structure supported by block partitioning can be adaptive, which means that the partitioning information structure described above can also be configured differently. (The example described below is assumed to be example 3.) In other words, in the above example, if partitioning of some tree methods (QT) has not been performed in the previous step, the partitioning process can be performed in the current step without considering some tree methods (QT). In addition, partitioning information related to related tree methods (e.g., information on whether partitioning is performed, partitioning direction information, etc. In this example, <qt>In this case, the information about whether or not the data is split can be removed.

[0129] The above example concerns adaptive partitioning information configuration for cases where block partitioning is permitted (e.g., the block size is within the range between the maximum and minimum values, the partitioning depth of each tree method does not reach the maximum depth <allowable depth>, etc.), but adaptive partitioning information configuration is also possible when block partitioning is restricted (e.g., the block size is not within the range between the maximum and minimum values, the partitioning depth of each tree method reaches the maximum depth, etc.).

[0130] As mentioned above, in the present invention, the tree-based division can be performed using a recursive method. For example, if the division flag of a coding block with a division depth of k is 0, the coding block is coded in a coding block with a division depth of k, and if the division flag of a coding block with a division depth of k is 1, the coding block is coded in N sub-coding blocks (where N is an integer equal to or greater than 2, such as 2, 3, 4, etc.) with a division depth of k+1 according to the division method.

[0131] The sub-coding block is again set as a coding block (k+1) and can be divided into a sub-coding block (k+2) through the above process. Such a hierarchical division method can be determined according to division settings such as a division range and an allowable division depth.

[0132] In this case, the bitstream structure for expressing the partition information may be selected from one or more scanning methods. For example, the bitstream of the partition information may be configured based on the order of partition depth, or the bitstream of the partition information may be configured based on whether or not it is partitioned.

[0133] For example, when the order of division depth is used as a criterion, this refers to a method of obtaining division information at the current level depth based on the first block, and then obtaining division information at the next level depth. When the criterion is whether or not there is division, this refers to a method of preferentially obtaining additional division information for blocks divided based on the first block, and other additional scanning methods can be considered.

[0134] The maximum block size and the minimum block size can be set to a common setting regardless of the type of tree (or all trees), or can be set individually for each tree, or can be set to a common setting for two or more trees. In this case, the maximum block size can be set to be equal to or smaller than the maximum coding block. If the maximum block size according to a given first tree is not equal to the maximum coding block, an implicit division is performed using a given second tree method until the maximum block size of the first tree is reached.

[0135] A common partition depth may be supported regardless of the type of tree, or an individual partition depth may be supported for each tree, or a common partition depth may be supported for two or more trees, or a partition depth may be supported for some trees and not for other trees.

[0136] Explicit syntax elements for the configuration information may be supported, and some configuration information may be defined implicitly.

[0137] (Index-based partitioning)

[0138] In the index-based division of the present invention, a constant split index (CSI) scheme and a variable split index (VSI) scheme can be supported.

[0139] The CSI scheme may be a scheme in which k sub-blocks are obtained by division in a predetermined direction. Here, k may be an integer equal to or greater than 2, such as 2, 3, or 4. In particular, the CSI scheme may be a division scheme in which the size and shape of the sub-blocks are determined based on the value of k, regardless of the size and shape of the block. Here, the predetermined direction may be one or a combination of two or more directions selected from the group consisting of horizontal, vertical, and diagonal (e.g., upper left to lower right or lower left to upper right).

[0140] The index-based CSI partitioning scheme of the present invention may include z partition candidates in either the horizontal or vertical direction, where z is an integer equal to or greater than 2, such as 2, 3, or 4, and one of the horizontal or vertical lengths of each sub-block may be the same and the other may be the same or different. The horizontal or vertical length ratio of the sub-blocks may be A1:A2:...:AZ, where A1 to AZ may be integers equal to or greater than 1, such as 1, 2, or 3.

[0141] In addition, candidates for division into x and y in the horizontal and vertical directions, respectively, may be included. In this case, x and y may be integers equal to or greater than 1, such as 1, 2, 3, 4, etc., but restrictions may be imposed when x and y are both 1 (because a already exists). Although FIG. 4 shows a case where the horizontal or vertical length ratio of each sub-block is the same, candidates including cases where they are different may also be included.

[0142] In addition, it may include candidates that are divided into w in either one of some diagonal directions (top left to bottom right) or some diagonal directions (bottom left to top right), where w may be an integer greater than or equal to 2, such as 2 or 3.

[0143] Referring to Fig. 4, the partitioning pattern may be divided into symmetrical partitioning pattern (b) and asymmetrical partitioning pattern (d, e) according to the length ratio of each subblock, and into partitioning pattern (k, m) biased in a specific direction and partitioning pattern (k) arranged in the center. The partitioning pattern may be defined according to various coding elements including the subblock shape as well as the length ratio of the subblocks, and the supported partitioning pattern may be implicitly or explicitly determined according to the coding setting. Therefore, a candidate group in the index-based partitioning method may be determined based on the supported partitioning pattern.

[0144] Meanwhile, the VSI method may be a method in which one or more sub-blocks are obtained through division in a predetermined direction while the width w or height h of the sub-block is fixed, where w and h may be integers equal to or greater than 1, such as 1, 2, 4, 8, etc. In particular, the VSI method may be a division method in which the number of sub-blocks is determined based on the size and shape of the block and the value of w or n.

[0145] The index-based VSI partitioning method of the present invention may include candidates that are partitioned by fixing either the horizontal or vertical length of the subblock, or may include candidates that are partitioned by fixing the horizontal and vertical lengths of the subblock. Since the horizontal or vertical length of the subblock is fixed, it may have a feature that allows equal partitioning in the horizontal or vertical direction, but is not limited thereto.

[0146] If the block before division is M×N and the horizontal length of the sub-block is fixed (w), or the vertical length is fixed (h), or the horizontal and vertical lengths are fixed (w, h), the number of sub-blocks obtained can be (M*N) / w, (M*N) / h, or (M*N) / w / h, respectively.

[0147] Depending on the coding settings, only the CSI scheme may be supported, or only the VSI scheme may be supported, or both schemes may be supported, and information about the supported schemes may be implicitly or explicitly specified.

[0148] In the present invention, it is assumed that the CSI scheme is supported.

[0149] Depending on the encoding settings, a set of candidates may consist of two or more of the index splits.

[0150] For example, a candidate group such as {a, b, c}, {a, b, c, n}, {a to g, n} may be constructed, but this may be an example in which a candidate group is constructed based on block shapes that are predicted to occur frequently based on general statistical characteristics, such as a block shape that is divided into two in the horizontal or vertical direction, or a block shape that is divided into two in both the horizontal and vertical directions.

[0151] Alternatively, a set of candidates such as {a, b}, {a, o}, {a, b, o} or {a, c}, {a, p}, {a, c, p} can be constructed, which include candidates that are divided into two and four in the horizontal and vertical directions, respectively. This can be an example of constructing a set of candidates based on block shapes that are predicted to be frequently divided in a particular direction.

[0152] Alternatively, a candidate group such as {a, o, p} or {a, n, q} can be constructed, but an example of constructing the candidate group may be a block shape that is predicted to have a large number of divisions having a smaller size than the block before division.

[0153] Alternatively, a candidate group such as {a, r, s} can be constructed, but it may be determined that an optimal division result that can be obtained in a rectangular shape has been obtained from the block before division using another method (tree method), and a non-rectangular division form is constructed as a candidate group.

[0154] As in the above examples, various candidate set configurations are possible, and more than one candidate set configuration may be supported by considering various coding factors.

[0155] Once the candidate set has been constructed, various partition configurations are possible.

[0156] For example, index selection information can be generated from a candidate group including a non-split candidate (a) and split candidates (b to s).

[0157] Alternatively, information indicating whether or not a division is to occur (whether or not the division type is a) can be generated, and if a division is to occur (if not a), index selection information can be generated from a candidate group consisting of candidates to be divided (b to s).

[0158] The partition information can be configured in various ways other than those described above, and binary bits can be assigned to the index of each candidate in the candidate group using various methods such as fixed length binarization, variable length binarization, etc. If the number of candidate groups is two, one bit can be assigned to the index selection information, and if the number is three or more, one or more bits can be assigned to the index selection information.

[0159] Unlike the tree-based partitioning method, the index-based partitioning method can be a method of selectively configuring partitioning patterns that are predicted to occur frequently into a candidate set.

[0160] Also, since the amount of bits for expressing index information can increase according to the number of supported candidate groups, this method may be suitable for single hierarchical division (e.g., division depth is limited to 0) rather than hierarchical division (recursive division) of a tree-based method. In other words, this method may support one division operation, or may be a method in which sub-blocks obtained through index-based division cannot be further divided.

[0161] In this case, it may mean that further division into blocks of the same type having a smaller size is not possible (for example, a coding block obtained by an index division method cannot be further divided into coding blocks), but it is also possible to set that further division into blocks of other types is not possible (for example, division of a coding block into not only coding blocks but also prediction blocks is not possible). Of course, it is not limited to the above example, and other modified examples are possible.

[0162] Next, a case will be considered in which block division is determined mainly based on the type of block among the coding elements.

[0163] First, coding blocks are obtained through a splitting process. Here, the splitting process can use a tree-based splitting method, and splitting results such as a (no split), n (QT), b, c (BT), i, l (TT) as shown in Figure 4 can be obtained depending on the type of tree. Depending on the coding settings, various combinations of each tree type such as QT / QT+BT / QT+BT+TT are possible.

[0164] The example described below shows the process in which a prediction block and a transformation block are finally divided based on the coding block obtained by the above process, and assumes that prediction, transformation and inverse transformation processes are performed based on each division size.

[0165] As an example (1), a prediction block may be set with the same size as the coding block to perform a prediction process, and a transformation block may be set with the same size as the coding block (or the prediction block) to perform a transformation and inverse transformation process. Since the prediction block and the transformation block are set based on the coding block, there is no separate partition information.

[0166] As an example (2), a prediction block may be set with the same size as the coding block, and a prediction process may be performed. In the case of a transformation block, a transformation block may be obtained through a division process based on the coding block (or the prediction block), and a transformation and inverse transformation process may be performed based on the obtained size.

[0167] Here, the splitting process can use a tree-based splitting method, and splitting results such as a (no split), b, c (BT), i, l (TT), and n (QT) can be obtained according to the type of tree in Fig. 4. Depending on the encoding settings, various combinations of each tree type such as QT / BT / QT+BT / QT+BT+TT are possible.

[0168] Here, the splitting process can use an index-based splitting method, and depending on the type of index, splitting results such as a (no split), b, c, d, etc. in Figure 4 can be obtained. Depending on the encoding settings, various candidate sets such as {a, b, c} and {a, b, c, d} can be constructed.

[0169] As an example (3), in the case of a prediction block, a prediction block can be obtained by performing a division process based on a coding block, and a prediction process can be performed based on the obtained size. In the case of a transformation block, the size of the coding block can be set as it is, and transformation and inverse transformation processes can be performed. This example may correspond to a case where the prediction block and the transformation block have an independent relationship with each other.

[0170] Here, the splitting process can use an index-based splitting method, and splitting results such as a (no split), b to g, n, r, and s in Fig. 4 can be obtained according to the type of index. Depending on the coding settings, various candidate groups such as {a, b, c, n}, {a to g, n}, and {a, r, s} can be configured.

[0171] As an example (4), in the case of a prediction block, a prediction block can be obtained by performing a division process based on a coding block, and a prediction process can be performed based on the obtained size. In the case of a transformation block, the size of the prediction block can be set as it is, and transformation and inverse transformation processes can be performed. This example may be a case where the transformation block is set as the size of the obtained prediction block, or vice versa (the prediction block is set as the size of the transformation block).

[0172] Here, the splitting process can use a tree-based splitting method, and splitting forms such as a (no split), b, c (BT), and n (QT) in Fig. 4 can be obtained according to the type of tree. Various combinations of each tree type such as QT / BT / QT+BT are possible according to the encoding settings.

[0173] Here, an index-based splitting method can be used for the splitting process, and split forms such as a (no split), b, c, n, o, and p in Fig. 4 can be obtained according to the type of index. Depending on the coding setting, various candidate groups such as {a, b}, {a, c}, {a, n}, {a, o}, {a, p}, {a, b, c}, {a, o, p}, {a, b, c, n}, and {a, b, c, n, p} can be configured. In addition, among the index-based splitting methods, the VSI method may be used alone or in combination with the CSI method to configure the candidate group.

[0174] As an example (5), in the case of a prediction block, a prediction block can be obtained by performing a division process based on a coding block, and a prediction process can be performed based on the obtained size. In addition, in the case of a transformation block, a prediction block can be obtained by performing a division process based on a coding block, and a transformation process and an inverse transformation process can be performed based on the obtained size. This example may be a case where a prediction block and a transformation block are each divided based on a coding block.

[0175] Here, the partitioning process can use a tree-based partitioning method or an index-based partitioning method, and a candidate group can be constructed in the same or similar manner as in the fourth example.

[0176] The above examples describe some cases that may occur depending on whether the division process of each type of block is shared, but the present invention is not limited to these examples and various modified examples are possible. In addition, block division settings may be determined taking into account not only the type of block but also various coding elements.

[0177] In this case, the coding elements may include image type (I / P / B), color components (YCbCr), block size / shape / position, block horizontal / vertical length ratio, block type (coding block, prediction block, transformation block, quantization block, etc.), division state, coding mode (Intra / Inter), prediction related information (intra-screen prediction mode, inter-screen prediction mode, etc.), transformation related information (transformation technique selection information, etc.), quantization related information (quantization area selection information, quantized transformation coefficient coding information, etc.), etc.

[0178] In an image encoding method according to an embodiment of the present invention, the inter prediction may be configured as follows. The inter prediction of the prediction unit may include a reference picture construction step, a motion estimation step, a motion compensation step, a motion information determination step, and a motion information encoding step. Also, the image encoding device may be configured to include a reference picture construction unit, a motion estimation unit, a motion compensation unit, a motion information determination unit, and a motion information encoding unit that realize the reference picture construction step, the motion estimation step, the motion compensation step, the motion information determination step, and the motion information encoding step. Some of the above-mentioned processes may be omitted, or other processes may be added, and the processes may be changed to another order other than the order described above.

[0179] In an image decoding method according to an embodiment of the present invention, the inter prediction may be configured as follows. The inter prediction of the predictor may include a motion information decoding step, a reference picture construction step, and a motion compensation step. Also, the image decoding apparatus may be configured to include a motion information decoding unit, a reference picture construction unit, and a motion compensation unit that realize the motion information decoding step, the reference picture construction step, and the motion compensation step. Some of the above-described processes may be omitted, or other processes may be added, and the above-described order may be changed to another order.

[0180] The reference picture construction unit and the motion compensation unit of the image decoding device have the same functions as the corresponding units of the image coding device, so detailed description is omitted, and the motion information decoding unit can be performed by using the method used in the motion information coding unit in reverse. Here, the prediction block generated via the motion compensation unit can be transmitted to the adder.

[0181] FIG. 5 is an exemplary diagram showing various cases in which a prediction block is obtained through inter-picture prediction according to the present invention.

[0182] Referring to Fig. 5, unidirectional prediction can obtain a prediction block (A. forward prediction) from previously coded reference pictures T-1, T-2, or can obtain a prediction block (B. backward prediction) from subsequently coded reference pictures T+1, T+2. Bidirectional prediction can generate a prediction block (C, D) from multiple previously coded reference pictures T-2 to T+2. In general, P image type can support unidirectional prediction, and B image type can support bidirectional prediction.

[0183] As in the above example, pictures referenced for encoding the current picture can be obtained from memory, and a reference picture list can be constructed based on the current picture T, including reference pictures whose time order or display order is before the current picture and reference pictures whose time order or display order is after the current picture.

[0184] Inter prediction E can be performed on the current image as well as previous or subsequent images based on the current image. Performing inter prediction on the current image can be called non-directional prediction. This can be supported in I image type or P / B image type. The supported image type can be determined according to the encoding setting. Performing inter prediction on the current image generates a prediction block using spatial correlation, and the only difference is that inter prediction is performed on another image for the purpose of using temporal correlation, and the prediction method (e.g., reference image, motion vector, etc.) can be the same.

[0185] Here, it is assumed that the image types capable of performing inter prediction are P and B pictures, but other various additional or alternative image types may also be applied. For example, a certain image type may not support intra prediction but may support only inter prediction, may support only inter prediction in a certain direction (backward), or may support only inter prediction in a certain direction.

[0186] The reference picture configuration unit can configure and manage reference pictures used for encoding the current picture through a reference picture list. At least one reference picture list can be configured according to encoding settings (e.g., image type, prediction direction, etc.), and a prediction block can be generated from the reference pictures included in the reference picture list.

[0187] In the case of unidirectional prediction, inter prediction can be performed using at least one reference picture included in reference picture list 0 (L0) or reference picture list 1 (L1). In addition, in the case of bidirectional prediction, inter prediction can be performed using at least one reference picture included in a composite list LC generated by combining L0 and L1.

[0188] For example, unidirectional prediction may be divided into forward prediction Pred_L0 using a forward reference picture list L0 and backward prediction Pred_L1 using a backward reference picture list L1. Bidirectional prediction Pred_BI may use both the forward reference picture list L0 and the backward reference picture list L1.

[0189] Alternatively, bidirectional prediction may also include copying the forward reference picture list L0 to the backward reference picture list L1 to perform two or more forward predictions, and bidirectional prediction may also include copying the backward reference picture list L1 to the forward reference picture list L0 to perform two or more backward predictions.

[0190] The prediction direction can be represented by flag information indicating the direction (e.g., inter_pred_idc, which is assumed to be adjustable by predFlagL0, predFlagL1, and predFlagBI). predFlagL0 indicates whether it is forward prediction or not, and predFlagL1 indicates whether it is backward prediction or not. Bidirectional prediction can be indicated by indicating the prediction status via predFlagBI, or by activating predFlagL0 and predFlagL1 simultaneously (e.g., when each flag is 1).

[0191] Although the present invention will be described mainly in terms of a case where the prediction is a forward prediction using a forward reference picture list and is a unidirectional prediction, the present invention can be applied in the same manner or with modifications to the other cases.

[0192] In general, a method may be used in which an optimal reference picture for a picture to be encoded is determined by an encoder and information on the reference picture is explicitly transmitted to a decoder, and thus a reference picture configuration unit may manage a picture list referenced for inter prediction of a current picture and set rules for reference picture management in consideration of limited memory size.

[0193] The transmitted information can be defined as an RPS (Reference Picture Set), and pictures selected in the RPS are classified as reference pictures and stored in a memory (or DPB), and pictures not selected in the RPS are classified as non-reference pictures and can be removed from the memory after a certain time. The memory can store a preset number of pictures (e.g., 14, 15, 16 pictures or more), and the size of the memory can be set according to the level and image resolution.

[0194] FIG. 6 is an example diagram of constructing a reference picture list according to an embodiment of the present invention.

[0195] 6, generally, reference pictures T-1 and T-2 existing before the current picture are assigned to L0, and reference pictures T+1 and T+2 existing after the current picture are assigned to L1 and managed. When constructing L0, if the number of reference pictures allowed for L0 is not met, reference pictures of L1 can be assigned. Similarly, when constructing L1, if the number of reference pictures allowed for L1 is not met, reference pictures of L0 can be assigned.

[0196] Also, the current picture may be included in at least one reference picture list. For example, the current picture may be included in L0 or L1, and L0 may be constructed by adding a reference picture (or the current picture) whose temporal order is T to the reference pictures before the current picture, and L1 may be constructed by adding a reference picture whose temporal order is T to the reference pictures after the current picture.

[0197] The composition of the reference picture list can be determined depending on the coding settings.

[0198] The current picture may not be included in the reference picture list and may be managed through a separate memory separate from the reference picture lists, or the current picture may be included in at least one reference picture list and managed.

[0199] For example, it can be determined by a signal (curr_pic_ref_enabled_flag) indicating whether the current picture is included in the reference picture list, where the signal can be information that is implicitly determined or explicitly generated.

[0200] In detail, when the signal is deactivated (e.g., curr_pic_ref_enabled_flag=0), the current picture is not included as a reference picture in any reference picture list, and when the signal is activated (e.g., curr_pic_ref_enabled_flag=1), whether the current picture is included in a given reference picture list is implicitly determined (e.g., added only to L0, added only to L1, or can be added simultaneously to L0 and L1) or can be explicitly determined by generating a related signal (e.g., curr_pic_ref_from_l0_flag, curr_pic_ref_from_l1_flag). The signals can be supported in units of sequences, pictures, subpictures, slices, tiles, bricks, etc.

[0201] Here, the current picture may be located at the first or last position in the reference picture list as shown in Fig. 6, and the arrangement order in the list may be determined according to encoding settings (e.g., image type information, etc.) For example, the current picture may be located at the first position for an I type and at the last position for a P / B type, but is not limited thereto, and other variations are possible.

[0202] Alternatively, a separate reference picture memory may be supported depending on a signal (ibc_enabled_flag) indicating whether block matching (or template matching) is supported in the current picture, where the signal may be information that is implicitly determined or explicitly generated.

[0203] In detail, when the signal is inactivated (e.g., ibc_enabled_flag=0), it means that block matching is not supported in the current picture, and when the signal is activated (e.g., ibc_enabled_flag=1), block matching is supported in the current picture and a reference picture memory for this purpose can be supported. In this example, it is assumed that additional memory is provided, but it is also possible to set to directly support block matching in an existing memory supported for the current picture without providing additional memory.

[0204] The reference picture construction unit may include a reference picture interpolation unit, and may determine whether to perform an interpolation process for a pixel in a decimal unit according to the interpolation accuracy of the inter-picture prediction. For example, when the interpolation accuracy is an integer unit, the reference picture interpolation process may be omitted, and when the interpolation accuracy is a decimal unit, the reference picture interpolation process may be performed.

[0205] In the case of an interpolation filter used in the reference picture interpolation process, it may be implicitly determined according to the encoding setting, or it may be explicitly determined from among a plurality of interpolation filters. The configuration of the plurality of interpolation filters may support fixed candidates or support adaptive candidates according to the encoding setting, and the number of candidates may be 2, 3, 4 or an integer greater than 1. The explicitly determined unit may be determined from among a sequence, a picture, a sub-picture, a slice, a tile, a rebrick, a block, etc.

[0206] Here, the coding settings may be determined by an image type, a color component, state information of a target block (e.g., a block size, a shape, a horizontal / vertical length ratio, etc.), settings of inter-frame prediction (e.g., a motion information coding mode, a motion model selection information, a motion vector precision selection information, a reference picture, a reference direction, etc.), etc. The motion vector precision may mean the precision of a motion vector (i.e., pmv+mvd), but may be replaced with the precision of a motion vector prediction value pmv or a motion vector difference value mvd.

[0207] Here, the interpolation filter may have a filter length of k-tap, where k may be an integer of 2, 3, 4, 5, 6, 7, 8 or more. The filter coefficients may be derived from a mathematical formula having various coefficient characteristics such as a Wiener filter or a Kalman filter. Filter information (e.g., filter coefficients, tap information, etc.) used for the interpolation may be implicitly determined or derived, or related information may be explicitly generated. In this case, the filter coefficients may be configured to include 0.

[0208] The interpolation filter may be applied to a given pixel unit, which may be limited to integer or fractional units, or may be applied to both integer and fractional units.

[0209] For example, an interpolation filter can be applied to k integer unit pixels adjacent to a pixel to be interpolated (that is, a fractional unit pixel) in the horizontal or vertical direction.

[0210] Alternatively, an interpolation filter can be applied to p integer unit pixels and q fractional unit pixels (p+q=k) adjacent to the target pixel in the horizontal or vertical direction. In this case, the precision of the fractional unit referenced for the interpolation (e.g., 1 / 4 unit) can be expressed with the same precision as the target pixel or with a lower precision (e.g., 2 / 4 → 1 / 2).

[0211] When only one of the x and y components of the pixel to be interpolated is located in a decimal unit, the interpolation can be performed based on k pixels adjacent in the component direction of the decimal unit (e.g., the x axis is horizontal and the y axis is vertical). When both the x and y components of the pixel to be interpolated are located in decimal units, the primary interpolation can be performed based on x pixels adjacent in either the horizontal or vertical direction, and the interpolation can be performed based on y pixels adjacent in the remaining direction. In this case, the case where x and y are the same as k will be described, but the present invention is not limited to this and x and y may be different.

[0212] The pixel to be interpolated may be obtained by an interpolation accuracy (1 / m), where m may be an integer of 1, 2, 4, 8, 16, 32 or more. The interpolation accuracy may be implicitly determined according to an encoding setting, or related information may be explicitly generated. The encoding setting may be defined based on an image type, a color component, a reference picture, a motion information encoding mode, motion model selection information, etc. The explicitly determined unit may be determined from among a sequence, a sub-picture, a slice, a tile, a brick, etc.

[0213] Through the above process, an interpolation filter setting for a target block can be obtained, and reference picture interpolation can be performed based on the obtained setting. Also, a detailed interpolation filter setting based on the interpolation filter setting can be obtained, and reference picture interpolation can be performed based on the detailed interpolation filter setting. That is, it is assumed that the obtained interpolation filter setting for a target block may be a single fixed candidate or may be a possibility of using multiple candidates. It is assumed that the interpolation accuracy in the example described below is 1 / 16 units (for example, the number of pixels to be interpolated is 15).

[0214] As an example of the detailed interpolation filter setting, one of a plurality of candidates can be adaptively used for a first pixel unit at a predetermined position, and one predetermined interpolation filter can be used for a second pixel unit at a predetermined position.

[0215] The first pixel unit can be defined among the total fractional pixels (in this example, 1 / 16 to 15 / 16) supported by the interpolation accuracy, and the number of pixels included in the first pixel unit is a, where a can be defined between 0, 1, 2, ... (m-1).

[0216] The second pixel unit may include pixels of the entire decimal unit excluding the first pixel unit, and the number of pixels included in the second pixel unit may be derived by subtracting the pixels of the first pixel unit from the total number of pixels to be interpolated. In this example, the pixel unit is divided into two, but the present invention is not limited thereto, and it may be divided into three or more.

[0217] For example, the first pixel unit may be a unit expressed in multiples of 1 / 2, 1 / 4, 1 / 8, etc. As an example, the first pixel unit may be a pixel at a position of 8 / 16 in the case of 1 / 2 units, {4 / 16, 8 / 16, 12 / 16} in the case of 1 / 4 units, and {2 / 16, 4 / 16, 6 / 16, 8 / 16, 10 / 16, 12 / 16, 14 / 16} in the case of 1 / 8 units.

[0218] The detailed interpolation filter settings can be determined according to the coding settings, which can be defined by the image type, color components, state information of the current block, inter prediction settings, etc. In the following, examples of detailed interpolation filter settings according to various coding elements are considered. For convenience of explanation, it is assumed that a is 0 and a is equal to or greater than c (c is an integer equal to or greater than 1).

[0219] For example, a can be (1,0), (0,1), (1,1) for color components (luminance, chrominance); or a can be (1,0), (0,1), (1,1), (1,3) for motion information coding modes (merged mode, competitive mode); or a can be (0,1,1), (1,0,0), (1,1,1), (1,3,7) for motion model selection information (movement motion, non-movement motion A, non-movement motion B); or a can be (0,0,1), (0,1,0), (1,0,0), (0,1,1), (1,0,1), (1,1,0), (1,1,1) for motion vector precision (1 / 2, 1 / 4, 1 / 8). Or, in case of reference pictures (current picture, other pictures), it can be (0,1), (1,0), (1,1).

[0220] As described above, the interpolation process can be performed by selecting one from a plurality of interpolation accuracies, and if an interpolation process based on adaptive interpolation accuracy is supported (e.g., if adaptive_ref_resolution_enabled_flag.0, a predetermined interpolation accuracy is used, and if it is 1, one of a plurality of interpolation accuracies is used), accuracy selection information (e.g., ref_resolution_idx) can be generated.

[0221] The motion estimation and compensation processes can be performed according to the interpolation accuracy, and the representation unit and storage unit for the motion vectors can also be determined based on the interpolation accuracy.

[0222] For example, if the interpolation accuracy is 1 / 2 unit, the motion estimation and compensation process is performed in 1 / 2 unit, and the motion vector is expressed in 1 / 2 unit and can be used in the encoding process. Also, the motion vector is stored in 1 / 2 unit and can be referenced in the motion information encoding process of other blocks.

[0223] Alternatively, if the interpolation accuracy is 1 / 8 unit, the motion estimation and compensation process is performed in 1 / 8 unit, and the motion vector is expressed in 1 / 8 unit and can be used in the encoding process and stored in 1 / 8 unit.

[0224] In addition, the motion estimation and compensation process and the motion vectors can be performed, represented and stored in units different from the interpolation accuracy, such as integer, 1 / 2, or 1 / 4 units, which can be adaptively determined depending on the inter-frame prediction method / settings (e.g., motion estimation / compensation method, motion model selection information, motion information coding mode, etc.).

[0225] For example, assuming that the interpolation accuracy is 1 / 8, in the case of a moving motion model, the motion estimation and compensation process is performed in 1 / 4 units, and the motion vector is expressed in 1 / 4 units (this example assumes the units in the encoding process) and can be stored in 1 / 8 units. In the case of a non-moving motion model, the motion estimation and compensation process is performed in 1 / 8 units, and the motion vector is expressed in 1 / 4 units and can be stored in 1 / 8 units.

[0226] For example, assuming that the interpolation accuracy is 1 / 8, in the case of block matching, the motion estimation and compensation process is performed in 1 / 4 units, and the motion vector is expressed in 1 / 4 units and can be stored in 1 / 8 units, whereas in the case of template matching, the motion estimation and compensation process is performed in 1 / 8 units, and the motion vector is expressed in 1 / 8 units and can be stored in 1 / 8 units.

[0227] As an example, assuming that the interpolation accuracy is 1 / 16 units, in the competitive mode, the motion estimation and compensation process is performed in 1 / 4 units, and the motion vector is expressed in 1 / 4 units and can be stored in 1 / 16 units. In the merge mode, the motion estimation and compensation process is performed in 1 / 8 units, and the motion vector is expressed in 1 / 4 units and can be stored in 1 / 16 units. In the skip mode, the motion estimation and compensation process is performed in 1 / 16 units, and the motion vector is expressed in 1 / 4 units and can be stored in 1 / 16 units.

[0228] In summary, the motion estimation and compensation, the motion vector representation, and the storage unit can be adaptively determined based on the inter-prediction method or setting, and the interpolation accuracy. In particular, the motion estimation and compensation and the motion vector representation unit can be adaptively determined according to the inter-prediction method or setting, and the storage unit of the motion vector can be generally determined according to the interpolation accuracy, but is not limited thereto and various modified examples are possible. In addition, although the above example is an example according to one category (e.g., motion model selection information, motion estimation / compensation method, etc.), the setting can be determined by mixing two or more categories.

[0229] Also, as described above, the interpolation accuracy information has a predetermined value or is selected from among a plurality of accuracies, but the reference picture interpolation accuracy can be determined based on a motion estimation and compensation setting supported according to an inter-frame prediction method or setting. For example, when a motion model supports up to 1 / 8 units and a non-motion model supports up to 1 / 16 units, an interpolation process can be performed according to the accuracy unit of the non-motion motion model having the highest accuracy.

[0230] That is, reference picture interpolation may be performed depending on settings for supported precision information such as a moving motion model, a non-moving motion model, a competition mode, a merge mode, a skip mode, etc. In this case, the precision information may be implicitly or explicitly determined, and when related information is explicitly generated, it may be included in units such as a sequence, a picture, a sub-picture, a slice, a tile, a brick, etc.

[0231] The motion estimation unit refers to a process of estimating (or searching) whether a target block has a high correlation with a predetermined block of a predetermined reference picture. The size and shape (M×N) of the target block on which prediction is performed can be obtained from the block division unit. As an example, the target block can be determined to be in the range of 4×4 to 128×128. Inter prediction may generally be performed in units of prediction blocks, but may be performed in units of coding blocks, transform blocks, etc. depending on the setting of the block division unit. Estimation is performed within an estimable range of the reference region, and at least one motion estimation method can be used. The motion estimation method can define an estimation order and conditions for pixel units.

[0232] The motion estimation may be performed based on a motion estimation method. For example, the area to be compared for the motion estimation process may be a target block in the case of block matching, and may be a predetermined area (template) set around the target block in the case of template matching. In the former case, a block having the highest correlation within an estimable range of the target block and the reference area may be found, and in the latter case, an area having the highest correlation within an estimable range of the template and the reference area defined according to the encoding setting may be found.

[0233] Motion estimation may be performed based on a motion model. In addition to a translation motion model that considers only translation, motion estimation and compensation may be performed using additional motion models. For example, motion estimation and compensation may be performed using a motion model that considers not only translation but also rotation, perspective, zoom-in / out, etc. This can help improve coding performance by generating a prediction block reflecting the various types of motion that occur according to the regional characteristics of an image.

[0234] FIG. 7 is a conceptual diagram illustrating a non-moving motion model according to an embodiment of the present invention.

[0235] 7, an example of expressing motion information based on motion vectors V0 and V1 at a predetermined position is shown as an example of an affine model. Since motion can be expressed based on multiple motion vectors, accurate motion estimation and compensation are possible.

[0236] As in the above example, inter-frame prediction is performed based on a predefined motion model, but inter-frame prediction based on an additional motion model can also be supported. Here, it is assumed that the predefined motion model is a translation motion model and the additional motion model is an affine model, but the present invention is not limited thereto and various modifications are possible.

[0237] In the case of a moving motion model, motion information (assuming one-way prediction) can be expressed based on one motion vector, and the control point (reference point) for indicating the motion information is assumed to be the upper left coordinate, but is not limited to this.

[0238] In the case of a motion model other than motion, it can be expressed by various configurations of motion information. In this example, a configuration is assumed in which one motion vector (based on the upper left coordinate) is expressed by additional information. Some motion estimation and compensation mentioned in the examples described below may not be performed in units of blocks, but may be performed in units of a predetermined sub-block. In this case, the size and position of the predetermined sub-block may be determined based on each motion model.

[0239] 8 is an exemplary diagram illustrating motion estimation in sub-block units according to an embodiment of the present invention, in particular, motion estimation in sub-block units using an affine model (two motion vectors).

[0240] In the case of the translation motion model, the motion vectors of the pixels included in the target block can be the same, that is, a motion vector can be applied collectively to each pixel, and motion estimation and compensation can be performed using a single motion vector V0.

[0241] In the case of a non-movement motion model (affine model), the pixel-by-pixel motion vectors included in the target block may not be the same, and individual pixel-by-pixel motion vectors may be required. In this case, a pixel-by-pixel or sub-block-by-subblock motion vector can be derived based on the motion vectors V0 and V1 at predetermined control point positions of the target block, and motion estimation and compensation can be performed using the induced motion vector.

[0242] For example, the motion vector {e.g., (Vx, Vy)} of a subblock or pixel unit in the target block can be derived by the formula Vx=(V1x-V0x)×x / M-(V1y-V0y)×y / N+V0x, Vy=(V1y-V0y)×x / M+(V1x-V0x)×y / N+V0y. In the above formula, V0 {in this example, (V0x, V0y)} means the motion vector of the upper left side of the target block, and V1 {in this example, (V1x, V1y)} means the motion vector of the upper right side of the target block. Considering the complexity, the motion estimation and motion compensation of the non-movement motion model can be performed on a subblock basis.

[0243] Here, the size of the sub-block (M×N) may be determined according to the coding setting, and may have a fixed size or may be set to an adaptive size. Here, M and N may be integers of 2, 4, 8, 16 or more, and M and N may be the same or may not be the same. The size of the sub-block may be explicitly generated in units of a sequence, a picture, a sub-picture, a slice, a tile, a brick, etc., or may be implicitly determined by a common agreement between an encoder and a decoder, or may be determined by coding settings.

[0244] Here, the coding settings can be defined by one or more elements such as state information of the current block, image type, color components, and inter-frame prediction setting information (motion information coding mode, reference picture information, interpolation accuracy, motion model selection information, etc.).

[0245] The above example describes a process of deriving the size of a sub-block according to a predetermined non-motion motion model and performing motion estimation and compensation based on the size of the sub-block. As in the above example, motion estimation and compensation can be performed on a sub-block or pixel basis according to a motion model, and detailed description thereof will be omitted.

[0246] In the following, various examples of motion information that are configured according to a motion model are considered.

[0247] For example, in the case of a motion model that expresses rotational motion, a translational motion of a block can be expressed by one motion vector, and the rotational motion can be expressed by rotational angle information. The rotational angle information can be measured with a predetermined position (e.g., the upper left coordinate) as a reference (0 degrees), and can be expressed as k candidates (k is an integer of 1, 2, 3 or more) having predetermined intervals (e.g., angle difference values ​​of 0 degrees, 11.25 degrees, 22.25 degrees, etc.) in a predetermined angle range (e.g., between -90 degrees and 90 degrees).

[0248] Here, the rotation angle information can be coded by itself during the motion information coding process, or can be coded (eg, prediction+difference value information) based on motion information (eg, motion vectors, rotation angle information) of adjacent blocks.

[0249] Alternatively, one motion vector can represent the translational motion of a block, and one or more additional motion vectors can represent the rotational motion of the block, where the number of additional motion vectors can be 1, 2 or an integer greater than 1, and the control point of the additional motion vector can be determined from the upper right, lower left, and lower right coordinates, or other coordinates within the block can be set as the control point.

[0250] Here, the additional motion vector can be coded by itself during the motion information coding process, or can be coded (e.g., prediction + difference value information) based on motion information of an adjacent block (e.g., a motion vector based on a motion model or a non-motion motion model), or can be coded (e.g., prediction + difference value information) based on other motion vectors in the block representing rotational motion.

[0251] For example, in the case of a motion model expressing a size adjustment or scaling motion such as a zoom-in / zoom-out situation, a block movement motion can be expressed by one motion vector, and the size adjustment motion can be expressed by scaling information, which can be expressed as scaling information indicating expansion or contraction in the horizontal or vertical direction based on a predetermined position (e.g., the upper left coordinate).

[0252] Here, scaling can be applied in at least one direction of the horizontal or vertical direction. In addition, individual scaling information applied to the horizontal or vertical direction can be supported, or scaling information applied commonly can be supported. The width and height of the block to which the scaling is applied can be added to a predetermined position (upper left coordinate) to determine a position for motion estimation and compensation.

[0253] Here, the scaling information can be coded by itself during the motion information coding process, or can be coded (eg, prediction+difference value information) based on motion information (eg, motion vectors, scaling information) of neighboring blocks.

[0254] Alternatively, one motion vector can represent the translation of a block, and one or more additional motion vectors can represent the size adjustment of the block, where the number of additional motion vectors can be 1, 2 or an integer greater than 1, and the control point of the additional motion vector can be determined among the coordinates of the upper right, lower left, and lower right, or the coordinates within other blocks can be set as the control point.

[0255] Here, the additional motion vector can be coded by itself during the motion information coding process, or can be coded (e.g., prediction + difference value information) based on motion information of an adjacent block (e.g., a motion vector based on a motion motion model or a non-motion motion model), or can be coded (e.g., prediction + difference value) based on a predetermined coordinate within the block (e.g., the lower right coordinate).

[0256] Although the above example has been described with respect to an expression for indicating a certain movement, it is also possible to express the movement information for expressing a plurality of movements.

[0257] For example, in the case of a motion model that expresses various and complex motions, a block translation motion can be expressed by one motion vector, a rotation motion can be expressed by rotation angle information, and a size adjustment can be expressed by scaling information. A detailed description of each motion can be provided by referring to the above examples, so that a detailed description will be omitted.

[0258] Alternatively, one motion vector can represent the translation motion of a block, and one or more additional motion vectors can represent other motions of the block, where the number of additional motion vectors can be one, two or more integers, and the control points of the additional motion vectors can be determined among the coordinates of the upper right, lower left, and lower right, or the coordinates within other blocks can be set as the control points.

[0259] Here, the additional motion vector can be coded by itself during the motion information coding process, or can be coded (e.g., prediction + difference value information) based on motion information of adjacent blocks (e.g., motion vectors based on a motion motion model or a non-motion motion model), or can be coded (e.g., prediction + difference value information) based on other motion vectors within the block representing various motions.

[0260] The above description may be about an affine model, and will be focused on the case where there is one or two additional motion vectors. In summary, it is assumed that the number of motion vectors used by a motion model is one, two, or three, and that it can be regarded as an individual motion model according to the number of motion vectors used to express motion information. It is also assumed that when there is one motion vector, it is a predefined motion model.

[0261] A plurality of motion models for inter prediction may be supported, and may be determined by a signal (e.g., adaptive_motion_mode_enabled_flag) indicating support of an additional motion model. Here, if the signal is 0, a predefined motion model may be supported, and if the signal is 1, a plurality of motion models may be supported. The signal may be generated in units of a sequence, a picture, a subpicture, a slice, a tile, a brick, a block, etc., and if it is not possible to confirm separately, a signal value may be assigned according to a predefined setting. Alternatively, whether or not to support may be implicitly determined according to an encoding setting. Alternatively, the case of implicit or explicit support may be determined according to an encoding setting. Here, the encoding setting may be defined by one or more elements such as an image type, an image type (e.g., 0 for a general image, 1 for a 360-degree image), a color component, etc.

[0262] Through the above process, it can be determined whether or not multiple motion models are supported. In the following, it is assumed that there are two or more additionally supported motion models, and that it is determined that multiple motion models are supported in units of a sequence, a picture, a sub-picture, a slice, a tile, a brick, etc., but there may be some exceptions. In the example described below, it is assumed that motion models A, B, and C can be supported, A is a basically supported motion model, and B and C are additionally supported motion models.

[0263] Configuration information for supported motion models can be generated in the above units, i.e., supported motion models such as {A, B}, {A, C}, {A, B, C,}, etc. can be configured.

[0264] For example, the configuration candidates can be assigned indexes (0 to 2) to select them. If index 2 is selected, a motion model configuration that supports {A, C} can be determined, and if index 3 is selected, {A, B, C} can be determined.

[0265] Alternatively, information indicating whether a certain motion model is supported can be supported individually. That is, a support / non-support flag for B and a support / non-support flag for C can be generated. If both flags are 0, it may be the case that only A is supported. This example may be an example in which processing is performed without generating information indicating whether multiple motion models are supported.

[0266] Once the candidate set of supported motion models is constructed as in the above example, any motion model from the candidate set can be explicitly determined and used on a block-by-block basis, or can be used implicitly.

[0267] In general, the motion estimation unit may be a component present in the encoding device, but may also be a component that may be included in the decoding device according to a prediction method (e.g., template matching, etc.). For example, in the case of template matching, the decoder may perform motion estimation via a neighboring template of the target block to obtain motion information of the target block. In this case, motion estimation related information (e.g., motion estimation range, motion estimation method <scanning order>, etc.) may be implicitly determined or explicitly generated and may be included in units such as a sequence, a picture, a sub-picture, a slice, a tile, a brick, etc.

[0268] The motion compensation unit refers to a process for obtaining data of a part of blocks of a predetermined reference picture determined through a motion estimation process as a prediction block of a target block. In particular, the motion compensation unit can generate a prediction block of a target block from at least one region (or block) of at least one reference picture based on motion information (e.g., reference picture information, motion vector information, etc.) obtained through the motion estimation process.

[0269] Motion compensation may be performed based on the motion compensation method as follows.

[0270] In the case of block matching, based on the motion vector (Vx, Vy) of the target block (M×N) explicitly obtained from the reference picture and the coordinates (Px+Vx, Py+Vy) obtained via the upper left coordinates (Px, Py) of the target block, data of the area M to the right and N below can be compensated for with the predicted block of the target block.

[0271] In the case of template matching, the data of the area M to the right and N below can be compensated for with the predicted block of the target block based on the coordinates (Px+Vx, Py+Vy) obtained via the motion vector (Vx, Vy) of the target block (M×N) implicitly obtained from the reference picture and the coordinates (Px, Py) of the upper left corner of the target block.

[0272] Also, motion compensation can be performed based on a motion model as follows.

[0273] In the case of a moving motion model, based on the coordinates (Px+Vx, Py+Vy) obtained via one of the motion vectors (Vx, Vy) of a target block (M×N) explicitly obtained from a reference picture and the coordinates (Px, Py) of the upper left corner of the target block, data of the corresponding area M to the right and N below can be compensated for by the prediction block of the target block.

[0274] In the case of a non-movement motion model, the data of the corresponding area M / m to the right and N / n to the bottom can be compensated with the prediction block of the subblock based on the motion vectors (Vmx, Vny) of m×n subblocks implicitly acquired through a plurality of motion vectors (V0x, V0y) and (V1x, V1y) of the target block (M×N) explicitly acquired in the reference picture and the coordinates (Pmx+, Vnx, Pmy+, Vny) acquired through the upper left coordinates (Pmx, Pny) of each subblock. That is, the prediction blocks of the subblocks can be collected and compensated with the prediction block of the target block.

[0275] The motion information determination unit may perform a process for selecting optimal motion information of the target block. In general, the motion information that is optimal in terms of coding cost may be determined using a rate-distortion technique that considers block distortion (e.g., distortion of the target block and the reconstructed block, SAD (Sum of Absolute Difference), SSD (Sum of Square Difference), etc.) and the amount of bits generated by the motion information. A predicted block generated based on the motion information determined through the above process may be transmitted to the subtraction unit and the addition unit. Also, the motion information determination unit may be configured to be included in a decoding device according to some prediction method (e.g., template matching, etc.), in which case the motion information may be determined based on the block distortion.

[0276] The motion information decision unit may consider inter-frame prediction related setting information such as a motion compensation method, a motion model, etc. For example, when a plurality of motion compensation methods are supported, motion compensation method selection information and the corresponding motion vector, reference picture information, etc. may be optimal motion information. Or, when a plurality of motion models are supported, motion model selection information and the corresponding motion vector, reference picture information, etc. may be optimal motion information.

[0277] The motion information encoding unit may encode the motion information of the target block obtained through the motion information determination process. At this time, the motion information may be composed of information on an image and an area referred to for predicting the target block. In particular, the motion information may be composed of information on the referenced image (e.g., reference image information, etc.) and information on the referenced area (e.g., motion vector information, etc.).

[0278] In addition, inter prediction related setting information (or selection information, e.g., motion estimation / compensation method, motion model selection information, etc.) may be included in the motion information of the target block. Based on the inter prediction related setting, information on the reference image and region (e.g., the number of motion vectors, etc.) may be configured.

[0279] The information on the reference image and the reference region can be configured as one combination to encode the motion information, and the combination of the information on the reference image and the reference region can be configured as a motion information encoding mode.

[0280] Here, the information on the reference image and the reference region can be obtained based on neighboring blocks or predetermined information (eg, images coded before or after the current picture, a zero motion vector, etc.).

[0281] The adjacent block is a block that belongs to the same space as the target block and is closest to the target block.<inter_blk_A> ,A block that belongs to the same space as the target block and is far away<inter_blk_B> , and blocks that belong to a space other than the target block<inter_blk_C> Based on adjacent blocks (candidate blocks) belonging to at least one of these categories, information regarding the reference image and the reference region can be obtained.

[0282] For example, the motion information of the target block may be encoded based on the motion information of the candidate block or the reference picture information, or the motion information of the target block may be encoded based on information derived from the motion information of the candidate block or the reference picture information (or information obtained through an intermediate value, a transformation process, etc.). That is, the motion information of the target block may be predicted from the candidate block and information related thereto may be encoded.

[0283] In the present invention, the motion information of the current block may be coded based on one or more motion information coding modes, which may be defined in various ways and may include one or more of a skip mode, a merge mode, a competition mode, etc.

[0284] Based on the above-mentioned template matching tmp, it may be combined with the motion information coding mode, or may be supported in a separate motion information coding mode, or may be included in all or some of the detailed configurations of the motion information coding mode. This is on the assumption that template matching is determined to be supported in a higher unit (e.g., picture, sub-picture, slice, etc.), and a flag indicating whether or not template matching is supported may be considered as a part of the setting of inter prediction.

[0285] Based on the above-mentioned method ibc for performing block matching in the current picture, it may be combined with the motion information coding mode, or may be supported in a separate motion information coding mode, or may be included in all or some of the detailed configurations of the motion information coding mode. This is on the assumption that block matching is determined to be supported in the current picture in a higher unit, and a flag indicating whether or not it is supported may be considered as a part of the setting of inter prediction.

[0286] Based on the above-mentioned motion model (affine), it may be combined with the motion information coding mode, or may be supported as a separate motion information coding mode, or may be included in all or some of the detailed configurations of the motion information coding mode. This is based on the assumption that a motion model other than motion is defined to be supported in a higher unit, and a flag indicating whether or not it is supported may be considered as a part of the setting of inter prediction.

[0287] For example, individual motion information coding modes such as temp_inter, temp_tmp, temp_ibc, and temp_affine may be supported. Or, combined motion information coding modes such as temp_inter_tmp, temp_inter_ibc, temp_inter_affine, and temp_inter_tmp_ibc may be supported. Or, among the motion information prediction candidates constituting temp, a template-based candidate, a candidate based on a method of performing block matching within the current picture, and an affine-based candidate may be included.

[0288] Here, temp can mean skip mode (skip), merge mode (merge), or competition mode (comp). For example, in the case of skip mode, skip_inter, skip_tmp, skip_ibc, and skip_affine are supported, in the case of merge mode, merge_inter, merge_tmp, merge_ibc, and merge_affine are supported, and in the case of competition mode, comp_inter, comp_tmp, comp_ibc, and comp_affine are supported.

[0289] If a skip mode, a merge mode, and a competition mode are supported and a candidate that considers the above factors is included in a group of motion information prediction candidates for each mode, one mode can be selected according to a flag that distinguishes the skip mode, the merge mode, and the competition mode. For example, if a flag indicating whether or not the skip mode is supported and has a value of 1, the skip mode can be selected, and if a flag indicating whether or not the merge mode is supported and has a value of 0, the merge mode can be selected, and if it has a value of 1, the competition mode can be selected. And, a group of motion information prediction candidates for each mode can include candidates based on inter, tmp, ibc, and affine.

[0290] Alternatively, when a plurality of motion information coding modes are supported under one common mode, in addition to a flag for selecting one of the skip mode, merge mode, and competition mode, an additional flag for distinguishing a detailed mode of the selected mode may be supported. For example, when the merge mode is selected, a flag for selecting from merge_inter, merge_tmp, merge_ibc, merge_affine, etc., which are detailed modes related to the merge mode, may be additionally supported. Alternatively, a flag indicating whether or not it is merge_inter may be supported, and if it is not merge_inter, a flag for selecting from merge_tmp, merge_ibc, merge_affine, etc. may be additionally supported.

[0291] All or some of the motion information encoding mode candidates may be supported according to an encoding setting, where the encoding setting may be defined by one or more elements such as state information of a target block, an image type, an image category, a color component, an inter-frame prediction support setting (e.g., whether template matching is supported, whether block matching is supported in the current picture, a non-motion motion model support element, etc.), etc.

[0292] As an example, a supported motion information encoding mode may be determined according to the size of a block. In this case, a support range of the block size may be determined according to a first threshold size (minimum value) or a second threshold size (maximum value), and each threshold size may be expressed as W, H, W×H, or W*H, where W is the width (W) and height (H) of the block. In the case of the first threshold size, W and H may be integers of 4, 8, 16 or more, and W*H may be an integer of 16, 32, 64 or more. In the case of the second threshold size, W and H may be integers of 16, 32, 64 or more, and W*H may be an integer of 64, 128, 256 or more. The above range may be determined by either the first threshold size or the second threshold size, or by both.

[0293] At this time, the magnitude of the threshold may be fixed or adaptive depending on the image (e.g., image type, etc.) At this time, the magnitude of the first threshold may be set based on the size of the smallest coding block, the smallest prediction block, the smallest transform block, etc., and the magnitude of the second threshold may be set based on the size of the largest coding block, the largest prediction block, the largest transform block, etc.

[0294] For example, a supported motion information coding mode may be determined according to the image type. In this case, the I image type may include at least one of a skip mode, a merge mode, and a competitive mode. In this case, a method of performing block matching (or template matching) in a current picture, an individual motion information coding mode related to an affine model (hereinafter referred to as a "element") may be supported, or a motion information coding mode in which two or more elements are combined may be supported. Or, in a certain motion information coding mode, other elements may be configured as a motion information prediction candidate group.

[0295] The P / B image type may include at least one of a skip mode, a merge mode, and a competitive mode. In this case, a motion information coding mode may be supported for a general inter-picture prediction, a template matching, a block matching performed on a current picture, and an affine model (hereinafter, referred to as a term "element"), or a motion information coding mode may be supported by combining two or more elements. Or, in a certain motion information coding mode, other elements may be configured as a motion information prediction candidate group.

[0296] FIG. 9 is a flow chart illustrating encoding of motion information according to an embodiment of the present invention.

[0297] Referring to FIG. 9, the motion information coding mode of a current block can be confirmed (S900).

[0298] A motion information coding mode may be defined by combining and setting predetermined information (e.g., motion information, etc.) used in inter-frame prediction. The predetermined information may include one or more of a predicted motion vector, a motion vector differential value, a differential motion vector accuracy, reference image information, a reference direction, motion model information, information on the presence or absence of a residual component, etc. The motion information coding mode may include at least one of a skip mode, a merge mode, and a competitive mode, and may also include other additional modes.

[0299] Depending on the motion information coding mode, the configuration and setting of explicitly generated information and implicitly defined information can be determined. In this case, in the case of explicit information, each piece of information can be generated individually or in a combined form (e.g., index form).

[0300] For example, the skip mode or merge mode may be defined based on (one) predetermined index information for a predicted motion vector, reference image information, and reference direction. At this time, the index may be configured as one or more candidates, and each candidate may be set based on the motion information of a predetermined block (e.g., a predicted motion vector, reference image information, and reference direction of the corresponding block may be configured as one combination). Also, the motion vector difference value may be implicitly processed (e.g., a zero vector).

[0301] As an example, the competitive mode may define a predicted motion vector, reference image information, and reference direction based on (one or more) predetermined index information. At this time, the index may be supported individually for each piece of information, or may be supported as a combination of two or more pieces of information. At this time, the index may be configured as one or more candidates, and each candidate may be set based on the motion information of a predetermined block (e.g., predicted motion vector, etc.), or may be configured with a previously set value (e.g., processing the distance from the current picture to 1, 2, 3, etc., processing the reference direction information to L0, L1, etc.) (e.g., reference image information, reference direction information, etc.). In addition, a motion vector differential value may be explicitly generated, and differential motion vector accuracy information may be additionally generated according to the motion vector differential value (e.g., when it is not a zero vector).

[0302] As an example, in skip mode, the presence or absence of residual components is implicitly processed (e.g.,<cbf_flag=0> ), and the motion model may be implicitly processed to a predetermined value (eg, to support a parallel motion model).

[0303] For example, in the merge mode or the competitive mode, information on the presence or absence of residual components may be explicitly generated, and a motion model may be explicitly selected. In this case, information for distinguishing the motion models may be generated within one motion information coding mode, or a separate motion information coding mode may be supported.

[0304] In the above description, when one index is supported, index selection information is not generated, and when two or more indexes are supported, index selection information can be generated.

[0305] In the present invention, a candidate group consisting of two or more indexes is called a motion information prediction candidate group. In addition, an existing motion information coding mode may be changed or a new motion information coding mode may be supported according to various predetermined information and settings.

[0306] Referring to FIG. 9, a predicted motion vector for a current block can be derived (S910).

[0307] The motion vector predictor may be set to a predefined value, or may be selected from a group of motion vector predictor candidates. In the former case, the motion vector predictor is implicitly determined, and in the latter case, index information for selecting the motion vector predictor may be explicitly generated.

[0308] Here, in the case of the already set value, it can be set based on the motion vector of one block at a predetermined position (e.g., a block in the left, top, top left, top right, bottom left direction, etc.), or it can be set based on the motion vectors of two or more blocks, or it can be set to a default value (e.g., a zero vector, etc.).

[0309] The predicted motion vector has the same or different candidate group configuration settings depending on the motion information coding mode. As an example, the skip mode / merge mode / competition mode may support a, b, and c prediction candidates, respectively, and the number of each candidate may be configured to be the same or different. In this case, a, b, and c may be configured as integers equal to or greater than 1, such as 2, 3, 5, 6, and 7. The candidate group configuration order, settings, and the like will be described through other embodiments. In the example described below, the candidate group configuration settings of the skip mode will be described assuming the same case as the merge mode, but are not limited thereto, and some may have different configurations.

[0310] The predicted motion vector obtained based on the index information can be used as is to reconstruct the motion vector of the current block, or can be adjusted based on the reference direction and the distance between the reference pictures.

[0311] For example, in the case of competitive mode, reference image information may be generated separately, but if the distance between the current picture and the reference picture of the target mode is different from the distance between the picture containing the predicted motion vector candidate block and the reference picture of the block, this may be adjusted according to the distance between the current picture and the reference picture of the target block.

[0312] Also, the number of motion vectors derived based on the motion model may be different from each other, that is, in addition to the motion vector at the upper left control point position, motion vectors at the upper right and lower left control point positions may be further derived based on the motion model.

[0313] Referring to FIG. 9, a motion vector differential value of a current block can be restored (S920).

[0314] The motion vector difference value can be induced to a zero vector value in the skip mode and merge mode, and the difference value information of each component can be restored in the competition mode.

[0315] The motion vectors may be represented based on a predetermined precision that has already been set (e.g., based on the interpolation precision, or based on the motion model selection information, etc.), or may be represented based on any one of a plurality of precisions, and predetermined information for that purpose may be generated. In this case, the predetermined information may be related to the selection of the motion vector precision.

[0316] For example, when one component of a motion vector is 32 / 16, based on an implicit precision setting of 1 / 16 pixel units, motion component data for 32 can be obtained. Or, based on an explicit precision setting of 2 pixel units, the motion component can be converted to 2 / 1, and motion component data for 2 can be obtained.

[0317] The above configuration may be a method for precision processing on a motion vector obtained by adding a motion vector predictor and a motion vector differential value that have the same precision or are obtained through the same precision conversion process.

[0318] In addition, the motion vector difference value may be represented according to a preset precision or may be represented with one of a plurality of precisions, in which case precision selection information may be generated.

[0319] The precision may include at least one of 1 / 16, 1 / 8, 1 / 4, 1 / 2, 1, 2, and 4 pixel units, and the number of candidates may be an integer of 1, 2, 3, 4, 5, or more. Here, the supported precision candidate group configuration (e.g., divided into number, candidates, etc.) may be explicitly defined in units of sequence, picture, sub-picture, slice, tile, brick, etc., or may be implicitly defined depending on the encoding setting. Here, the encoding setting may be defined by at least one element such as image type, reference image, color component, motion model selection information, etc.

[0320] Next, a case where candidates for accuracy are configured according to various coding elements will be described. The accuracy will be described assuming an explanation regarding motion vector differential values, but this can be similarly or identically applied to motion vectors.

[0321] For example, in the case of a moving motion model, a candidate group can be configured as {1 / 4, 1}, {1 / 4, 1 / 2}, {1 / 4, 1 / 2, 1}, {1 / 4, 1, 4}, {1 / 4, 1 / 2, 1, 4}, etc., and in the case of a non-moving motion model, a precision candidate group can be configured as {1 / 16, 1 / 4, 1}, {1 / 16, 1 / 8, 1}, {1 / 16, 1, 4}, {1 / 16, 1 / 4, 1, 2}, {1 / 16, 1 / 4, 1, 4}, etc. This assumes that the minimum precision supported is in 1 / 4 pixel units in the former case and in 1 / 16 pixel units in the latter case, and additional precisions other than the minimum precision can be included in the candidate group.

[0322] Alternatively, when the reference image is a different picture, a candidate set can be configured as {1 / 4, 1 / 2, 1}, {1 / 4, 1, 4}, {1 / 4, 1 / 2, 1, 4}, etc., and when the reference image is the current picture, a candidate set can be configured as {1, 2}, {1, 4}, {1, 2, 4}, etc. In this case, the latter case can be a configuration that does not perform decimal unit interpolation for block matching, etc., and when decimal unit interpolation is performed, a configuration can be made that includes decimal unit precision candidates.

[0323] Alternatively, if the color components are luminance components, possible candidate groups are {1 / 4, 1 / 2, 1}, {1 / 4, 1, 4}, {1 / 4, 1 / 2, 1, 4}, etc., and if the color components are chrominance components, possible candidate groups are {1 / 8, 1 / 4}, {1 / 8, 1 / 2}, {1 / 8, 1}, {1 / 8, 1 / 4, 1 / 2}, {1 / 8, 1 / 2, 2}, {1 / 8, 1 / 4, 1 / 2, 2}, etc. In this case, a candidate group proportional to the luminance component can be formed according to the composition ratio of the color components (e.g., 4:2:0, 4:2:2, 4:4:4, etc.), or a separate candidate group can be formed.

[0324] Although the above example describes a case where there are multiple precision candidates for each coding element, the case where there is only one candidate configuration (ie, representation with a minimum precision that has already been set) is also possible.

[0325] In summary, based on the differential motion vector accuracy in competitive mode, a motion vector differential value with minimum accuracy can be restored, and a motion vector differential value with zero vector value can be induced in skip mode and merge mode.

[0326] Here, the number of restored motion vector differential values ​​may differ depending on the motion model, i.e., in addition to the motion vector differential value at the upper left control point position, motion vector differential values ​​at the upper right and lower left control point positions may be further derived depending on the motion model.

[0327] Also, one differential motion vector precision can be applied to multiple motion vector differential values, or an individual differential motion vector precision can be applied to each motion vector differential value, which can be determined according to coding settings.

[0328] Also, if at least one motion vector differential value is not 0, information on the differential motion vector precision may be generated, and if not, it may be omitted, but is not limited thereto.

[0329] Referring to FIG. 9, the motion vector of the current block can be restored (S930).

[0330] The motion vector of the target block can be restored by adding the predicted motion vector and the motion vector difference value obtained through the previous process of this step. In this case, if the motion vector is restored by selecting one of a plurality of precisions, a precision unification process can be performed.

[0331] For example, the motion vector obtained by adding the predicted motion vector and the motion vector difference value (related to the motion vector precision) can be restored based on the precision selection information. Or, the motion vector difference value (related to the difference motion vector precision) can be restored based on the precision selection information, and the restored motion vector difference value can be added to the predicted motion vector to obtain the motion vector.

[0332] Referring to FIG. 9, motion compensation can be performed based on the motion information of the current block (S940).

[0333] The motion information may include reference image information other than the above-mentioned predicted motion vector and motion vector differential value, reference direction, motion model information, etc. In the case of the skip mode and the merge mode, some of the information (e.g., reference image information, reference direction, etc.) may be implicitly determined, and the competitive mode may explicitly process related information.

[0334] A prediction block can be obtained by performing motion compensation based on the motion information obtained through the above process.

[0335] Referring to FIG. 9, the residual components of the current block may be decoded (S950).

[0336] The current block can be reconstructed by adding the residual component obtained through the above process to the prediction block. In this case, depending on the motion information coding mode, the presence or absence of the residual component can be processed explicitly or implicitly.

[0337] FIG. 10 is a layout diagram of a target block and its adjacent blocks according to an embodiment of the present invention.

[0338] Referring to FIG. 10, blocks adjacent to the target block in the left, upper, upper left, upper right, lower left, etc. directions are<inter_blk_A> In the space (Col_Pic) that is not the same in time, the target block and the corresponding block are located in the center, left, right, top, bottom, top left, top right, bottom left, bottom right, etc.<inter_blk_B> can be configured as a candidate block for prediction of motion information (eg, motion vectors, etc.) of the target mode.

[0339] The case of inter_blk_A may be an example in which the direction of adjacent blocks is determined based on an encoding order such as raster scan or z-scan, and existing directions are removed according to various scan orders, or adjacent blocks to the right, bottom, and bottom-right can be further configured as candidate blocks.

[0340] Referring to FIG. 10, Col_Pic may be an adjacent image before or after the current image (e.g., when the interval between images is 1), and the corresponding block may be set to have the same position in the image as the target block.

[0341] Alternatively, Col_Pic may be an image in which the spacing between images is already defined based on the current image (for example, the spacing between images is z, where z is an integer of 1, 2, or 3), and the corresponding block may be set to a position moved a predetermined displacement vector (Disparity Vector) from a predetermined coordinate (for example, the upper left side) of the target block, but the displacement vector may be set to a previously defined value.

[0342] Alternatively, Col_Pic can be set based on motion information (e.g., a reference image) of adjacent blocks, and the displacement vector can be set based on motion information (e.g., a motion vector) of adjacent blocks to determine the position of the corresponding block.

[0343] In this case, k adjacent blocks can be referenced, where k can be an integer of 1, 2 or more. If k is 2 or more, Col_Pic and the displacement vector can be obtained based on the calculation of the maximum, minimum, median, weighted average, etc. of the motion information (e.g., reference image or motion vector) of the adjacent blocks. For example, the displacement vector can be set as the motion vector of the left or upper block, and can be set as the median or average value of the motion vectors of the left and lower left blocks.

[0344] The setting of the temporal candidates may be determined based on the setting of the motion information configuration, etc. For example, the position of Col_Pic, the position of the corresponding block, etc. may be determined depending on whether the motion information to be included in the motion information prediction candidate group is configured in block units or subblock units, such as block unit motion information or subblock unit motion information. As an example, when the motion information in subblock units is acquired, a block at a position moved by a predetermined displacement vector may be set as the position of the corresponding block.

[0345] The above example shows a case where information regarding the position of Col_Pic and the corresponding block is implicitly defined, and explicit related information can also occur in units of sequences, pictures, slices, tile groups, tiles, bricks, etc.

[0346] Meanwhile, when a division unit partition such as a sub-picture, slice, or tile is not limited to one picture (e.g., the current picture) but is shared between images, if the position of the block corresponding to Col_Pic is different from the division unit to which the target block belongs (i.e., if it is on or off the boundary of the division unit to which the target block belongs), it can be determined based on motion information of a subsequent candidate based on a predetermined priority order, or can be set to the same position in the image as the target block.

[0347] The motion information of the spatially adjacent block and the temporally adjacent block may be included in a motion information prediction candidate group of the target block, which are called spatial candidates and temporal candidates. The spatial candidates and the temporal candidates may be a and b, and a and b may be integers of 1 to 6 or more. In this case, a may be equal to or greater than b.

[0348] The number of spatial candidates and temporal candidates supported may be fixed or may be variable (eg, including 0) depending on the configuration of the preceding motion information prediction candidates.

[0349] Also, blocks that are not immediately adjacent to the target block<inter_blk_C> can be taken into consideration in forming the candidate group.

[0350] As an example, motion information of blocks located at a predetermined interval with respect to the target block (e.g., the coordinates of the upper left side) may be included in the candidate group. The interval may be p or q (depending on each component of the motion vector), where p and q may be integers of 2 or more, such as 0, 2, 4, 8, 16, 32, etc., and p and q may be the same or different. In other words, assuming that the coordinates of the upper left side of the target block are (m, n), motion information of blocks including positions (m±p, n±q) may be included in the candidate group.

[0351] For example, motion information of blocks that have been encoded before the target block may be included in the candidate group. In this case, a predetermined number of blocks that have been encoded recently based on a predetermined encoding order (e.g., raster scan, Z scan, etc.) based on the target block may be considered as candidates, and blocks with older encoding order may be removed from the candidates as the encoding progresses in a first-in, first-out manner such as FIFO.

[0352] In the above example, this means that the candidate group includes motion information of a non-neighboring block that has already been coded based on the target mode, and therefore this is statistically referred to as a candidate.

[0353] In this case, the target block can be set as the reference block for obtaining statistical candidates, but the target block may be set to a higher block (e.g., the target block is a coding block). <1> / Predicted Blocks <2> If , the upper block is the largest coding block <1> / Encoding block <2> It is also possible to set k statistical candidates, where k is an integer between 1 and 6 or greater.

[0354] The candidate group may include motion information obtained by combining motion information such as the spatial candidates, the temporal candidates, the statistical candidates, etc. For example, a candidate may be obtained by applying a weighted average to each component of a plurality of (two, three or more) pieces of motion information already included in the candidate group, or a candidate may be obtained by subjecting each component of a plurality of pieces of motion information to a process such as median, maximum, minimum, etc. This is called a combination candidate, and r combination candidates may be supported, where r may be an integer of 1, 2, 3 or more.

[0355] If the candidates used for combination are not identical to the information of the reference image, the reference image information of the combination candidate can be set based on the reference image information of one of the candidates, or can be set to an already defined value.

[0356] The statistical candidates or combination candidates may be supported in a fixed number, or may be supported in a variable number according to the configuration of the preceding motion information prediction candidates.

[0357] Also, default candidates having preset values ​​such as (s, t) may be included in the candidate group, and a variable number (0, 1 or more integers) may be supported according to the configuration of the preceding motion information prediction candidate. In this case, s and t may be set to values ​​including 0, and may be set based on various block sizes (e.g., the horizontal or vertical length of the maximum coding / prediction / transformation block, the minimum coding / prediction / transformation block, etc.).

[0358] A group of motion information prediction candidates may be configured from the candidates according to a motion information coding mode, and other additional candidates may be included. Also, the configuration of the group of candidates may be the same or different according to the motion information coding mode.

[0359] For example, the skip mode and merge mode may form a common candidate set, and the competition mode may form a separate candidate set.

[0360] Here, the configuration setting of the candidate group can be defined by the category and position of the candidate block (e.g., determined from among left / top / top left / top right / bottom left directions, and the sub-block position from which motion information is obtained in the determined direction), the number of candidates (e.g., total number, maximum number per category, etc.), and the candidate configuration method (e.g., priority per category, priority within a category, etc.).

[0361] The number (total number) of motion information prediction candidate groups may be k, where k may be an integer of 1 to 6 or more. In this case, when the number of candidate groups is one, candidate group selection information is not generated, meaning that motion information of a previously defined candidate block is set as predicted motion information, and when the number of candidate groups is two or more, candidate group selection information may be generated.

[0362] The category of the candidate block may be any one of inter_blk_A, inter_blk_B, and inter_blk_C, in which inter_blk_A may be a category that is basically included, and another category may be a category that is additionally supported, but is not limited thereto.

[0363] The above description may be related to the motion information prediction candidate configuration for a non-movement motion model, and the non-movement motion model (affine model) may also use / reference the same or similar candidate blocks for candidate group configuration. Meanwhile, in the case of an affine model, since the number of motion vectors and the motion characteristics are different from those of the motion model, the candidate group configuration may be different from that of the motion model.

[0364] For example, when the motion model of the candidate block is an affine model, the configuration of the motion vector set of the candidate block can be included in the candidate group as it is. As an example, when the coordinates of the upper left side and the upper right side of the candidate block are used as control points, the motion vector set of the coordinates of the upper left side and the upper right side of the candidate block can be included in the candidate group as one candidate.

[0365] Alternatively, when the motion model of the candidate block is a moving motion model, a combination of motion vectors of the candidate block set based on the position of the control point can be configured as one set, and included in the candidate group. As an example, when the coordinates of the upper left, upper right, and lower left are set as control points, the motion vector of the upper left control point can be predicted based on the motion vectors of the left, upper, and upper left blocks of the target block (for example, in the case of a moving motion model), the motion vector of the upper right control point can be predicted based on the motion vectors of the upper and upper right blocks of the target block (for example, in the case of a moving motion model), and the motion vector of the lower left control point can be predicted based on the motion vectors of the left and lower left blocks of the target block. In the above example, the case where the motion model of the candidate block set based on the position of the control point is a moving motion model has been described, but even if it is an affine model, it is also possible to obtain only the motion vector of the control point position by acquiring or guiding it. In other words, for the motion vector of the control point of the upper left, upper right, and lower left of the target block, a motion vector can be acquired or induced from the control points of the upper left, upper right, lower left, and lower right of the candidate block.

[0366] In summary, if the motion model of the candidate block is an affine model, the motion vector set of the block can be included in the candidate group (A), and if the motion model of the candidate block is a translation motion model, the motion vector set considered as a candidate for the motion vector of a specified control point of the target block and obtained according to the combination of each control point can be included in the candidate group (B).

[0367] In this case, the candidate group can be constructed using only one of the methods A and B, or the candidate group can be constructed using both the methods A and B. The method A can be constructed first and the method B can be constructed later, but is not limited thereto.

[0368] FIG. 11 shows an example diagram of a statistical candidate according to an embodiment of the present invention.

[0369] Referring to FIG. 11, blocks A to L in FIG. 11 refer to blocks that have been coded and are spaced apart from each other by a predetermined interval based on a target mode.

[0370] When candidate blocks for motion information of a target block are limited to blocks adjacent to the target block, it may not reflect various characteristics of the image. For this reason, blocks that have been previously coded in the current picture may also be considered as candidate blocks. Inter_blk_B has already been mentioned as a category related to this. This is called a statistical candidate.

[0371] Since the number of candidate blocks included in the motion information prediction candidate set is limited, an efficient statistical candidate management for this purpose may be necessary.

[0372] (1) A block at a previously set position having a predetermined distance from the target block can be considered as a candidate block.

[0373] As an example, based on a specific coordinate of the target block (e.g., the upper left coordinate), the x component can be limited and blocks with a certain interval can be considered as candidate blocks. Examples include (-4, 0), (-8, 0), (-16, 0), and (-32, 0).

[0374] As an example, based on a certain coordinate of the target block, blocks having a certain interval can be considered as candidate blocks by limiting the y component, and examples include (0, -4), (0, -8), (0, -16), and (0, -32).

[0375] For example, blocks having a certain interval other than 0 as x and y components based on a certain coordinate of a target block can be considered as candidate blocks, and examples include (-4, -4), (-4, -8), (-8, -4), (-16, 16), (16, -16), etc. In this example, the x component may have a positive sign depending on the y component.

[0376] Depending on the coding settings, the candidate blocks that are considered as statistical candidates may be restricted. For an example, see FIG.

[0377] A candidate block can be classified into a statistical candidate according to whether it belongs to a predetermined unit, where the predetermined unit can be determined from among a maximum coding block, a brick, a tile, a slice, a sub-picture, a picture, and the like.

[0378] For example, when selecting candidate blocks by limiting the blocks that belong to the same maximum coding block as the target block, A to C may be the candidates.

[0379] Alternatively, when candidate blocks are selected by limiting the selection to the largest coded block to which the target block belongs and one largest coded block to the left of the target block, A to C, H, and I may be the candidates.

[0380] Alternatively, when candidate blocks are selected by limiting the selection to the largest coded block to which the current block belongs and blocks that belong to the largest coded block above, A to C, E, and F may be the candidates.

[0381] Alternatively, when selecting candidate blocks by limiting the blocks to those belonging to the same slice as the target block, A to I may be the candidates.

[0382] A candidate block can be classified as a statistical candidate depending on whether it is located in a predetermined direction with respect to a target block, where the predetermined direction can be defined as the left, top, top-left, top-right, or the like.

[0383] For example, if candidate blocks are selected by limiting them to blocks located to the left and above the target block, B, C, F, I, and L may be candidates.

[0384] Alternatively, when the candidate blocks are selected by limiting the blocks located to the left, upper, upper left, and upper right directions based on the target block, A to L may be the candidates.

[0385] As in the above example, even if a candidate block considered as a statistical candidate is selected, a priority order for including the candidate block in the motion information prediction candidate group may exist. That is, as many candidates as are supported by the statistical candidate may be selected from the priority order.

[0386] The priority order may be determined by various coding elements, which may be defined based on the distance between the candidate block and the target block (e.g., whether the distance is short or not; the distance between the blocks can be confirmed based on the x and y components), the relative direction of the candidate block with respect to the target block (e.g., left, top, top left, top right directions; the order is left → top → top right → left, etc.), etc.

[0387] (2) A block that has been completely coded in accordance with a predetermined coding order based on a target block can be considered as a candidate block.

[0388] It should be understood that the examples described below assume a raster scan (e.g., largest coding block), Z scan (e.g., coding block, predicted block, etc.), or the contents described below can be modified depending on the scan order.

[0389] In the above-mentioned example (1), a priority order for constructing a group of motion information prediction candidates can be supported, and similarly, in the present embodiment, a predetermined priority order can be supported. The priority order can be determined according to various coding elements, but for convenience of explanation, it is assumed that the priority order is determined according to the coding order. In the following example, the candidate block and the priority order will be considered together.

[0390] Depending on the coding settings, the candidate blocks that are considered as statistical candidates can be restricted.

[0391] A candidate block can be classified as a statistical candidate depending on whether it belongs to a predetermined unit, where the predetermined unit can be determined from among a maximum coding block, a brick, a tile, a slice, a sub-picture, a picture, and the like.

[0392] For example, when a block belonging to the same picture as the current block is selected as a candidate block, the candidate blocks and priorities may be JDEFGKLHIABC.

[0393] Alternatively, when a block belonging to the same slice as the target block is selected as a candidate block, the candidate blocks and priorities such as DEF, GHI, ABC can be selected as the candidate blocks.

[0394] A candidate block can be classified as a statistical candidate depending on whether it is located in a predetermined direction with respect to a target block, where the predetermined direction can be defined as a left direction, an up direction, or the like.

[0395] For example, when candidate blocks are selected by limiting the selection to blocks located to the left of a target block, candidate blocks and priorities such as KLHIABC may be selected.

[0396] Alternatively, when candidate blocks are selected by limiting the blocks located above the target block, candidate blocks and priorities such as EF, A, B, C, and the like may be selected.

[0397] The above description is given on the assumption that the candidate blocks and the priority are combined, but the present invention is not limited to this, and various candidate blocks and priorities can be set.

[0398] It has been mentioned through the above description of (1) and (2) that the priority order included in the motion information prediction candidate group can be supported. The priority order can be supported more than one, and one priority order for all the candidate blocks can be supported, or the candidate blocks can be classified into two or more categories and individual priorities according to the categories can be supported. The former case can be an example of selecting k candidate blocks in one priority order, and the latter case can be an example of selecting p, q (p+q=k) candidate blocks in each priority order (e.g., two categories). In this case, the category can be classified based on a predetermined coding element, and the coding element can include whether the candidate block belongs to a predetermined unit, whether the candidate block is located in a predetermined direction based on the target block, etc.

[0399] An example of selecting a candidate block depending on whether it belongs to a predetermined unit (e.g., division unit) as described above through (1) and (2) has been described. Apart from the division unit, the candidate block may be selected by limiting it to blocks that belong to a predetermined range based on the target mode. For example, it may be confirmed by boundary points that define the range of the minimum and maximum values ​​of the x or y components, such as (x1, y1), (x2, y2), (x3, y3), etc. The values ​​of x1 to y3, etc. may be integers such as 0, 4, 8, etc. (based on absolute values).

[0400] A statistical candidate can be supported based on either one of the above settings (1) or (2), or a statistical candidate in a mixed form of (1) and (2) can be supported. From the above explanation, how to select a candidate block for a statistical candidate has been considered. The management and update of the statistical candidate and the case where the statistical candidate is included in a motion information prediction candidate group will be described later.

[0401] The motion information prediction candidate group may be generally configured on a block-by-block basis, because there is a high possibility that the motion information of adjacent blocks around the target block is the same or similar. In other words, in the case of spatial candidates and temporal candidates, the group may be configured based on the target mode.

[0402] Meanwhile, the statistical candidate can be configured based on a predetermined unit since the candidate block is not adjacent to the target block, and the predetermined unit can be a higher-order block including the target block.

[0403] For example, when the current block is a predicted block, statistical candidates can be formed in units of predicted blocks, or statistical candidates can be formed in units of coding blocks.

[0404] Alternatively, if the target block is a coding block, the statistical candidates may be configured in units of coding blocks, or in units of ancestor blocks (e.g., depth information differs from that of the target block by 1 or more). In this case, the ancestor block may include the maximum coding block, or may be a unit obtained based on the maximum coding block (e.g., an integer multiple of the maximum coding block).

[0405] As in the above example, statistical candidates can be constructed based on the current block or a higher unit, and in the example described below, it is assumed that the statistical candidates are constructed based on the current block.

[0406] Furthermore, memory initialization for statistical candidates may be performed based on a predetermined unit. The predetermined unit may be determined from among a picture, a subpicture, a slice, a tile, a brick, a block, etc., and in the case of a block, the predetermined unit may be set based on the largest coding block. For example, memory initialization for statistical candidates may be performed based on an integer number (1, 2, or an integer greater than 1) of the largest coding block, a row or column unit of the largest coding block, etc.

[0407] Next, statistical candidate management and update will be described. A memory for managing statistical candidates can be prepared and can store motion information of up to k blocks. Here, k can be an integer of 1 to 6 or more. The motion information stored in the memory can be determined from among a motion vector, a reference picture, a reference direction, etc. Here, the number of motion vectors (e.g., 1 to 3, etc.) can be determined based on motion model selection information. For convenience of explanation, the following description will focus on a case where a block that has been coded according to a predetermined coding order based on a target block is considered as a candidate block.

[0408] (Case 1) Motion information of a block that has been previously coded according to the coding order may be included as a candidate in the previous order. Also, when the maximum number of candidates is reached and an additional candidate is added, the candidate in the previous order may be removed and the order may be increased by 1. In the following example, x means that a blank has not yet been configured as a candidate.

[0409] (Example) 1order-[a, x, x, x, x, x, x, x, x, x] 2order-[a, b, x, x, x, x, x, x, x, x] ... 9order-[a, b, c, d, e, f, g, h, i, x] 10order-[a, b, c, d, e, f, g, h, i, j] → j added. Fully filled 11order-[b, c, d, e, f, g, h, i, j, k] → k added. Remove leading a, move order.

[0410] (Case 2) If there is overlapping motion information, the existing candidates are removed and the order of the existing candidates is adjusted by moving them up.

[0411] At this time, the redundancy means that the motion information is the same. This can be defined according to the motion information coding mode. For example, in the case of the merge mode, it is possible to determine whether or not there is a redundancy based on a motion vector, a reference picture, a reference direction, etc., and in the case of the competitive mode, it is possible to determine whether or not there is a redundancy based on a motion vector and a reference picture. At this time, in the case of the competitive mode, if the reference pictures of each candidate are the same, it is possible to determine whether or not there is a redundancy through a comparison of the motion vectors, and if the reference pictures of each candidate are different, it is possible to determine whether or not there is a redundancy through a comparison of the motion vectors scaled based on the reference pictures of each candidate, but is not limited thereto.

[0412] (Example) 1order-[a, b, c, d, e, f, g, x, x, x] 2order-[a, b, c, d, e, f, g, d, x, x] → d is included, but duplicated - [a, b, c, e, f, g, d, x, x, x] → Remove existing d, move order

[0413] (Case 3) If there is duplicated motion information, the candidate can be marked and managed as a candidate to be stored for a long time (long-term candidate) based on a predetermined condition.

[0414] A separate memory for the long-term candidates can be supported, and in addition to motion information, information such as occurrence frequency can be additionally stored and updated. In the following examples, the memory for long-term candidates can be represented by ().

[0415] (Example) 1order-[(), b, c, d, e, f, g, h, i, j, k]

[0416] 2order - [(), b, c, d, e, f, g, h, i, j, k] → e is the next order. Duplication -[(e), b, c, d, f, g, h, i, j, k] → Move to the beginning. Long-term marking

[0417] 3order-[(e), c, d, f, g, h, i, j, k, l] → add l and remove b. ... 10order-[(e), l, m, n, o, p, q, r, s, t] → l is the next order. - [(e,l),m,n,o,p,q,r,s,t] → long term marking 11order-[(e,l),m,n,o,p,q,r,s,t] →l is the next order. 3 duplicates - [(l,e),m,n,o,p,q,r,s,t] → Change the order of long-term candidates

[0418] The above example shows a case where long-term candidates are managed together with existing statistical candidates (short-term candidates), but they can be managed separately from short-term candidates, and the number of long-term candidates can be 0, 1, 2 or an integer greater than 1.

[0419] Short-term candidates may be managed in a first-in-first-out manner based on the coding order, while long-term candidates may be managed according to frequency based on candidates that have redundancy among short-term candidates. For long-term candidates, the candidate order may be changed according to frequency during memory update, but is not limited to this.

[0420] The above example may be a part of the case where the candidate blocks are classified into two or more categories and priority according to the category is supported. When the motion information prediction candidate group includes statistical candidates, a and b short-term candidates and long-term candidates may be configured according to the priority within each candidate. Here, a and b may be integers of 0, 1, 2 or more.

[0421] The above cases 1 to 3 are examples based on statistical candidates, and the configuration of (case 1+case 2) or (case 1+case 3) is possible, and the statistical candidates can be managed in various other modified and added forms.

[0422] FIG. 12 is a conceptual diagram of a statistical candidate based on a non-moving motion model according to an embodiment of the present invention.

[0423] The statistical candidate may be a candidate supported not only by the motion model but also by the non-motion model. In the case of the motion model, there are many blocks that can be referenced as spatial or temporal candidates because of high occurrence frequency, but in the case of the non-motion model, there are few occurrence frequency, so a statistical candidate other than the spatial or temporal candidate may be necessary.

[0424] Referring to FIG. 12, a block that belongs to the same space as the target block and is far away is<inter_blk_B> 13 shows a case where motion information {TV0, TV1} of a target block is predicted based on motion information {PV0, PV1} of the target block (hmvp).

[0425] Compared with the statistical candidates of the above-mentioned mobile motion model, the number of motion vectors in the motion information stored in the memory may be increased. Therefore, the basic concept of the statistical candidates may be the same as or similar to the mobile motion model, and a related explanation can be induced by a configuration in which the number of motion vectors is different. Next, the following will focus on the differences due to non-mobile motion models. In the example below, the case of an affine model with three control points (e.g., v0, v1, v2) is assumed.

[0426] In the case of the affine model, the motion information stored in the memory can be determined from among a reference picture, a reference direction, and motion vectors v0, v1, and v2. Here, the motion vectors can be stored in various forms. For example, the motion vectors can be stored as they are, or predetermined difference value information can be stored.

[0427] FIG. 13 is a diagram illustrating an example of the configuration of motion information of each control point position stored as a statistical candidate according to an embodiment of the present invention.

[0428] Referring to Figure 13, (a) shows a case where the motion vectors of candidate blocks v0, v1, and v2 are represented as v0, v1, and v2, and (b) shows a case where the motion vector of candidate block v0 is represented as v0, and the motion vectors of v1 and v2 are represented as difference values ​​v*1 and v*2 from the motion vector of v0.

[0429] In other words, the motion vector of each control point position can be stored as is, or the difference value with respect to the motion vector of another control point position can be stored. This is an example of a configuration that takes into consideration the memory management aspect, and various variations are possible.

[0430] Whether or not a statistical candidate is supported may be determined by explicitly generating related information for each sequence, picture, slice, tile, brick, etc., or may be determined implicitly according to an encoding setting. The encoding setting may be defined by the various encoding elements described above, and therefore a detailed description thereof will be omitted.

[0431] Here, the statistical candidates may be supported or not depending on the motion information coding mode, or may be supported or not depending on the motion model selection information. For example, the statistical candidates may be supported from among merge_inter, merge_ibc, merge_affine, comp_inter, comp_ibc, and comp_affine.

[0432] If statistical candidates are supported for moving and non-moving motion models, memory for multiple statistical candidates can be supported.

[0433] Next, a method for forming a group of motion information prediction candidates according to a motion information encoding mode will be described.

[0434] A group of motion information prediction candidates for a competitive mode (hereinafter, a competitive mode candidate group) may include k candidates, where k may be an integer of 2, 3, 4 or more. The competitive mode candidate group may include at least one of a spatial candidate or a temporal candidate.

[0435] The spatial candidate can be derived from at least one of the blocks adjacent to the reference block in the left, upper, upper left, upper right, lower left, etc. Alternatively, at least one candidate can be derived from the block adjacent to the left (left, lower left block) and the block adjacent to the upper side (upper left, upper, upper right block), which will be described later assuming this setting.

[0436] There may be more than one priority order for forming the candidate set: for adjacent regions to the left, a bottom-left-left priority order may be established, and for adjacent regions to the top, a top-right-top-top-top-left priority order may be established.

[0437] Although the above example may be a configuration in which spatial candidates are derived only from blocks with the same reference picture as the current block, spatial candidates may also be derived through a scaling process (hereinafter denoted as *) based on the reference picture of the current block. In this case, the order of priority can be set as left-bottom-left-left*-bottom-left* or left-bottom-left-bottom-left*-left* in the leftward adjacent region, and the order of priority can be set as top-right-top-top-top-top*-top*-top-left* or top-right-top-top-top-top-left-top*-top*-top-right* in the upward adjacent region.

[0438] The temporal candidates can be derived from at least one of blocks adjacent to the block corresponding to the reference block in the center, left, right, top, bottom, top left, top right, bottom left, bottom right, etc. There can be a priority order for configuring the candidate group, and the priority order can be set as center-bottom left-right-bottom, bottom left-center-top left, etc.

[0439] When the sum of the maximum allowable number of spatial candidates and the maximum allowable number of temporal candidates is set to be less than the number of competitive mode candidates, temporal candidates can be included in the candidate group regardless of the composition of the candidate group of spatial candidates.

[0440] Based on the priority and the availability of each candidate block, and the maximum allowed number of temporal candidates (q, an integer between 1 and the number of competitive mode candidate groups), all or some of the candidates may be included in the candidate group.

[0441] Here, if the maximum allowable number of spatial candidates is set to be equal to the number of merging mode candidates, the temporal candidates cannot be included in the candidate group, and if the maximum allowable number of spatial candidates is not met, the temporal candidates can be included in the candidate group. This example assumes the latter case.

[0442] Here, the motion vector of the temporal candidate can be obtained based on the motion vector of the candidate block and the distance between the current image and the reference image of the target block, and the reference image of the temporal candidate can be obtained based on the distance between the current image and the reference image of the target block, or based on the reference image of the temporal candidate, or based on an already defined reference image (e.g., reference picture index is 0).

[0443] If the competitive mode candidate group is not filled with spatial candidates, temporal candidates, etc., the candidate group can be completed through a default candidate including a zero vector.

[0444] The competitive mode is centered on the description of comp_inter, and in the case of comp_ibc or comp_affine, a candidate pool can be formed with similar or different candidates.

[0445] For example, in the case of comp_ibc, a candidate group can be configured based on predetermined candidates selected from spatial candidates, statistical candidates, combination candidates, default candidates, etc. In this case, the candidate group can be configured with spatial candidates as a priority, and then the candidate group can be configured in the order of statistical candidates, combination candidates, default candidates, etc., but is not limited to this and various orders are possible.

[0446] Alternatively, in the case of comp_affine, the candidate group may be configured based on a predetermined candidate selected from among spatial candidates, temporal candidates, statistical candidates, combination candidates, default candidates, etc. In particular, the candidate (e.g., one block) motion vector set candidates may be configured into a candidate group. <1> Alternatively, candidates based on the positions of the control points (e.g., two or more blocks) may be combined into a candidate group. <2> It is possible to <1> The candidate for is included in the candidate set first, and the next <2> The order in which candidates for are included is possible but not limited to this.

[0447] Detailed explanations regarding the configuration of the various competitive mode candidates mentioned above can be found in comp_inter, so detailed explanations will be omitted.

[0448] In the process of constructing a competitive mode candidate group, if there is overlapping motion information among the candidates included earlier, the candidate with the next highest priority can be included in the candidate group while maintaining the candidate included earlier.

[0449] Here, before constructing the candidate group, the motion vectors in the candidate group can be scaled based on the distance between the reference picture of the target block and the current picture. For example, if the distance between the reference picture of the target block and the current picture is the same as the distance between the reference picture of the candidate block and the picture to which the candidate block belongs, the motion vector can be included in the candidate group, and if they are not the same, the motion vector scaled according to the distance between the reference picture of the target block and the current picture can be included in the candidate group.

[0450] At this time, redundancy means that the motion information is the same, which can be defined by the motion information coding mode. In the competitive mode, it can be determined whether there is redundancy based on the motion vector, the reference picture, the reference direction, etc. For example, it can be determined that there is no redundancy if at least one component of the motion vector is different. The redundancy confirmation process may be generally performed when a new candidate is included in the candidate group, but it can also be modified to be omitted.

[0451] A set of motion information prediction candidates for a merge mode (hereinafter, a merge mode candidate set) may include k candidates, where k may be an integer of 2, 3, 4, 5, 6 or more. The merge mode candidate set may include at least one of a spatial candidate or a temporal candidate.

[0452] The spatial candidates can be derived from at least one of blocks adjacent to the reference block in the left, top, top left, top right, bottom left direction, etc. Priorities for configuring the candidate group can be set, such as left-top-bottom left-top right-top left, left-top-top right-bottom left-top left, top-left-bottom left-top left-top right, etc.

[0453] Based on the priority, the availability of each candidate block (e.g., determined based on the coding mode, block position, etc.), and the maximum allowable number of spatial candidates (p, an integer between 1 and the number of merging mode candidate groups), all or some of the candidates may be included in the candidate group. Based on the maximum allowable number and availability, the candidates may not be included in the candidate group in the order of tl-tr-bl-t3-l3; if the maximum allowable number is 4 and the availability of all candidate blocks is true, the motion information of tl is not included in the candidate group, and if the availability of some candidate blocks is false, the motion information of tl may be included in the candidate group.

[0454] The temporal candidates can be derived from at least one of blocks adjacent to the block corresponding to the reference block in the center, left, right, top, bottom, top left, top right, bottom left, bottom right, etc. There can be a priority for the configuration of the candidate group, and the priority can be set as center-bottom left-right-bottom, bottom left-center-top left, etc.

[0455] Based on the priority and the availability of each candidate block, and the maximum allowed number of temporal candidates (q, an integer between 1 and the number of merging mode candidate groups), all or some of the candidates can be included in the candidate group.

[0456] Here, the motion vector of the temporal candidate can be obtained based on the motion vector of the candidate block, and the reference image of the temporal candidate can be obtained based on the reference image of the candidate block or based on an already defined reference image (e.g., the reference picture index is 0).

[0457] The priority order included in the merge mode candidate group can be set to spatial candidate-temporal candidate or vice versa, and a priority order having a mixed configuration of spatial candidate and temporal candidate can be supported. In this example, the case of spatial candidate-temporal candidate is assumed.

[0458] In addition, the merging mode candidate group may further include a statistical candidate or a combination candidate. The statistical candidate and the combination candidate may be configured after the spatial candidate and the temporal candidate, but are not limited thereto, and various priorities may be set.

[0459] The statistical candidates can manage up to n pieces of motion information, and z pieces of motion information among them can be included in the merging mode candidate group as statistical candidates. z can be variable depending on the candidate configurations already included in the merging mode candidate group, and can be an integer of 0, 1, 2 or more, and can be less than or equal to n.

[0460] The combination candidates may be derived by combining n candidates already included in the merging mode candidate group, where n may be an integer of 2, 3, 4 or more. The number (n) of combination candidates may be information that is explicitly generated in units of a sequence, a picture, a sub-picture, a slice, a tile, a brick, a block, etc., or may be implicitly determined according to the encoding setting. In this case, the encoding setting may be defined based on one or more factors such as the size, shape, position, image type, color components, etc. of the reference block.

[0461] Also, the number of combination candidates may be determined based on the number of candidates that are not filled in the merging mode candidate group. In this case, the number of candidates that are not filled in the merging mode candidate group may be a difference value between the number of merging mode candidate group and the number of candidates that are already filled. In other words, if the configuration of the merging mode candidate group is already completed, a combination candidate may not be added. If the configuration of the merging mode candidate group is not completed, a combination candidate may be added, but if the number of candidates that are filled in the merging mode candidate group is one or less, a combination candidate is not added.

[0462] If the merging mode candidate group is not filled with spatial candidates, temporal candidates, statistical candidates, combination candidates, etc., the construction of the candidate group can be completed via default candidates including a zero vector.

[0463] The above merge mode is mainly explained with respect to merge_inter, and in the case of merge_ibc or merge_affine, a candidate group can be formed from different or similar candidates.

[0464] For example, in the case of merge_ibc, a candidate group can be configured based on predetermined candidates selected from spatial candidates, statistical candidates, combination candidates, default candidates, etc. In this case, the candidate group can be configured with priority given to spatial candidates, and then the candidate group can be configured in the order of statistical candidates, combination candidates, default candidates, etc., but is not limited to this and various orders are possible.

[0465] Alternatively, in the case of merge_affine, the candidate group may be configured based on a predetermined candidate selected from spatial candidates, temporal candidates, statistical candidates, combination candidates, default candidates, etc. In particular, the candidate (e.g., one block) motion vector set candidates may be configured into a candidate group. <1> Alternatively, candidates based on the positions of the control points (e.g., two or more blocks) may be combined to form a candidate group. <2> It is possible to <1> The candidates for are included in the candidate set first, then <2> The order in which candidates for are included is possible but not limited to this.

[0466] The details regarding the configuration of the various merge mode candidates mentioned above can be derived from merge_inter, so a detailed description will be omitted.

[0467] In the process of constructing a merging mode candidate group, if there is overlapping motion information among the previously included candidates, the previously included candidate can be maintained and the next highest priority candidate can be included in the candidate group.

[0468] At this time, redundancy means that the motion information is the same, which can be defined by the motion information coding mode. In the case of the merging mode, it can be determined whether or not there is redundancy based on the motion vector, the reference picture, the reference direction, etc. For example, it can be determined that there is no redundancy if at least one component of the motion vector is different. The redundancy confirmation process may be generally performed when a new candidate is included in the candidate group, but it can also be modified to be omitted.

[0469] 14 is a flowchart of a motion information encoding process according to an embodiment of the present invention. In particular, the process may be encoding of motion information of a current block in a competitive mode.

[0470] A motion vector prediction candidate list for the current block can be generated (S1400). The competitive mode candidates mentioned above can refer to the motion vector candidate list, and a detailed description thereof will be omitted.

[0471] The motion vector difference value of the target block can be restored (S1410). The difference values ​​for the x and y components of the motion vector can be restored separately, and the difference value of each component can have a value of 0 or more.

[0472] A prediction candidate index of a target mode can be selected from a motion vector prediction candidate list (S1420). A motion vector prediction value of a target block can be derived based on a motion vector obtained from the candidate list according to the candidate index. If one pre-set motion vector prediction value can be derived, the process of selecting a prediction candidate index and index information can be omitted.

[0473] A motion vector difference precision information can be derived (S1430). Precision information commonly applied to the x and y components of the motion vector can be derived, or precision information applied to each component can be derived. If the motion vector difference value is 0, the precision information can be omitted, and this process can also be omitted.

[0474] An adjustment offset for the motion vector predictor may be derived (S1440). The offset may be a value that is added to or subtracted from the x or y component of the motion vector predictor. The offset may be supported for only the x or y component, or may be supported for both the x and y components.

[0475] Assuming the motion vector predictor is (pmv_x, pmv_y) and the adjusted offset is offset_x, offset_y, the motion vector predictor can be adjusted (or obtained) to (pmv_x+offset_x, pmv_y+offset_y).

[0476] Here, the absolute values ​​of offset_x and offset_y are each an integer of 0, 1, 2 or more, and may have values ​​(1, -1, +2, -2, etc.) in which sign information is taken into consideration. Also, offset_x and offset_y may be determined based on a predetermined accuracy. The predetermined accuracy may be determined from among 1 / 16, 1 / 8, 1 / 4, 1 / 2, 1 pixel units, etc., and may also be determined based on interpolation accuracy, motion vector accuracy, etc.

[0477] For example, if the interpolation accuracy is 1 / 4 pixel units, the absolute value and sign information can be combined to 0, 1 / 4, -1 / 4, 2 / 4, -2 / 4, 3 / 4, -3 / 4, etc.

[0478] Here, offset_x and offset_y can support a and b, respectively, and a and b can be integers of 0, 1, 2 or more. a and b can have fixed values ​​or can have variable values. Also, a and b can have the same or different values.

[0479] Whether or not an adjustment offset for a motion vector predictor is supported may be explicitly supported in units of a sequence, a picture, a subpicture, a slice, a tile, a brick, etc., or may be implicitly determined according to an encoding setting. In addition, the setting of the adjustment offset (e.g., a value range, the number, etc.) may be determined according to an encoding setting.

[0480] The encoding settings can be determined taking into consideration at least one of the encoding elements such as image type, color components, status information of the target block, motion model selection information (e.g., whether it is a motion model or not), reference picture (e.g., whether it is a current picture or not), and differential motion vector precision selection information (e.g., whether it is a specified unit among 1 / 4, 1 / 2, 1, 2, or 4 units).

[0481] For example, whether or not to support the adjustment offset and the offset setting may be determined according to the size of the block. In this case, the support and setting range of the block size may be determined according to a first threshold size (minimum value) or a second threshold size (maximum value), and each threshold size may be expressed as W, H, W×H, or W*H, where W is the width (W) and height (H) of the block. In the case of the first threshold size, W and H may be integers of 4, 8, 16 or more, and W*H may be an integer of 16, 32, 64 or more. In the case of the second threshold size, W and H may be integers of 16, 32, 64 or more, and W*H may be an integer of 64, 128, 256 or more. The above range may be determined by either the first threshold size or the second threshold size, or by both.

[0482] In this case, the threshold value may be fixed or adaptive based on the image (e.g., image type, etc.) In this case, the first threshold value may be set based on the size of the smallest coding block, the smallest prediction block, the smallest transform block, etc., and the second threshold value may be set based on the size of the largest coding block, the largest prediction block, the largest transform block, etc.

[0483] In addition, the adjustment offset can be applied to all candidates included in the motion information prediction candidate group, or only to some candidates. In the example described below, it is assumed that the adjustment offset is applied to all candidates included in the candidate group, but the candidate to which the adjustment offset is applied can be selected from 0, 1, 2, or the maximum number of the candidate group.

[0484] If adjustment offset is not supported, this process and the adjustment offset information can be omitted.

[0485] The motion vector prediction value and the motion vector differential value are added to restore the motion vector of the target block (S1450). At this time, a process of unifying the motion vector prediction value or the motion vector differential value to the motion vector precision may precede the above-mentioned motion vector scaling process or may be performed as this process.

[0486] The above configurations and sequences are merely some examples, and are not limited to these, and various modifications are possible.

[0487] FIG. 15 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention.

[0488] For ease of explanation, it is assumed that two motion vector prediction candidates are supported and one component (either x or y component) is compared. It is also assumed that the interpolation accuracy or motion vector accuracy is in 1 / 4 pixel units. In addition, it is assumed that the differential motion vector accuracy is supported in 1 / 4, 1, and 4 pixel units (for example, when it is 1 / 4, <0> , when it is 1 <10> , when it is 4 <11> In addition, the motion vector difference value is assumed to be processed by unary binarization (for example, 0: <0> , 1: <10> , 2: <110> etc.)

[0489] Referring to FIG. 15, the actual motion vector (X) has a value of 2, candidate 1 (A) has a value of 0, and candidate 2 (B) has a value of 1 / 4.

[0490] (When differential motion vector accuracy is not supported) Since the distance (da) between A and X is 8 / 4 (9 bits), and the distance (db) between B and X is 7 / 4 (8 bits), from the viewpoint of bit generation, B can be selected as the prediction candidate.

[0491] (When differential motion vector accuracy is supported) Since da is 8 / 4, 1 pixel unit accuracy (2 bits) and 2 / 1 difference value information (3 bits) are generated, so a total of 5 bits can be generated. Meanwhile, since db is 7 / 4, 1 / 4 pixel unit accuracy (1 bit) and 7 / 4 difference value information (8 bits) are generated, so a total of 9 bits can be generated. From the perspective of bit amount generation, A can be selected as the prediction candidate.

[0492] When differential motion vector accuracy is not supported as in the above example, it is advantageous to select a candidate with a short distance from the motion vector of the target block as the prediction candidate, and when differential motion vector accuracy is supported, it may be important to select a prediction candidate based not only on the distance from the motion vector of the target block but also on the amount of information generated based on accuracy information.

[0493] 16 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention, assuming that differential motion vector accuracy is supported.

[0494] Since da is 33 / 4, a total of 35 bits can be generated, including 1 / 4 pixel unit accuracy (1 bit) and difference value information (34 bits) at 33 / 4. Meanwhile, since db is 21 / 4, a total of 23 bits can be generated, including 1 / 4 pixel unit accuracy (1 bit) and difference value information (22 bits) at 21 / 4. From the perspective of bit generation, the prediction candidate can be selected as B.

[0495] 17 is a diagram illustrating an example of motion vector predictor candidates and a motion vector of a current block according to an embodiment of the present invention, assuming that differential motion vector accuracy and adjusted offset information are supported.

[0496] In this example, it is assumed that the adjustment offset has candidates of 0 and +1, and a flag (1 bit) indicating whether the adjustment offset has been applied and offset selection information (1 bit) are generated.

[0497] 17, A1 and B1 may be motion vector predictors obtained based on prediction candidate indexes, and A2 and B2 may be new motion vector predictors obtained by correcting the adjustment offset to A1 and B1. Assume that the distances between A1, A2, B1, and B2 and X are da1, da2, db1, and db2, respectively.

[0498] (1) Since da1 is 33 / 4, a total of 37 bits can be generated, including 1 / 4 pixel unit accuracy (1 bit), offset application flag (1 bit), offset selection information (1 bit), and 33 / 4 difference value information (34 bits).

[0499] (2) Since da2 is 32 / 4, a total of 7 bits can be generated: 4 pixel unit accuracy (2 bits), offset application flag (1 bit), offset selection information (1 bit), and 2 / 1 difference value information (3 bits).

[0500] (3) Since db1 is 21 / 4, a total of 25 bits can be generated, including 1 / 4 pixel unit accuracy (1 bit), offset application flag (1 bit), offset selection information (1 bit), and 21 / 4 difference value information (22 bits).

[0501] (4) Since db2 is 20 / 4, a total of 10 bits can be generated, including 1 pixel unit accuracy (2 bits), offset application flag (1 bit), offset selection information (1 bit), and 5 / 1 difference value information (6 bits).

[0502] From the viewpoint of bit generation, the prediction candidate can be selected as A, and the offset selection information can be selected as the first index (+1 in this example).

[0503] Considering the above example, when deriving a motion vector difference value based on a conventional motion vector predictor, a small vector value difference may generate many bits. This problem can be solved by adjusting the motion vector predictor.

[0504] In order to apply an adjustment offset to a motion vector predictor, a predetermined flag may be supported, which may include an offset application flag, offset selection information, and the like.

[0505] (If only offset application flag is supported)

[0506] When the offset application flag is 0, no offset is applied to the motion vector predicted value, and when the offset application flag is 1, a preset offset can be added to or subtracted from the predicted motion vector value.

[0507] (If only offset selection information is supported)

[0508] The offset set based on the offset selection information can be added to or subtracted from the predicted motion vector value.

[0509] (If offset application flag and offset selection information are supported)

[0510] When the offset application flag is 0, no offset is applied to the motion vector predicted value, and when the offset application flag is 1, an offset set based on the offset selection information can be added to or subtracted from the predicted motion vector value. This setting is assumed in the examples described later.

[0511] Meanwhile, the offset application flag and offset selection information can be used as information that is generated unnecessarily depending on the situation. That is, even if an offset is not applied, if the offset-related information already has a zero value or the amount of information is reduced based on the maximum precision information, the offset-related information may be rather inefficient. Therefore, it is necessary to support a setting where the offset-related information is generated implicitly according to a certain condition, rather than always being generated explicitly.

[0512] Next, it is assumed that the motion-related coding order proceeds as follows: (restoration of motion vector differential value → acquisition of differential motion vector precision). In this example, it is assumed that differential motion vector precision is supported when any one of the x and y components of the motion vector differential value is not 0.

[0513] When the motion vector differential value is not 0, the differential motion vector precision can be selected from among {1 / 4, 1 / 2, 1, 4} pixel units.

[0514] If the selected information belongs to a predetermined category, information regarding the adjustment offset can be implicitly omitted, and if the selected information does not belong to a predetermined category, information regarding the adjustment offset can be explicitly generated.

[0515] Here, the category may include one of the differential motion vector accuracy candidates, and may be configured as various categories such as {1 / 4}, {1 / 4, 1 / 2}, {1 / 4, 1 / 2, 1}, etc. Here, the minimum accuracy may be included in the category.

[0516] For example, if the differential motion vector precision is in quarter pixel units (e.g., minimum precision), the offset application flag is implicitly set to 0 (i.e., not applied), if the differential motion vector precision is not in quarter pixel units, the offset application flag is explicitly generated, and if the offset application flag is 1 (i.e., offset applied), offset selection information can be generated.

[0517] Alternatively, if the differential motion vector precision is 4-pixel units (e.g., maximum precision), the offset application flag may be explicitly generated, and the offset selection information may be generated if the offset application flag is 1. Alternatively, if the differential motion vector precision is not 4-pixel units, the offset information may be implicitly set to 0.

[0518] The above example assumes, but is not limited to, a case in which offset-related information is implicitly omitted when the differential motion vector points to the minimum precision, and offset-related information is explicitly generated when the differential motion vector points to the maximum precision.

[0519] FIG. 18 is an exemplary diagram of an arrangement of a plurality of motion vector predictors according to an embodiment of the present invention.

[0520] In the above example, the overlap check has been described when forming a motion information prediction candidate group. Here, overlap means that the motion information is the same, and it has been described that it is possible to determine that there is no overlap when at least one component of the motion vector is different.

[0521] A plurality of candidates for a motion vector predictor may not overlap with each other through a process of checking for overlap. However, if the components of the plurality of candidates are very similar (i.e., the x or y components of each candidate are within a predetermined range, and the width or height of the predetermined range is an integer of 1, 2 or more. Alternatively, the range may be set based on offset information), the motion vector predictor and the offset-compensated motion vector predictor may overlap. Various settings for this purpose are possible.

[0522] As an example (C1), in the motion information prediction candidate group construction step, a new candidate can be included in the candidate group if it does not overlap with the candidates already included. In other words, if it is the same as the above-mentioned existing description and does not overlap with the candidate itself, it can be configured to be included in the candidate group.

[0523] As an example (C2), if a new candidate and a candidate (group_A) obtained by adding an offset thereto and a candidate (group_B) obtained by adding an offset thereto do not overlap by a predetermined number of times, they can be included in the candidate group. The predetermined number can be an integer of 0, 1 or more. If the predetermined number is 0, the new candidate may not be included in the candidate group if there is even one overlap. Alternatively, as (C3), a predetermined offset (which is a different concept from the adjustment offset) can be added to the new candidate to include it in the candidate group, and the offset can be a value that allows group_A to be configured so as not to overlap with group_B.

[0524] Referring to FIG. 18, multiple motion vectors belonging to categories A, B, C, and D (in this example, AX, BX, CX, DX, and X are 1 and 2. For example, A1 is a candidate included in the candidate group before A2) may satisfy the non-overlapping condition (candidate motion vectors differ in even one component).

[0525] In this example, it is assumed that the offset is −1, 0, and 1 for the x and y components, respectively, and the offset-corrected motion vector predicted value can be represented by * in the drawing. Also, the dashed lines (rectangles) indicate ranges (e.g., group_A, group_B, etc.) that can be obtained by adding an offset to a certain motion vector predicted value.

[0526] In the case of the A category, it may be the case that A1 and A2 do not overlap, and group_A1 and group_A2 do not overlap.

[0527] In the case of the B, C, and D categories, this may apply when B1 / C1 / D1 and B2 / C2 / D2 do not overlap, and group_B1 / C1 / D1 and group_B2 / C2 / D2 partially overlap.

[0528] Here, the case of category B could be an example of forming a candidate group without taking any special measures (C1).

[0529] Here, in the case of the C category, C2 is determined to have duplication in the duplication check step, and C3, which has the next highest priority, may be included in the candidate group (C2).

[0530] Here, in the case of the D category, D2 is determined to have redundancy in the redundancy check step, and D2 is corrected so that group_D2 does not overlap with group_D1 (i.e., D3, where D3 is not a motion vector present in the candidate group configuration priority) (C3).

[0531] The prediction mode candidates may be configured from the various categories, or various methods other than those mentioned above may be applied.

[0532] 19 is a flowchart of a motion information encoding process in a merging mode according to an embodiment of the present invention. In particular, the process may be related to motion information of a current block in a merging mode.

[0533] A motion information prediction candidate list for the target block can be generated (S1900). The above-mentioned merging mode candidate group can refer to the motion information prediction candidate list, and a detailed description thereof will be omitted.

[0534] A prediction candidate index of a target mode can be selected from a motion information prediction candidate list (S1910). A motion vector prediction value of a target block can be derived based on motion information acquired from the candidate list according to the candidate index. If one pre-set motion information prediction value can be derived, the prediction candidate index selection process and index information can be omitted.

[0535] An adjustment offset for the motion vector predictor may be derived (S1920). The offset may be a value that is added to or subtracted from the x or y component of the motion vector predictor. The offset may be supported for only one of the x and y components, or for both the x and y components.

[0536] The adjustment offset in this process may be the same or similar to the adjustment offset described above, so a detailed description will be omitted and differences will be described later.

[0537] Assuming that the motion vector predictor is (pmv_x, pmv_y) and the adjustment offset is offset_x, offset_y, the motion vector predictor can be adjusted (or obtained) to (pmv_x+offset_x, pmv_y+offset_y).

[0538] Here, the absolute values ​​of offset_x and offset_y may be integers such as 0, 1, 2, 4, 8, 16, 32, 64, 128, etc., and may have values ​​that are taken into consideration together with sign information. Also, offset_x and offset_y may be determined based on a predetermined precision. The predetermined precision may be determined from among 1 / 16, 1 / 8, 1 / 4, 1 / 2, 1 pixel units, etc.

[0539] For example, if the motion vector precision is in 1 / 4 pixel units, the absolute value and sign information can be combined to 0, 1 / 4, -1 / 4, 1 / 2, -1 / 2, 1, -1, 2, -2, etc.

[0540] Here, offset_x and offset_y can support a and b, respectively, where a and b can be integers such as 0, 1, 2, 4, 8, 16, and 32. a and b can have fixed values ​​or can have variable values. Also, a and b can have the same or different values.

[0541] Whether or not an adjustment offset for a motion vector predictor is supported may be explicitly supported in units of a sequence, a picture, a subpicture, a slice, a tile, a brick, etc., or may be implicitly determined according to an encoding setting. In addition, the setting of the adjustment offset (e.g., a value range, the number, etc.) may be determined according to an encoding setting.

[0542] The encoding settings can be determined taking into account at least one of the encoding elements such as image type, color components, state information of the target block, motion model selection information (e.g., whether or not a motion model), and reference picture (e.g., whether or not a current picture).

[0543] For example, whether or not to support the adjustment offset and the offset setting may be determined according to the size of the block. In this case, the support and setting range of the block size may be determined according to a first threshold size (minimum value) or a second threshold size (maximum value), and each threshold size may be expressed as W, H, W×H, or W*H, where W is the width (W) and height (H) of the block. In the case of the first threshold size, W and H may be integers of 4, 8, 16 or more, and W*H may be an integer of 16, 32, 64 or more. In the case of the second threshold size, W and H may be integers of 16, 32, 64 or more, and W*H may be an integer of 64, 128, 256 or more. The range may be determined according to either the first threshold size or the second threshold size, or both.

[0544] At this time, the magnitude of the threshold may be fixed or may be adaptive depending on the image (e.g., image type, etc.) At this time, the magnitude of the first threshold may be set based on the size of the smallest coding block, the smallest prediction block, the smallest transformation block, etc., and the magnitude of the second threshold may be set based on the size of the largest coding block, the largest prediction block, the largest transformation block, etc.

[0545] In addition, the adjustment offset can be applied to all candidates included in the motion information prediction candidate group, or only to some candidates. In the example described below, it is assumed that the adjustment offset is applied to all candidates included in the candidate group, but the candidate to which the adjustment offset is applied can be selected from 0, 1, 2, or the maximum number of the candidate group.

[0546] A predetermined flag for applying an adjustment offset to a motion vector predictor can be supported, and the predetermined flag can be configured as an offset application flag, offset absolute value information, offset sign information, or the like.

[0547] If the adjustment offset is not supported, this process and the adjustment offset information can be omitted.

[0548] A motion vector of a target mode can be restored through the motion vector predictor (S1930). Motion information other than the motion vector (eg, reference picture, reference direction, etc.) can be obtained based on the prediction candidate index.

[0549] The above configurations and sequences are merely examples, and are not limited thereto, and various modifications are possible. The background of supporting the adjustment offset in the merging mode has been described above through various examples of the competition mode, so a detailed description will be omitted.

[0550] The methods of the present invention may be embodied in the form of program instructions that can be executed by various computing means and stored on computer readable media. The computer readable media may include, alone or in combination with program instructions, data files, data structures, and the like. The program instructions stored on the computer readable media may be those specially designed and constructed for the purposes of the present invention, or they may be well known and available to those skilled in the computer software arts.

[0551] Examples of computer readable media may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter, etc. The above-mentioned hardware devices may be configured to operate as at least one software module to perform the operations of the present invention, and vice versa.

[0552] Furthermore, the above-described methods and devices may be realized in such a manner that all or part of their configurations or functions are combined or separated.

[0553] While the present invention has been described above with reference to preferred embodiments, it will be understood by those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit and scope of the present invention as set forth in the following claims. [Industrial Applicability]

[0554] The present invention can be used for encoding / decoding images.< / qt>

Claims

1. constructing a predictive motion candidate list for a current block; deriving a predicted motion vector from the motion candidate list based on a prediction candidate index; recovering predicted motion vector adjustment offset information; restoring a motion vector of a current block based on the predicted motion vector and the predicted motion vector adjustment offset information; The inter prediction method, wherein the motion candidate list includes at least one of spatial candidates, temporal candidates, statistical candidates, or combination candidates.

2. The inter prediction method according to claim 1 , wherein the predicted motion vector adjustment offset is determined based on at least one of an offset application flag or offset selection information.

3. The method of claim 1 , wherein the information on whether the predicted motion vector adjustment offset information is supported is included in at least one of a sequence, a picture, a sub-picture, a slice, a tile, or a brick.

4. If the current block is coded in a merge mode, the motion vector of the current block is restored using a zero vector; The method of claim 1 , wherein if the current block is coded in a competitive mode, the motion vector of the current block is reconstructed using a motion vector differential value.

Images