Method for encoding / decoding an image and method for storing a bit stream

Abstract

The present invention relates to a method for encoding / decoding an image and a method for storing a bitstream. The method for encoding an image comprises: generating geometric modification information of a current block; determining a first geometric modification flag of a current parameter set; determining a second geometric modification flag when the first geometric modification flag indicates that geometric modification is enabled for a block using the current parameter set; determining geometric modification type information of the current block; determining a reference geometric modification index indicating reference geometric modification information; determining predicted geometric modification information of the current block based on the reference geometric modification information of the current block; determining residual geometric modification information; encoding a bitstream including the first geometric modification flag, the second geometric modification flag, the reference geometric modification index, the residual geometric modification information, and the geometric modification type information, wherein the geometric modification information is used to perform affine modification on a reference area referenced by the current block.

Description

[0001] This application is a divisional application of a patent application for an invention titled "Method and Apparatus for Encoding / Decoding Images Using Geometrically Modified Pictures" with an application date of November 18, 2016, an application number of 201680066455.0. Technical Field

[0002] The present invention generally relates to a method and apparatus for encoding / decoding images by using geometrically modified pictures, wherein a geometrically modified picture is generated by geometrically modifying a reference picture. Background Art

[0003] As high-definition (HD) broadcasting has spread nationwide and worldwide, many users have become accustomed to images with high resolution and high picture quality. Therefore, many institutions are driving the development of the next generation of image devices. In addition, with the growing interest in ultra-high definition (UHD) with a resolution several times higher than that of HDTV, there is a need for technologies capable of compressing and processing images with higher resolution and higher image quality.

[0004] As image compression technologies, there are various techniques: such as inter-frame prediction techniques, in which pixel values included in a current picture are predicted based on pictures before or after the current picture; intra-frame prediction techniques, in which pixel values included in a current picture are predicted using pixel information in the current image; transform and quantization techniques for compressing the energy of a residual signal; and entropy coding techniques, in which short codes are assigned to values with high occurrence frequencies and long codes are assigned to values with low occurrence frequencies. By using these image compression techniques, image data can be transmitted and stored in a state where the image data is effectively compressed.

[0005] When global motion is included in a reference picture referred to during inter-frame prediction, the similarity between the reference picture and the current picture decreases. The reduced similarity between the reference picture and the current picture may lead to a decrease in prediction efficiency. Also, when intra-frame prediction is performed, since the prediction direction of intra-frame prediction of a current block is restricted, the efficiency of intra-frame prediction may deteriorate. Therefore, improvements are needed to solve the above problems. Summary of the Invention

[0006] Technical Problem

[0007] An object of the present invention is to propose a method and apparatus for effectively encoding / decoding images.

[0008] In addition, the present invention provides a method and apparatus for effectively performing intra-frame prediction and / or inter-frame prediction.

[0009] Furthermore, the present invention provides a method and apparatus for generating a geometrically modified picture by geometrically modifying a reference picture.

[0010] In addition, the present invention provides a method and an apparatus for effectively signaling information related to geometrically modified pictures.

[0011] In addition, the present invention provides a method and an apparatus for performing intra prediction and / or inter prediction by referring to geometrically modified pictures.

[0012] Technical solution

[0013] According to one aspect of the present invention, there is provided a method for encoding an image. The method may include: generating a geometrically modified picture by geometrically modifying a reference picture; and generating a first prediction block of a current block by performing inter prediction with reference to the geometrically modified picture.

[0014] According to the encoding method of the present invention, the method may further include: generating a second prediction block of the current block by performing inter prediction with reference to the reference picture; and selecting a final prediction block for motion compensation of the current block from the first prediction block and the second prediction block.

[0015] According to the encoding method of the present invention, the final prediction block may be selected by selecting one having less error between the current block among the first prediction block and the second prediction block.

[0016] According to the encoding method of the present invention, the method may further include generating geometric modification information for generating the geometrically modified picture. And the generation of the geometrically modified picture may be performed based on the geometric modification information and the reference picture.

[0017] According to the encoding method of the present invention, the method may further include encoding the geometric modification information.

[0018] According to the encoding method of the present invention, encoding the geometric modification information may include reconfiguring the geometric modification information, and the reconfiguring the geometric modification information may include simplifying or predicting the geometric modification information.

[0019] According to the encoding method of the present invention, the geometric modification information may be generated based on a change in pixel information between a current picture including the current block and the reference picture.

[0020] According to the encoding method of the present invention, the geometric modification information may be generated based on matching information between feature points extracted from each of the current picture and the reference picture.

[0021] According to the encoding method of the present invention, the generating of the geometrically modified picture may include: identifying a point in a reference picture that corresponds to a point in the geometrically modified picture; and setting the pixel information of the corresponding point in the reference picture as the pixel information of the point in the geometrically modified picture.

[0022] According to the encoding method of the present invention, when the point corresponding to the point in the geometrically modified picture does not exist in the reference picture, the point in the reference picture that is closest to the corresponding point may be identified as the corresponding point.

[0023] According to another aspect of the present invention, a method for decoding an image is provided. The method may include generating a geometrically modified picture by geometrically modifying a reference picture; and generating a predicted block of a current block by performing inter-frame prediction on the reference geometrically modified picture.

[0024] According to the decoding method of the present invention, the method may further include: receiving motion compensation related information; and determining whether the geometrically modified picture is used for motion compensation of the current block based on the motion compensation related information, and when determining, when it is determined that the reference geometrically modified picture is used for motion compensation of the current block, the generating of the geometrically modified picture and the generating of the predicted block may be performed.

[0025] According to the decoding method of the present invention, the determining may be performed based on whether the motion compensation related information includes geometric modification information or based on information about whether a reference geometrically modified picture is referenced, and the information is included in the motion compensation related information.

[0026] According to the decoding method of the present invention, when determining, when it is determined that a reference geometrically modified picture is referenced in the motion compensation of the current block, the method may further include reconstructing geometric modification information based on information about the geometric modification information included in the motion compensation related information, and may perform the generating of the geometrically modified picture based on the reference picture and the reconstructed geometric modification information.

[0027] According to the decoding method of the present invention, the information about the geometric modification information may include residual geometric modification information or a scaling factor, and the reconstructing of the geometric modification information may be performed based on at least one of the residual geometric modification information, the scaling factor, and previously stored geometric modification information.

[0028] According to the decoding method of the present invention, the motion compensation related information may include information about previously stored geometric modification information, and the information about previously stored geometric modification information may be information for identifying one geometric modification information among one or more previously stored geometric modification information.

[0029] According to the decoding method of the present invention, generating the geometrically modified picture may include: identifying a point in a reference picture that corresponds to a point in the geometrically modified picture; and setting the pixel information of the corresponding point in the reference picture as the pixel information of the point in the geometrically modified picture.

[0030] According to the decoding method of the present invention, when the point corresponding to the point in the geometrically modified picture does not exist in the reference picture, the point in the reference picture that is closest to the corresponding point may be identified as the corresponding point.

[0031] According to the decoding method of the present invention, when the corresponding point in the reference picture has real-valued coordinates, one or more points in the reference picture that have integer coordinates and are adjacent to the corresponding point may be identified, and the pixel information of the corresponding point in the reference picture may be derived based on the pixel information of the one or more points that have integer coordinates.

[0032] According to the decoding method of the present invention, generating the geometrically modified picture is performed in at least one of a picture, a slice, a tile, a coding unit, and a prediction unit.

[0033] Advantageous Effects

[0034] According to the present invention, an image can be efficiently encoded / decoded.

[0035] In addition, according to the present invention, inter prediction and / or intra prediction can be efficiently performed.

[0036] Furthermore, according to the present invention, a geometrically modified picture can be generated by geometrically modifying a reference picture.

[0037] In addition, according to the present invention, information related to the geometrically modified picture can be efficiently signaled.

[0038] Moreover, according to the present invention, intra prediction and / or inter prediction can be performed by referring to the geometrically modified picture. Description of the Drawings

[0039] Figure 1 is a block diagram showing the configuration of an image encoding apparatus to which an embodiment of the present invention is applied.

[0040] Figure 2 is a block diagram showing the configuration of an image decoding apparatus to which an embodiment of the present invention is applied.

[0041] Figure 3 is a diagram schematically showing a partition structure of an image when the image is encoded.

[0042] Figure 4It is a diagram showing the form of prediction units (PUs) that can be included in a coding unit (CU).

[0043] Figure 5 It is a diagram showing the form of transform units (TUs) that can be included in a coding unit (CU).

[0044] Figure 6 It is a diagram showing an example of intra prediction processing.

[0045] Figure 7 It is a diagram showing an example of inter prediction processing.

[0046] Figure 8 It is a diagram showing a transfer modification of an embodiment of geometric modification of an image according to the present invention.

[0047] Figure 9 It is a diagram showing a size modification of an embodiment of geometric modification of an image according to the present invention.

[0048] Figure 10 It is a diagram showing a rotation modification of an embodiment of geometric modification of an image according to the present invention.

[0049] Figure 11 It is a diagram showing an affine modification of an embodiment of geometric modification of an image according to the present invention.

[0050] Figure 12 It is a diagram showing a projection modification of an embodiment of geometric modification of an image according to the present invention.

[0051] Figure 13 It is a diagram showing an example of a method for implementing a homography according to the present invention.

[0052] Figure 14 It is an example method for deriving a relationship between two corresponding points between two images according to the present invention.

[0053] Figure 15 It is a diagram showing a method for generating a geometrically modified image based on a geometric modification matrix and an original image according to the present invention.

[0054] Figure 16 It is a diagram showing a method for generating a geometrically modified image by using an inverse mapping according to the present invention.

[0055] Figure 17 It is a diagram showing a method for generating a geometrically modified image based on a geometric modification matrix and an original image according to the present invention, where the geometric modification matrix may correspond to geometric modification information.

[0056] Figure 18 It is a diagram showing a reference according to an embodiment of the present invention Figure 17The figure of bilinear interpolation among various interpolation methods shown.

[0057] Figure 19 It is a figure showing a method for performing motion prediction according to the present invention, in which a video encoder generates geometric modification information and a geometric modification picture, and performs motion prediction by using a reference picture and the geometric modification picture.

[0058] Figure 20 It is a block diagram showing the configuration of an image encoding apparatus to which another embodiment of the present invention is applied.

[0059] Figure 21 It is shown according to Figure 20 The figure of the configuration and operation of the geometric modification picture generation unit 2010 of the example encoding apparatus shown.

[0060] Figure 22 It is shown according to Figure 20 The figure of the configuration and operation of the geometric modification picture predictor 2015 of the example encoding apparatus shown.

[0061] Figure 23 It is a flowchart showing motion prediction according to an embodiment of the present invention.

[0062] Figure 24 It is a flowchart showing a method for generating a geometric modification picture according to an embodiment of the present invention.

[0063] Figure 25 It is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0064] Figure 26 It is a flowchart showing an encoding method for geometric modification information according to an embodiment of the present invention.

[0065] Figure 27 It is a figure showing various examples for modifying geometric modification information to reduce the amount of bits for encoding the geometric modification information.

[0066] Figure 28 It is a figure showing a motion compensation method, in which a decoder generates a geometric modification picture from a reference picture by using geometric modification information, and performs motion compensation by using the reference picture and the geometric modification picture.

[0067] Figure 29 It is a block diagram showing the configuration of a decoding apparatus to which another embodiment of the present invention is applied.

[0068] Figure 30 It is shown Figure 29 The figure of the operation and configuration of the geometric modification picture generator 2970 of the decoding apparatus shown.

[0069] Figure 31 is a diagram showing Figure 29 the operation and configuration of the geometric modified picture predictor 2960 of the decoding device shown in the figure.

[0070] Figure 32 is a flowchart showing the motion compensation of a decoder according to an embodiment of the present invention.

[0071] Figure 33 is a conceptual diagram showing the extended intra prediction according to an embodiment of the present invention.

[0072] Figure 34 is a diagram showing the operation and configuration of the extended intra prediction unit according to an embodiment of the present invention.

[0073] Figure 35 is a diagram showing the extended intra prediction according to an embodiment of the present invention.

[0074] Figure 36 is a diagram showing the intra prediction direction according to the present invention.

[0075] Figure 37 is a diagram showing an embodiment of performing the extended intra prediction according to the present invention.

[0076] Detailed implementation mode

[0077] Since various modifications can be made to the present invention and there are various embodiments of the present invention, examples will now be provided with reference to the accompanying drawings and will be described in detail. However, the present invention is not limited thereto, and the exemplary embodiments can be interpreted to include all modifications, equivalents, or alternatives within the technical concept and technical scope of the present invention. In all aspects, similar reference numerals refer to the same or similar functions. In the drawings, for clarity, the shapes and sizes of the elements may be exaggerated, and the same reference numerals are used throughout to designate the same or similar elements. In the following detailed description of the present invention, reference is made to the accompanying drawings which illustrate, in an illustrative manner, specific embodiments in which the present invention can be practiced. The embodiments are described in sufficient detail to enable those skilled in the art to practice the present disclosure. It should be understood that the various embodiments of the present disclosure, although different, are not necessarily mutually exclusive. For example, the specific features, structures, and characteristics described herein in connection with one embodiment can be implemented in other embodiments without departing from the spirit and scope of the present disclosure. Additionally, it should be understood that the positions or arrangements of the individual elements within each disclosed embodiment can be modified without departing from the spirit and scope of the present disclosure. Therefore, the following detailed description should not be considered limiting in nature, and the scope of the present disclosure, as well as the full scope equivalent to the scope claimed by the claims, is defined only by the appended claims, appropriately interpreted.

[0078] The terms "first", "second", etc. used in the specification may be used to describe various components, but these components should not be construed as being limited to these terms. These terms are only used to distinguish one component from other components. For example, a "first" component may be referred to as a "second" component without departing from the scope of the present invention, and a "second" component may similarly be referred to as a "first" component. The term "and / or" includes combinations of multiple items or any one of multiple items.

[0079] When referring to an element being "coupled" or "connected" to another element, this may mean that it is directly coupled or connected to the other element, but it should be understood that there may be another element therebetween. On the other hand, when referring to an element being "directly coupled" or "directly connected" to another element, it should be understood that there are no other elements therebetween.

[0080] In addition, the constituent parts shown in the embodiments of the present invention are independently shown to represent different characteristic functions from each other. Therefore, this does not mean that each constituent part is constituted by separate hardware or software constituent units. In other words, for convenience, each constituent part includes each of the listed constituent parts. Therefore, at least two of the constituent parts of each constituent part may be combined to form one constituent part, or one constituent part may be divided into multiple constituent parts to perform each function. Embodiments in which each constituent part is combined and embodiments in which one constituent part is divided are also included in the scope of the present invention without departing from the essence of the present invention.

[0081] The terms used in this specification are only for describing specific embodiments and are not intended to limit the present invention. Expressions used in the singular form include expressions in the plural form unless they have a clearly different meaning in the context. In this specification, it should be understood that terms such as "including", "having", etc. are intended to indicate the existence of features, numbers, steps, actions, elements, parts, or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, elements, parts, or combinations thereof may exist or may be added. In other words, when a specific element is referred to as being "included", elements other than the corresponding element are not excluded, but additional elements may be included in the embodiments of the present invention or within the scope of the present invention.

[0082] In addition, some constituent parts may not be indispensable constituent parts for performing the basic functions of the present invention, but are only selective constituent parts for improving its performance. In addition to the constituent parts for improving performance, the present invention can be implemented by only including the basic indispensable constituent parts for implementing the present invention. Structures that only include the indispensable constituent parts, in addition to the selective constituent parts for only improving performance, are also included in the scope of the present invention.

[0083] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. When describing the exemplary embodiments of the present invention, well-known functions or configurations will not be described in detail because they may unnecessarily obscure the understanding of the present invention. The same component elements in the drawings are denoted by the same reference numerals, and repeated descriptions of the same elements will be omitted.

[0084] In addition, hereinafter, an image may refer to a picture constituting a video, or may refer to a video. For example, "encoding and / or decoding an image" may refer to "encoding and / or decoding a video", or may refer to "encoding and / or decoding a single image among the images constituting a video". Here, a picture may refer to an image.

[0085] Encoder: may refer to an encoding device.

[0086] Decoder: may refer to a decoding device.

[0087] Parsing: may refer to determining the value of a syntax element by performing entropy decoding, or may refer to an entropy decoder.

[0088] Block: may refer to samples of an MxN matrix, where M and N are positive integers. A block may refer to a sample matrix of a two-dimensional matrix.

[0089] Unit: may refer to a unit for encoding or decoding an image. When encoding and decoding an image, a unit may be a region generated by partitioning the image. Alternatively, when an image is subdivided and encoded or decoded, a unit may refer to a divided unit of an image. While encoding and decoding, a predetermined process may be performed for each unit. A single unit may be divided into smaller sub-units. The unit may also refer to a block, macroblock (MB), coding unit (CU), prediction unit (PU), transform unit (TU), coding block (CB), prediction block (PB), or transform block (TB) according to its function. A unit may refer to an object including a luminance component block, its corresponding chrominance component block, and syntax elements for each block for indicating differentiation from a block. The unit may have different sizes and shapes. Specifically, the shape of the unit may include two-dimensional forms such as a rectangle, cube, trapezoid, triangle, pentagon, etc. In addition, the shape of the unit may include geometric figures. Furthermore, the unit information may include at least one of a unit type (such as a coding unit, prediction unit, transform unit, etc.), unit size, unit depth, and the sequence of unit encoding and decoding.

[0090] Reconstructed neighboring unit: may refer to a reconstructed unit that has been encoded or decoded spatially / temporally and is adjacent to the encoding / decoding target unit.

[0091] Depth: Indicates the degree of partitioning of a unit. In a tree structure, the highest node can refer to the root node, and the lowest node can refer to the leaf node.

[0092] Symbol: Can refer to the syntax elements and coding parameters for encoding / decoding the target unit, the values of transform coefficients, etc.

[0093] Parameter set: Can correspond to the header information in the structure within the bitstream. At least one of the video parameter set, sequence parameter set, picture parameter set, and adaptive parameter set can be included in the parameter set. Additionally, the parameter set can include the information of slice headers and tile headers.

[0094] Bitstream: Can refer to a bit string including encoded image information.

[0095] Coding parameters: Can include not only the information encoded by the encoder and sent to the decoder together with the syntax elements, but also the information that can be derived during the encoding or decoding process, or can refer to the parameters necessary for encoding and decoding. For example, the coding parameters can include at least one of the following values and / or statistics: intra prediction mode, inter prediction mode, intra prediction direction, motion information, motion vector, reference image index, inter prediction direction, inter prediction indicator, reference image list, motion vector predictor, motion merge candidate, transform type, transform size, information on whether to use additional transforms, in-loop filter information, information on whether there is a residual signal, quantization parameter, context model, transform coefficient, transform coefficient level, encoded block pattern, encoded block flag, image display / output order, slice information, tile information, picture type, information on whether to use the motion merge mode, information on whether to use the skip mode, block size, block depth, block partition information, unit size, unit partition information, etc.

[0096] Prediction unit: Can refer to the basic unit during inter prediction or intra prediction and when compensating for the prediction. The prediction unit can be divided into multiple partitions. Each of the partitions can also be the basic unit during inter prediction or intra prediction and when performing the compensation for the prediction. The partitioned prediction unit can also refer to the prediction unit. Additionally, a single prediction unit can be divided into smaller sub-units. The prediction unit can have various sizes and shapes. Specifically, the shape of the unit can include two-dimensional forms such as rectangles, cubes, trapezoids, triangles, pentagons, etc. Additionally, the shape of the unit can include geometric figures.

[0097] Prediction unit partition: Can refer to the partitioning form of the prediction unit.

[0098] Reference picture list: May refer to a list including at least one reference picture for inter - frame prediction or motion compensation. The types of reference lists may include a combined list (LC), L0 (list 0), L1 (list 1), L2 (list 2), L3 (list 3), etc. At least one reference picture list may be used for inter - frame prediction.

[0099] Inter - frame prediction indicator: May refer to the inter - frame prediction direction (unidirectional prediction, bidirectional prediction) for encoding / decoding a target block. Alternatively, the indicator may refer to the number of reference pictures used to generate a prediction block for the encoding / decoding target block, or may refer to the number of prediction blocks used when performing motion compensation for the encoding / decoding target block.

[0100] Reference picture index: May refer to the index of a specific picture within a reference picture list.

[0101] Reference picture: May refer to a reference picture referred to by a specific unit for inter - frame prediction or motion compensation. Optionally, the reference image may refer to a reference picture.

[0102] Motion vector: Refers to a two - dimensional matrix for inter - frame prediction or motion compensation, or may be the offset between an encoding / decoding target image and a reference image. For example, (mvX, mvY) may indicate a motion vector, where mvX may be the horizontal component and mvY may be the vertical component.

[0103] Motion vector candidate: May refer to a unit that becomes a prediction candidate when predicting a motion vector, or may refer to the motion vector of a unit.

[0104] Motion vector candidate list: May refer to a list configured with motion vector candidates.

[0105] Motion vector candidate index: May refer to an indicator indicating a motion vector candidate within a motion vector candidate list, or may refer to the index of a motion vector predictor.

[0106] Motion information: May refer to information including at least one of a motion vector, a reference picture index, an inter - frame prediction indicator, reference picture list information, a reference picture, a motion vector candidate, a motion vector candidate index, etc.

[0107] Transformation unit: When performing operations such as transformation, inverse transformation, quantization, inverse quantization, and encoding / decoding of residual signals like transformation coefficients, may refer to a basic unit. A single unit may be divided into smaller sub - units. The unit may have different sizes and shapes. Specifically, the shape of the unit may include two - dimensional forms such as rectangles, cubes, trapezoids, triangles, pentagons, etc. Additionally, the shape of the unit may also include geometric figures.

[0108] Scaling: It may refer to the process of multiplying a factor by a transform coefficient level, and the result can generate a transform coefficient. Scaling may also refer to inverse quantization.

[0109] Quantization parameter: It may refer to a value used to scale the transform coefficient level in quantization and inverse quantization. Here, the quantization parameter may be a value mapped to the quantization step size.

[0110] Delta quantization parameter: It may refer to the residual value between the predicted quantization parameter of the encoding / decoding target unit and the quantization parameter.

[0111] Scanning: It may refer to a method of sorting the coefficient order within a block or matrix. For example, sorting a two-dimensional matrix into a one-dimensional matrix may refer to scanning or inverse scanning.

[0112] Transform coefficient: It may be the coefficient value generated after performing a transform. In the present invention, the transform coefficient level quantized by applying quantization to the transform coefficient may be included in the transform coefficient.

[0113] Non-zero transform coefficient: It may refer to a transform coefficient whose value or magnitude is not 0.

[0114] Quantization matrix: It may refer to a matrix used for quantization and inverse quantization to improve the quality of an image. The quantization matrix may also refer to a scaling list.

[0115] Quantization matrix coefficient: It may refer to each element of the quantization matrix. The quantization matrix coefficient may also refer to a matrix coefficient.

[0116] Default matrix: It may refer to a predetermined quantization matrix defined in advance in the encoder and decoder.

[0117] Non-default matrix: It may refer to a quantization matrix sent / received by the user and not defined in advance in the encoder and decoder.

[0118] Figure 1 It is a block diagram showing the configuration of an image encoding device to which an embodiment of the present invention is applied.

[0119] Encoding device 100 may be a video encoding device or an image encoding device. The video may include at least one image. Encoding device 100 may encode at least one image of the video in chronological order.

[0120] Reference Figure 1 , encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

[0121] The encoding device 100 can encode an input image in an intra mode, an inter mode, or both. Additionally, the encoding device 100 can generate a bitstream by encoding the input image and can output the generated bitstream. When the intra mode is used as the prediction mode, the switch 115 can be switched to intra. When the inter mode is used as the prediction mode, the switch 115 can be switched to inter. Here, the intra mode can be referred to as the intra prediction mode, and the inter mode can be referred to as the inter prediction mode. The encoding device 100 can generate a prediction signal for an input block of the input image. The prediction signal as a block unit can be referred to as a prediction block. Additionally, after generating the prediction block, the encoding device 100 can encode the residual value between the input block and the prediction block. The input image can be referred to as the current image that is the target of the current encoding. The input block can be referred to as the current block or the encoding target block that is the target of the current encoding.

[0122] When the prediction mode is the intra mode, the intra prediction unit 120 can use the pixel values of previously encoded blocks adjacent to the current block as reference pixels. The intra prediction unit 120 can perform spatial prediction by using the reference pixels for spatial prediction and can generate prediction samples for the input block by using the spatial prediction. Here, intraprediction may mean intra-frame prediction.

[0123] When the prediction mode is the inter mode, the motion prediction unit 111 can search for the region that best matches the input block of the reference image during the motion prediction process and can derive a motion vector by using the searched region. The reference image can be stored in the reference picture buffer 190.

[0124] The motion compensation unit 112 can generate a prediction block by performing motion compensation by using the motion vector. Here, the motion vector can be a two-dimensional vector used in inter prediction. Additionally, the motion vector can indicate the offset between the current image and the reference image. Here, inter prediction may refer to inter-frame prediction.

[0125] When the value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 can generate a prediction block by applying an interpolation filter to a partial region in the reference image. To perform inter prediction or motion compensation, based on the coding unit, the motion prediction method of the prediction unit included in the coding unit and the compensation method of the motion prediction can be determined among the skip mode, the merge mode, and the AMVP mode. Additionally, inter prediction or motion compensation can be performed according to the mode.

[0126] The subtractor 125 can generate a residual block by using the residual between the input block and the prediction block. The residual block may be referred to as a residual signal.

[0127] The transform unit 130 can generate transform coefficients by transforming the residual block and can output the transform coefficients. Here, the transform coefficients may be coefficient values generated by transforming the residual block. In the transform skip mode, the transform unit 130 can skip the transformation of the residual block.

[0128] The quantized transform coefficient levels can be generated by applying quantization to the transform coefficients. Hereinafter, in the embodiments of the present invention, the quantized transform coefficient levels may be referred to as transform coefficients.

[0129] The quantization unit 140 can generate the quantized transform coefficient levels by quantizing the transform coefficients according to quantization parameters and can output the quantized transform coefficient levels. Here, the quantization unit 140 can quantize the transform coefficients by using a quantization matrix.

[0130] According to the probability distribution, the entropy coding unit 150 can generate a bitstream by performing entropy coding on the values calculated by the quantization unit 140 or the coding parameter values calculated in the coding process, etc., and can output the bitstream. The entropy coding unit 150 can perform entropy coding on the information for decoding the image and the information of the pixels of the image. For example, the information for decoding the image may include syntax elements, etc.

[0131] When applying entropy coding, symbols are represented by allocating a small number of bits to symbols with a high occurrence probability and a large number of bits to symbols with a low occurrence probability, thereby reducing the size of the bitstream of the symbols to be encoded. Therefore, the compression performance of image coding can be increased by entropy coding. For entropy coding, the entropy coding unit 150 can use coding methods such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). For example, the entropy coding unit 150 can perform entropy coding by using a variable length coding / code (VLC) table. In addition, the entropy coding unit 150 can derive the binarization method of the target symbol and the probability model of the target symbol / bin, and can perform arithmetic coding using the derived binarization method or the derived probability model thereafter.

[0132] To encode the transform coefficient levels, the entropy encoding unit 150 may change the two-dimensional block-form coefficients into a one-dimensional vector form by using a transform coefficient scanning method. For example, by scanning the coefficients of the block using a top-right scan, the two-dimensional block-form coefficients can be changed into a one-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, instead of using a top-right scan, a vertical scan that scans the two-dimensional block-form coefficients in the column direction and a horizontal scan that scans the two-dimensional block-form coefficients in the row direction may be used. In other words, the scanning method can be determined among a top-right scan, a vertical scan, and a horizontal scan according to the size of the transform unit and the intra prediction mode.

[0133] The encoding parameters may include not only the information encoded by the encoder and then transmitted to the decoder together with the syntax elements, but also the information that can be derived during the encoding or decoding process, or the parameters that can refer to the necessary parameters for encoding and decoding. For example, the encoding parameters may include at least one of the following values or statistics: intra prediction mode, inter prediction mode, intra prediction direction, motion information, motion vector, reference image index, inter prediction direction, inter prediction indicator, reference image list, motion vector predictor, motion merge candidate, transform type, transform size, information on whether to use an additional transform, in-loop filter information, information on whether there is a residual signal, quantization parameter, context model, transform coefficient, transform coefficient level, encoded block pattern, encoded block flag, image display / output order, slice information, tile information, picture type, information on whether to use a motion merge mode, information on whether to use a skip mode, block size, block depth, block partitioning information, unit size, unit partitioning information, etc.

[0134] The residual signal may mean the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming the difference between the original signal and the predicted signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the predicted signal. A residual block may be a residual signal, which is a block unit.

[0135] When the encoding device 100 performs encoding by using inter prediction, the encoded current image may be used as a reference image for other images to be processed later. Therefore, the encoding device 100 may decode the encoded current image and may store the decoded image as a reference image. To perform decoding, inverse quantization and inverse transformation may be performed on the encoded current image.

[0136] The quantization coefficients may be dequantized by the inverse quantization unit 160 and may be inverse-transformed by the inverse transform unit 170. The dequantized and inverse-transformed coefficients may be added to the prediction block by the adder 175, whereby a reconstructed block may be generated.

[0137] The reconstructed block can pass through filter unit 180. Filter unit 180 can apply at least one of a deblocking filter, sample adaptive offset (SAO), and adaptive loop filter (ALF) to the reconstructed block or the reconstructed image. Filter unit 180 can be referred to as an in-loop filter.

[0138] The deblocking filter can remove block distortion occurring at the boundary between blocks. To determine whether the deblocking filter is operated, it can be determined whether the deblocking filter is applied to the current block based on pixels in several rows or columns included in the block. When the deblocking filter is applied to a block, a strong filter or a weak filter can be applied according to the required deblocking filtering strength. Additionally, when performing vertical filtering and horizontal filtering during the application of the deblocking filter, the horizontal direction filtering and the vertical direction filtering can be processed in parallel.

[0139] The sample adaptive offset can add an optimal offset value to the pixel value to compensate for coding errors. The sample adaptive offset can utilize pixels to correct the offset between the deblocked image and the original image. To perform offset correction for a specific picture, a method of applying offset correction considering the edge information of each pixel can be used, or a method of partitioning the pixels of the image into a predetermined number of regions, determining the regions to which offset correction is to be performed, and applying offset correction to the determined regions can be used.

[0140] The adaptive loop filter can perform filtering based on values obtained by comparing the reconstructed image and the original image. The pixels of the image can be partitioned into predetermined groups, a single filter applied to each group can be determined, and different filtering can be performed for each group. Information regarding whether to apply the adaptive loop filter can be sent to each coding unit (CU). The shape and filter coefficients of the adaptive loop filter applicable to each block can vary. Additionally, an adaptive loop filter having the same form (fixed form) can be applied regardless of the characteristics of the target block.

[0141] The reconstructed block that has passed through filter unit 180 can be stored in reference picture buffer 190.

[0142] Figure 2 is a block diagram showing the configuration of an image decoding apparatus to which an embodiment of the present invention is applied.

[0143] The decoding apparatus 200 can be a video decoding apparatus or an image decoding apparatus.

[0144] Reference Figure 2 , the decoding apparatus 200 can include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

[0145] The decoding device 200 can receive the bitstream output from the encoding device 100. The decoding device 200 can decode the bitstream in an intra mode or an inter mode. In addition, the decoding device 200 can generate a reconstructed image through decoding and can output the reconstructed image.

[0146] When using the intra mode as the prediction mode used in decoding, the switch can be switched to intra. When using the inter mode as the prediction mode used in decoding, the switch can be switched to inter.

[0147] The decoding device 200 can obtain the reconstructed residual block from the input bitstream and can generate a prediction block.

[0148] When obtaining the reconstructed residual block and the prediction block, the decoding device 200 can generate a reconstructed block as the decoding target block by adding the reconstructed residual block and the prediction block. The decoding target block can be referred to as the current block.

[0149] The entropy decoding unit 210 can generate symbols by performing entropy decoding on the bitstream according to the probability distribution. The generated symbols can include symbols in the form of quantized transform coefficient levels.

[0150] Here, the method of entropy decoding can be similar to the above-mentioned method of entropy encoding. For example, the method of entropy decoding can be the inverse process of the above-mentioned method of entropy encoding.

[0151] To decode the transform coefficient levels, the entropy decoding unit 210 can change the one-dimensional block-form coefficients into two-dimensional vector form by using a transform coefficient scanning method. For example, by scanning the coefficients of the block using a right-up scan, the one-dimensional block-form coefficients can be changed into two-dimensional vector form. Depending on the size of the transform unit and the intra prediction mode, vertical scan and horizontal scan can be used instead of using the right-up scan. In other words, the scanning method can be determined among the right-up scan, the vertical direction scan, and the horizontal direction scan according to the size of the transform unit and the intra prediction mode.

[0152] The quantized transform coefficient levels can be dequantized by the inverse quantization unit 220 and can be inverse-transformed by the inverse transform unit 230. The quantized transform coefficient levels are dequantized and inverse-transformed to generate a reconstructed residual block. Here, the inverse quantization unit 220 can apply a quantization matrix to the quantized transform coefficient levels.

[0153] When using the intra mode, the intra prediction unit 240 can generate a prediction block by performing spatial prediction using the pixel values of the previously decoded blocks around the decoding target block.

[0154] When using an inter-frame mode, the motion compensation unit 250 can generate a prediction block by performing motion compensation using both a motion vector and a reference image stored in the reference picture buffer 270. When the value of the motion vector is not an integer, the motion compensation unit 250 can generate a prediction block by applying an interpolation filter to a partial region in the reference image. To perform motion compensation, based on the coding unit, a motion prediction method and a compensation method for motion prediction of a prediction unit included in the coding unit can be determined among a skip mode, a merge mode, and an AMVP mode. Additionally, inter-frame prediction or motion compensation can be performed according to the mode. Here, the current picture reference mode can mean a prediction mode that uses a previously reconstructed region within the current picture having a decoding target block. The previously reconstructed region may not be adjacent to the decoding target block. To specify the previously reconstructed region, a fixed vector can be used for the current picture reference mode. Additionally, a flag or an index for indicating whether the decoding target block is a block decoded in the current picture reference mode can be signaled, and the flag or the index can be derived by using the reference picture index of the decoding target block. The current picture for the current picture reference mode can exist at a fixed position (e.g., the position where refIdx = 0 or the last position) within a reference picture list for the decoding target block. Additionally, it can be variably positioned within the reference picture list, and for this, an additional reference picture index for indicating the position of the current picture can be signaled.

[0155] The reconstructed residual block can be added to the prediction block by an adder 255. The block generated by adding the reconstructed residual block and the prediction block can pass through a filter unit 260. The filter unit 260 can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed image. The filter unit 260 can output a reconstructed image. The reconstructed image can be stored in the reference picture buffer 270 and can be used in inter-frame prediction.

[0156] Figure 3 is a diagram schematically showing a partition structure of an image when encoding and decoding the image. Figure 3 Schematically shows an example of partitioning a single unit into multiple lower-level units.

[0157] To effectively partition an image, a coding unit (CU) can be used during both encoding and decoding. A unit can refer to 1) a syntax element, and 2) a block including a sample image. For example, "partitioning of a unit" can refer to "partitioning of a block corresponding to the unit". Block partition information can include depth information of the unit. The depth information can indicate the number and / or the degree of partitioning in the unit.

[0158] Reference Figure 3, the image 300 is partitioned in the order of the largest coding unit (hereinafter referred to as LCU), and the partition structure is determined based on the LCU. Here, the LCU can be used as a coding tree unit (CTU). A single unit can include depth information based on a tree structure and can be hierarchically partitioned. Each of the partition units in the lower layer can include depth information. The depth information indicates the number of partitions and / or the degree of partitioning in the unit and can thus include the unit size information of the lower layer.

[0159] The partition structure can refer to the distribution of coding units (CUs) within the LCU 310. The CU can be a unit for efficiently encoding an image. The distribution can be determined based on whether a single CU will be partitioned into multiple (positive integers greater than 2, including 2, 4, 8, 16, etc.). The width size and height size of each partitioned CU can be half of the width size and height size of a single CU. Alternatively, depending on the number of partition units, the width size and height size of each partitioned CU can be less than the width size and height size of a single CU. Similarly, the partitioned CU can be recursively partitioned into multiple CUs, and each CU reduces the width size and height size by half from the partitioned CU.

[0160] Here, the partitioning of the CU can be recursively performed until a predetermined depth. The depth information can be information indicating the size of the CU. The depth information of each CU can be stored therein. For example, the depth of the LCU can be 0, and the depth of the smallest coding unit (SCU) can be a predetermined maximum depth. Here, the LCU can be a CU having the largest CU size as described above, and the SCU can be a CU having the smallest CU size.

[0161] Whenever the LCU 310 is partitioned and its width size and height size are reduced, the depth of the CU increases by 1. A CU for which partitioning has not been performed can have a size of 2N×2N for each depth, and a CU for which partitioning has been performed can be partitioned from a CU having a size of 2N×2N into multiple CUs, where each of the multiple CUs has a size of N×N. Whenever the depth increases by 1, the size of N is halved.

[0162] Reference Figure 3 , the size of the LCU with the minimum depth 0 can be 64×64 pixels, and the size of the SCU with the maximum depth 3 can be 8×8 pixels. Here, the LCU with 64×64 pixels can be represented by depth 0, the CU with 32×32 pixels can be represented by depth 1, the CU with 16×16 pixels can be represented by depth 2, and the SCU with 8×8 pixels can be represented by depth 3.

[0163] In addition, information on whether a specific CU will be partitioned can be represented by 1-bit partitioning information for each CU. Except for the SCU, all CUs may contain partitioning information. For example, when a CU is not partitioned, the partitioning information may be 0. Alternatively, when a CU is partitioned, the partitioning information may be 1.

[0164] Figure 4 is a diagram showing the form of prediction units (PUs) that can be included in a CU.

[0165] A CU that is no longer partitioned from the LCU and is partitioned can be partitioned into at least one PU. Such a process may also be referred to as partitioning.

[0166] A prediction unit (PU) can be a basic unit of prediction. The PU can be encoded and decoded in any one of a skip mode, an inter prediction mode, and an intra prediction mode. The PU can be partitioned in various forms according to each mode.

[0167] As Figure 4 shown, in the skip mode, there may be no partitioning within the CU. Additionally, a 2N×2N mode 410 having the same size as the CU can be supported within the CU without partitioning.

[0168] In the inter prediction mode, 8 partitioning forms can be supported within the CU. For example, a 2N×2N mode 410, a 2N×2N mode 415, an N×2N mode 420, an N×N mode 425, a 2N×nU mode 430, a 2N×nD mode 435, an nL×2N mode 440, and an nR×2N mode 445.

[0169] Figure 5 is a diagram showing the form of transform units (TUs) that can be included in a CU.

[0170] A transform unit (TU) can be a basic unit within the CU for transform, quantization, inverse transform, and inverse quantization processes. The TU can have a rectangular or square form. The TU can be independently determined by the size and / or form of the CU.

[0171] A CU that is no longer partitioned from the LCU and is partitioned can be partitioned into one or more TUs. Here, the partitioning structure of the TU can be a quadtree structure. For example, as Figure 5As shown, depending on the quadtree structure, a single CU 510 can be partitioned one or more times such that the CU 510 is formed by TUs of various sizes. Alternatively, a single CU 510 can be partitioned into at least one TU based on the number of horizontal and / or vertical lines for partitioning the CU. The CU can be partitioned into TUs that are symmetric to each other, or can be partitioned into TUs that are asymmetric to each other. To partition into asymmetric TUs, information on the size and form of the TU can be signaled, or this information can be derived from the information on the size and form of the CU.

[0172] While performing the transform, the residual block can be transformed by using one of the predetermined methods. For example, the predetermined methods can include discrete cosine transform (DCT), discrete sine transform (DST), or Karhunen - Loève transform (KLT). To determine the method for transforming the residual block, the method can be determined by using at least one of the inter - prediction mode information of the prediction unit, the intra - prediction mode information of the prediction unit, or the size and form of the transform block. Alternatively, in some cases, information indicating the method can be signaled.

[0173] Figure 6 is a diagram showing an example of an intra - prediction mode.

[0174] The number of intra - prediction modes can vary according to the size of the prediction unit (PU), or can be fixed to a number N, regardless of the size of the prediction unit (PU). Here, the number N can include 35 and 67, or can be a positive integer greater than 1. For example, the predetermined intra - prediction modes of the encoder / decoder can include 2 non - directional modes and 65 directional modes, as Figure 6 shown. The two non - directional modes can include the DC mode and the plane mode.

[0175] The number of intra - prediction modes can be different according to the type of color component. For example, the number of intra - prediction modes can vary whether the color component is a luminance signal or a chrominance signal.

[0176] The PU can have a square of size NxN or 2Nx2N. The NxN size may include 4x4, 8x8, 16x16, 32x32, 64x64, 128x128, etc. Alternatively, the PU can have a size of MxN. Here, M and N can be positive integers greater than 2, and M and N can be different numbers. The unit of the PU can be the size of at least one of the CU, PU, and TU.

[0177] Intra - coding and / or decoding can be performed by using the sampled values or coding parameters included in adjacent reconstructed units.

[0178] In intra prediction, a reference sampling filter may be applied to reference pixels to generate a prediction block by using at least one of the sizes of an encoding / decoding target block. The types of reference filters applied to the reference pixels may be different. For example, the reference filter may vary according to the intra prediction mode of the encoding / decoding target block, the size / form of the encoding / decoding target block, or the position of the reference pixels. "The types of reference filters may be different" may mean that the filter coefficients of the reference filter, the number of filter taps, the filter strength, or the number of filtering processes may be different.

[0179] To perform intra prediction, the intra prediction mode of a current prediction unit may be predicted by the intra prediction modes of adjacent prediction units adjacent to the current prediction unit. When the intra prediction mode of the current prediction unit is predicted by using the intra prediction mode information of an adjacent prediction unit and the two modes are the same, information indicating that the two modes are the same may be sent by using a predetermined flag. Alternatively, when the modes are different, all the prediction mode information within the encoding / decoding target block may be encoded by entropy coding.

[0180] Figure 7 is a diagram showing an example of an inter prediction process.

[0181] Figure 7 The rectangle of may refer to an image (or picture). Additionally, Figure 7 The arrow of may indicate a prediction direction. In other words, an image may be encoded and / or decoded according to the arrow direction. Depending on the encoding type, each image may be classified as an I picture (intra picture), a P picture (unidirectional prediction picture), a B picture (bidirectional prediction picture), etc. Each picture may be encoded and decoded according to the encoding type of each picture.

[0182] When the encoding target image is an I picture, intra encoding may be performed on the target image itself while performing inter prediction. When the encoding target image is a P picture, the target image may be encoded by using inter prediction or motion compensation of a reference image in the forward direction. When the encoding target image is a B picture, the target image may be encoded by using inter prediction or motion compensation of reference pictures in the forward direction and the backward direction. Alternatively, the target image may be encoded by using inter prediction of reference images in the forward direction and the backward direction. Here, in the case of an inter prediction mode, the encoder may perform inter prediction or motion compensation, and the decoder may perform motion compensation in response to the encoder. Images of P pictures and B pictures that are encoded and / or decoded by using reference images are used for inter prediction.

[0183] Hereinafter, inter prediction according to an embodiment will be described in detail.

[0184] Inter-frame prediction or motion compensation can be performed by using a reference image and motion information. Additionally, inter-frame prediction can use the skip mode described above.

[0185] The reference picture can be at least one of a previous picture of the current picture or a subsequent picture of the current picture. Here, in inter-frame prediction, a block of the current picture can be predicted based on the reference picture. Here, a region within the reference picture can be specified by using a reference picture index refIdx indicating the reference picture and a motion vector, which will be described later.

[0186] In inter-frame prediction, a reference block corresponding to the current block within the reference picture can be selected. A predicted block of the current block can be generated by using the selected reference block. The current block can be the currently encoded or decoded target block among the blocks of the current picture.

[0187] Motion information can be derived from the inter-frame prediction processing of the encoding device 100 and the decoding device 200. Additionally, the derived motion information can be used for inter-frame prediction. Here, the encoding device 100 and the decoding device 200 can improve the encoding and / or decoding efficiency by using the motion information of reconstructed adjacent blocks and / or the motion information of collocated blocks (col blocks). A collocated block can be a block within a reconstructed collocated picture (col picture) that spatially corresponds to the encoded / decoded target block. A reconstructed adjacent block can be a block within the current picture, and a reconstructed block obtained by encoding and / or decoding. Additionally, a reconstructed block can be a block adjacent to the encoded / decoded target block and / or a block located at the outer corner of the encoded / decoded target block. Here, a block located at the outer corner of the encoded / decoded target block can be a block adjacent in the vertical direction, and the block adjacent in the vertical direction is adjacent to the encoded / decoded target block in the horizontal direction. Alternatively, a block located at the outer corner of the encoded / decoded target block can be a block adjacent in the horizontal direction, and the block adjacent in the horizontal direction is adjacent to the encoded / decoded target block in the vertical direction.

[0188] Each of the encoding device 100 and the decoding device 200 can determine a predetermined relative position based on a block existing at a position within the collocated picture that spatially corresponds to the current block. The predetermined relative position can be inside and / or outside the block existing at the position that spatially corresponds to the current block. Additionally, the encoding device 100 and the decoding device 200 can derive a collocated block based on the determined relative position. Here, the collocated picture can be at least one picture among the reference pictures included in the reference picture list.

[0189] The method for deriving motion information can vary according to the prediction mode of the encoded / decoded target block. For example, prediction modes applied to inter-frame prediction can include an advanced motion vector predictor (AMVP) mode, a merge mode, etc. Here, the merge mode can refer to a motion merge mode.

[0190] For example, in the case of applying the Advanced Motion Vector Predictor (AMVP) mode, the encoding device 100 and the decoding device 200 may generate a list of predicted motion vector candidates by using the motion vectors of the recovered neighboring blocks and / or the motion vectors of the collocated blocks. In other words, the motion vectors of the recovered neighboring blocks and / or the motion vectors of the collocated blocks may be used as predicted motion vector candidates. Here, the motion vectors of the collocated blocks may refer to temporal motion vector candidates, and the motion vectors of the recovered neighboring blocks may refer to spatial motion vector candidates.

[0191] The encoding device 100 may generate a bitstream, and the bitstream may include a motion vector candidate index. In other words, the encoding device 100 may perform entropy coding on the motion vector candidate index to generate a bitstream. The motion vector candidate index may indicate the optimal predicted motion vector selected from among the predicted motion vector candidates included in the list of motion vector candidates. The motion vector candidate index may be sent from the encoding device 100 to the decoding device 200 via the bitstream.

[0192] The decoding device 200 may perform entropy decoding on the motion vector candidate index via the bitstream, and by using the entropy-decoded motion vector candidate index, select a motion vector candidate for the block to be decoded from among the motion vector candidates included in the list of motion vector candidates.

[0193] The encoding device 100 may calculate the Motion Vector Difference (MVD) between the motion vector of the block to be encoded and the motion vector candidate, and may perform entropy coding on the Motion Vector Difference (MVD). The bitstream may include the entropy-coded MVD. The MVD is sent to the decoding device 200 via the bitstream. Here, the decoding device 200 may perform entropy decoding on the MVD from the bitstream. The decoding device 200 may derive the motion vector of the block to be decoded by adding the decoded MVD and the motion vector candidate.

[0194] The bitstream may include a reference picture index indicating a reference picture. The reference picture index may be entropy-coded and sent from the encoding device 100 to the decoding device 200 via the bitstream. The decoding device 200 may predict the motion vector of the current block by using the motion information of neighboring blocks, and may derive the motion vector of the block to be decoded by using the predicted motion vector and the residual of the predicted motion vector. The decoding device 200 may generate a predicted block of the block to be decoded based on the derived motion vector and the reference picture index information.

[0195] As another method of deriving motion information, a merge mode may be used. The merge mode may refer to the motion merging of multiple blocks. The merge mode may refer to applying the motion information of a single block to another block. When the merge mode is applied, the encoding device 100 and the decoding device 200 may generate a merge candidate list by using the motion information of the recovered neighboring blocks and / or the motion information of the collocated blocks. Here, the motion information may include at least one of 1) a motion vector, 2) a reference picture index, and 3) an inter prediction indicator. The prediction indicator may indicate unidirectional (LO prediction, L1 prediction) or bidirectional.

[0196] Here, the merge mode may be applied in units of an encoding unit or a prediction unit (PU). In the case where the merge mode is executed by the CU unit or the PU unit, the encoding device 100 may generate a bitstream by entropy encoding predetermined information and send the bitstream to the decoding device 200. The bitstream may include the predetermined information. The predetermined information may include 1) a merge flag indicating whether the merge mode is used for the merge of each block partition, and 2) a merge index including information on which block among the neighboring blocks adjacent to the encoding target block is merged. For example, the neighboring blocks adjacent to the encoding target block may include the left neighboring block of the current block, the upper neighboring block of the encoding target block, the temporally neighboring blocks of the encoding target block, etc.

[0197] The merge candidate list may represent a list in which motion information is stored. The merge candidate list may be generated before the merge mode is executed. The motion information stored in the merge candidate list may be at least one of the following information: the motion information of the neighboring blocks adjacent to the encoding / decoding target block, the motion information of the collocated blocks corresponding to the encoding / decoding target block in the reference picture, the newly generated motion information by combining the motion information that already exists in the merge motion candidate list, and a zero merge candidate. Here, the motion information of the neighboring blocks adjacent to the encoding / decoding target block may refer to a spatial merge candidate, and the motion information of the collocated blocks corresponding to the encoding / decoding target block in the reference picture may refer to a temporal merge candidate.

[0198] In the case of the skip mode, the skip mode applies the motion information of the neighboring block to the encoding / decoding target block. The skip mode may be one of the other modes for inter prediction. When the skip mode is used, the encoding device 100 may generate a bitstream by entropy encoding the information of the neighboring blocks that can be used for the encoding target block and send the bitstream to the decoding device 200. The encoding device 100 may not send other information such as syntax information to the decoding device 200. The syntax information may include at least one of the residual information of the motion vector, the coding block flag, and the transform coefficient level.

[0199] Figures 8 to 18 is a diagram showing a method of generating a geometrically modified image by geometrically modifying an image.

[0200] Geometric modification of an image can refer to geometrically modifying the light information of the image. The light information can refer to the brightness, color, or chromaticity of each point of the image. Alternatively, the light information can refer to the pixel values in a digital image. Geometric modification can refer to a parallel shift of each point within the image, rotation of the image, resizing of the image, etc.

[0201] Figures 8 to 12 are diagrams respectively showing the geometric modification of an image according to the present invention. (x, y) of each figure refers to the point of the original image before modification. (x', y') refers to the point corresponding to the point (x, y) after modification. Here, the corresponding point refers to the point that moves the light information of (x, y) through geometric modification.

[0202] Figure 8 is a diagram showing the translation modification of an embodiment of the geometric modification of an image according to the present invention.

[0203] In Figure 8 , tx refers to the displacement of each point that has been translated on the x-axis, and ty represents the displacement of each point that has been translated on the y-axis. Therefore, the point (x', y') within the image is derived by adding tx and ty to the point (x, y) that was a point within the image before modification. The translation modification can be represented by the matrix shown in Figure 8 .

[0204] Figure 9 is a diagram showing the size modification of an embodiment of the geometric modification of an image according to the present invention.

[0205] In Figure 9 , sx refers to the size modification multiple in the x-axis direction, and sy represents the size modification multiple in the y-axis direction. The size modification multiple can refer to the size ratio of the image before modification to the image after modification. When the size modification multiple is equal to 1, it means that the size of the image before modification is equal to the size of the image after modification. When the size modification multiple is greater than 1, it means that the size of the image after modification is enlarged. When the size modification multiple is less than 1, it means that the size of the image after modification is reduced. The size modification multiple always has a value greater than 0. Therefore, the point (x', y') within the modified image with modified size is derived by multiplying sx and sy by the point (x, y) within the image before modification. The size modification can be represented by the matrix shown in Figure 9 .

[0206] Figure 10 is a diagram showing the rotation modification of an embodiment of the geometric modification of an image according to the present invention.

[0207] In Figure 10 , θ refers to the rotation angle of the image. In Figure 10In the embodiment, rotation is performed centered on the point (0, 0) of the pre-modification image. The point (x', y') within the rotated and modified image after modification can be derived by using θ and trigonometric functions. The rotation modification can be represented by the matrix shown in Figure 10 as shown.

[0208] Figure 11 FIG. is a diagram showing an affine modification of an embodiment of a geometric modification of an image according to the present invention.

[0209] Affine modification refers to the case where translation modification, size modification, and rotation modification are performed in combination. The geometric modification of the affine modification may vary according to the order of the translation modification, size modification, and / or rotation modification applied to the image. Depending on the application order among the multiple modifications that make up the affine modification and the complex of each modification, the image may be modified in the form of skew, translation modification, size modification, and rotation modification.

[0210] In Figure 11 , M i may be a 3×3 matrix for translation modification, size modification, or rotation modification. According to the order of the modifications that make up the affine modification, a 3x3 matrix can be obtained by matrix-multiplying each matrix for the modification with each other. In Figure 11 , the matrix A may correspond to a 3x3 matrix obtained by matrix-multiplying the matrix M 1 to the matrix M n . The matrix A may be composed of elements a 1 to a 6 . The matrix p is a point within the pre-modification image, where the modification is represented by a matrix. The matrix p' is a point within the post-modification image and corresponds to the point p within the pre-modification image. Therefore, the affine modification can be represented as the matrix equation p' = Ap.

[0211] Figure 12 FIG. is a diagram showing a projection modification of an embodiment of a geometric modification of an image according to the present invention.

[0212] Projection modification may be an extended affine modification, where perspective modification is added to the affine modification. When an object in three-dimensional space is projected onto a two-dimensional plane, perspective modification may occur according to the viewing angle of the camera or observer. In perspective modification, distant objects are represented as small and nearby objects are represented as large.

[0213] In Figure 12 , the matrix H may be used for projection modification. The elements h 1 to h 6 that make up the matrix H may correspond to the elements a Figure 11 of the matrix A that makes up the 1 affine modification.6 Thus, the projection modification may include an affine modification. The elements h 7 and h 8 of the matrix H may be elements related to the perspective modification.

[0214] The geometric modification of an image is a method of modifying the image geometry into a specific form. The points in the geometrically modified image corresponding to the points in the image before the geometric modification can be calculated by the geometric modification defined in the matrix. In contrast, homography refers to a method of inversely deriving the mutual geometric modification matrix from two images that respectively have points corresponding to each other.

[0215] Figure 13 is a diagram showing an example of a method for implementing homography according to the present invention.

[0216] Homography can derive the geometric modification relationship between two images based on the recognition of two points located in the two images and corresponding to each other. To this end, feature point matching can be used. The feature points of an image refer to the points in the image that have descriptive features.

[0217] In steps S1301 and S1302, the homography implementation method may extract feature points from the original image and the geometrically modified image. The feature points of the image can be extracted differently according to the extraction method or according to the purpose of use. The points where the brightness value in the image changes significantly, the center points of regions with a specific shape, or the outer corner points of the target in the image can be used as feature points. Feature points can be extracted by using algorithms such as Scale-Invariant Feature Transform (SIFT), Speeded-Up Robust Features (SURF), blob detection, etc.

[0218] In step S1303, the homography implementation method may match the feature points based on the feature points extracted from the original image and the geometrically modified image. Specifically, each extracted feature point is descriptive, and the feature points between the two images can be matched by finding points with similar descriptive information. The matched feature points can be used as the points corresponding to each other between the original image and the geometrically modified image.

[0219] However, the feature point matching may not match points that actually correspond to each other. Therefore, in step S1304, valid feature points among the derived feature points can be selected. The method of selecting valid feature points can vary according to the calculation algorithm. For example, the following methods can be used, such as a method of excluding feature points that do not satisfy the reference line based on the description information, a method of excluding feature points with very low similarity through the distribution of the matching results, or a method using the Random Sample Consensus (RANSAC) algorithm. The homography implementation method can selectively execute step S1304 according to the matching result of the feature points. In other words, step S1304 may not be executed depending on the situation. Alternatively, steps S1303 and S1304 can be combined. Or, the homography implementation method can perform the matching process of the valid feature points without executing steps S1303 and S1304.

[0220] In step S1305, the homography implementation method can derive the relational expression between the original image and the geometrically modified image by using the selected valid points. In step S1306, the homography implementation method can derive the geometric matrix by using the derived formula. Or, the homography implementation method can not execute step S1306 and output the information of the derived formula obtained in step S1305 in a different form except for the geometric modification matrix.

[0221] Figure 14 is an exemplary method for deriving the relational expression between two corresponding points in two images according to the present invention.

[0222] The geometric modification of the image can be performed using a 3×3 matrix H. Therefore, a system of simultaneous equations including the elements h 1 to h 9 of the matrix H can be derived from the matrix formula p’ = Hp. Here, p means a point in the original image, and p’ represents the point corresponding to point p in the geometrically modified image. By dividing all the elements of the matrix H by h 9 , the equation can be simply calculated by fixing H 9 to 1. In addition, the number of unknowns can be reduced from 9 to 8.

[0223] Figure 14 The elements k 1 to k 8 correspond to the values of h 1 to h 8 divided by h 9 . In which h 9 is changed to 1, and h 1 to h 8 are respectively changed to k 1 to k 8 , the geometric matrix can perform the same geometric modification. Therefore, it may be necessary to calculate 8 unknown values. InFigure 14 In [the above], the final formula for a pair of points that match each other in the form of a single point can be expressed in two forms, x' and y'. At least 4 pairs of points that match each other may be required because there are 8 unknown values. However, as described above, a pair of points may not match each other. Or, a pair of points may be mismatched. Such errors can occur even when valid feature points are selected. By using many pairs of points that match each other while calculating the geometric modification matrix, such errors can be reduced. Therefore, considering these features, the number of pairs of points to be used can be determined.

[0224] Figure 15 FIG. [number] is a diagram showing a method for generating a geometrically modified image based on a geometric modification matrix and an original image according to the present invention.

[0225] As Figure 15 shown, by using the light information of points within the original image, the generation of the geometrically modified image can correspond to the generation of the light information of corresponding points within the geometrically modified image. Figure 15 In [the figure], (x0, y0), (x1, y1), and (x2, y2) refer to different points within the original image. Additionally, (x'0, y'0), (x'1, y'1), and (x'2, y'2) are the points within the geometrically corrected image that correspond to (x0, y0), (x1, y1), and (x2, y2), respectively. The function f calculates the corresponding x' coordinate on the x-axis within the geometrically modified image by using the point (x, y) within the original image and additional information α for geometric modification. The function g calculates the corresponding y' coordinate on the y-axis within the geometrically modified image by using the point (x, y) within the original image and additional information β for geometric modification. When expressing (x, y), (x', y'), the function f, and the function g in matrix form, the matrix H can represent the geometric modification method. Therefore, the points corresponding to each other within the original image and the geometrically modified image can be found by using the matrix H.

[0226] Figure 15 The geometric modification method in [the figure] may be problematic in a discrete sampled image signal because light information is only included in points with integer coordinates of the discrete image signal. Therefore, when the points within the geometrically corrected image and corresponding to the points within the original image have real coordinates, the light information of the closest integer coordinates is assigned to the points within the geometrically corrected image. Thus, the light information may be overlapped onto a part of the points within the geometrically corrected image with real coordinates, or the light information may not be assigned. In this case, an inverse mapping can be used.

[0227] Figure 16 FIG. [number] is a diagram showing a method for generating a geometrically modified image by using an inverse mapping according to the present invention.

[0228] Figure 16The dashed rectangular area refers to the actually observed area. Points within the original image corresponding to each point within the dashed rectangular area can be derived. Thus, the light information of the original image can be assigned to all points within the geometrically modified image. However, the point (x3, y3) corresponding to (x'3, y'3) may lie outside the original image. In such a case, the light information of the original image may not be assigned to the point (x'3, y'3). Among the points to which the light information of the original image is not assigned, the adjacent light information of the original image can be assigned. In other words, the light information of the closest point (e.g., (x4, y4)) within the original image can be assigned.

[0229] Figure 17 FIG. is a diagram showing a method of generating a geometrically modified image based on a geometric modification matrix and an original image according to the present invention, where the geometric modification matrix can correspond to geometric modification information.

[0230] In step S1701, the generation method may receive an input original image, a geometric modification matrix, and / or information about the current point of the geometrically modified image. The generation method may calculate the point of the original image corresponding to the current point of the geometrically modified image by using the original image and the geometric modification matrix. The calculated corresponding point of the original image may be a real corresponding point with real coordinates.

[0231] In step S1702, the generation method may determine whether the calculated corresponding point lies within the original image.

[0232] In step S1702, when the calculated corresponding point does not lie within the original image, in step S1703, the generation method may replace the corresponding point with the point closest to the calculated corresponding point within the original image.

[0233] In step S1702, when the calculated corresponding point lies within the original image, the generation method may perform step S1704. When the calculated corresponding point is replaced in step 1703, the generation method may perform step S1704.

[0234] In step S1704, when the corresponding point has real coordinates, the generation method may identify the closest point with integer coordinates. When the corresponding point has integer coordinates, the generation method may skip steps S1704 and S1705 and perform step S1706.

[0235] In step S1705, the generation method may generate the light information of the point with real coordinates by inserting the light information (e.g., pixel value) of the identified point with integer coordinates. As the insertion method, Lanczos interpolation, S-Spline interpolation, or bicubic interpolation can be used.

[0236] In step S1706, the generation method may check whether all points in the geometrically modified image have completed their geometric modification. Then, the generation method may finally output the generated geometrically modified image.

[0237] When it is determined in step S1706 that the geometric modification is not completed, in step S1707, the generation method may change the current point of the geometrically modified image to another point, and steps S1701 to S1706 may be repeated.

[0238] Figure 18 is a diagram showing bilinear interpolation among various interpolation methods explained with reference to Figure 17 an embodiment of the present invention.

[0239] In Figure 18 the real coordinate (x, y) may correspond to Figure 17 the real corresponding point mentioned in step S1704 of Figure 17 . The four points (i, j), (i, j + 1), (i + 1, j), and (i + 1, j + 1) adjacent to the coordinate (x, y) may correspond to

[0240] the closest points with integer coordinates mentioned in step S1704 of

[0241] The I(x, y) may refer to the light information of the point (x, y), such as brightness. a refers to the x-axis distance between i and x, and b refers to the y-axis distance between j and y. 1 - a refers to the x-axis distance between i + 1 and x, and 1 - b refers to the y-axis distance between j + 1 and y. The light information of the point (x, y) may be calculated from the light information of the points (i, j), (i, j + 1), (i + 1, j), and (i + 1, j + 1) by using the ratio of a to 1 - a in the x-axis and the ratio of b to 1 - b in the y-axis. When the inter-frame prediction unit of a video encoder performs motion prediction, the inter-frame prediction unit may predict an encoding target region (current region or current block) within an encoding target picture (current picture) by referring to a reference picture. Here, when the time interval between the reference picture and the encoding target picture is large, or when global motion such as rotation, zooming in, zooming out, or a change in the viewing angle of an object has occurred between the two images, the pixel similarity between the two images decreases. Therefore, the prediction accuracy may decrease, and the encoding efficiency may decrease. In such a case, the encoder may calculate the change in motion between the encoding target picture and the reference picture, and geometrically modify the reference picture such that the reference picture has a form similar to the encoding target picture. The reference picture may be geometrically modified in units of frames, slices, and / or blocks. The picture generated by geometrically modifying the reference picture may be defined as a geometrically modified picture. The motion prediction accuracy is improved by referring to the geometrically modified picture instead of the reference picture.According to the present invention, a reference picture and / or a geometrically modified picture are not limited to the frame size or picture size of a frame configuring a video. In the present invention, a partial area of a reference picture having a picture (frame) size defined in a parameter set may refer to the reference picture. A geometrically modified picture generated by modifying the reference picture or a partial area of the reference picture may have a size corresponding to the size of the image before being modified. In other words, when the image before being modified is a reference picture having a picture (frame) size, its geometrically modified picture may have the picture (frame) size. When the image before being modified is a partial area of a reference picture having a picture (frame) size, the geometrically modified picture may have a size corresponding to the partial area of the reference picture.

[0242] When a video encoder has performed encoding by referring to a geometrically modified picture, a video decoder may receive information required for geometric modification. The information required for geometric modification may refer to geometric modification information. The video decoder may generate a geometrically modified picture from a reference picture based on the received geometric modification information. The video decoder may perform inter prediction by referring to the generated geometrically modified picture.

[0243] Figure 19 is a diagram showing motion prediction, in which a video encoder generates geometric modification information and performs motion prediction by using a reference picture and a geometrically modified picture.

[0244] A geometrically modified picture generator of a video encoder may generate geometric modification information from a reference picture. The geometrically modified picture generator of the video encoder may generate a geometrically modified picture by using the geometric modification information. The video encoder may store the generated geometrically modified picture such that a geometrically modified picture predictor uses the geometrically modified picture as a reference signal. The geometrically modified picture may be stored in at least one of a reconstructed picture buffer DPB, a reference picture list, and a geometrically modified picture buffer of the video encoder. The geometrically modified picture predictor of the video encoder may perform inter prediction by using the generated geometrically modified picture and / or the reference picture. An encoder including the geometrically modified picture generator, the geometrically modified picture buffer, and / or the geometrically modified picture predictor will be described later.

[0245] As Figure 19As shown, geometric modification information can be generated based on an encoded target picture and a reference picture. The reference picture can be selected from a list of reference pictures configured with at least one already decoded reconstructed picture. The reference picture can be a reference picture with global motion. Geometric modification information can be generated by reflecting the global motion between the encoded target picture and the reference picture with global motion. The geometric modification information can be configured or generated in units of the entire image or a part of the image (such as a frame, a slice, a block, etc.). When generating the geometric modification information, the whole or a part of the global motion can be reflected. The global motion can be a motion related to the geometric modification of the above-mentioned image, such as translation, enlargement, reduction, rotation, etc. A geometric modification picture can be generated based on the reference picture and the generated geometric modification information. The geometric modification picture can be a geometric modification picture reflecting the global motion.

[0246] As a method of configuring geometric modification information related to the translation, enlargement, reduction, or rotation of pixels between an encoded target picture and a reference picture, the homography described in the reference can be used. Figure 13 The homography can provide geometric modification information reflecting the rotation, scaling, and translation between two images to explain the two-dimensional geometric modification relationship of a planar target.

[0247] By using the geometric modification information to rotate, enlarge, reduce, or translate an image, Figure 19 the image within the reference picture of can be geometrically modified to Figure 19 the image within the geometric modification picture of. Figure 19 The reference picture of and Figure 19 the geometric modification picture of can have rectangles of different shapes through geometric modification. Figure 19 Any coordinate (a, b) within the reference picture of becomes (a’, b’) through the relational expression derived in, and (a, b) can correspond to Figure 14 the coordinate (a’, b’) within the geometric modification picture of. Figure 19 the coordinate (a’, b’) within the geometric modification picture of.

[0248] Therefore, the video encoder can generate an image similar to the encoded target picture from the reference picture by calculating the geometric modification information between the encoded target picture and the reference picture. Specifically, Figure 19 the similarity between region A within the geometric modification picture of and Figure 19 region B within the encoded target picture of is very high. In other words, the pixel value similarity between region A and region B is very high, so the prediction accuracy of the encoder can be improved by referring to the geometric modification picture while performing motion prediction.

[0249] As described above, when performing inter-frame prediction in a video encoder, the video encoder may refer to a reference picture configured with a decoded picture. However, since the reference picture is not the same as the picture to be encoded, there may be a change in pixel values during the time interval between the reference picture and the picture to be encoded. Therefore, the video encoder preferably refers to a geometrically modified picture generated based on geometric modification information reflecting the change in pixel values between the reference picture and the picture to be encoded. The change in pixel values may include, for example, something caused by global motion. While performing motion prediction, an optimal prediction signal can be generated by referring to the reference picture and the geometrically modified picture.

[0250] Specifically, since the geometrically modified picture is an image reflecting the change in pixel values between the reference picture and the picture to be encoded, the pixel distribution similarity between the picture to be encoded and the geometrically modified picture is very high. However, when new pixels that did not exist in the previous image appear in the image, such as the appearance of a new object, or the error of the geometric modification information is large, errors or noise may appear in the geometrically modified picture. Here, the motion prediction referring to the geometrically modified picture may degrade. Therefore, when performing motion prediction, the first motion prediction is performed by referring to both the reference picture and the geometrically modified picture. Then, the information with high prediction accuracy can be updated as the optimal prediction information. Thus, a reduction in encoding performance caused by noise or errors in the geometrically modified picture can be prevented. The video encoder may encode the motion prediction information with the best encoding efficiency and send the encoded information through a bitstream. Here, the video encoder may not encode the geometric modification information of the area not referred to by the geometrically modified picture. In other words, the encoding efficiency can be improved by encoding only the necessary geometric modification information.

[0251] Figure 20 FIG. is a block diagram showing the configuration of an image encoding apparatus to which another embodiment of the present invention is applied.

[0252] Figure 20 The encoding apparatus shown may include a geometrically modified picture generation unit 2010, a geometrically modified picture predictor 2015, an extended intra-frame prediction unit 2020, a subtractor 2025, a transformation unit 2030, a quantization unit 2040, an entropy encoding unit 2050, an inverse quantization unit 2060, an inverse transformation unit 2070, an adder 2075, a deblocking filter unit 2080, and a sample adaptive offset unit 2090.

[0253] The geometric modification picture generation unit 2010 can generate a geometric modification picture 2012 by calculating geometric modification information, and the calculated geometric modification information reflects the change in pixel values between the coded target picture 2011 and the reference pictures in the reference picture list stored in the reconstructed picture buffer 2013. The generated geometric modification picture 2012 can be stored in the geometric modification picture buffer 2016.

[0254] The geometric modification picture predictor 2015 can include a geometric modification picture buffer 2016 and an inter-frame prediction unit 2017. The geometric modification picture buffer 2016 can store the geometric modification pictures generated in the geometric modification picture generation unit 2010. The inter-frame prediction unit 2017 can perform motion prediction by using the reference pictures in the reference picture list stored in the reconstructed picture buffer 2013 as reference signals. When a geometric modification picture is referred to while performing motion prediction, the geometric modification information used to generate the geometric modification picture can be sent to the entropy coding unit 2050 and encoded in the entropy coding unit 2050.

[0255] The geometric modification picture generation unit 2010 can perform reconfiguration of the geometric modification information, which will be described later. The reconfiguration of the geometric modification information can be performed when generating the geometric modification picture.

[0256] The extended intra-frame prediction unit 2020 can perform extended intra-frame prediction by referring to the already encoded / decoded signals of the current picture and the geometric modification picture. The extended intra-frame prediction unit 2020 will be described later.

[0257] Figure 21 is a diagram showing the configuration and operation of the geometric modification picture generation unit 2010 of the exemplary encoding device according to Figure 20 as shown.

[0258] The geometric modification picture generation unit 2010 can generate a geometric modification picture 2108, which can be used as a reference signal in the geometric modification picture predictor 2015. The geometric modification information configurator 2105 can configure geometric modification information 2106 by receiving the reference picture 2103 of the reference picture list 2102 configured by the reconstructed picture buffer 2101 and the coded target picture 2104. The picture geometric modification calculator 2107 can generate a geometric modification picture 2108 by modifying the reference picture 2103 by using the geometric modification information 2106.

[0259] The geometric modification information configurator 2105 can calculate the change in pixel values between the reference picture 2103 and the encoded target picture 2104, and specifically, calculate the global motion information. The geometric modification information configurator 2105 can find the matching feature points between the two images and calculate the motion information by calculating the changes in translation, rotation, and / or size between the matching feature points. The geometric modification information configurator 2105 can configure and output geometric modification information capable of generating a geometric modification picture based on the calculated global motion information.

[0260] The picture geometric modification calculator 2107 can receive the reference picture 2103 and the geometric modification information 2106 to generate a geometric modification picture 2108. The picture geometric modification calculator 2107 can generate the geometric modification picture 2108 from the reference picture 2103 by using the geometric modification information 2106. The generated geometric modification picture 2108 can be stored in the geometric modification picture buffer 2016 and can be referenced by the inter-frame prediction unit 2017 while performing inter-frame prediction.

[0261] Figure 22 is a diagram showing the configuration and operation of the geometric modification picture predictor 2015 of the example encoding device according to Figure 20 shown.

[0262] The reference picture 2202 in the reference picture list 2201 and the geometric modification picture 2204 in the geometric modification picture buffer 2203 can be input to the inter-frame predictor 2205 and used as a reference signal for motion prediction.

[0263] While performing motion prediction, the inter-frame predictor 2205 can derive the best motion prediction information 2206 by referring to both the reference picture 2202 and the geometric modification picture 2204. The best motion prediction information 2206 can refer to motion prediction information with high prediction accuracy and the best coding efficiency. When the best motion prediction information 2206 is derived by referring to the geometric modification image 2204, the geometric modification information can be encoded and sent through the bitstream.

[0264] Reference Figures 20 to 22 The configuration of the encoding device shown is only one of various embodiments of the present invention and is not limited thereto. Figures 20 to 22 Some configurations of the encoding device shown can be combined or omitted with other configurations. Alternatively, other configurations can be added. Additionally, a part of the multiple configurations included in the geometric modification picture generation unit 2010 and the geometric modification picture predictor 2015 can be configured independently of the geometric modification picture generation unit 2010 and the geometric modification picture predictor 2015. Alternatively, it can be included in a sub-configuration of another configuration or combined with another configuration.

[0265] Figure 23 is a flowchart showing motion prediction according to an embodiment of the present invention.

[0266] In steps S2301 and S2302, an encoding target picture and a reference picture can be specified. The reference picture can be selected from a list of reference pictures.

[0267] In step S2303, geometric modification information can be generated based on the encoding target picture and the reference picture. The geometric modification information can be generated by using the above-described method.

[0268] In step S2304, a geometric modification picture can be generated based on the generated geometric modification information. The generated geometric modification picture can be stored in a geometric modification picture buffer.

[0269] In step S2305, motion prediction can be performed by referring to the reference picture and / or the geometric modification picture.

[0270] In step S2306, the best prediction information can be stored and updated based on a reference signal having the best encoding efficiency. The rate distortion cost (RD Cost) can be used as an index for determining the best encoding efficiency.

[0271] In step S2307, when not all reference pictures are applied to steps S2301 to S2306, steps S2302 to S2306 are repeated. In step S2308, when all reference pictures are applied, the finally determined best motion prediction information and / or geometric modification information can be encoded. When a geometric modification picture is used while performing motion prediction, only the geometric modification information can be encoded and transmitted.

[0272] Figure 24 is a flowchart showing a method for generating a geometric modification picture according to an embodiment of the present invention.

[0273] In step S2401, the generation method can receive an input of an encoding target picture and a reference picture selected from a list of reference pictures.

[0274] In step S2402, the generation method can extract feature points from the two pictures and match the extracted feature points. The extracted feature points can be used to calculate the change in pixel values between the two pictures, specifically, to calculate global motion information. As described above, a feature point can refer to a pixel that can be distinguished from adjacent pixels. In step S2403, geometric modification information reflecting the rotation, magnification, reduction, and / or translation of the feature points within the two images can be calculated by matching the extracted feature points. The geometric modification information can be calculated by using various algorithms. For example, the Figure 13 homography can be used.

[0275] In step S2404, the generation method can generate a geometrically modified picture. By applying the geometric modification information to each pixel of the reference picture, the geometrically modified picture can be generated by deriving each pixel corresponding to each pixel of the reference picture within the geometrically modified picture.

[0276] In step S2405, the generation method can store the generated geometrically modified picture. For example, the generated geometrically modified picture can be stored in a geometrically modified picture buffer. However, it is not limited thereto, and the geometrically modified picture can be stored in a reconstructed picture buffer or a reference picture list.

[0277] Figure 25 is a flowchart showing an inter-frame prediction method according to an embodiment of the present invention.

[0278] In steps S2501 and S2502, the inter-frame prediction method performs motion prediction by referring to a reference picture and motion prediction by referring to a geometrically modified picture. The order of the motion prediction is not limited thereto. In other words, the motion prediction by referring to the geometrically modified picture can be performed first. Alternatively, the two motion predictions can be performed simultaneously. The reference picture can be selected from a reference picture list.

[0279] In step S2503, the inter-frame prediction method can determine which one of the motion prediction by referring to the reference picture and the motion prediction by referring to the geometrically modified picture is more efficient.

[0280] When it is determined in step S2503 to perform the motion prediction by referring to the reference picture, in step S2504, the inter-frame prediction method can store or update the reference picture information as the best prediction information.

[0281] When it is determined in step S2503 to perform the motion prediction by referring to the geometrically modified picture, in step S2505, the inter-frame prediction method can store or update the geometrically modified picture information as the best prediction information.

[0282] In step S2506, when the geometrically modified picture information is finally encoded as the best prediction information, the inter-frame prediction method can encode the geometric modification information.

[0283] Reference Figure 26 and Figure 27 , a method for effectively encoding geometric modification information is described.

[0284] The geometric modification information can be configured with multiple factors, such as Figure 14matrix. Additionally, these factors may be real numbers. When a large number of bits are used to encode a matrix configured with real number factors, the encoding efficiency may decrease. Various methods can be applied to reduce the amount of bits while transmitting geometric modification information. For example, the real number factors can be approximated to integer factors. Alternatively, the geometric modification information can be simplified by reducing a part of the generated geometric modification information. Alternatively, the geometric modification information to be encoded can be predicted from pre-used geometric modification information, and only the residual value between the geometric modification information can be transmitted. In addition to the described methods, various methods can be used to reduce the amount of bits. Alternatively, one or more of the above methods can be applied together.

[0285] Figure 26 is a flowchart showing an encoding method of geometric modification information according to an embodiment of the present invention.

[0286] In step S2601, the encoding method can receive an input of geometric modification information to be encoded.

[0287] In step S2602, the encoding method can simplify the geometric modification information. In step S2603, the encoding method can predict the geometric modification information. Steps S2602 and S2603 can be executed in reverse. Alternatively, one of steps S2602 and S2603 can be selectively executed.

[0288] In step S2602, the geometric modification information can be simplified by approximating the real number factors constituting the geometric modification information to integer factors. The number of bits for expressing these factors can be reduced by approximation. To reduce the number of bits, various operations such as rounding, raising, lowering, discarding, and truncating based on a predetermined number of digits can be used. As an example of approximation, a real number can be converted to an integer.

[0289] In step S2603, the encoding method can predict the geometric modification information based on the previously used geometric modification information. The encoding method can transmit the residual value between the predicted and the previously used information. The previously used geometric modification information can refer to the geometric modification information just used before or the geometric modification information of a previously encoded picture. When multiple geometric modification information is available for predicting the geometric modification information, the encoding method can additionally encode the information indicating the referred geometric modification information. Here, the decoder can determine the referred geometric modification information based on the additionally transmitted information and decode the geometric modification information based on this information. When only specific geometric modification information (e.g., the geometric modification information just used before) is available as the geometric modification information, the encoding method may not additionally transmit the information indicating the referred geometric modification information.

[0290] In step S2604, the encoding method can encode the information required to reconstruct the geometric modification information.

[0291] Figure 27 FIG. is a diagram showing various examples for modifying geometric modification information to reduce the amount of bits for encoding the geometric modification information.

[0292] Figure 27 (1) is an example of converting a real number factor to an integer factor.

[0293] Figure 27 (2) is an example of simplifying the geometric modification information by removing a part of it. Among the removed factors, 0.1 and 0.7 as the left factors are factors for the projection modification described for reference Figure 12 The factors for the projection modification statistically appear at small values close to zero. The factors close to zero have a small influence when generating the geometric modification picture. Therefore, even if omitted, it will not have an adverse effect on the accuracy of the geometric modification picture. In addition, among the three removed factors, the rightmost factor 1 may not be sent to the decoder because it is always fixed to 1 while generating the geometric modification information.

[0294] Figure 27 (3) is an example of configuring geometric modification information B by multiplying the geometric modification information A by a coefficient 2. The geometric modification information A and the coefficient 2 can be sent to the decoder instead of sending the geometric modification information A and B. The modified information B can be reconstructed by sending the geometric modification information A and the coefficient 2.

[0295] Figure 27 (4) is an example of configuring geometric modification information B by using the geometric modification information A as the reference geometric modification information. The geometric modification information A and the remaining geometric modification information B d are encoded. B d can correspond to the difference between the geometric modification information A and B. The decoder can reconstruct the geometric modification information B by adding the geometric modification information A and B d together.

[0296] Figure 28 FIG. is a diagram showing a motion compensation method, in which the decoder generates a geometric modification picture from a reference picture by using the geometric modification information, and performs motion compensation by using the reference picture and the geometric modification picture.

[0297] The video decoder according to the present invention can receive the geometric modification information generated by the video encoder through a bitstream.

[0298] A video decoder may generate a geometrically modified picture by applying geometric modification information to a reference picture selected from a reference picture list. The geometrically modified picture may be generated by a geometrically modified picture generator of the decoder to be described later. The unit of the geometrically modified picture generated by the geometrically modified picture generator may vary according to the unit determined when encoding the geometrically modified picture, and the unit may be a frame, a slice, a block, etc.

[0299] The method of generating a geometrically modified picture by applying geometric modification information to a reference picture may correspond to the method of generating a geometrically modified picture in the encoder as described above. However, different from the video encoder, the decoder receives the geometric modification information through a bitstream. Therefore, the configuration complexity of the video decoder does not increase significantly. The generated geometrically modified picture may be stored in a geometrically modified picture buffer to be described later.

[0300] The video decoder may refer to both the reference picture and the geometrically modified picture for inter - frame prediction of a target picture. The inter - frame prediction referring to the reference picture and / or the geometrically modified picture may be performed by an inter - frame prediction unit of the decoder to be described later. The inter - frame prediction unit of the decoder may perform inter - frame prediction by using the reference picture of the reconstruction picture buffer DPB and / or the geometrically modified picture of the geometrically modified picture buffer as a reference signal.

[0301] Information on whether to use the reference picture or the geometrically modified picture for inter - frame prediction of a target picture or a target region to be decoded may be signaled through a bitstream. Such information may be signaled through additional syntax elements. Alternatively, such information may be signaled by the presence of the geometric modification information.

[0302] Based on the information indicating that the target picture or target region to be decoded is predicted by referring to the geometrically modified picture, the video decoder may receive the geometric modification information, generate the geometrically modified picture, and / or perform inter - frame prediction by referring to the geometrically modified picture.

[0303] Based on the information indicating that the target picture or target region to be decoded is predicted by referring to the reference picture, the video decoder may not receive the geometric modification information, may not generate the geometrically modified picture, and / or may not perform inter - frame prediction by referring to the geometrically modified picture.

[0304] Figure 29 FIG. [FIG. number not provided in the original] is a block diagram showing the configuration of a decoding apparatus to which another embodiment of the present invention is applied.

[0305] Figure 29The decoder shown may include an entropy decoding unit 2910, an inverse quantization unit 2920, an inverse transform unit 2930, a subtractor 2935, a filter unit 2940, an extended intra prediction unit 2950, a geometrically modified picture predictor 2960, and a geometrically modified picture generator 2970. The decoder may output a decoded picture 2980 by receiving a bitstream 2900.

[0306] The geometrically modified picture generator 2970 may generate a geometrically modified picture 2972 by using geometric modification information extracted from the bitstream 2900 and reference pictures in a reference picture list that is stored in a reconstructed picture buffer 2971 and entropy decoded.

[0307] The geometrically modified picture predictor 2960 may be configured with a geometrically modified picture buffer 2961 for storing the geometrically modified picture 2972 and an inter prediction unit 2962.

[0308] The geometrically modified picture 2972 generated in the geometrically modified picture generator 2970 may be stored in the geometrically modified picture buffer 2961. The geometrically modified picture 2972 stored in the geometrically modified picture buffer 2961 may be used as a reference signal in the inter prediction unit 2962.

[0309] The inter prediction unit 2962 may reconstruct a picture to be decoded based on information sent from the encoder, by using a reference picture and / or a geometrically modified picture as a reference signal for motion prediction.

[0310] Figure 30 is a diagram showing Figure 29 the operations and configuration of the geometrically modified picture generator 2970 of the decoding apparatus shown.

[0311] The geometric modification calculator 3005 of the geometrically modified picture generator 2970 may generate a geometrically modified picture 3006 by receiving geometric modification information 3004 and a reference picture 3003. The reference picture 3003 may be selected from a reference picture list 3002 configured in a reconstructed picture buffer 3001. Information for selecting the reference picture 3003 may be included in the bitstream and sent through the bitstream. The information for selecting the reference picture 3003 may be sent as an additional syntax element, or may be explicitly or implicitly included in the geometric modification information 3004.

[0312] As described above, the generated geometrically modified picture 3006 may be output to the geometrically modified picture buffer 2961.

[0313] Figure 31 is a diagram showing Figure 29 the operations and configuration of the geometrically modified picture predictor 2960 of the decoding apparatus shown.

[0314] The inter - frame predictor 3105 of the geometric - modification picture predictor 2960 can perform inter - frame prediction by referring to a reference picture 3102 and / or a geometric - modification picture 3104, and output a prediction signal 3106. Information on whether to use the reference picture 3102 or the geometric - modification picture 3104 or both for inter - frame prediction of the target picture or the target region to be decoded can be signaled via a bitstream. As described above, such information signaling can be performed via additional syntax elements. Alternatively, such information signaling can be performed via the presence of geometric - modification information.

[0315] The reference picture 3102 can be selected from a reference - picture list 3101. The geometric - modification picture 3104 can be selected from a geometric - modification picture buffer 3103. The geometric - modification picture 3104 can be configured with parts required for inter - frame prediction.

[0316] Reference Figures 29 to 31 The configuration of the decoding device shown is merely one of various embodiments of the present invention, but is not limited thereto. Figures 29 to 31 Some configurations of the encoding device shown can be combined with or omitted from other configurations. Alternatively, other configurations can be added. Additionally, a part of the multiple configurations included in the geometric - modification picture generator 2970 and the geometric - modification picture predictor 2960 can be configured independently of the geometric - modification picture generator 2970 and the geometric - modification picture predictor 2960. Alternatively, it can be included in a sub - configuration of another configuration or combined with another configuration.

[0317] Figure 32 is a flowchart showing motion compensation of a decoder according to an embodiment of the present invention.

[0318] In step S3201, the decoder can obtain motion - compensation - related information by parsing a bitstream. The motion - compensation - related information can include at least one of reference - picture information and geometric - modification information. The reference - picture information can be information for specifying one reference picture within the reference pictures included in the reference - picture list. Alternatively, the reference - picture information can be information indicating whether to use a reference image or a geometric - modification picture during motion compensation. Alternatively, the reference - picture information can be information indicating whether to use both a reference image and a geometric - modification picture during motion compensation. Alternatively, the reference - picture information can be information including at least two of the above - mentioned information. The reference - picture information can be signaled via a single syntax element or via multiple single syntax elements. The reference - picture information can be signaled via a method explicitly or implicitly defined by an encoder and a decoder. The use of a reference picture or a geometric - modification picture during inter - frame prediction can be signaled via the presence of geometric - modification information.

[0319] In step S3202, based on the motion compensation related information, the decoder can determine whether a geometrically modified picture is referred to when performing motion compensation. As described above, the decoder can make a decision based on the presence of the geometric modification information and / or the reference picture information. When the decoder uses the presence of the geometric modification information, for example, when receiving the geometric modification information, the decoder can determine to perform motion prediction by referring to the geometrically modified picture.

[0320] If the result in step S3202 is "yes", then in step S3203, the decoder can generate a geometrically modified picture. The geometrically modified picture can be generated based on the reference picture selected from the reference picture list and the geometric modification information parsed from the bitstream. Here, the decoder can generate a part of the geometrically modified picture required for motion compensation.

[0321] In step S3204, the decoder can perform motion compensation by referring to the generated geometrically modified picture.

[0322] Alternatively, if the result in step S3202 is "no", then in step S3205, the decoder can perform motion compensation by referring to the reference picture selected from the reference picture list.

[0323] In step S3203, the reference Figure 32 The explained motion compensation is based on whether to use the geometrically modified picture to generate the geometrically modified picture (S3202). However, it is not limited thereto, and when receiving the geometric modification information, the decoder can generate the geometrically modified picture regardless of its reference.

[0324] As described above, the geometrically modified picture can be generated from the reference picture by using the geometric modification information. Thus, the inter-frame prediction accuracy can be improved by referring to the reference picture and / or the geometrically modified picture while performing inter-frame prediction.

[0325] The prediction using the geometrically modified picture is not limited to inter-frame prediction. For example, intra-frame prediction can be performed by using the geometrically modified picture. Hereinafter, refer to Figures 33 to 37 Describe the extended intra-frame prediction using the geometrically modified picture according to an embodiment of the present invention.

[0326] Figure 33 is a conceptual diagram showing the extended intra-frame prediction according to an embodiment of the present invention. In Figure 33 the current block may refer to the decoding target block.

[0327] As Figure 33As shown in (1), the region that is the decoded region of the current block can be referred to for intra prediction of the current block. In other words, the left region, upper region, upper left region, and / or upper right region of the current block can be referred to for intra prediction. However, since the right region, lower region, lower left region, and / or lower right region of the current block have not been decoded yet, these regions cannot be referred to for intra prediction.

[0328] As Figure 33 shown in (2), for the right region, lower region, lower left region, and / or lower right region of the current block that cannot be referred to, the signal of the decoded picture corresponding to the current picture, or the signal of the geometrically modified picture generated by geometrically modifying the decoded picture, can be referred to as the reference signal for intra prediction. The geometrically modified picture can be a geometrically modified picture generated by reconfiguring the reference picture of the current picture using geometric modification information such that the reconfigured reference picture is similar to the current picture.

[0329] As Figure 33 shown in (3), by using Figure 33 all the reference signals of (1) and 33(2), all the reference signals of the region surrounding the current block can be obtained. Therefore, intra prediction of the current block can be performed in all directions (the left, right, upper left, upper right, right, lower, lower left, and / or lower right regions of the current block). Therefore, the intra prediction efficiency can be improved.

[0330] Figure 34 is a diagram showing the operation and configuration of an extended intra prediction unit according to an embodiment of the present invention.

[0331] Figure 34 The extended intra prediction unit can correspond to Figure 20 the extended intra prediction unit of the encoder shown in

[0332] The extended intra prediction unit may include a reference signal selector 3404 and a prediction executor 3405. The reference signal selector 3404 may receive the current block 3403 to be decoded. Additionally, the reference signal selector 3404 may receive the current picture 3401 and / or the geometrically modified picture 3402 that serves as a reference signal for intra prediction. The received current picture 3401 may be a signal of a region that was encoded / decoded earlier than the current block 3403. The received geometrically modified picture 3402 may be a reference image of the current picture 3401. Alternatively, the received geometrically modified picture 3402 may be a geometrically modified picture generated by geometrically modifying a reference picture of the current picture 3401 using geometric modification information. The information about the current block 3403 received by the reference signal selector 3404 may be information about the position and / or size of the current block 3403 within the current picture 3401. The reference signal selector 3404 may select a reference signal from the current picture 3401 and the geometrically modified picture 3402. Alternatively, the reference signal selector 3404 may select the current picture 3401 or the geometrically modified picture 3402. For example, when performing intra prediction from the right region of the current block 3403, the reference signal selector 3404 may select the geometrically modified picture 3402 as the reference picture.

[0333] When a reference picture is selected, the prediction executor 3405 may perform intra prediction and generate a predicted block (prediction signal 3406) of the current block 3403. Additionally, the decoder may generate intra prediction information 3407 required for intra prediction. The intra prediction information 3407 may include an intra prediction direction or a reference signal, etc.

[0334] Intra prediction may be performed for all available reference signals, and the reference signal with the best efficiency may be selected.

[0335] The extended intra prediction unit of the decoder may selectively refer to the decoded region of the current picture or the geometrically modified picture (or reference picture) based on the intra prediction information sent from the encoder. For example, when the intra prediction direction indicated by the intra prediction information is the right region of the current block, the extended intra prediction unit may perform intra prediction by using the geometrically modified picture. When referring to the geometrically modified picture, the decoder may generate only the geometrically modified picture. As described above, the information about whether to use the geometrically modified picture may be determined by the intra prediction direction. Alternatively, the information about whether to use the geometrically modified picture may be determined by the presence of geometric modification information. Alternatively, the information about whether to use the geometrically modified picture may be signaled by one or more additional syntax elements, or may be signaled using a method explicitly or implicitly defined by the encoder and the decoder.

[0336] In addition, the method for generating a geometrically modified picture to be referenced during intra prediction according to the present invention may be the same as the method for generating a geometrically modified picture to be referenced during inter prediction according to the present invention. In addition, the method for configuring and using geometric modification information may be the same as the method for inter prediction.

[0337] Figure 35 FIG. is a diagram showing extended intra prediction according to an embodiment of the present invention.

[0338] In step S3501, the extended intra prediction selects a reference region to be used for intra prediction. The reference region can be selected from all directions of the current block. The intra prediction direction can be expressed as directions subdivided at a predetermined interval, and each direction can be expressed as an index. In addition, all other possible expressions can be used. The encoder can perform intra prediction in all directions sequentially or in parallel. In the decoder, a single intra prediction direction can be specified based on the intra prediction information.

[0339] In step S3502, the extended intra prediction can determine whether the reference region indicated by the intra prediction direction within the current picture is valid. In other words, if the selected reference region is within the region that has been encoded / decoded before the current block, the reference region is valid; otherwise, the reference region is invalid. The result of step S3502 can be determined by selecting the reference region in step S3501. For example, it can be determined whether the reference region is valid for the current picture according to the angle, index, or direction used to indicate the selected reference region.

[0340] In step S3504, when the reference region is valid, the extended intra prediction can perform intra prediction of the current block by referring to the already decoded signal. In step S3503, when the reference region is invalid, the extended intra prediction can perform intra prediction of the current block by referring to the geometrically modified picture.

[0341] Figure 36 FIG. is a diagram showing an intra prediction direction according to the present invention.

[0342] In Figure 36 , prediction modes 0 and 1 are non-angle modes. Prediction mode 0 refers to the intra-plane mode, and prediction mode 1 refers to the DC mode (intra DC). In Figure 36 (1), prediction modes 2 to 34 refer to angle modes. In Figure 36 (2), prediction modes 2 to 65 refer to angle modes. In Figure 36 , the arrow can indicate the reference signal for intra prediction of the current block.

[0343] Figure 36 (1) is a diagram showing an intra prediction direction when only the already decoded signal of the current picture can be referenced. As Figure 36(1) As shown, for intra prediction of prediction modes 2 to 34, only the left, upper, upper-left, and / or upper-right signals of the current block can be referred to.

[0344] Figure 36 (2) is a diagram showing the intra prediction direction when the decoded signal, decoded picture, or geometrically modified picture of the current picture can be referred to. As Figure 36 (2) shows, for intra prediction of prediction modes 2 to 65, not only the left, upper, upper-left, and / or upper-right signals of the current block can be referred to, but also the right, lower, lower-left, and / or lower-right signals of the current block can be referred to.

[0345] Figure 37 is a diagram showing an embodiment of performing extended intra prediction according to the present invention.

[0346] In Figure 37 , the adjacent pixels in the current picture refer to the pixels in the current picture that are referred to when performing intra prediction. The adjacent pixels in the geometrically modified picture refer to the pixels in the geometrically modified picture (or reference picture) corresponding to the current picture. The area without any information refers to the area that does not have pixel information that can be referred to when performing intra prediction among the adjacent pixels. Figure 37 (a) and the arrow with a line in 37(b) indicate the intra prediction direction when performing intra prediction by referring to the adjacent pixels in the current picture. Here, the intra prediction direction can correspond to Figure 36 the prediction modes 2 to 34. Figure 37 (d) The arrow with a dotted line indicates the intra prediction direction when performing intra prediction by referring to the adjacent pixels in the geometrically modified picture (or reference picture) corresponding to the current picture. Here, the intra prediction direction can correspond to Figure 36 the prediction modes 35 to 65. Figure 37 (c) The arrow with a dotted line indicates the direction in which intra prediction is not possible due to the absence of a reference signal.

[0347] In Figure 37 (a) and 37(c), intra prediction is performed by referring to the adjacent pixels in the current picture. Since there is a reference signal, intra prediction in the intra prediction direction shown in Figure 37 (a) can be performed. However, since the reference signal does not exist, intra prediction in the intra prediction direction shown in Figure 37 (c) may not be performed.

[0348] In Figure 37 (b) and Figure 37 (d), intra prediction is performed by referring to not only the adjacent pixels in the current picture but also the adjacent pixels in the geometrically modified picture (or reference picture) corresponding to the current picture. Figure 37Intra prediction in the intra prediction direction shown in (b) can be performed by referring to adjacent pixels within the current picture. Figure 37 Intra prediction in the intra prediction direction shown in (d) can be performed by referring to adjacent pixels within the geometrically modified picture. According to Figure 37 the intra prediction of (b) and 37(d), the prediction accuracy can be improved by performing intra prediction that refers to adjacent pixels within the current picture and adjacent pixels within the geometrically modified picture, thereby improving the coding efficiency.

[0349] According to the present invention, while generating the geometrically modified picture, the geometrically modified picture can be configured in a coding unit (CU), a prediction unit (PD), a slice unit, or a picture frame unit. When a change in pixel value occurs in the picture frame unit of the slice unit, it is preferable to generate the geometrically modified picture in a large unit such as the slice unit or the picture frame unit rather than a small unit such as the CU or the PU. Therefore, duplicate geometric modification information generated when generating the geometrically modified picture in a small unit can be avoided, and it can be more efficient in terms of complexity. Additionally, when the geometrically modified picture is not referred to in the region of the frame unit or the slice unit, information on whether the geometrically modified picture is referred to is included in the frame unit or the slice unit. However, the encoder / decoder may not configure unnecessary modification information. For example, a picture parameter set (PPS), syntax configuration, and semantics of the slice header can be applied.

[0350] In Tables 1 and 2, the "modification_image_generation_flag" can be information on whether to use the geometrically modified picture as a reference picture when predicting motion information. The "modification_image_generation_flag" can be included in a sequence unit, a frame unit, a slice unit, etc., and includes information on whether the geometrically modified picture is used for each unit. When the geometrically modified picture is used as a reference picture for each unit, the "modification_image_generation_flag" can be set to "1". When the geometrically modified picture is not used as a reference picture, the "modification_image_generation_flag" can be set to "0". Alternatively, the "modification_image_generation_flag" can be configured to have the opposite value in each case. The "modification_image_generation_flag" can be used as information on whether the geometrically modified picture is used in each unit, or as information on whether the geometric modification information is included in the bitstream.

[0351] Tables 1 and 2 are examples of syntax configurations when each of the "modification_image_generation_flag" is sent through the PPS and the slice header.

[0352] [Table 1]

[0353]

[0354]

[0355] [Table 2]

[0356]

[0357]

[0358] "image_modification_info" can be a syntax element related to geometric modification information and configured with a matrix or non-linear data. "image_modification_info" can be a syntax element present in the bitstream when using geometrically modified pictures in each unit such as a frame unit, slice unit, etc. (when "modification_image_generation_flag" is 1). Tables 1 and 2 show examples when "modification_image_generation_flag" has geometric modification information configured in matrix form. As described above, the geometric modification information matrix can include 9 coefficients. Since a fixed value can be used for one of the 9 coefficients, 8 coefficients can be sent. By approximation or prediction of the coefficients, the matrix coefficients to be sent can be reduced to 8 or less. In Tables 1 and 2, the matrix coefficients can be sent in the form of "image_modification_info[x][y][z]". Here, x can refer to the index of the reference picture list, y can refer to the index of the reference picture within the reference picture list, and z can refer to the index of the matrix coefficient. Therefore, all reference pictures may each have a different matrix.

[0359] When the encoder determines that the geometrically modified picture configured in the coding unit and / or prediction unit is the best case, as shown in Tables 3 and 4, "modification_image_generation_flag" and "image_modification_info" can be included as syntax elements of the CU and / or PU. In Tables 3 and 4, the CU can include the reference picture information it refers to, and thus may require a single geometric modification information for geometrically modifying the corresponding reference picture. Therefore, different from Tables 1 and 2, the matrix coefficients are sent in the form of "image_modification_info[z]". Here, z refers to the index of the matrix coefficient.

[0360] Tables 3 and 4 are examples of the syntax configuration when signaling each of "modification_image_generation_flag" and "image_modification_info" through the CU and PU.

[0361] [Table 3]

[0362]

[0363] [Table 4]

[0364]

[0365] When the PPS or slice header includes information on geometric modification information as shown in Tables 1 and 2, the geometric modification information may not be sent in the units of the CU or PU. Therefore, the CU or PU can be configured as shown in Tables 5 and 6. In Tables 5 and 6, a "modification_image_reference_flag" of 1 may indicate that the geometrically modified picture is used as a reference signal, or a "modification_image_reference_flag" of 0 may indicate that the geometrically modified picture is not used as a reference signal. Alternatively, the "modification_image_reference_flag" can be configured to have the opposite value in each case. When the geometrically modified picture is used as reference information in the CU or PU (or when the "modification_image_generation_flag" is "1"), the CU or PU may include "reference_modification_info" of the information required for reference. The information required for reference may include a reference index, a residual signal, etc.

[0366] Tables 5 and 6 are examples of syntax configurations when each "reference_modification_info" is signaled through the CU and PU.

[0367] [Table 5]

[0368]

[0369]

[0370] [Table 6]

[0371]

[0372] Unidirectional prediction or bidirectional prediction can be used in motion prediction. When a geometrically modified picture is used in motion prediction, the motion prediction type can be signaled in the CU and PU through the syntax configurations of Tables 7 and 8.

[0373] Specifically, a geometrically modified picture can be generated by using geometric modification information signaled in an upper layer such as a slice header or PPS. In addition, prediction types such as bidirectional prediction using a geometrically modified picture, unidirectional prediction using a geometrically modified picture, or prediction without using a geometrically modified picture can be signaled in the CU or PU. In addition, information on whether a geometrically modified picture is used in each direction can be signaled through flag information.

[0374] In Tables 7 and 8, "modification_image_reference_type" can be a flag indicating one of bidirectional prediction using geometrically modified pictures, unidirectional prediction using geometrically modified pictures, or prediction without using geometrically modified pictures. "ref_0_modification_flag" can be a flag indicating whether the current decoded picture references a geometrically modified picture of a reference picture within the first reference picture list of two reference picture lists. "ref_1_modification_flag" can be a flag indicating whether the current decoded picture references a geometrically modified picture of a reference picture within the second reference picture list of two reference picture lists. When "ref_X_modification_flag" (where X can be 0 or 1) is set to true (1), prediction can be performed by referring to the geometrically modified picture of the corresponding reference picture. When "ref_X_modification_flag" is set to false (0) (where X can be 0 or 1), prediction can be performed without referring to the geometrically modified picture of the corresponding reference picture.

[0375] When "modification_image_modification_type" in a CU or a PU is "NONE_USE", prediction in both directions can be performed by setting "ref_0_modification_flag" and "ref_1_modification_flag" to false, without referring to the geometrically modified pictures of the reference pictures.

[0376] When "modification_image_modification_type" is "REF_0_USE", prediction in list 0 direction can be performed by setting "ref_0_modification_flag" to true and "ref_1_modification_flag" to false, referring to the geometrically modified pictures.

[0377] When "modification_image_modification_type" is "REF_1_USE", prediction in list 1 direction can be performed by setting "ref_0_modification_flag" to false and "ref_1_modification_flag" to true, referring to the geometrically modified pictures.

[0378] When "modification_image_modification_type" is "BOTH_USE", prediction in both directions can be performed by setting "ref_0_modification_flag" and "ref_1_modification_flag" to true, referring to the geometrically modified pictures of the reference pictures.

[0379] Tables 7 and 8 are examples of syntax configurations when each of "modification_image_reference_type" is signaled via a CU and a PU.

[0380] [Table 7]

[0381]

[0382]

[0383] [Table 8]

[0384]

[0385] As an application example of the present invention, geometrically modified pictures with geometric modifications in the upper layer (such as slice headers, PPS, etc.) and geometric modification information sent from the upper layer can be corrected and used in the CU layer and / or the PU layer. As shown in Tables 9 and 10, the CU layer and / or the PU layer may include "modification_image_using_revision_flag" and "modification_revision_info".

[0386] "modification_image_using_revision_flag" being 1 may indicate that the CU layer or the PU layer includes information to be corrected. Here, "modification_revision_info" can be additionally signaled.

[0387] "modification_image_using_revision_flag" being 0 may indicate that the CU layer or the PU layer does not include information to be corrected. Since there is no information to correct, "modification_revision_info" may not be signaled.

[0388] "modification_image_using_revision_flag" and "modification_revision_info" can be directly signaled as syntax elements, as shown in the examples of Tables 9 and 10. Alternatively, "modification_image_using_revision_flag" and "modification_revision_info" can be explicitly and / or implicitly signaled according to predefined rules defined in the encoder and the decoder.

[0389] Tables 9 and 10 are examples of syntax configurations when each of "modification_image_using_revision_flag" and "modification_revision_info" is signaled via a CU and a PU.

[0390] [Table 9]

[0391]

[0392] [Table 10]

[0393]

[0394] In the above embodiments, the method is described based on a flowchart having a series of steps or units. However, the present invention is not limited to the order of these steps, but rather some steps may be executed simultaneously with other steps or in a different order. Additionally, those of ordinary skill in the art should understand that the steps in the flowchart do not exclude each other, and other steps may be added to the flowchart, or some steps may be deleted from the flowchart without affecting the scope of the present invention.

[0395] The above description includes examples of various aspects. Of course, it is not possible to describe every possible combination of components or methods for the purpose of describing each aspect, but those of ordinary skill in the art will recognize that many further combinations and permutations are possible. Therefore, this specification is intended to cover all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims.

[0396] A computer-readable storage medium may include program instructions, data files, data structures, etc., either alone or in combination. The program instructions recorded on the computer-readable storage medium may be program instructions specifically designed and constructed for the present invention or any program instructions known to those skilled in the computer software art. Examples of computer-readable storage media include magnetic recording media (such as hard disks, floppy disks, and magnetic tapes); optical data storage media (such as CD-ROMs or DVD-ROMs); magneto-optical media (such as optical disks); hardware devices specifically configured to store and implement program instructions (such as read-only memory (ROM), random access memory (RAM), and flash memory). Examples of program instructions include not only machine language code formatted by a compiler but also high-level language code that can be implemented by a computer using an interpreter. The hardware device may be configured to operate through one or more software modules (or vice versa) to perform the processing according to the present invention.

[0397] Although the present invention has been described in terms of specific items such as detailed elements and limited embodiments and drawings, they are provided only to assist in a more general understanding of the present invention, and the present invention is not limited to the above embodiments. Those skilled in the art to which the present invention pertains will understand that various modifications and changes can be made from the above description.

[0398] Therefore, the spirit of the present invention should not be limited to the above embodiments, and the entire scope of the appended claims and their equivalents will fall within the scope and spirit of the present invention.

[0399] Industrial Applicability

[0400] The present invention can be used for encoding / decoding images.

Claims

1. A method for encoding an image, the method comprising: Generate geometry modification information of the current block; determining a first geometry modification flag of a current parameter set, the first geometry modification flag indicating whether geometry modification is enabled for a block using the current parameter set; When the first geometry modification flag is encoded with a value indicating that geometry modification is enabled for the block using the current parameter set, determining a second geometry modification flag indicating whether geometry modification is used for motion compensation of the current block among the blocks using the current parameter set; determining geometry modification type information of the current block, the geometry modification type information indicating a number of affine references used for the motion compensation of the current block; Determining predicted geometry modification information of the current block based on reference geometry modification information of the current block; determining a reference geometry modification index indicating the reference geometry modification information among a plurality of reference geometry modification information, the reference geometry modification information comprising a plurality of the affine references as many as the number of the affine references; determining residual geometry modification information indicating a difference between the predicted geometry modification information of the current block and the geometry modification information of the current block; as well as encoding a bit stream including the first geometry modification flag, the second geometry modification flag, the reference geometry modification index, the residual geometry modification information, and the geometry modification type information, The geometric modification information enables the prediction block of the current block to be derived from an affine-modified shape region in a reference picture referenced by the current block. 2 . The method of claim 1 , wherein encoding the geometry modification information comprises reconfiguring the geometry modification information, and the reconfiguring the geometry modification information comprises simplifying or predicting the geometry modification information. 3 . The method of claim 1 , wherein the geometry modification information is generated based on a change in pixel information between a current picture including the current block and the reference picture. 4 . The method of claim 3 , wherein the geometry modification information is generated based on matching information between feature points extracted from each of the current picture and the reference picture.

5. The method of claim 1, wherein the method further comprises: identifying a point within the reference picture, the point corresponding to a point within the geometrically modified picture; as well as The pixel information of the corresponding point in the reference picture is set as the pixel information of the point in the geometry modified picture. 6 . The method of claim 5 , wherein when a point corresponding to the point within the geometry-modified picture does not exist within the reference picture, a point within the reference picture that is closest to the corresponding point is identified as the corresponding point.

7. A method for decoding an image, the method comprising: Obtaining a first geometry modification flag of a current parameter set from a bitstream, the first geometry modification flag indicating whether geometry modification is enabled for a block using the current parameter set; When the first geometry modification flag indicates that geometry modification is enabled for the block using the current parameter set, obtaining a second geometry modification flag of a current block among the blocks using the current parameter set from the bitstream, the second geometry modification flag indicating whether geometry modification is used for motion compensation of the current block; Obtaining geometry modification type information of the current block from the bitstream, the geometry modification type information indicating the number of affine references used for the motion compensation of the current block; Obtaining, from the bitstream, a reference geometry modification index indicating one reference geometry modification information among a plurality of reference geometry modification information; Retrieving residual geometry modification information from the bitstream, the residual geometry modification information indicating a difference between predicted geometry modification information of the current block and geometry modification information of the current block; When the second geometry modification flag indicates that geometry modification is used for motion compensation of the current block, determining reference geometry modification information of the current block from previously used geometry modification information, the reference geometry modification information including a plurality of the affine references indicated by the geometry modification type information; Determining the predicted geometry modification information of the current block based on the reference geometry modification information specified by the reference geometry modification index; Determining the geometry modification information of the current block based on the predicted geometry modification information of the current block and the residual geometry modification information of the current block; as well as generating a prediction block of the current block by performing inter-frame prediction based on a reference picture of the current block and the geometry modification information of the current block, The geometric modification information enables the prediction block to be derived from an affine-modified shape region in the reference picture referenced by the current block.

8. The method of claim 7, wherein the method further comprises: identifying a point within the reference picture, the point corresponding to a point within the geometrically modified picture; as well as The pixel information of the corresponding point in the reference picture is set as the pixel information of the point in the geometry modified picture.

9. The method of claim 8, wherein when a point corresponding to the point within the geometry-modified picture does not exist within the reference picture, a point within the reference picture that is closest to the corresponding point is identified as the corresponding point.

10. The method of claim 8, wherein when the corresponding point within the reference picture has real coordinates, one or more points within the reference picture each having integer coordinates and adjacent to the corresponding point are identified, and pixel information of the corresponding point within the reference picture is derived based on the pixel information of each of the one or more points having integer coordinates.

11. The method of claim 8, wherein the generating of the geometry modified picture is performed in at least one of a picture, a slice, a tile, a coding unit, and a prediction unit.

12. A method for storing a bit stream, comprising: Generate a bit stream including a first geometry modification flag, a second geometry modification flag, geometry modification type information, a reference geometry modification index, and residual geometry modification information; as well as storing the bitstream, in, The first geometry modification flag indicates whether geometry modification is enabled for the block using the current parameter set, the second geometry modification flag indicating whether geometry modification is used for motion compensation of a current block of the blocks using the current parameter set; The geometry modification type information indicates the number of affine references used for the motion compensation of the current block; The reference geometry modification index indicates a piece of reference geometry modification information among a plurality of reference geometry modification information; The reference geometry modification information specified by the geometry modification index is used to determine the predicted geometry modification information of the current block and includes a plurality of the affine references indicated by the geometry modification type information; The residual geometry modification information indicates a difference between the predicted geometry modification information of the current block and the geometry modification information of the current block; The geometric modification information enables the prediction block to be derived from an affine modified region in a reference picture referenced by the current block, and The second geometry modification flag is determined when the first geometry modification flag indicates that geometry modification is enabled for the block using the current parameter set.

13. A method for decoding an image, the method comprising: Receiving a bitstream including a plurality of flag syntax elements corresponding to a plurality of reference pictures, respectively, wherein each of the plurality of flag syntax elements indicates whether the bitstream includes a global geometry modification information syntax element for the corresponding reference picture; If a flag syntax element indicates that the bitstream includes global geometry modification information for a reference picture corresponding to the flag syntax element, reading from the bitstream a global geometry modification information syntax element associated with the reference picture corresponding to the flag syntax element; reconstructing geometry modification information of the reference picture corresponding to the flag syntax element based on the geometry modification information syntax element of the reference picture corresponding to the flag syntax element; geometrically modifying the reference picture corresponding to the flag syntax element based on the reconstructed geometry modification information of the reference picture corresponding to the flag syntax element to generate a prediction block; Generate a residual block by performing dequantization and inverse transformation on the residual data; reconstructing a current block based on the residual block and the prediction block; as well as A loop filter is applied to the current block.

14. The method of claim 13, wherein reconstructing the geometry modification information comprises: The geometry modification information is further reconstructed based on previously stored geometry modification information.

15. The method of claim 13, wherein geometrically modifying the reference picture corresponding to the flag syntax element comprises: identifying a point within the reference picture that corresponds to the flag syntax element, wherein the point within the reference picture that corresponds to the flag syntax element corresponds to a point within the prediction block; as well as Pixel information of the prediction block is determined based on the one point in the reference picture corresponding to the flag syntax element.

16. The method of claim 15, wherein the geometry modification information represents a relationship between the one point within the prediction block and the one point within the reference picture corresponding to the flag syntax element.

17. A method for encoding an image, the method comprising: determining a plurality of flag syntax elements corresponding to a plurality of reference pictures, respectively, wherein each of the plurality of flag syntax elements indicates whether a bitstream includes a global geometry modification information syntax element for the corresponding reference picture; If the flag syntax element is determined to indicate that the bitstream includes global geometry modification information for a reference picture corresponding to the flag syntax element, determining a global geometry modification information syntax element associated with the reference picture corresponding to the flag syntax element; determining, based on the geometry modification information syntax element of the reference picture corresponding to the flag syntax element, geometry modification information for the reference picture corresponding to the flag syntax element; geometrically modifying the reference picture corresponding to the flag syntax element based on the geometry modification information of the reference picture corresponding to the flag syntax element to generate a prediction block; Generate a residual block by performing dequantization and inverse transformation on the residual data; reconstructing a current block based on the residual block and the prediction block; as well as A loop filter is applied to the current block.

Images