Method and apparatus for encoding / decoding image

Abstract

To provide a video encoding / decoding method and apparatus for performing intra-prediction based on the availability of a reference pixel so as to improve encoding performance.SOLUTION: An intra-prediction method for a color copy mode checks a reference pixel region designated to obtain correlation information, determines a reference pixel processing setting based on a determination of availability of the reference pixel region, and performs intra-prediction according to the determined reference pixel processing. In addition, a candidate list for motion information prediction of the current block is generated, a control point vector of the current block is derived based on the candidate list and a candidate index, a motion vector of the current block is derived based on the control point vector of the current block, and inter prediction is performed on the current block using the motion vector.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The present invention relates to a video encoding / decoding method and apparatus. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images has been increasing in various application fields, and as a result, highly efficient image compression techniques have been discussed.

[0003] There are various video compression technologies, such as inter-frame prediction technology that predicts pixel values ​​contained in a current picture from pictures before or after the current picture, intra-frame prediction technology that predicts pixel values ​​contained in a current picture using pixel information within the current picture, and entropy coding technology that assigns short codes to values ​​that occur frequently and long codes to values ​​that occur less frequently.Using these video compression technologies, video data can be effectively compressed and transmitted or stored. Summary of the Invention [Problem to be solved by the invention]

[0004] SUMMARY OF THE INVENTION In order to solve the above problems, an object of the present invention is to provide an image encoding / decoding method and apparatus for performing intra prediction based on the availability of reference pixels.

[0005] An object of the present invention is to provide an inter-prediction method and device.

[0006] SUMMARY OF THE INVENTION An object of the present invention is to provide a method and apparatus for sub-block-based motion compensation.

[0007] The present invention aims to provide a method and apparatus for determining affine candidates. [Means for solving the problem]

[0008] To achieve the above object, a method for performing intra-frame prediction according to one embodiment of the present invention may include a step of checking a reference pixel area designated to obtain correlation information, a step of determining a reference pixel processing setting based on a determination of the usability of the reference pixel area, and a step of performing intra-frame prediction using the determined reference pixel processing setting.

[0009] The video encoding / decoding method and apparatus according to the present invention may generate a candidate list for motion information prediction of a current block, derive a control point vector of the current block based on the candidate list and a candidate index, derive a motion vector of the current block based on the control point vector of the current block, and perform inter prediction on the current block using the motion vector.

[0010] In the video encoding / decoding device according to the present invention, the candidate list can include a plurality of affine candidates.

[0011] In the video encoding / decoding apparatus according to the present invention, the affine candidates may include at least one of spatial candidates, temporal candidates, or configuration candidates.

[0012] In the video encoding / decoding apparatus according to the present invention, the motion vector of the current block may be derived in units of sub-blocks of the current block.

[0013] In the video encoding / decoding apparatus according to the present invention, the spatial candidates may be determined in consideration of whether the boundary of the current block is adjacent to a coding tree block boundary (CTU boundary).

[0014] In the video encoding / decoding apparatus according to the present invention, the configuration candidates may be determined based on at least two combinations of control point vectors corresponding to the respective corners of the current block. [Effects of the Invention]

[0015] When the method of performing intra prediction based on the availability of reference pixels according to the present invention is used, coding performance can be improved.

[0016] According to the present invention, it is possible to improve video encoding / decoding performance by using inter prediction based on an affine model.

[0017] According to the present invention, it is possible to improve the accuracy of prediction by performing inter prediction in units of sub-blocks.

[0018] According to the present invention, the encoding / decoding efficiency of inter prediction can be improved by efficient affine candidate determination. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram of a video encoding and decoding system according to an embodiment of the present invention; [Figure 2] 1 is a block diagram illustrating the configuration of a video encoding device according to an embodiment of the present invention. [Figure 3] 1 is a block diagram of a video decoding device according to an embodiment of the present invention; [Figure 4] 1 is an exemplary diagram illustrating an intra-frame prediction mode according to an embodiment of the present invention; [Figure 5] FIG. 1 is a conceptual diagram illustrating intra-frame prediction for directional and non-directional modes according to an embodiment of the present invention. [Figure 6] FIG. 10 is a conceptual diagram illustrating intra-screen prediction in a color copy mode according to an embodiment of the present invention. [Figure 7] 1 is an exemplary diagram showing corresponding blocks and adjacent areas of each color space in relation to a color copy mode according to an embodiment of the present invention; [Figure 8] 10 is an exemplary diagram illustrating area setting for obtaining correlation information in a color copy mode according to an embodiment of the present invention; [Figure 9] 1 is an exemplary diagram illustrating a configuration of reference pixels used in intra-frame prediction according to an embodiment of the present invention; [Figure 10] 1 is a conceptual diagram illustrating blocks adjacent to a current block of intra-frame prediction according to an embodiment of the present invention; [Figure 11] 10A and 10B are exemplary diagrams illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention; [Figure 12] 10A and 10B are exemplary diagrams illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention; [Figure 13] 10A and 10B are exemplary diagrams illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention; [Figure 14] 1 is a flowchart illustrating an intra-screen prediction method in a color copy mode according to an embodiment of the present invention. [Figure 15] 10A and 10B are exemplary diagrams illustrating prediction in a color copy mode according to an embodiment of the present invention; [Figure 16] 10A and 10B are exemplary diagrams illustrating prediction in a color copy mode according to an embodiment of the present invention; [Figure 17] 10A and 10B are exemplary diagrams illustrating prediction in a color copy mode according to an embodiment of the present invention; [Figure 18] 10 is a flowchart of a process for performing correction in a color copy mode according to an embodiment of the present invention. [Figure 19] 10A and 10B are exemplary diagrams illustrating types of filters applied to a pixel to be corrected according to an embodiment of the present invention; [Figure 20] 10A and 10B are exemplary diagrams illustrating types of filters applied to a pixel to be corrected according to an embodiment of the present invention; [Figure 21] 1 is a diagram illustrating an inter-frame prediction method according to an embodiment of the present invention; [Figure 22] FIG. 10 is a diagram illustrating a method for deriving affine candidates from spatial / temporal neighboring blocks according to an embodiment of the present invention. [Figure 23] 10 is a diagram illustrating a method for deriving candidates configured by combining motion vectors of spatial / temporal neighboring blocks according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0020] The video encoding / decoding method and apparatus according to the present invention may include a step of checking a reference pixel area designated for obtaining correlation information, a step of determining a reference pixel processing setting based on a determination of the usability of the reference pixel area, and a step of performing intra-frame prediction using the determined reference pixel processing setting.

[0021] The video encoding / decoding method and apparatus according to the present invention may generate a candidate list for motion information prediction of a current block, derive a control point vector of the current block based on the candidate list and a candidate index, derive a motion vector of the current block based on the control point vector of the current block, and perform inter prediction on the current block using the motion vector.

[0022] In the video encoding / decoding device according to the present invention, the candidate list can include a plurality of affine candidates.

[0023] In the video encoding / decoding apparatus according to the present invention, the affine candidates may include at least one of spatial candidates, temporal candidates, or configuration candidates.

[0024] In the video encoding / decoding apparatus according to the present invention, the motion vector of the current block may be derived in units of sub-blocks of the current block.

[0025] In the video encoding / decoding apparatus according to the present invention, the spatial candidates may be determined in consideration of whether the boundary of the current block is adjacent to a coding tree block boundary (CTU boundary).

[0026] In the video encoding / decoding apparatus according to the present invention, the configuration candidates may be determined based on a combination of at least two control point vectors corresponding to the respective corners of the current block.

[0027] Although the present invention can be modified in various ways and can have various embodiments, specific embodiments will be illustrated in the drawings and described in detail. However, it should be understood that this is not intended to limit the present invention to the specific embodiments, and that the present invention includes all modifications, equivalents, and alternatives that fall within the spirit and technical scope of the present invention.

[0028] Terms such as "first," "second," "A," and "B" may be used to describe various elements, but the elements should not be limited to these terms. These terms are used solely to distinguish one element from another. For example, a first element may be referred to as a "second element," and similarly, a second element may be referred to as a "first element," without departing from the scope of the present invention. The term "and / or" includes a combination of multiple related items or any of multiple related items.

[0029] When a component is referred to as being "coupled" or "connected" to another component, it should be understood that the component may be directly coupled or connected to the other component, but that there may be additional components in between. On the other hand, when a component is referred to as being "directly coupled" or "directly connected" to another component, it should be understood that there are no additional components in between.

[0030] The terms used in the present invention are used only to describe specific embodiments and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In the present invention, terms such as "comprise" or "have" are intended to specify the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0031] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with the meaning they have in the context of the relevant art, and unless expressly defined in this invention, they should not be interpreted as having an ideal or overly formal meaning.

[0032] Typically, an image may be configured with one or more color spaces depending on the color format of the image. Depending on the color format, it may be configured with one or more pictures of a certain size or one or more pictures of different sizes. For example, in a YCbCr color configuration, color formats such as 4:4:4, 4:2:2, 4:2:0, and monochrome (composed of only Y) can be supported. For example, in the case of YCbCr 4:2:0, it may be configured with one luminance component (Y in this example) and two chrominance components (Cb / Cr in this example). Here, the composition ratio of the chrominance component to the luminance component may be 1:2 horizontally and vertically. For example, in the case of 4:4:4, the horizontal and vertical dimensions may have the same composition ratio. When configured with one or more color spaces as in the above example, the picture may be divided into each color space.

[0033] Images can be classified into I, P, B, etc. depending on the image type (e.g., picture type, slice type, tile type, etc.). The I image type may refer to an image that is encoded / decoded by itself without using a reference picture, the P image type may refer to an image that is encoded / decoded using a reference picture but allows only forward prediction, and the B image type may refer to an image that is encoded / decoded using a reference picture and allows forward / backward prediction, but some of these types may be combined (combining P and B) or other image types may be supported depending on the encoding / decoding setting.

[0034] FIG. 1 is a conceptual diagram of a video encoding and decoding system according to an embodiment of the present invention.

[0035] Referring to FIG. 1, the video encoding device 105 and the decoding device 100 may be a user terminal such as a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a PlayStation Portable (PSP), a wireless communication terminal, a smartphone, or a TV, or a server terminal such as an application server or a service server, and may include various devices including a communication device such as a communication modem for communicating with various devices or wired and wireless communication networks, memories 120, 125 for storing various programs and data for inter or intra prediction for encoding or decoding video, or processors 110, 115 for executing programs, performing calculations, and control.

[0036] In addition, the video encoded into a bitstream by the video encoding device 105 can be transmitted to the video decoding device 100 in real time or non-real time via a wired or wireless communication network such as the Internet, a short-range wireless communication network, a wireless LAN network, a WiBro network, or a mobile communication network, or via various communication interfaces such as a cable or a Universal Serial Bus (USB), and can be decoded by the video decoding device 100 to restore the video and be played back. In addition, the video encoded into a bitstream by the video encoding device 105 can be transmitted from the video encoding device 105 to the video decoding device 100 via a computer-readable recording medium.

[0037] The video encoding device and the video decoding device may be separate devices, but may be implemented as a single video encoding / decoding device. In this case, some components of the video encoding device may be substantially the same technical elements as some components of the video decoding device, and may be implemented to include at least the same structure or perform at least the same function.

[0038] Therefore, in the following detailed description of technical elements and their operating principles, redundant descriptions of corresponding technical elements will be omitted. Also, since a video decoding device corresponds to a computer device that applies the video encoding method performed in a video encoding device to decoding, the following description will focus on the video encoding device.

[0039] The computer device may include a memory for storing a program or software module implementing the video encoding method and / or the video decoding method, and a processor coupled to the memory for executing the program. Here, the video encoding device may be referred to as an encoder, and the video decoding device may be referred to as a decoder.

[0040] FIG. 2 is a block diagram showing the configuration of a video encoding device according to an embodiment of the present invention.

[0041] Referring to FIG. 2, the video encoding device 20 may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, an encoded picture buffer 240, and an entropy encoding unit 245.

[0042] The prediction unit 200 may be implemented using a prediction module, which is a software module, and may generate a predicted block for a block to be coded using intra prediction or inter prediction. The prediction unit 200 may generate a predicted block by predicting a current block to be coded in an image. In other words, the prediction unit 200 may generate a predicted block having a predicted pixel value of each pixel generated by predicting a pixel value of each pixel of a current block to be coded in an image using intra prediction or inter prediction. In addition, the prediction unit 200 may transmit information required for generating the predicted block, such as information about a prediction mode such as an intra prediction mode or an inter prediction mode, to an encoding unit, and cause the encoding unit to encode information about the prediction mode. Here, a processing unit in which prediction is performed, a prediction method, and a processing unit in which specific content is determined may be determined according to encoding / decoding settings. For example, a prediction method, a prediction mode, etc. may be determined in a prediction unit, and prediction may be performed in a transform unit.

[0043] The inter prediction unit may be classified into a translation motion model and an affine motion model depending on the motion prediction method. The translation motion model performs prediction by considering only translation, while the affine motion model may perform prediction by considering not only translation but also motions such as rotation, perspective, and zoom in / out. Assuming unidirectional prediction, the translation motion model may require one motion vector, while the affine motion model may require one or more motion vectors. In the affine motion model, each motion vector may be information applied to a predetermined position of the current block, such as the upper left vertex or upper right vertex of the current block, and the position of the area to be predicted of the current block may be obtained in pixel units or sub-block units using the corresponding motion vector. Depending on the motion model, the inter prediction unit may apply some processes, described below, in common and some processes individually.

[0044] The inter prediction unit may include a reference picture construction unit, a motion estimation unit, a motion compensation unit, a motion information determination unit, and a motion information encoding unit. The reference picture construction unit may include pictures encoded before or after the current picture in reference picture lists (L0, L1). A prediction block may be obtained from the reference pictures included in the reference picture lists, and the current picture may also be constructed from reference pictures according to an encoding setting and included in at least one of the reference picture lists.

[0045] In the inter prediction unit, the reference picture construction unit may include a reference picture interpolation unit, and may perform an interpolation process for a small number of unit pixels depending on the interpolation accuracy. For example, an interpolation filter based on an 8-tap DCT may be applied to the luminance component, and an interpolation filter based on a 4-tap DCT may be applied to the chrominance component.

[0046] In the inter-frame prediction unit, the motion estimation unit is a process of searching for blocks that have a high correlation with the current block using a reference picture, and various methods such as FBMA (Full search-based block matching algorithm) and TSS (Three step search) can be used. The motion compensation unit is a process of obtaining a prediction block through the motion estimation process.

[0047] In the inter prediction unit, the motion information determination unit may perform a process for selecting optimal motion information for a current block, and the motion information may be coded using motion information coding modes such as skip mode, merge mode, and competition mode. The modes may be configured by combining modes supported by a motion model, and examples thereof may include skip mode (motion), skip mode (non-motion), merge mode (motion), merge mode (non-motion), competition mode (motion), and competition mode (non-motion). Depending on the coding setting, some of the modes may be included in the candidate group.

[0048] The motion information coding mode can obtain a predicted value of motion information (motion vector, reference picture, prediction direction, etc.) of the current block from at least one candidate block, and when two or more candidate blocks are supported, optimal candidate selection information can be generated. The skip mode (without residual signal) and the merge mode (with residual signal) can use the predicted value as the motion information of the current block, and the competitive mode can generate difference value information between the motion information of the current block and the predicted value.

[0049] The candidate set for the motion information predictor of the current block can have various adaptive configurations depending on the motion information coding mode. The candidate set can include motion information of blocks spatially adjacent to the current block (e.g., left, upper, upper left, upper right, lower left blocks, etc.), motion information of blocks temporally adjacent to the current block, or a combination of spatial and temporal candidates.

[0050] The temporally adjacent blocks include blocks in other images that correspond to (or are equivalent to) the current block, and may refer to blocks located on the left, right, upper, lower, upper left, upper right, lower left, lower right, etc., around the current block. The combined motion information may refer to information obtained as an average, median, etc. from the motion information of spatially adjacent blocks and the motion information of temporally adjacent blocks.

[0051] There may be a priority order for constructing a group of motion information predictor candidates. The order of steps included in constructing a group of predictor candidates may be determined according to the priority order, and the construction of the group of candidates may be completed when the number of candidates (determined by the motion information coding mode) is reached according to the priority order. Here, the priority order may be determined in the order of motion information of spatially adjacent blocks, motion information of temporally adjacent blocks, and combined motion information of spatial and temporal candidates, but other variations are also possible.

[0052] For example, among spatially adjacent blocks, the candidate group may include blocks in the order of left-top-top-top-bottom-left-top block, etc., and among temporally adjacent blocks, the candidate group may include blocks in the order of bottom-right-middle-right-bottom block, etc.

[0053] The subtraction unit 205 may subtract the predicted block from the current block to generate a residual block. In other words, the subtraction unit 205 may calculate the difference between the pixel value of each pixel of the current block to be encoded and the predicted pixel value of each pixel of the predicted block generated by the prediction unit to generate the residual block, which is a residual signal in block form. In addition, the subtraction unit 205 may generate the residual block in units other than the block unit obtained by the block division unit described below.

[0054] The transform unit 210 can transform a signal belonging to the spatial domain into a signal belonging to the frequency domain, and the signal obtained through the transform process is called a transformed coefficient. For example, the transform unit 210 can obtain a transformed block having transform coefficients by transforming a residual block having a residual signal transmitted from the subtraction unit. The input signal is determined by a coding setting and is not limited to a residual signal.

[0055] The transform unit may transform the residual block using a transform technique such as a Hadamard transform, a discrete sine transform (DST based-transform), or a discrete cosine transform (DCT based-transform), but is not limited to these, and may use various transform techniques that are improvements or modifications of these.

[0056] At least one of the transformation techniques may be supported, and at least one detail transformation technique may be supported for each transformation technique, where the detail transformation techniques may be configured such that a portion of the basis vectors differs in each transformation technique.

[0057] For example, in the case of DCT, one or more detail transformation techniques of DCT-I to DCT-VIII can be supported, and in the case of DST, one or more detail transformation techniques of DST-I to DST-VIII can be supported. A group of candidate transformation techniques can be configured by configuring some of the detail transformation techniques. For example, DCT-II, DCT-VIII, and DST-VII can be configured as a group of candidate transformation techniques to perform transformation.

[0058] The transformation can be performed in the horizontal and vertical directions. For example, a one-dimensional transformation can be performed in the horizontal direction using the DCT-II transformation technique, and a one-dimensional transformation can be performed in the vertical direction using the DST-VIII transformation technique, resulting in a total two-dimensional transformation, which can convert pixel values ​​in the spatial domain into the frequency domain.

[0059] The transformation can be performed using a single fixed transformation technique, or by adaptively selecting a transformation technique according to the encoding / decoding setting. Here, in the adaptive case, the transformation technique can be selected using an explicit or implicit method. In the explicit case, selection information for each transformation technique or transformation technique set applied in the horizontal and vertical directions can be generated in units such as blocks. In the implicit case, the encoding setting can be defined according to the image type (I / P / B), color components, block size, shape, intra-frame prediction mode, etc., and a predefined transformation technique can be selected accordingly.

[0060] Also, some of the conversions may be omitted depending on the encoding settings, which means that one or more horizontal / vertical units may be omitted explicitly or implicitly.

[0061] In addition, the transform unit can transmit the information necessary to generate the transform block to the encoder to encode it, and then record the resulting information into a bitstream and transmit it to the decoder, and the decoding unit of the decoder can parse the information and use it in the inverse transform process.

[0062] The quantization unit 215 may quantize an input signal. Here, a signal obtained through the quantization process is called a quantized coefficient. For example, a residual block having residual transform coefficients transmitted from a transform unit may be quantized to obtain a quantized block having quantized coefficients. The input signal is determined by a coding setting and is not limited to residual transform coefficients.

[0063] The quantization unit may quantize the transformed residual block using a quantization technique such as Dead Zone Uniform Threshold Quantization, Quantization Weighted Matrix, etc., but is not limited thereto, and may use various improved and modified quantization techniques.

[0064] The quantization process can be omitted depending on the encoding setting. For example, the quantization process can be omitted (including the inverse process) depending on the encoding setting (e.g., a quantization parameter of 0, i.e., a lossless compression environment). As another example, the quantization process can be omitted if the compression performance through quantization is not achieved due to the characteristics of the image. Here, the region in the quantization block (M×N) where the quantization process is omitted can be the entire region or a partial region (M / 2×N / 2, M×N / 2, M / 2×N, etc.), and the quantization omission selection information can be determined implicitly or explicitly.

[0065] The quantization unit can transmit the information necessary to generate a quantization block to the encoding unit to encode it, and then record the resulting information into a bitstream and transmit it to the decoder, and the decoding unit of the decoder can parse the information and use it in the inverse quantization process.

[0066] Although the above example has been described under the assumption that the residual block is transformed and quantized by the transform unit and the quantization unit, it is also possible to transform the residual signal to generate a residual block having transform coefficients without performing the quantization process, to perform only the quantization process without converting the residual signal of the residual block into transform coefficients, or to perform neither the transform nor the quantization process, which can be determined by the encoder settings.

[0067] The inverse quantization unit 220 inverse quantizes the residual block quantized by the quantization unit 215. That is, the inverse quantization unit 220 inverse quantizes the quantized frequency coefficient sequence to generate a residual block having frequency coefficients.

[0068] The inverse transform unit 225 inversely transforms the residual block dequantized by the inverse quantization unit 220. That is, the inverse transform unit 225 inversely transforms the frequency coefficients of the dequantized residual block to generate a residual block having pixel values, i.e., a restored residual block. Here, the inverse transform unit 225 can perform the inverse transform by using the transform method used by the transform unit 210 in reverse.

[0069] The adder 230 reconstructs the current block by adding the predicted block predicted by the predictor 200 and the residual block reconstructed by the inverse transformer 225. The reconstructed current block is stored as a reference picture (or reference block) in the coding picture buffer 240 and can be used as a reference picture when encoding the next block of the current block or other future blocks or pictures.

[0070] The filter unit 235 may include one or more post-processing filter processes, such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF). The deblocking filter can remove block artifacts that occur at boundaries between blocks from a restored picture. The ALF can perform filtering based on a value obtained by comparing the restored image with the original image after the blocks are filtered through the deblocking filter. The SAO can restore an offset difference between the original image and a residual block to which the deblocking filter has been applied in pixel units. Such post-processing filters can be applied to a restored picture or block.

[0071] The coded picture buffer 240 may store blocks or pictures reconstructed through the filter unit 235. The reconstructed blocks or pictures stored in the coded picture buffer 240 may be provided to a prediction unit 200 that performs intra prediction or inter prediction.

[0072] The entropy coding unit 245 scans the generated quantized frequency coefficient sequence in various scanning methods to generate a quantized coefficient sequence, and outputs the quantized coefficient sequence by encoding it using an entropy coding technique, etc. The scan pattern can be set to one of various patterns such as zigzag, diagonal, raster, etc. Also, the entropy coding unit 245 can generate coded data including coding information transmitted from each component and output it as a bitstream.

[0073] FIG. 3 is a block diagram of a video decoding device according to an embodiment of the present invention.

[0074] Referring to FIG. 3, the video decoding device 30 may include an entropy decoding unit 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, an adder / subtractor 325, a filter 330, and a decoded picture buffer 335.

[0075] The prediction unit 310 may further include an intra-frame prediction module and an inter-frame prediction module.

[0076] First, when a video bitstream is received from the video encoding device 20 , it can be transmitted to the entropy decoding unit 305 .

[0077] The entropy decoder 305 can decode the bitstream and generate decoded data including quantized coefficients and decoding information transmitted to each component.

[0078] The prediction unit 310 may generate a prediction block based on data received from the entropy decoding unit 305. Here, the prediction unit 310 may also construct a reference picture list using a default construction method based on reference images stored in the decoded picture buffer 335.

[0079] The inter prediction unit may include a reference picture construction unit, a motion compensation unit, and a motion information decoding unit, some of which may perform the same process as the encoder, and some of which may perform the reverse process.

[0080] The inverse quantization unit 315 can inverse quantize the quantized transform coefficients provided as a bitstream and decoded by the entropy decoding unit 305 .

[0081] The inverse transform unit 320 may apply an inverse transform technique such as an inverse DCT, an inverse integer transform, or a similar concept to the transform coefficients to generate residual blocks.

[0082] Here, the inverse quantization unit 315 and the inverse transform unit 320 may be implemented in various ways, inversely performing the processes performed by the transform unit 210 and the quantization unit 215 of the video encoding device 20. For example, the inverse quantization unit 315 and the inverse transform unit 320 may use the same processes and inverse transforms as those shared by the transform unit 210 and the quantization unit 215, or may inversely perform the transform and quantization processes using information about the transform and quantization processes from the video encoding device 20 (e.g., transform size, transform shape, quantization type, etc.).

[0083] The residual block that has undergone the inverse quantization and inverse transform processes can be added to the prediction block derived by the prediction unit 310 to generate a reconstructed image block. This addition can be performed by the adder / subtractor 325.

[0084] The filter 330 may apply a deblocking filter to the restored image block to remove blocking artifacts if necessary, and may also use other loop filters before or after the decoding process to improve video quality.

[0085] The restored and filtered image blocks can be stored in a decoded picture buffer 335 .

[0086] Although not shown in the drawings, the video encoding / decoding apparatus may further include a picture dividing unit and a block dividing unit.

[0087] The picture division unit can divide (or partition) a picture into at least one processing unit such as a color space (e.g., YCbCr, RGB, or XYZ), a tile, a slice, or a basic coding unit (or a maximum coding unit), and the block division unit can divide the basic coding unit into at least one processing unit (e.g., encoding, prediction, transform, quantization, entropy, and in-loop filter unit, etc.).

[0088] The basic coding unit can be obtained by dividing a picture at regular intervals in the horizontal and vertical directions. Based on this, division into tiles, slices, etc. can be performed, but is not limited to this. The division units such as tiles and slices can be configured as integer multiples of the basic coding block, but exceptional cases may occur for division units located at image boundaries. For this reason, the size of the basic coding block may be adjusted.

[0089] For example, a picture can be partitioned into basic coding units and then split into the units, or a picture can be partitioned into the units and then split into basic coding units. In the present invention, the partition and division order of each unit are assumed to be the former case, but the present invention is not limited thereto, and the latter case is also possible depending on the encoding / decoding setting. In the latter case, the size of the basic coding unit can be modified to be adaptive depending on the division unit (tile, etc.). That is, it means that basic coding blocks having different sizes for each division unit can be supported.

[0090] In the present invention, the following examples will be described based on the case where a picture is divided into basic coding units. The above-mentioned basic setting may mean a case where a picture is not divided into tiles or slices, or a case where a picture is one tile or one slice. However, it should be understood that the various embodiments described below can be applied in the same way or with modifications to a case where each division unit (tile, slice, etc.) is first divided and then divided into basic coding units according to the obtained unit (i.e., a case where each division unit is not an integer multiple of the basic coding unit).

[0091] Among the division units, a slice may be composed of a set of at least one block that is contiguous according to a scan pattern, a tile may be composed of a rectangular set of spatially adjacent blocks, and other additional division units may be supported and configured according to their definition. Slices and tiles may be division units supported for purposes such as parallel processing, and for this reason, references between division units may be restricted (i.e., not referenceable).

[0092] For slices, division information for each unit can be generated using information about the starting position of consecutive blocks, and for tiles, information about horizontal and vertical division lines or tile position information (e.g., upper left, upper right, lower left, lower right positions) can be generated.

[0093] Here, slices and tiles can be divided into multiple units according to encoding / decoding settings.

[0094] For example, some units may be a unit containing configuration information that affects the encoding / decoding process (i.e., a tile header or a slice header), and some units can be a unit that does not contain configuration information, or some unit is a unit that cannot refer to other units during the encoding / decoding process, and some units can be a unit that can be referred to. Also, some units is another unit Can be a hierarchical relationship involving some units is another unit can have an equal relationship with

[0095] Here, A and B can be a slice and a tile (or a tile and a slice), or A and B can consist of one slice and one tile. For example, A can be a slice / tile <type 1> and B can be a slice / tile <type 2>.

[0096] Here, Type 1 and Type 2 can each be a single slice or tile, or Type 1 can be multiple slices or tiles (including Type 2) (a set of slices or tiles), and Type 2 can be a single slice or tile.

[0097] As already mentioned above, the present invention will be described assuming that a picture is composed of one slice or tile, but if two or more division units occur, the above description can be applied to the following embodiments. Also, A and B are examples of characteristics that a division unit may have, and examples in which A and B are combined are also possible.

[0098] Meanwhile, the image may be divided into blocks of various sizes by the block dividing unit. Here, a block may be composed of one or more blocks (e.g., one luminance block and two chrominance blocks) according to a color format, and the size of the block may be determined according to the color format. For convenience of explanation, the following description will be given based on a block of one color component (luminance component).

[0099] Although the following description focuses on one color component, it should be understood that it can be changed and applied to other color components in proportion to the ratio of the color format (for example, in the case of YCbCr4:2:0, the ratio of the width and height of the luminance component and chrominance component is 2:1). Also, it should be understood that while block division that depends on other color components (for example, in the case of Cb / Cr, depending on the result of Y block division) is possible, independent block division is also possible for each color component. Also, it is possible to use one common block division setting (taking into account the proportionality to the length ratio), but it should also be understood that individual block division settings can be used for each color component.

[0100] A block may have a variable size such as M×N (M and N are integers such as 4, 8, 16, 32, 64, 128, etc.) and may be a unit (coding block) for performing coding. In particular, it may be a basic unit for prediction, transformation, quantization, entropy coding, etc., and is generally referred to as a block in the present invention. Here, the term "block" does not only refer to a rectangular block but should be understood as a broad concept including regions of various shapes such as triangles and circles, but the present invention will mainly describe the case of a rectangular block.

[0101] The block division unit can be set in association with each component of the video encoding device and decoding device, and the size and shape of the block can be determined through this process. Here, the set block can be defined differently depending on the component, and can correspond to a prediction block in the case of a predictor, a transform block in the case of a transformer, a quantization block in the case of a quantizer, etc. However, without being limited thereto, additional block units can be defined according to other components. In the present invention, although it is assumed that the input and output of each component are blocks (i.e., rectangular), some components can have inputs / outputs of other shapes (e.g., square, triangular, etc.).

[0102] The size and shape of the initial (or starting) block of the block division unit can be determined from the higher unit. For example, in the case of a coding block, the basic coding block can be the initial block, and in the case of a prediction block, the coding block can be the initial block. Also, in the case of a transformation block, the coding block or the prediction block can be the initial block, which can be determined by the encoding / decoding setting.

[0103] For example, if the coding mode is intra, the prediction block may be a superordinate unit of the transform block, and if the coding mode is inter, the prediction block may be a unit independent of the transform block. An initial block, which is a starting unit of division, may be divided into smaller-sized blocks, and once an optimal size and shape are determined by dividing the block, the block may be determined as an initial block of the lower unit. The initial block, which is a starting unit of division, may be considered as an initial block of the upper unit. Here, the upper unit may be a coding block, and the lower unit may be a prediction block or a transform block, but is not limited thereto. Once the initial block of the lower unit is determined as in the above example, a division process may be performed to find a block of optimal size and shape like the upper unit.

[0104] In summary, the block division unit can divide a basic coding unit (or a maximum coding unit) into at least one coding unit (or a sub-coding unit). Furthermore, the coding unit can be divided into at least one prediction unit and at least one transform unit. The coding unit can be divided into at least one coding block, and the coding block can be divided into at least one prediction block and at least one transform block. The prediction unit can be divided into at least one prediction block, and the transform unit can be divided into at least one transform block.

[0105] Here, some blocks may be combined with other blocks to perform a single division process. For example, when a coding block and a transform block are combined into a single unit, a division process is performed to obtain an optimal block size and shape, which may be the optimal size and shape of not only the coding block but also the transform block. Alternatively, a coding block and a transform block may be combined into a single unit, a prediction block and a transform block may be combined into a single unit, a coding block, a prediction block and a transform block may be combined into a single unit, or other block combinations are possible. However, whether or not to combine blocks is not applied collectively within an image (picture, slice, tile, etc.), but may be adaptively determined according to detailed conditions of the block unit (e.g., image type, coding mode, block size / shape, prediction mode information, etc.).

[0106] As described above, when a block having an optimal size and shape is found, mode information (e.g., partition information, etc.) for the block can be generated. The mode information can be recorded in a bitstream together with information generated in a component to which the block belongs (e.g., prediction-related information, transformation-related information, etc.) and transmitted to a decoder, where it can be parsed in the same level unit and used in the video decoding process.

[0107] The following describes a division method, and for convenience of explanation, it is assumed that the initial block is square, but this is not limited to this, as it can be applied equally or similarly when the initial block is rectangular.

[0108] Although various methods for block division can be supported, the present invention will be described with emphasis on tree-based division, and at least one tree division can be supported. Here, the tree method can be a quad tree (QT), a binary tree (BT), a ternary tree (TT), etc. Supporting one tree method is called single tree division, and supporting two or more tree methods is called multi-tree method.

[0109] Quad tree division refers to a method of dividing a block into two parts horizontally and vertically, binary tree division refers to a method of dividing a block into two parts in one direction, horizontally or vertically, and ternary tree division refers to a method of dividing a block into three parts horizontally or vertically.

[0110] In the present invention, when a block before division is M×N, it is assumed that quad tree division divides it into four M / 2×N / 2 blocks, binary tree division divides it into two M / 2×N or M×N / 2 blocks, and ternary tree division divides it into M / 4×N / M / 2×N / M / 4×N or M×N / 4 / M×N / 2 / M×N / 4 blocks. However, the division results are not limited to the above, and various variations are possible.

[0111] Depending on the encoding / decoding configuration, one or more tree partitioning methods may be supported, for example, quad tree partitioning may be supported, or quad tree partitioning and binary tree partitioning may be supported, or quad tree partitioning and ternary tree partitioning may be supported, or quad tree partitioning, binary tree partitioning and ternary tree partitioning may be supported.

[0112] The above example is an example in which the basic partitioning method is a quad tree, and binary tree partitioning and ternary tree partitioning are included as additional partitioning methods depending on whether other trees are supported, but various modifications are possible. Here, information on whether other trees are supported (bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, etc., which can have a value of 0 or 1, where 0 means not supported and 1 means supported) can be implicitly determined by encoding / decoding settings or explicitly determined in units of sequences, pictures, slices, tiles, etc.

[0113] The division information can include information about whether division is possible (tree_part_flag, or qt_part_flag, bt_part_flag, tt_part_flag, bt_tt_part_flag. It can have a value of 0 or 1, where 0 means no division and 1 means division). In addition, depending on the division method (binary tree, ternary tree), information about the division direction (dir_part_flag, or bt_dir_part_flag, tt_dir_part_flag, bt_tt_dir_part_flag. It can have a value of 0 or 1, where 0 means <horizontal> and 1 means <vertical>) can be added, which can be information that can be generated when division is performed.

[0114] When multiple tree divisions are supported, various division information configurations are possible. The following describes an example of how division information is configured at a single depth level (i.e., the supported division depth is set to one or more, and recursive division is possible, but for the sake of convenience, this example will be described).

[0115] In example (1), information on whether or not division is possible is confirmed. If division is not to be carried out, division is terminated.

[0116] If splitting is to be performed, the selection information for the split type (for example, tree_idx. If it is 0, it is QT, if it is 1, it is BT, if it is 2, it is TT) is checked. Here, the split direction information is further checked depending on the selected split type, and the next step is proceeded to (if additional splitting is possible due to reasons such as the split depth not reaching the maximum, it starts again from the beginning, and if splitting is not possible, it ends the splitting).

[0117] In example (2), information on whether or not the partial tree method (QT) can be split is checked, and then the next step is performed. If splitting is not to be performed, information on whether or not the partial tree method (BT) can be split is checked. If splitting is not to be performed, information on whether or not the partial tree method (TT) can be split is checked. If splitting is not to be performed, the splitting is terminated.

[0118] If a partial tree type (QT) split is to be performed, proceed to the next step. If a partial tree type (BT) split is to be performed, check the split direction information and proceed to the next step. If a partial tree split type (TT) split is to be performed, check the split direction information and proceed to the next step.

[0119] In example (3), information on whether or not splitting is possible for some tree methods (QT) is checked. If splitting is not to be performed, information on whether or not splitting is possible for some tree methods (BT and TT) is checked. If splitting is not to be performed, splitting is terminated.

[0120] If a partial tree-type (QT) division is to be performed, the process proceeds to the next step. If a partial tree-type (BT and TT) division is to be performed, the process checks the division direction information and proceeds to the next step.

[0121] The above examples show cases where tree division priority exists (examples 2 and 3) or does not exist (example 1), but various variations are possible. Also, while the above examples illustrate cases where the division of the current stage is independent of the division results of the previous stage, it is also possible for the division of the current stage to be dependent on the division results of the previous stage.

[0122] For example, in cases 1 to 3, if some tree-based division (QT) was performed in the previous stage and then moved to the current stage, the same tree-based division (QT) can be supported in the current stage.

[0123] On the other hand, if you have moved to the current stage after performing other tree-based divisions (BT or TT) without performing partial tree-based divisions (QT) in the previous stage, it is also possible to set partial tree-based divisions (BT and TT) to be supported in subsequent stages, including the current stage, except for partial tree-based divisions (QT).

[0124] In this case, the tree structure supporting block division is adaptive, which means that the above-mentioned division information structure can also be configured differently (assuming the example below is Example 3). That is, in the above example, if division of some tree methods (QT) was not performed in the previous step, the division process can be performed in the current step without considering some tree methods (QT). Also, division information for related tree methods (e.g., information on whether or not to divide, division direction information, etc.) is also available. In this example, <qt>In this case, the information about whether or not the data can be divided can be removed.

[0125] The above example is a case of adaptive partition information configuration when block partitioning is allowed (e.g., the block size is within the range between the maximum and minimum values, and the partition depth of each tree method does not reach the maximum depth <allowable depth>), but adaptive partition information configuration is also possible when block partitioning is restricted (e.g., the block size is not within the range between the maximum and minimum values, and the partition depth of each tree method reaches the maximum depth).

[0126] As described above, in the present invention, tree-based partitioning can be performed in a recursive manner. For example, if the partition flag of a coding block with a partition depth of k is 0, the coding block is encoded into a coding block with a partition depth of k, and if the partition flag of a coding block with a partition depth of k is 1, the coding block is encoded into N sub-coding blocks (where N is an integer equal to or greater than 2, such as 2, 3, or 4) with a partition depth of k+1 according to a partitioning scheme.

[0127] The sub-coding block can be further set as a coding block (k+1) and divided into a sub-coding block (k+2) through the above process, and such a hierarchical division method can be determined by division settings such as the division range and the allowable division depth.

[0128] Here, the bitstream structure for expressing the partition information can be selected from one or more scanning methods. For example, the bitstream of the partition information can be configured based on the order of partition depth or based on whether or not the partition is possible.

[0129] For example, when based on the order of division depth, it is a method of acquiring division information at the current level depth based on the first block and then acquiring division information at the next level depth, and when based on whether division is possible, it means a method of preferentially acquiring additional division information for divided blocks based on the first block, and other additional scanning methods can be considered. In the present invention, it is assumed that a bitstream of division information is constructed based on whether division is possible.

[0130] As described above, various cases for block division have been described, and fixed or adaptive settings for block division can be supported.

[0131] Here, the setting for block partitioning may explicitly include related information in units of a sequence, a picture, a slice, a tile, etc. Alternatively, the block partitioning setting may be implicitly determined by the encoding / decoding setting. Here, the encoding / decoding setting may be configured by one or a combination of two or more of various encoding / decoding elements such as an image type (I / P / B), a color component, a partition type, and a partition depth.

[0132] In an image encoding method according to an embodiment of the present invention, intra prediction may be configured as follows. The intra prediction of the predictor may include a reference pixel configuration step, a prediction block generation step, a prediction mode determination step, and a prediction mode encoding step. In addition, the image encoding device may be configured to include a reference pixel configuration unit, a prediction block generation unit, and a prediction mode encoding unit that implement the reference pixel configuration step, the prediction block generation step, the prediction mode determination step, and the prediction mode encoding step. Some of the above-described steps may be omitted or other steps may be added, and the steps may be arranged in a different order than the above-described order.

[0133] FIG. 4 is an exemplary diagram illustrating an intra-frame prediction mode according to an embodiment of the present invention.

[0134] Referring to Figure 4, 67 prediction modes are configured as a prediction mode candidate group for intra prediction. It is assumed that 65 of the prediction modes are directional modes and 2 are non-directional modes (DC, Planar), but various configurations are possible without being limited thereto. Here, the directional modes may be classified by gradient (e.g., dy / dx) or angle information (degree). Furthermore, all or some of the prediction modes may be included in a prediction mode candidate group for a luminance component or a chrominance component, and additional modes may be included in the prediction mode candidate group.

[0135] In the present invention, the direction of the directional mode may mean a straight line, and a curved directional mode may also be configured as a prediction mode. In addition, the non-directional mode may include a DC mode in which a predicted block is obtained from the average (or weighted average, etc.) of pixels of neighboring blocks (e.g., left, upper, upper left, upper right, lower left block, etc.) adjacent to the current block, and a planar mode in which a predicted block is obtained by linear interpolation of pixels of neighboring blocks.

[0136] Here, in the case of DC mode, the reference pixels used to generate the predicted block can be obtained from blocks consisting of various combinations such as the left side, the top side, the left side + top side, the left side + bottom left side, the top side + top right side, and the left side + top + bottom left side + top right side, and the reference pixel acquisition block position can be determined according to the encoding / decoding setting defined by the image type, color component, block size / shape / position, etc.

[0137] Here, in the case of planar mode, pixels used to generate a predicted block can be obtained from a region composed of reference pixels (e.g., left side, upper side, upper left side, upper right side, lower left side, etc.) and a region not composed of reference pixels (e.g., right side, lower side, lower right side, etc.), and in the case of a region not composed of reference pixels (i.e., not coded), it can be obtained implicitly by using one or more pixels from the region composed of reference pixels (e.g., by directly copying, weighted averaging, etc.), or information about at least one pixel in the region not composed of reference pixels can be explicitly generated. Thus, a predicted block can be generated using the region composed of reference pixels and the region not composed of reference pixels in this way.

[0138] FIG. 5 is a conceptual diagram illustrating intra-frame prediction for directional and non-directional modes according to an embodiment of the present invention.

[0139] 5(a) shows intra-frame prediction in vertical (5a), horizontal (5b), and diagonal (5c-5e) direction modes. FIG. 5(b) shows intra-frame prediction in DC mode. FIG. 5(c) shows intra-frame prediction in Planar mode.

[0140] The present invention may include additional non-directional modes other than those described above. Although the present invention will be described focusing on the linear directional mode and the DC and planar non-directional modes, modifications and application to other cases are also possible.

[0141] 4 may be prediction modes that are fixedly supported regardless of the block size, or the prediction modes supported depending on the block size may differ from those shown in FIG.

[0142] For example, the number of prediction mode candidate groups may be adaptive (e.g., the angles between prediction modes are equally spaced but the angles are set differently, such as 9, 17, 33, 65, 129, etc. based on directional mode), or the number of prediction mode candidate groups may be fixed but may have other configurations (e.g., directional mode angles, non-directional types, etc.).

[0143] 4 may be fixed prediction modes supported regardless of the block type, or the prediction modes supported depending on the block type may be different from those shown in FIG.

[0144] For example, the number of prediction mode candidates may be adaptive (e.g., the number of prediction modes derived horizontally or vertically may be set to a smaller or larger number depending on the width-to-height ratio of the block), or the number of prediction mode candidates may be fixed but may have other configurations (e.g., the number of prediction modes derived horizontally or vertically may be set more precisely depending on the width-to-height ratio of the block).

[0145] Alternatively, a larger number of prediction modes may be supported for longer blocks and a smaller number for shorter blocks. When the block length is long, the prediction mode interval may be modes located to the right of mode 66 in Fig. 4 (e.g., modes having angles of +45 degrees or more relative to mode 50, i.e., modes numbered 67 to 80) or modes located to the left of mode 2 (e.g., modes having angles of -45 degrees or more relative to mode 18, i.e., modes numbered -1 to -14). This can be determined according to the ratio of the width and height of the block, and the opposite situation is also possible.

[0146] In the present invention, the description will be focused on the case where the prediction mode is fixedly supported (regardless of any encoding / decoding factors) as shown in FIG. 4, but it is also possible to set a prediction mode that is adaptively supported depending on the encoding setting.

[0147] In addition, when classifying prediction modes, horizontal and vertical modes (modes 18 and 50), some diagonal modes (Diagonal up right <mode 2>, Diagonal down right <mode 34>, Diagonal down left <mode 66>, etc.) can be used as criteria, and this can be a classification method based on some directions (or angles such as 45 degrees, 90 degrees, etc.).

[0148] In addition, some modes located at both ends of the directional modes (modes 2 and 66) may be reference modes for prediction mode classification, which is an example possible when the intra-frame prediction mode configuration is as shown in Figure 4. That is, when the prediction mode configuration is adaptive, the reference mode may be changed. For example, mode 2 may be replaced with a mode having a number smaller or larger than 2 (-2, -1, 3, 4, etc.), or mode 66 may be replaced with a mode having a number smaller or larger than 66 (64, 66, 67, 68, etc.).

[0149] In addition, additional prediction modes for color components can be included in the prediction mode candidate set. Next, a color copy mode and a color mode will be described as examples of the prediction modes.

[0150] (Color copy mode)

[0151] Prediction modes related to methods of obtaining data for generating a prediction block from a region located in another color space may be supported.

[0152] For example, a prediction mode for obtaining data for generating a prediction block in another color space using correlation between color spaces can be an example of this.

[0153] 6 is a conceptual diagram illustrating intra-frame prediction for a color copy mode according to an embodiment of the present invention. Referring to FIG. 6, a current block C in a current color space M can be predicted using data from a corresponding region D in another color space N at the same time t.

[0154] Here, in the case of YCbCr, the correlation between color spaces may refer to the correlation between Y and Cb, Y and Cr, or Cb and Cr. That is, in the case of a chrominance component (Cb or Cr), a reconstructed block of a luminance component (Y) corresponding to the current block may be used as a prediction block for the current block (chrominance versus luminance is the basic setting in the example described below). Alternatively, a reconstructed block of a part of the chrominance components (Cb or Cr) corresponding to the current block of a part of the chrominance components (Cr or Cb) can be used as a prediction block of the current block.

[0155] Here, the area corresponding to the current block may have the same absolute position within each image in some color formats (e.g., YCbCr4:4:4, etc.), or may have the same relative position within each image in some color formats (e.g., YCbCr4:2:0, etc.). Corresponding positions can be determined according to the ratio of width to height according to the color format, and the pixel of the current image and the corresponding pixel in another color space can be obtained by multiplying or dividing each component of the coordinates of the current pixel by the ratio of width to height according to the color format.

[0156] For convenience of explanation, the description will focus on the case of a certain color format, 4:4:4, but it should be understood that the position of the corresponding area in other color spaces can be determined by the ratio of width to height depending on the color format.

[0157] In color copy mode, a restored block of another color space can be used as a predicted block, or a block acquired by considering the correlation between color spaces can be used as a predicted block. A block acquired by considering the correlation between color spaces means a block acquired by correcting an existing block. In detail, in the formula {P=a*R+b}, a and b represent values ​​used for correction, and R and P represent the value acquired in another color space and the predicted value of the current color space, respectively. Here, P represents a block acquired by considering the correlation between color spaces.

[0158] In this example, it is assumed that data obtained using the correlation between color spaces is used as a predicted value of the current block, but the data can also be used as a correction value applied to an existing predicted value of the current block, i.e., the predicted value of the current block can be corrected with a residual value of another color space.

[0159] Although the present invention will be described assuming the former case, it is not limited to this case and the same modifications or modifications can be applied to the case where the value is used as a correction value.

[0160] Whether the color copy mode is supported can be determined explicitly or implicitly depending on the encoding / decoding settings. Here, the encoding / decoding settings can be defined by a combination of one or more of the image type, color components, block position / size / shape, and block width-to-height ratio. In the explicit case, related information can be included in units such as sequence, picture, slice, and tile. In addition, whether the color copy mode is supported can be determined implicitly depending on the encoding / decoding settings in some cases, and related information can be explicitly generated in other cases.

[0161] In color copy mode, the correlation information between color spaces (such as a and b) can be generated explicitly or implicitly by the encoding / decoding settings.

[0162] Here, the areas compared (or referenced) to obtain correlation information can be the current block (C in FIG. 6) and the corresponding area in another color space (D in FIG. 6), or the adjacent area of ​​the current block (e.g., the left, upper, upper left, upper right, or lower left block centered on C in FIG. 6) and the adjacent area of ​​the corresponding area in another color space (e.g., the left, upper, upper left, upper right, or lower left block centered on D in FIG. 6).

[0163] In the above description, the former case corresponds to an example of explicitly processing the correlation information because the correlation information must be directly obtained from the current block using the data of the corresponding block. That is, the correlation information must be generated because the data of the current block has not yet been encoded. The latter case corresponds to an example of implicitly processing the correlation information because the correlation information can be indirectly obtained from the neighboring area of ​​the current block using the data of the neighboring area of ​​the corresponding block.

[0164] In summary, in the former case, correlation information is obtained by comparing the current block with the corresponding block, and in the latter case, correlation information is obtained by comparing the current block with its adjacent region, and the data obtained by applying the correlation information to the corresponding block can be used as predicted pixels for the current block.

[0165] In the former case, the correlation information can be directly encoded, or the correlation information obtained by comparing adjacent regions can be used as a predicted value and information about the difference value can be encoded. The correlation information can be information that can be generated when a color copy mode is selected as a prediction mode.

[0166] Here, the latter case can be understood as an example of an implicit case where there is no additional information generated except that the color copy mode is selected as the optimal mode from the prediction mode candidate group, i.e., this can be an example that is possible under a setting that supports one correlation information.

[0167] If two or more pieces of correlation information are supported, the color copy mode is selected as the optimum mode, and selection information about the correlation information may be required. As in the above example, it is also possible to combine explicit and implicit cases depending on the encoding / decoding settings.

[0168] In the present invention, the case where correlation information is indirectly acquired will be mainly described. The acquired correlation information may be N or more pieces (N is an integer equal to or greater than 1, such as 1, 2, or 3). The setting information regarding the number of pieces of correlation information may be included in units of sequences, pictures, slices, tiles, etc. It should be understood that in some of the examples described below, supporting k pieces of correlation information may be the same as supporting k color copy modes.

[0169] 7 is an example diagram showing corresponding blocks and adjacent areas in each color space in a color copy mode according to an embodiment of the present invention. Referring to FIG. 7, the corresponding instances (p and q) between pixels in a current color space M and another color space N are shown, and can be understood as examples that can occur in the case of some color formats (4:2:0). Corresponding relationship 7a can be identified for obtaining correlation information, and corresponding relationship 7b can be identified for applying a predicted value.

[0170] Next, we will explain how to obtain correlation information in color copy mode. To obtain correlation information, pixel values ​​of pixels within a predetermined area (all or part of each adjacent area of ​​the block corresponding to the current block) in each color space can be compared (or used) (i.e., a 1:1 pixel value comparison process can be performed). Here, the pixel values ​​to be compared can be obtained based on the corresponding pixel positions in each color space. The pixel values ​​can be values ​​derived from at least one pixel in each color space.

[0171] For example, in the case of some color formats such as 4:4:4, the pixel value of one pixel in the chrominance space and the pixel value of one pixel in the luminance space can be used as the pixel value corresponding to the correlation information acquisition process, or in the case of some color formats such as 4:2:0, the pixel value of one pixel in the chrominance space and pixel values ​​derived from one or more pixels in the luminance space (i.e., obtained by a downsampling process) can be used as the pixel value corresponding to the correlation information acquisition process.

[0172] Specifically, in the former case, p[x,y] in the chrominance space can be compared with q[x,y] in the luminance space, where the pixel value can be the brightness value of a single pixel. In the latter case, p[x,y] in the chrominance space can be compared with q[2x,2y], q[2x,2y+1], q[2x+1,2y], q[2x+1,2y+1], etc. in the luminance space.

[0173] Here, since a 1:1 pixel value comparison must be performed, in the case of a luminance space, one of the plurality of pixels can be used as a value to be compared with the pixel value of a chrominance pixel. That is, the brightness value of one of the plurality of pixels is used as is. Alternatively, one pixel value can be derived from two or more pixels (k pixels, where k is an integer equal to or greater than 2, such as 2, 4, or 6) among the plurality of pixels and used as a value to be compared. That is, a weighted average (weights can be assigned to each pixel equally or unevenly) can be applied to two or more pixels.

[0174] When there are multiple corresponding pixels as in the above example, the pixel value of one predefined pixel or pixel values ​​derived from two or more pixels can be used as the comparison value. Here, one of the above two methods for deriving pixel values ​​to be compared in each color space according to encoding / decoding settings can be used alone or in combination.

[0175] The following explanation assumes that the pixel value of one pixel in the current color space is used for comparison, and one or more pixels in another color space can be used to derive pixel values. For example, assume that the color format is YCbCr4:2:0, the current color space is a chrominance space, and the other color space is a luminance space. The method for deriving pixel values ​​will be explained mainly for the other color space.

[0176] For example, it may be determined based on the shape of the block (ratio of width to height). As a detailed example, p[x, y] in the chrominance space adjacent to the longer side of the current block (or the block to be predicted) may be compared with q[2x, 2y] in the luminance space, and p[x, y] in the chrominance space adjacent to the shorter side may be compared with the average of q[2x, 2y] and q[2x+1, 2y] in the luminance space.

[0177] Here, adaptive settings are possible, such as applying the above content to some block shapes (rectangles) regardless of the ratio of width to height, or applying it only when the ratio of width to height is above / exceeding a certain ratio (k:1 or 1:k, for example, k is 2 or more, such as 2:1, 4:1, etc.).

[0178] For example, it can be determined by the size of the block. As a detailed example, if the size of the current block is a fixed size (M×N), for example, 2 m ×2 n If the size is greater than or equal to a certain size (where m and n are integers greater than or equal to 1, such as 2 to 6), p[x, y] in the chrominance space can be compared with q[2x+1, 2y] in the luminance space, and if the size is less than or equal to a certain size, p[x, y] in the chrominance space can be compared with the average of q[2x, 2y] and q[2x, 2y+1] in the luminance space.

[0179] Here, it is possible to adaptively set the boundary value for size comparison so that there is one as in the above example, or there are two or more (M1×N1, M2×N2, etc.).

[0180] The above examples are only some of the cases that can be considered in terms of the amount of calculation, and various modified examples including the opposite cases to the above examples are possible.

[0181] For example, it may be determined based on the position of the block. As a detailed example, if the current block is located within a predetermined region (assumed to be the largest coding block in this example), p[x, y] in the chrominance space may be compared with an average of q[2x, 2y], q[2x+1, 2y], q[2x, 2y+1], and q[2x+1, 2y+1] in the luma space. If the current block is located on the boundary of the predetermined region (assumed to be the upper left boundary in this example), p[x, y] in the chrominance space may be compared with q[2x+1, 2y+1] in the luma space. The predetermined region may refer to a region set based on a slice, tile, block, etc. In particular, it may be obtained based on an integer multiple of a slice, tile, or largest coding / prediction / transform block.

[0182] As another example, if the current block is located on a partial boundary of the region (assumed to be the upper boundary in this example), p[x, y] in the chrominance space adjacent to the partial boundary (upper side) can be compared with q[2x+1, 2y+1] in the luminance space, and p[x, y] in the chrominance space adjacent to the interior (left side) of the region can be compared with the average of q[2x, 2y], q[2x+1, 2y], q[2x, 2y+1], q[2x+1, 2y+1] in the luminance space.

[0183] The above examples are only some of the cases that can be considered in terms of memory, and various modifications including cases opposite to the above examples are possible.

[0184] Based on the above examples, various cases have been described regarding the derivation of pixel values ​​to be compared in each color space. As in the above examples, the setting of pixel value derivation for obtaining the correlation information can be determined by taking into consideration not only the size / shape / position of the block but also various encoding / decoding factors.

[0185] Based on the above example, the area to be compared to obtain correlation information is explained as using one or two reference pixel lines for each of the current block and the corresponding block as shown in Figure 7. That is, in the case of YCbCr4:4:4, one reference pixel line is used for each, and in the case of other formats, some color spaces are used as shown in Figure 7.<color N> However, the present invention is not limited to this and various modifications are possible.

[0186] The following description focuses on the reference pixel lines of the current color space, and it should be understood that for other color spaces, the reference pixel lines can be determined depending on the color format, for example, the same number of reference pixel lines can be used or twice as many reference pixel lines can be used.

[0187] In the color copy mode of the present invention, k reference pixel lines (where k is an integer equal to or greater than 1, such as 1 or 2) can be used (or compared) to obtain correlation information. The k reference pixel lines can be used either fixedly or adaptively. Various examples of setting the number of reference pixel lines will now be described.

[0188] For example, it can be determined based on the shape of the block (ratio of width to height). As a detailed example, two adjacent reference pixel lines on the longer side of the current block can be used, and one adjacent reference pixel line on the shorter side can be used.

[0189] Here, the above content can be applied to some block shapes (rectangles) regardless of the ratio of width to height, or can be applied only when the ratio of width to height is equal to or exceeds a certain ratio (k:1 or 1:k, where k is an integer equal to or greater than 2, for example, 2:1, 4:1, etc.). In addition, there are two or more boundary values ​​for the ratio of width to height, and extensions are possible, such as using two reference pixel lines adjacent to the longer (or shorter) side in the case of 2:1 or 1:2, and using three reference pixel lines adjacent to the longer (or shorter) side in the case of 4:1 or 1:4.

[0190] The above example may be an example where the longer (or shorter) side uses s reference pixel lines and the shorter (or longer) side uses t reference pixel lines depending on the ratio of horizontal to vertical length, where s is greater than or equal to t (i.e., s and t are integers greater than or equal to 1).

[0191] For example, it can be determined by the size of the block. As a detailed example, if the size of the current block is a fixed size (M×N), for example, 2 m ×2 n If m and n are integers greater than or equal to 1, such as 2 to 6, two reference pixel lines can be used, and if m and n are less than or equal to a certain size, one reference pixel line can be used.

[0192] Here, it is possible to adaptively set the boundary value for size comparison so that there is one as in the above example, or there are two or more (M1×N1, M2×N2, etc.).

[0193] For example, it can be determined based on the position of the block. As a detailed example, if the current block is located inside a predetermined region (which can be guided by the previous description related to obtaining correlation information, assumed to be the largest coding block in this example), two reference pixel lines can be used, and if the current block is located on the boundary of the predetermined region (assumed to be the upper left boundary in this example), one reference pixel line can be used.

[0194] As another example, if the current block is located at a partial boundary of the region (assumed to be the upper boundary in this example), one reference pixel line adjacent to the partial boundary (upper side) can be used, and two reference pixel lines adjacent to the interior (left side) of the region can be used.

[0195] The above examples are only some cases that can be considered in terms of accuracy of correlation information and memory, and various modifications including cases opposite to the above examples are possible.

[0196] Based on the above examples, various cases have been described regarding the setting of reference pixel lines used to obtain correlation information in each color space. As in the above examples, the setting of reference pixel lines for obtaining correlation information can be determined taking into consideration not only the size / shape / position of the block but also various encoding / decoding factors.

[0197] Next, we will explain other cases of the area to be compared (or referenced) to obtain correlation information. The area to be compared may be pixels adjacent to the current block in the current color space, such as the left side, top side, upper left side, upper right side, and lower left side.

[0198] Here, the comparison area may be set to include all blocks located on the left, top, upper left, top right, and bottom left. Alternatively, the reference pixel area may be configured as a combination of blocks located in some positions. For example, the comparison area may be configured as a combination of adjacent blocks such as left / top / left + top / left + top + top left / left + bottom left / top + top right / left + top left + bottom left / top + top left + top right / left + top + top right / left + top + bottom left.

[0199] FIG. 8 is an exemplary diagram illustrating the setting of a region for obtaining correlation information in a color copy mode according to an embodiment of the present invention. The cases of a to e in FIG. 8 may correspond to the examples previously described (left side + upper side, upper side + upper right side, left side + lower left side, left side + upper right side + upper right side, left side + upper + lower left side). In addition, a block at a certain position may be divided into one or more sub-blocks, some of which may be set as a region for obtaining correlation information (FIG. 8 f and g). That is, a region for obtaining correlation information may be set using one or more sub-blocks located in a certain direction. Alternatively, a region for obtaining correlation information may be set using one or more blocks located in a certain direction (a) and one or more sub-blocks located in a certain direction (b) (here, a and b refer to the other direction). In addition, examples of setting a region for obtaining correlation information using non-contiguous blocks (FIG. 8 h and i) are also possible.

[0200] In summary, the regions to be compared for obtaining correlation information can be configured as predefined regions, or can be configured as various combinations of some regions. That is, the regions to be compared can be configured either fixedly or adaptively depending on the encoding / decoding settings.

[0201] Next, various examples of which direction adjacent regions are configured as reference regions around a current block in a current color space will be described. Here, it is assumed that which direction adjacent regions are configured as reference regions in a corresponding block in another color space is determined based on the reference region configuration of the current color block. It is also assumed that the basic reference region is configured as a left and upper block.

[0202] For example, it can be determined based on the shape of the block (ratio of width to height). For a detailed example, if the current block is long horizontally, the left, upper, and upper right blocks can be set as the reference region, and if it is long vertically, the left, upper, and lower left blocks can be set as the reference region.

[0203] Here, the above content can be applied to some block shapes (rectangles) regardless of the ratio of width to height, or can be applied only when the ratio of width to height is equal to or exceeds a certain ratio (k:1 or 1:k, where k is an integer equal to or greater than 2, for example, 2:1, 4:1, etc.). In addition, there are two or more boundary values ​​for the ratio of width to height, and extensions are possible, such as setting the left, upper, and upper right blocks (or left, upper, and lower left) as the reference region when the ratio is 2:1 (or 1:2), and setting the upper and upper right blocks (or left, lower left) as the reference region when the ratio is 4:1 (or 1:4).

[0204] For example, it can be determined by the size of the block. As a detailed example, if the size of the current block is a fixed size (M×N), for example, 2 m ×2 n If the size of the pixel is greater than or equal to a certain size (where m and n are integers greater than or equal to 1, such as 2 to 6), the left and upper blocks are set as the reference region, and if the size is less than or equal to a certain size, the left, upper, and upper left blocks are set as the reference region.

[0205] Here, it is possible to adaptively set the boundary value for size comparison so that there is one as in the above example, or there are two or more (M1×N1, M2×N2, etc.).

[0206] For example, it can be determined based on the position of the block. As a detailed example, if the current block is located inside a preset region (which can be derived from the previous description related to obtaining correlation information, assumed to be the largest coding block in this example), the left, upper, upper left, upper right, and lower left blocks can be set as reference regions, and if the current block is located on the boundary of the preset region (assumed to be the upper left boundary in this example), the left and upper blocks can be set as reference regions.

[0207] As another example, if the current block is located at a boundary of a part of the region (assumed to be the upper boundary in this example), the left and lower left adjacent blocks within the region, excluding the block adjacent to the boundary (upper boundary), can be set as reference regions. That is, the left and lower left blocks can be set as reference regions.

[0208] The above examples are only some cases that can be considered in terms of the amount of calculation, memory, etc., and various modifications including the opposite cases to the above examples are possible.

[0209] Based on the above examples, various cases have been described for setting a reference area used to acquire correlation information in each color space. As in the above examples, the setting of a reference area for acquiring correlation information can be determined taking into consideration not only the size / shape / position of a block but also various encoding / decoding factors.

[0210] Also, the comparison area may be pixels adjacent to the current block in the current color space, where all or some of the reference pixels may be used to obtain correlation information.

[0211] For example, if the current block (based on color M in FIG. 7) is a block (i.e., 8×8) having a pixel range of (a, b) to (a+7, b+7), the area to be compared (the corresponding block is omitted as its explanation can be guided by the color format) is assumed to be one reference pixel line of the block to the left and above the current block.

[0212] Here, the area to be compared may include all pixels within the ranges of (a, b-1) to (a+7, b-1) and (a-1, b) to (a-1, b+7). Alternatively, it may include partial pixels within the ranges, i.e., (a, b-1), (a+2, b-1), (a+4, b-1), (a+6, b-1), (a-1, b), (a-1, b+2), (a-1, b+4), and (a-1, b+6). Alternatively, it may include partial pixels within the ranges, i.e., (a, b-1), (a+4, b-1), (a-1, b), and (a-1, b+4).

[0213] The above example can be applied for the purpose of reducing the amount of calculation required for obtaining correlation. As with the many examples described above, the setting of reference pixel sampling of the area to be compared for obtaining correlation information can take into account various encoding / decoding factors such as block size / shape / position, and related application examples can be derived from the previous examples, so detailed description will be omitted.

[0214] Based on the various examples described above, various factors that affect correlation information acquisition (e.g., corresponding pixel value induction, number of reference pixel lines, reference area direction setting, reference pixel sampling, etc.) have been described. There are various cases in which the above examples affect correlation information acquisition, either alone or in combination.

[0215] The above description can be understood as a pre-setting process for obtaining one piece of correlation information. Also, as mentioned above, one or more pieces of correlation information can be supported depending on the encoding / decoding setting, and two or more pieces of correlation information can be supported by setting two or more of the above-mentioned pre-settings (i.e., combinations of factors that affect the acquisition of correlation information).

[0216] In summary, parameter information based on correlation information can be derived from the neighboring regions of the current block and the neighboring regions of the corresponding block. That is, at least one parameter (e.g.,<a1、b1> ,<a2、b2> ,<a3、b3> , etc.), which can be used as values ​​to multiply or add to the pixels of the reconstructed block in other color spaces.

[0217] Next, a linear model applied in the color copy mode will be described. By applying the parameters obtained through the above process, prediction based on the following linear model can be performed.

[0218] pred_sample_C(i, j)=a×rec_sample_D(i, j)+b

[0219] In the above equation, pred_sample_C means a predicted pixel value of the current block in the current color space, and rec_sample_D means a restored pixel value of the corresponding block in another color space. a and b can be obtained by minimizing the regression error between the adjacent areas of the current block and the adjacent areas of the corresponding block, and can be calculated using the following equations.

number

[0220] In the above formula, D(n) means the adjacent area of ​​the corresponding block, C(n) means the adjacent area of ​​the current block, and N means a value set based on the horizontal or vertical length of the current block (in this example, assumed to be twice the minimum value of the horizontal or vertical length).

[0221] In addition, various methods can be used, such as a straight-line equation, which obtains correlation information based on the minimum and maximum values ​​of adjacent regions in each color space. Here, a pre-set model can be used to obtain the correlation information, or one of multiple models can be selected. Here, selecting one of multiple models means that model information can be considered as an encoding / decoding element for parameter information based on the correlation information. In other words, when multiple parameters are supported, even if the remaining correlation information-related settings are the same, different parameter information can be distinguished by using different models to obtain the correlation.

[0222] For some color formats (other than 4:4:4), one pixel in the current block can correspond to one or more (two, four, etc.) pixels in the corresponding block. For example, in the case of 4:2:0, p[x, y] in the chrominance space can correspond to q[2x, 2y], q[2x, 2y+1], q[2x+1, 2y], q[2x+1, 2y+1], etc. in the luma space.

[0223] For one predicted pixel value, a pixel value (or predicted value) of one pixel previously set from the corresponding plurality of pixels, or one pixel value from two or more pixels, can be derived (7b). That is, to obtain one predicted pixel value, a restored value before applying correlation information can be obtained from one or more corresponding pixels in another color space. Various cases are possible depending on the encoding / decoding setting, and related explanations can be given in the section regarding corresponding pixel value derivation (7a) for obtaining correlation information, so detailed explanations will be omitted. However, 7a and 7b can apply the same or different settings.

[0224] (Color mode)

[0225] Prediction modes related to methods of obtaining prediction modes for generating prediction blocks from regions located in other color spaces may be supported.

[0226] For example, a prediction mode for a method of obtaining a prediction mode for generating a prediction block in another color space using correlation between color spaces can be an example. That is, the color mode does not have a specific prediction direction or prediction method, but can be a mode that uses an existing prediction direction and method and is adaptively determined according to the prediction mode of a corresponding block in another color space.

[0227] Here, various color modes can be obtained by setting block division.

[0228] For example, in a setting where block division for some color components (chrominance) is implicitly determined based on the block division result for some color components (luminance) (i.e., when block division for the luminance component is explicitly determined), one block of some color components (chrominance) may correspond to one block of some color space (luminance). Therefore, (assuming a 4:4:4 case; for other formats, the explanation of this example can be guided by the ratio of width to height), if a current block (chrominance) has a pixel range of (a, b) to (a+m, b+n), any pixel position within the pixel range of (a, b) to (a+m, b+n) of the corresponding block (luminance) indicates one block, and therefore one prediction mode can be obtained from the block including the pixel.

[0229] Alternatively, if individual block division is supported for each color component (i.e., if block division for each color space is explicitly determined), one block of some color components (chrominance) may correspond to one or more blocks of some color space (luminance). Therefore, even if the current block (chrominance) has the same pixel range as in the above example, the corresponding block (luminance) may be composed of one or more blocks depending on the block division result. Therefore, depending on the pixel position within the pixel range of the current block, other prediction modes (ie, one or more modes) can be obtained from the corresponding block indicated by the pixel.

[0230] If one color mode is supported in the intra prediction mode candidate group for the chrominance component, it is possible to set from which position of the corresponding block the prediction mode is to be taken.

[0231] For example, prediction modes can be obtained from positions such as the center, upper left, upper right, lower left, and lower right of the corresponding block. That is, prediction modes are obtained in the order described above, but if the corresponding block is not used (for example, the coding mode is Inter), a prediction mode from the next corresponding position can be obtained. Alternatively, a prediction mode with a high frequency (2 or more times) can be obtained from the block at the position described above.

[0232] Alternatively, when multiple color modes are supported, it is possible to set from which prediction modes to use according to the priority order. Alternatively, a combination is possible in which some prediction modes are used according to the priority order and some prediction modes with high frequency are used from the block at the position. Here, the above-mentioned priority order is just one example, and various modifications are possible.

[0233] The color mode and the color copy mode may be prediction modes that can be supported for the chrominance component. For example, a group of prediction mode candidates for the chrominance component may be configured including horizontal, vertical, DC, planar, diagonal modes, etc. Alternatively, a group of intra-frame prediction mode candidates may be configured including the color mode and the color copy mode.

[0234] That is, it can be composed of directional + non-directional + color mode, or directional + non-directional + color copy mode, or it can be composed of directional + non-directional + color mode + color copy mode, or it can include modes for other additional color difference components.

[0235] Whether color mode and color copy mode are supported can be determined by the encoding / decoding settings, and this can be done implicitly or explicitly. Alternatively, a combination of explicit and implicit configurations can be used. In addition, detailed settings related to color mode and color copy mode (e.g., the number of supported modes) can be included and processed implicitly or explicitly.

[0236] For example, the related information may be explicitly included in units of a sequence, a picture, a slice, a tile, a block, etc., or may be implicitly determined by various encoding / decoding factors (e.g., an image type, a block position, a block size, a block shape, a ratio of width to height of a block, etc.), or may be implicitly determined under some conditions depending on the encoding / decoding factors, or may be explicitly generated under other conditions.

[0237] 9 is a diagram illustrating an example of a configuration of reference pixels used in intra prediction according to an embodiment of the present invention. The size and shape (M×N) of a prediction block can be obtained by a block division unit.

[0238] Block range information, which is defined as the sizes of the minimum and maximum blocks for intra prediction, may include information related to units such as a sequence, a picture, a slice, a tile, etc. Generally, size information may be set by specifying the width and height (e.g., 32x32, 64x64, etc.), but size information may also be set in the form of the product of the width and height. For example, if the product of the width and height is 64, the size of the minimum block may correspond to 4x16, 8x8, 16x4, etc.

[0239] In addition, size information can be set by specifying the width and height, or by setting size information in the form of a product. For example, if the product of the width and height is 4096 and the maximum value of one of the lengths is 64, the maximum block size can be 64 x 64.

[0240] As in the above example, the size and shape of the predicted block can be finally determined by combining block division information in addition to size information of the smallest and largest blocks. In the present invention, the product of the width and height of the predicted block must be greater than or equal to s (e.g., s is a multiple of 2, such as 16 or 32), and one of the width and height must be greater than or equal to k (e.g., k is a multiple of 2, such as 4 or 8). In addition, the width and height of the block can be defined as being less than or equal to v and w, respectively (e.g., v and w are multiples of 2, such as 16, 32, or 64), but various block ranges can be set without being limited thereto.

[0241] Intra-frame prediction may generally be performed in units of prediction blocks, but may also be performed in units of coding blocks, transform blocks, etc., depending on the setting of the block division unit. After checking the block information, the reference pixel construction unit can construct reference pixels to be used for prediction of the current block. Here, the reference pixels are stored in a temporary memory (for example, an array <array>It can be managed by a primary, secondary array, etc., and is generated and removed for each intra-prediction process of a block, and the size of the temporary memory can be determined depending on the configuration of reference pixels.

[0242] In this example, it is assumed that the left, upper, upper left, upper right, and lower left blocks of the current block are used to predict the current block, but the present invention is not limited to this, and other configurations of block candidate groups can also be used to predict the current block. For example, the neighboring block candidate group for the reference pixel may be an example of a case where a raster or Z scan is used, and some of the candidate groups can be removed depending on the scan order, or other block candidate groups (e.g., right, lower, and lower right blocks can be added) can be included.

[0243] In addition, when a certain prediction mode (color copy mode) is supported, a certain area of ​​another color space can be used for predicting the current block, and therefore this can also be considered as a reference pixel. The existing reference pixels (a spatially adjacent area of ​​the current block) and the additional reference pixels can be managed as one or managed separately (e.g., reference pixel A and reference pixel B. In other words, the reference pixel memory can be named separately just as the temporary memory is used separately).

[0244] For example, the temporary memory for the basic reference pixels (based on one reference pixel line) may have a size of <2 x blk_width + 2 x blk_height + 1>, and the temporary memory for the additional reference pixels (in the case of 4:4:4)<blk_width×blk_height> (In the case of 4:2:0, blk_width / 2×blk_height / 2 is required.) The size of the temporary memory is an example and is not limited to this.

[0245] In addition, to obtain correlation information, adjacent areas of the current block and the corresponding block to be compared (or referenced) can also be managed as reference pixels, which means that additional reference pixels can be managed according to the color copy mode.

[0246] In summary, adjacent areas of the current block can be included as reference pixels for intra-frame prediction of the current block, and depending on the prediction mode, even corresponding blocks in other color spaces and their adjacent areas can be included as reference pixels.

[0247] 10 is a conceptual diagram illustrating blocks adjacent to a target block of intra prediction according to an embodiment of the present invention. Specifically, the left side of FIG. 10 illustrates blocks adjacent to a current block in a current color space, and the right side illustrates corresponding blocks in another color space. For convenience of explanation, the following description will be given assuming that the blocks adjacent to the current block in the current color space are configured as basic reference pixels.

[0248] As shown in Figure 9, the reference pixels used for predicting the current block can be configured from adjacent pixels in the left, upper, upper left, upper right, and lower left blocks (Ref_L, Ref_T, Ref_TL, Ref_TR, and Ref_BL in Figure 9). Here, the reference pixels are generally configured from pixels in the neighboring block closest to the current block (a in Figure 9, which is referred to as the reference pixel line), but other pixels (b in Figure 9 and other pixels on the outer lines) can also be configured as reference pixels.

[0249] Pixels adjacent to the current block can be classified into at least one reference pixel line. The pixel closest to the current block is ref_0 {e.g., pixels whose distance from the boundary pixel of the current block is 1, such as p(-1,-1) to p(2m-1,-1), p(-1,0) to p(-1,2n-1)}, the next adjacent pixel {e.g., pixels whose distance from the boundary pixel of the current block is 2, such as p(-2,-2) to p(2m,-2), p(-2,-1) to p(-2,2n)} is ref_1, and the next adjacent pixel {e.g., pixels whose distance from the boundary pixel of the current block is 3, such as p(-3,-3) to p(2m+1,-3), p(-3,-2) to p(-3,2n+1)} is ref_2, etc. In other words, the reference pixel lines can be classified according to the distance between the boundary pixel of the current block and the adjacent pixel.

[0250] Here, the number of supported reference pixel lines may be N or more, and N may be an integer greater than or equal to 1, such as 1 to 5. Here, the reference pixel line candidates are generally included in the reference pixel line candidate group in order starting from the reference pixel line closest to the current block, but this is not limiting. For example, if N is 3,<ref_0、ref_1、ref_2> The candidate set can be constructed in this way, or<ref_0、ref_1、ref_3> ,<ref_0、ref_2、ref_3> ,<ref_1、ref_2、ref_3> The candidate group may be formed not sequentially, or by excluding the most adjacent reference pixel line.

[0251] Prediction can be performed using all reference pixel lines in the candidate group, or using some (one or more) of the reference pixel lines.

[0252] For example, one of a plurality of reference pixel lines may be selected according to encoding / decoding settings, and intra-frame prediction may be performed using the selected reference pixel line, or two or more of a plurality of reference pixel lines may be selected and intra-frame prediction may be performed using the selected reference pixel lines (e.g., by applying a weighted average to the data of each reference pixel line).

[0253] Here, the selection of the reference pixel line can be determined implicitly or explicitly. For example, the implicit case means that it is determined according to an encoding / decoding setting defined by one or a combination of two or more factors such as an image type, a color component, a block size / shape / position, etc. The explicit case means that reference pixel line selection information can be generated in units such as a block.

[0254] Although the present invention will be described mainly in terms of intra-frame prediction using the most adjacent reference pixel line, it should be understood that the various embodiments described below can be applied in the same or similar manner when multiple reference pixel lines are used.

[0255] The reference pixel construction unit for intra-frame prediction of the present invention may include a reference pixel generation unit, a reference pixel interpolation unit, a reference pixel filter unit, etc., and may include all or part of the above components.

[0256] The reference pixel construction unit can check the availability of reference pixels and classify them into available reference pixels and unavailable reference pixels. Here, the availability of reference pixels is determined to be unavailable when at least one of the following conditions is satisfied. Of course, the availability of reference pixels can be determined based on additional conditions not mentioned in the example described below, but the present invention will be described assuming that the availability of reference pixels is limited to the exemplary conditions described below.

[0257] For example, if it is located outside the picture boundary, if it does not belong to the same division unit as the current block (for example, units that cannot refer to each other, such as slices and tiles. However, if units such as slices or tiles have the characteristic of being able to refer to each other, exceptions are made even if they are not the same division unit), or if encoding / decoding is not completed, it can be determined to be unusable. In other words, if any of the above conditions is not met, it can be determined to be usable.

[0258] In addition, the use of reference pixels may be restricted depending on the encoding / decoding settings. For example, even if it is determined that the use of reference pixels is permitted according to the above conditions, the use of reference pixels may be restricted depending on whether or not constrained intra prediction (e.g., constrained_intra_pred_flag) is performed. Constrained intra prediction may be performed when it is desired to prohibit the use of blocks reconstructed by reference from other images as reference pixels when performing error-resilient encoding / decoding due to external factors such as a communication environment.

[0259] When constrained intra prediction is deactivated (e.g., I picture type, or P or B picture type with constrained_intra_pred_flag set to 0), reference pixel candidate blocks (but only if they meet the aforementioned conditions, such as being located inside the picture boundary) may be usable.

[0260] Alternatively, when constrained intra prediction is activated (e.g., constrained_intra_pred_flag is set to 1 for P or B video type), the usability of the reference pixel candidate block can be determined depending on the coding mode (Intra or Inter). Generally, it is usable in the case of Intra mode and unusable in the case of Inter mode. In the above example, it is assumed that the usability is determined depending on the coding mode, but the usability can also be determined depending on various other coding / decoding factors.

[0261] Since reference pixels are composed of one or more blocks, the reference pixel feasibility can be checked and classified into three cases: <all usable>, <some usable>, and <all unusable>. In the remaining cases excluding the case where all are usable, reference pixels at unusable candidate block positions can be filled or generated (A). Alternatively, reference pixels at unusable candidate block positions cannot be used in the prediction process, and prediction mode encoding / decoding can be performed excluding prediction modes that perform prediction from reference pixels at corresponding positions (B).

[0262] If the reference pixel candidate block is available, the pixel at the corresponding position can be included in the reference pixel memory of the current block by directly copying the pixel data or by using a process such as reference pixel filtering or reference pixel interpolation.

[0263] If the reference pixel candidate block is unusable, processing can be performed under reference pixel processing setting A or B. Next, a processing example when the reference pixel candidate block is unusable under each setting will be described.

[0264] (A) When the reference pixel candidate block is unavailable, the pixel at the corresponding position obtained by the reference pixel generation process can be included in the reference pixel memory of the current block.

[0265] Next, as an example of the reference pixel generation process, a method for generating reference pixels at unusable positions will be described.

[0266] For example, a reference pixel may be generated using an arbitrary pixel value. Here, the arbitrary pixel value may be a pixel value (e.g., a minimum value, a maximum value, a median value, etc.) belonging to a pixel value range (e.g., a pixel value range based on a bit depth or a pixel value range based on pixel distribution in a corresponding image). In particular, this may be an example that is applicable when all of the reference pixel candidate blocks are unavailable, but is not limited thereto and may also be applicable when only a portion of the reference pixel candidate blocks are unavailable.

[0267] Alternatively, reference pixels may be generated from an area where encoding / decoding of the video has been completed. Specifically, reference pixels may be generated based on at least one usable block (or usable reference pixels) adjacent to the unusable block. Here, at least one of methods such as extrapolation, interpolation, and copying may be used.

[0268] (B) When a reference pixel candidate block is unavailable, the use of prediction modes that use pixels at the corresponding position can be restricted. For example, when the reference pixel at the TR position in Fig. 9 is unavailable, the use of modes 51 to 66 (Fig. 4) that perform prediction using pixels at the corresponding position can be restricted, and the use of modes 2 to 50 (vertical modes) that perform prediction using reference pixels at positions T, TL, L, and BL that are not reference pixels at the TR position can be permitted (in this example, the explanation is limited to directional modes).

[0269] As another example, if reference pixels at all positions are unavailable, no allowable prediction mode may exist. In this case, a prediction block may be generated using any pixel value, as in some configurations of setting A, and a pre-set prediction mode (e.g., DC mode) may be set as the prediction mode of the corresponding block for reference in the prediction mode encoding / decoding process of the next block. In other words, the prediction mode encoding / decoding process may be implicitly omitted.

[0270] The above example may be related to a prediction mode encoding / decoding process. The following description will assume that the prediction mode encoding / decoding unit of the present invention supports setting A. If setting B is supported, some configuration of the prediction mode encoding / decoding unit may be changed. Since the case where reference pixels at all positions are unavailable has already been described above, the case where reference pixels at some positions are unavailable will be described below.

[0271] For example, assume that the MPM candidate group includes a prediction mode of a neighboring block. If the prediction mode of the neighboring block is a prediction mode that uses reference pixels at a block position that is unavailable for use in the current block, a process of excluding the corresponding mode from the MPM candidate group may be added. That is, a process of checking unavailable modes may be added to a redundancy checking process (to be described later) in the prediction mode encoding / decoding unit. Here, unavailable modes may be specified by various definitions, but in this example, they are assumed to be prediction modes that use reference pixels at an unavailable block position. Therefore, a process of forming an MPM candidate group for prediction modes may be performed according to priority, and whether a prediction mode is included in the MPM candidate group may be determined through a redundancy checking process and / or an unavailable mode checking process. Here, if a prediction mode of a corresponding order fails to pass the checking process, a prediction mode of the next priority may become a candidate for the MPM candidate group forming process.

[0272] Alternatively, assume that when reference pixels at positions TR, T, and TL in FIG. 9 are unavailable, the use of modes 19 to 66, which perform prediction using pixels at the corresponding positions, is restricted. In this case, modes 2 to 18, which perform prediction using reference pixels at positions L and BL, may be available prediction modes. In this case, assuming that the number of MPM candidates is six, the number of non-MPM candidates may be 12. Here, just as maintaining the number of MPM candidates at six (a larger number than all modes) is inefficient, when prediction mode use is restricted due to unavailable reference pixels, entropy encoding / decoding setting changes such as adjustment of the number of MPM candidates (e.g., p → q, p > q) and binarization (e.g., variable length binary coded A → variable length binary coded B) may also occur. In other words, this may be a situation in which adaptive prediction mode encoding / decoding is supported, and a detailed description thereof will be omitted.

[0273] In addition, since prediction modes whose use is restricted by unavailable reference pixels are also unlikely to occur in the non-MPM candidate group, it may be unnecessary to include such modes in the candidate group. This may be the case in which the entropy encoding / decoding settings, such as the number adjustment (e.g., s → t, s > t) and binarization (e.g., fixed-length binarization → variable-length binarization), of the non-MPM candidate group support adaptive prediction mode encoding / decoding, which can be changed like the MPM candidate group.

[0274] In the above examples, various processing examples have been described for cases where reference pixels are unavailable, which can occur not only in general prediction modes but also in color copy modes.

[0275] Next, when supporting a color copy mode, reference pixels are classified into usable reference pixels and unusable reference pixels based on the availability of the reference pixels, and various processing examples for the classification will be described.

[0276] 11 is an exemplary diagram illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention, assuming that the left and upper blocks of a current block (current color space) and a corresponding block (other color space) represent areas used to obtain correlation information and are in some color formats (YCbCr4:4:4).

[0277] As mentioned above, reference pixel availability can be determined based on the position of the current block (e.g., whether it is located outside the picture boundary). Figure 11 shows various examples of reference pixel availability that can be determined based on the position of the current block.

[0278] In Figure 11, the reference pixel availability of the current block is the same as that of the corresponding block (result), but it is assumed that the other color space is divided into the same division units (tiles, slices, etc.) as the current color space (however, it is necessary to consider the component ratio of the color format).

[0279] In Figure 11, a indicates the case where all reference pixels are usable, b and c indicate the case where some of the reference pixels are usable (the upper and left blocks, respectively), and d indicates the case where none of the reference pixels are usable.

[0280] Except for case a in Figure 11, at least one reference pixel belongs to the case where it is unusable, and therefore processing for this is necessary. When reference pixel processing A is set, the process of filling the unusable area can be performed by the reference pixel generation process. Here, the reference pixels of the current block can be processed by the reference pixel generation process (general intra-frame prediction) already described above.

[0281] The reference pixels of the corresponding block can be processed in the same way as the current block or differently. For example, if the reference pixel at position L2 (see FIG. 10) among the reference pixels of the current block is unavailable, it can be generated by an arbitrary pixel value or by available reference pixels. In particular, the available reference pixels can be located to the left / right or above / below the unavailable reference pixel (e.g., L1, L3 in this example), or can be located on the same reference pixel line (e.g., R3 in this example).

[0282] Meanwhile, the reference pixels of the corresponding block can also be generated using arbitrary pixel values ​​or usable reference pixels. However, the positions of the usable reference pixels can be the same as or different from those of the current block. In particular, the usable reference pixels can be located not only to the left / right / upper / lower of the unusable reference pixels but also in various directions, such as the upper left, upper right, lower left, and lower right. In the case of the current block, since encoding / decoding has not yet been completed, pixels a to p in FIG. 10 do not belong to the usable reference pixels. However, in the case of the corresponding block, since encoding / decoding has been completed, pixels aa to pp in FIG. 10 can also belong to the usable reference pixels. Therefore, reference pixels at unusable positions can be generated using various methods, such as interpolation, extrapolation, copying, and filtering of usable reference pixels.

[0283] Through the above process, reference pixels at unusable positions such as b to d in Figure 11 can be generated and stored in the reference pixel memory, and the reference pixels at the corresponding positions can be used to obtain correlation information as shown in a in Figure 11.

[0284] Next, the case of setting reference pixel processing B will be described. It is possible to restrict the use of reference pixels at unusable positions. In addition, it is possible to restrict the use of prediction modes that make predictions from reference pixels at unusable positions (applying adaptive prediction mode encoding / decoding, etc.), or to perform other processing.

[0285] First, we will explain the case where the use of reference pixels in unusable positions is restricted. As shown in Figure 11(a), the left and upper blocks of the current block and the corresponding block must be used to obtain correlation information, while Figure 11(b) and Figure 11(c) correspond to the case where some reference pixels are unusable. Here, the correlation information can be obtained using the available reference pixels without using the unavailable reference pixels. Meanwhile, consideration must be given to the case where there is insufficient data for obtaining the correlation information.

[0286] For example, if the left (N) and upper (M) blocks of the current block (M×N) are correlation information acquisition areas, the number of available reference pixels is k (k is greater than 0).<M+N> If the number of available reference pixels is less than k, the reference pixel cannot be used in the correlation information acquisition process.

[0287] Alternatively, if the number of available reference pixels in the left block is greater than / exceeds p (p is greater than 0 but less than N) and the number of available reference pixels in the upper block is greater than / exceeds q (q is greater than 0 but less than M), the corresponding reference pixels can be used in the correlation information acquisition process. If the number of available reference pixels in the left block is less than / equal to p or the number of available reference pixels in the upper block is less than / equal to q, the corresponding reference pixels cannot be used in the correlation information acquisition process.

[0288] In the former case, it may be classification based on boundary value conditions in the entire region for obtaining correlation information, and in the latter case, it may be classification based on boundary value conditions in a partial (partial) region for obtaining correlation information. In the latter case, it may be an example of classifying adjacent regions for obtaining correlation information into left, upper, upper left, upper right, and lower left positions, but it may also be an example applicable to classification based on various adjacent region divisions (for example, division into left and upper positions, where the upper + upper right block is classified as upper *, and the left + lower left block is classified as left *).

[0289] In the latter case, the boundary values ​​can be set to the same or different values ​​for each region. For example, if all reference pixels (N) are available in the left block, the corresponding reference pixels can be used in the correlation information acquisition process, and if at least one reference pixel is available in the upper block (i.e., if some reference pixels are available), the corresponding reference pixel can be used in the correlation information acquisition process.

[0290] In addition, the above example may support the same or different settings depending on the color copy mode (e.g., a, b, c, etc. in FIG. 8). Related settings may also be defined differently depending on other encoding / decoding settings (e.g., image type, block size, shape, position, block division type, etc.).

[0291] When the correlation information acquisition process is not performed (i.e., when even one reference pixel is not used in the correlation information acquisition process), correlation information can be acquired implicitly. For example, a and b can be set to 1 and 0, respectively, in the color copy mode equation (i.e., the data of the corresponding block can be used as the predicted value of the current block as is). Alternatively, correlation information of a block that has been encoded / decoded in color copy mode or pre-set correlation information can be used.

[0292] Alternatively, the predicted value of the current block may be filled with any value (e.g., minimum, median, maximum, etc. of the bit depth or pixel value range), which is similar to the method used when all reference pixels are unavailable in general intra prediction.

[0293] The case where correlation information is implicitly acquired or where the predicted value is filled with an arbitrary value as in the above example can be applied to the case shown as d in Figure 11. That is, this is because not even one reference pixel is used in the correlation information acquisition process.

[0294] Next, a case will be described in which the use of a prediction mode in which prediction is made from reference pixels at unusable positions is restricted.

[0295] 12 is an exemplary diagram illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention. In the example described below, it is assumed that the left, upper, upper right, and lower left blocks of a current block and a corresponding block are areas used to obtain correlation information. It is also assumed that three color copy modes (mode_A, mode_B, and mode_C) are supported, and each mode is classified into a mode in which correlation information is obtained for the left+upper, upper+upper right, and left+lower left blocks of each block.

[0296] 12, a in FIG. 12 shows a case where the current block is located inside an image (picture, slice, tile, etc.), b in FIG. 12 shows a case where the current block is located at the left boundary of an image, c in FIG. 12 shows a case where the current block is located at the top boundary of an image, and d in FIG. 12 shows a case where the current block is located at the top left boundary of an image. In other words, it is assumed that reference pixel availability is determined based on the position of the current block.

[0297] Referring to Figure 12, mode_A, mode_B, and mode_C can be supported in Figure 12a, mode_B can be supported in Figure 12b, mode_C can be supported in Figure 12c, and no mode can be supported in Figure 12d. That is, if even one reference pixel used for obtaining correlation information is unavailable, this can be a description of a case where the corresponding mode is not supported.

[0298] For example, when configuring a group of intra-frame prediction modes for a chrominance component, the group may include directional and non-directional modes, a color mode, and a color copy mode. Here, it is assumed that the group of candidates includes a total of seven prediction modes, including four prediction modes such as DC, planar, vertical, horizontal, and diagonal modes, one color mode, and three color copy modes.

[0299] As in the above example, the color copy mode in which correlation information is obtained from unavailable reference pixels can be excluded. In the situation of c in Figure 12, mode_c can be excluded and the remaining six prediction modes can form a candidate group for the chrominance component. That is, it is possible to adjust from a group of m prediction mode candidates to n (m>n, n is an integer equal to or greater than 1). This may also require changes to prediction mode index setting and entropy encoding / decoding such as binarization.

[0300] In the situation of FIG. 12d, mode_A, mode_B, and mode_C can be excluded, and a total of four candidate groups can be formed, including the remaining prediction modes, directional and non-directional modes, and color modes.

[0301] As in the above example, it is possible to configure an intra-frame prediction mode candidate group by applying a prediction mode use restriction setting based on unusable reference pixels.

[0302] As in the above example, when the reference pixel processing B setting is used, various processing methods can be supported. Depending on the encoding / decoding setting, reference pixel processing and intra prediction can be performed implicitly or explicitly based on one processing method.

[0303] The above example describes a case where the availability of reference pixels in an adjacent area is determined based on the position of a current block. That is, the current block and the corresponding block are located at the same position in an image (picture, slice, tile, largest coding block, etc.), and therefore, if a specific block (current block or corresponding block) is adjacent to an image boundary, the corresponding block is also located at the image boundary. Therefore, determining the availability of reference pixels based on the position of each block produces the same result.

[0304] Also, as mentioned above, restricted intra prediction is used as a criterion for determining the possibility of reference pixels, which may result in a possibility that the possibility of reference pixels in adjacent regions of the current block and the corresponding block may not be the same.

[0305] 13 is an exemplary diagram illustrating the availability of reference pixels in a color copy mode according to an embodiment of the present invention. In the example described below, it is assumed that the left and upper blocks of the current block and the corresponding block are areas used to obtain correlation information. That is, this is a description of mode_A in FIG. 12, and it is assumed that the contents described below can be applied identically or similarly to other color copy modes.

[0306] 13, the cases can be divided into (i) when all adjacent areas of both blocks (current block and corresponding block) are usable, (ii) when only a portion of the adjacent areas are usable, and (iii) when none of the adjacent areas are usable. Here, the case where only a portion of the adjacent areas are usable (ii) can be divided into (ii-1) when there is a commonly usable area in both blocks and (ii-2) when there is no commonly usable area. Here, the case where there is a commonly usable area in both blocks (ii-1) can be divided into (ii-1-1) when the corresponding areas in both blocks completely match and (ii-1-2) when only a portion of the adjacent areas are usable.

[0307] Referring to Figure 13, in the above classification, i and iii correspond to a and f in Figure 13, ii-2 corresponds to d and e in Figure 13, ii-1-1 corresponds to c in Figure 13, and ii-1-2 corresponds to b in Figure 13. Here, compared to the case where reference pixel possibility is determined based on the position of the current block (or corresponding block), ii-2 and ii-1-2 may be new situations that must be considered.

[0308] This process may be configured to include a step of determining the availability of reference pixels in a color copy mode. Depending on the determination result, an intra-frame prediction setting for the color copy mode, including processing related to reference pixels, may be determined. Next, an example of reference pixel processing and intra-frame prediction based on the availability of adjacent regions of a current block and a corresponding block will be described.

[0309] In the case of the reference pixel processing A setting, reference pixels at unusable positions can be filled using the various methods already described. However, detailed settings can be varied depending on the classification based on the availability of adjacent regions of the two blocks. The following description will focus on cases ii-2 and ii-1-2. Detailed descriptions of other classifications will be omitted as they may overlap with the above-described content of the present invention.

[0310] For example, d(ii-2) in Figure 13 may be a case where there is no area that can be used in common between the current block and the corresponding block. Also, b(ii-1-2) in Figure 13 may be a case where some adjacent areas that can be used in the current block and the corresponding block overlap. In this case, the data of the adjacent area at the unavailable position can be used in the correlation information acquisition process by filling in the data (e.g., filling in the available area) of the unavailable position.

[0311] Alternatively, e(ii-2) in Figure 13 may be a case where either the current block or the corresponding block has an adjacent area that is unavailable, i.e., one of the two blocks has no data to be used in the correlation information acquisition process, and the corresponding area can be filled with various data.

[0312] Here, the corresponding region can be filled using any value, such as the minimum value, median value, or maximum value based on the pixel value range (or bit depth) of the image.

[0313] Here, the corresponding area can be filled with a usable adjacent area of ​​another color space by copying, etc. In this case, since the adjacent areas of both blocks have the same image characteristics (i.e., the same data) through the above process, pre-set correlation information can be obtained. For example, by setting a to 1 and b to 0 in the correlation related equation, the data of the corresponding block is directly copied to the predicted value of the current block, and various other correlation information settings are possible.

[0314] In the case of the reference pixel processing B setting, the use of reference pixels at unusable positions or the use of the corresponding color copy mode can be restricted by the various methods already described above. However, detailed settings can be varied depending on the classification of the usability. The following description will focus on cases ii-2 and ii-1-2, and detailed descriptions of the other classifications will be omitted as they may overlap with the above-described content of the present invention.

[0315] For example, d and e (ii-2) in FIG. 13 may be cases where there is no area that can be used in common between the current block and the corresponding block. Since there is no overlapping area that can be compared to obtain correlation information between the two blocks, the use of the corresponding color copy mode may be restricted. Alternatively, the predicted value of the current block may be filled with an arbitrary value. In other words, this may mean that the correlation information obtaining process is not performed.

[0316] Alternatively, b(ii-1-2) in Figure 13 may be a case where a portion of the adjacent usable areas of the current block and the corresponding block overlap, so the correlation information acquisition process can be performed even if it is limited to a portion of the adjacent usable areas.

[0317] 14 is a flowchart illustrating an intra-frame prediction method for a color copy mode according to an embodiment of the present invention. Referring to FIG. 14, a designated reference pixel area can be identified to obtain correlation information (S1400). Then, a processing setting for the reference pixels can be determined based on whether the designated reference pixel area is usable (S1410). Then, intra-frame prediction can be performed according to the processing setting for the determined reference pixels (S1420). Here, correlation information can be obtained based on data of the reference pixel area that is usable according to the reference pixel processing setting, and a prediction block according to the color copy mode can be generated, or a prediction block filled with an arbitrary value can be generated.

[0318] In summary, when a color copy mode is supported for intra-frame prediction of chrominance components, a comparison area for obtaining correlation information designated by the color copy mode can be checked. Unlike general intra-frame prediction modes, the color copy mode can check not only adjacent areas of the current block (especially areas used for correlation information comparison) but also the availability of reference pixels of the corresponding block. Depending on a preset reference pixel processing setting or one of multiple reference pixel processing settings and the availability of the reference pixels, reference pixel processing and intra-frame prediction according to the various examples described above can be performed.

[0319] Here, the reference pixel processing setting may be implicitly determined by the image type, color components, block size / position / shape, block width-to-height ratio, coding mode, intra-frame prediction mode (e.g., the range, position, number of pixels, etc. of the area to be compared for obtaining correlation information in a color copy mode), limited intra-frame prediction setting, etc., or related information may be explicitly generated in units of a sequence, picture, slice, tile, etc. Here, the reference pixel processing setting may be defined by limiting the state information of the current block (or current image) or limiting the state information of the corresponding block (or other color image), or may be defined by combining multiple state information.

[0320] Although the reference pixel processing settings A and B have been described separately in the above example, the two settings can be used independently or in combination, which can also be determined based on the state information or explicit information.

[0321] After the reference pixel interpolation unit completes the construction of the reference pixels, it can generate a small number of reference pixels by linearly interpolating the reference pixels, or it can perform a reference pixel filtering process (to be described later) before performing the reference pixel interpolation process.

[0322] Here, the interpolation process is not performed in the case of horizontal, vertical, some diagonal modes (e.g., modes with a 45-degree difference between vertical and horizontal, such as diagonal up right, diagonal down right, and diagonal down left; corresponding to modes 2, 34, and 66 in FIG. 4), non-directional mode, and color copy mode, but the interpolation process can be performed in the case of other modes (other diagonal modes).

[0323] The pixel position to be interpolated (i.e., which decimal unit is to be interpolated, determined from 1 / 2 to 1 / 64, etc.) can be determined based on the prediction mode (e.g., the directionality of the prediction mode, such as dy / dx) and the positions of the reference pixel and the predicted pixel. Here, regardless of the decimal precision, one filter (e.g., assuming the same filter for determining the filter coefficients or filter tap lengths, but assuming a filter in which only coefficients are adjusted according to decimal precision <e.g., 1 / 32, 7 / 32, 19 / 32>) can be applied, or one of multiple filters (e.g., assuming filters in which the mathematical formulas used to determine the filter coefficients or filter tap lengths are differentiated) can be selected and applied according to the decimal precision.

[0324] In the former case, integer unit pixels may be used as input for fractional unit pixel interpolation, and in the latter case, input pixels may be different for each stage (for example, integer pixels are used for 1 / 2 unit, integer and 1 / 2 unit pixels are used for 1 / 4 unit, etc.), but are not limited thereto. In the present invention, the former case will be mainly described.

[0325] Fixed or adaptive filtering can be performed for reference pixel interpolation, which can be determined by encoding / decoding settings (e.g., one or a combination of two or more of the image type, color components, block position / size / shape, block width-to-height ratio, prediction mode, etc.).

[0326] Fixed filtering can perform reference pixel interpolation using one filter, while adaptive filtering can perform reference pixel interpolation using one of a plurality of filters.

[0327] In the case of adaptive filtering, one of multiple filters can be implicitly or explicitly determined depending on the encoding / decoding setting. Here, the filter types can be configured as a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, an 8-tap Kalman filter, etc., and the filter candidate set supported by the color component can be defined differently (for example, some of the filter types can be the same or different, or the filter tap length can be short or long, etc.).

[0328] The reference pixel filter unit can perform filtering on the reference pixels to reduce degradation remaining in the encoding / decoding process and thereby improve prediction accuracy. The filter used in this case can be, but is not limited to, a low-pass filter. Whether or not filtering is applied can be determined depending on the encoding / decoding setting (which can be derived from the above description). In addition, when filtering is applied, fixed filtering or adaptive filtering can be applied.

[0329] Fixed filtering means that reference pixel filtering is not performed or that reference pixel filtering is applied using one filter. Adaptive filtering means that whether filtering is applied is determined by the encoding / decoding settings, and if there are two or more supported filter types, one of them can be selected.

[0330] Here, the filter type can support a variety of filters classified by various filter coefficients, filter tap lengths, etc., such as a 3-tap filter such as [1, 2, 1] / 4, or a 5-tap filter such as [2, 3, 6, 3, 2] / 16.

[0331] The reference pixel interpolator and the reference pixel filter introduced in the reference pixel configuration step may be components necessary to improve prediction accuracy. The two processes may be performed independently, but they may also be combined (i.e., processed in one filtering).

[0332] The prediction block generator may generate a prediction block according to at least one prediction mode and may use reference pixels according to the prediction mode, where the reference pixels may be used in a method such as extrapolation (directional mode) or in a method such as interpolation, DC, or copy (non-directional mode) depending on the prediction mode.

[0333] Next, reference pixels used depending on the prediction mode will be described.

[0334] In the directional mode, modes between horizontal and some diagonal modes (Diagonal up right) (modes 2 to 17 in Figure 4) can use reference pixels from the lower left + left block (Ref_BL, Ref_L in Figure 10), horizontal modes can use the left block, modes between horizontal and vertical (modes 19 to 49 in Figure 4) can use the left + upper left + upper block (Ref_L, Ref_TL, Ref_T in Figure 10), vertical modes can use the upper block (Ref_L in Figure 10), and modes between vertical and some diagonal modes (Diagonal down left) (modes 51 to 66 in Figure 4) can use reference pixels from the upper + upper right block (Ref_T, Ref_TR in Figure 10).

[0335] In addition, in the non-directional mode, reference pixels located in one or more of the lower left, left, upper left, upper, and upper right blocks (Ref_BL, Ref_L, Ref_TL, Ref_T, and Ref_TR in FIG. 10) can be used. For example, various combinations of reference pixels, such as left, upper, left + upper, left + upper + upper left, left + upper + upper left + upper right + lower left, can be used for intra prediction, and this can be determined depending on the non-directional mode (DC, Planar, etc.). In the example described below, it is assumed that the DC mode uses the left + upper block, and the planar mode uses the left + upper + lower left + upper right blocks as reference pixels for prediction.

[0336] In addition, in the color copy mode, a reconstructed block in another color space (Ref_C in FIG. 10) can be used as a reference pixel. In the example described below, a case where the current block and the corresponding block are used as reference pixels for prediction will be mainly described.

[0337] Here, the reference pixels used in intra prediction may be divided into multiple concepts (or units). For example, the reference pixels used in intra prediction may be divided into one or more categories, such as first reference pixels and second reference pixels. For convenience of explanation, the reference pixels are divided into first reference pixels and second reference pixels, but it can be understood that other additional reference pixels are also supported.

[0338] Here, the first reference pixels may be pixels directly used in generating a predicted value of the current block, and the second reference pixels may be pixels indirectly used in generating a predicted value of the current block. Alternatively, the first reference pixels may be pixels used in generating a predicted value of all pixels in the current block, and the second reference pixels may be pixels used in generating a predicted value of some pixels in the current block. Alternatively, the first reference pixels may be pixels used in generating a primary predicted value of the current block, and the second reference pixels may be pixels used in generating a secondary predicted value of the current block. Alternatively, the first reference pixels may be pixels located based on the starting point (or origin) of the prediction direction of the current block, and the second reference pixels may be pixels located regardless of the prediction direction of the current block.

[0339] As described above, performing prediction using the second reference pixels can be called a prediction block (or correction) process. That is, the prediction block generator of the present invention can be configured to add a prediction block corrector.

[0340] Here, the reference pixels used in the prediction block generator and the prediction block corrector are not limited to the first and second reference pixels in each configuration. That is, the prediction block generator may perform prediction using the first reference pixels or the first and second reference pixels, and the prediction block corrector may perform prediction (or correction) using the second reference pixels or the first and second reference pixels. It should be understood that the present invention will be described by dividing into a plurality of pixel concepts for convenience of explanation.

[0341] In the intra prediction of the present invention, prediction can be performed using not only first reference pixels but also second reference pixels (i.e., correction), which can be determined by encoding / decoding settings. First, information on whether second reference pixels are supported (i.e., whether prediction value correction is supported) can be generated in units of a sequence, picture, slice, tile, etc. Even if it is explicitly or implicitly determined that second reference pixels are supported, whether they are supported in all blocks or in some blocks, and detailed settings related to the second reference pixels in supported blocks (related details can be seen below) can depend on encoding / decoding settings defined by image type, color components, block size / shape / position, block width / height ratio, encoding mode, intra prediction mode, restricted intra prediction settings, etc. Alternatively, related setting information can be explicitly determined in units of a sequence, picture, slice, tile, etc.

[0342] Here, the numbers of first and second reference pixels used for the predicted value of one pixel may be m and n, respectively, and m and n may have a pixel number ratio of (1:1), (1:2 or more), (2 or more:1), or (2 or more:2 or more), which may be determined depending on the prediction mode, the size / shape / position of the current block, pixel position, etc. That is, m may be an integer of 1 or more, such as 1, 2, or 3, and n may be an integer of 1 or more, such as 1, 2, 3, 4, 5, or 8.

[0343] When the weights applied to the first and second reference pixels (assuming one pixel each is used in this example) are p and q, p can be greater than or equal to q, p can have a positive value, and q can have a positive or negative value.

[0344] Next, a case will be described in which prediction is performed using second reference pixels in addition to generating a prediction block using first reference pixels.

[0345] For example, (see Figure 10) Diagonal up right direction mode is<Ref_BL+Ref_L> Block or<Ref_BL+Ref_L+Ref_T+Ref_TR> Prediction can be performed using the Ref_L block or the horizontal mode.<Ref_L+Ref_T+Ref_TL> Prediction can be performed using blocks. Diagonal down right direction mode is also available.<Ref_TL+Ref_T+Ref_L> Block or<Ref_TL+Ref_T+Ref_L+Ref_TR+Ref_BL> Prediction can be performed using blocks, and vertical mode can be performed using Ref_T blocks or<Ref_T+Ref_L+Ref_TL> Prediction can be performed using blocks. Diagonal down left direction mode is also available.<Ref_TR+Ref_T> Block or<Ref_TR+Ref_T+Ref_L+Ref_BL> Blocks can be used to perform predictions.

[0346] Another example is DC mode.<Ref_T+Ref_L> Block or<Ref_T+Ref_L+Ref_TL+Ref_TR+Ref_BL> Prediction can be performed using blocks. Also, the planar mode<Ref_T+Ref_L+Ref_TR+Ref_BL> Block or<Ref_T+Ref_L+Ref_TR+Ref_BL+Ref_TL> Blocks can be used to perform predictions.

[0347] Also, the color copy mode is Ref_C block or<Ref_C+(Ref_T or Ref_L or Ref_TL or Ref_TR or Ref_BL)> You can use blocks to perform predictions, or<Ref_C+(Def_T or Def_B or Def_L or Def_R or Def_TL or Def_TR or Def_BL or Def_BR)> 10, Def is a term used to refer to adjacent blocks of Ref_C (the current block and its corresponding block), and Def_T to Def_BR can be adjacent blocks in the upper, lower, left, right, upper left, upper right, lower left, or lower right direction of Ref_F. That is, in the color copy mode, prediction can be performed on the Ref_C block using reference pixels adjacent to the current block or reference pixels adjacent to Ref_C (the corresponding block).

[0348] The above examples show some examples in which prediction is performed using the first reference pixels or the first and second reference pixels, but the present invention is not limited thereto and various modifications are possible.

[0349] Generating or correcting a prediction block using multiple reference pixels can be performed to compensate for the shortcomings of existing prediction modes.

[0350] For example, in the directional mode, a mode may be supported for the purpose of increasing prediction accuracy by assuming the presence of an edge in a specific direction of the current block, but the accuracy of prediction may be reduced because changes within the block may not be accurately reflected using only reference pixels located at the start point of the prediction direction. Alternatively, in the color copy mode, prediction is performed by reflecting correlation information from another color image at the same time, but the accuracy of prediction may be reduced because degradation remaining at the block boundary in the other color image may be reflected. To solve this problem, the accuracy of prediction may be increased by additionally using second reference pixels.

[0351] Next, a case where prediction is performed using multiple reference pixels in a color copy mode will be described. Portions of the examples not shown in the drawings below can be explained with reference to FIG. 10. Here, the explanation will focus on conceptual aspects, assuming that the correlation information acquisition process of the color copy mode can be performed in a previous process or a subsequent process. Furthermore, except for acquiring a predicted value from another color space, the intra prediction in the color copy mode described below can be applied identically or similarly to other intra prediction modes.

[0352] 15 is an exemplary diagram illustrating prediction in a color copy mode according to an embodiment of the present invention, in which a predicted block can be generated using reference pixels of a corresponding block in the color copy mode.

[0353] In detail, correlation information can be obtained from the adjacent regions (specified by the color copy mode) of the current block and the corresponding block (p1). Then, data is obtained from the corresponding block (p2), and the previously obtained correlation information is applied (p3) to obtain a predicted block (pred_t). This can be compensated for with the predicted block (pred_f) of the current block (p4).

[0354] The above example can be explained from the color copy mode described above, so a detailed explanation will be omitted.

[0355] 16 is an example diagram illustrating prediction in a color copy mode according to an embodiment of the present invention. Referring to FIG. 16, in the color copy mode, a predicted block can be generated and corrected using a corresponding block and its adjacent reference pixels.

[0356] Specifically, correlation information can be obtained from the adjacent regions of the current block and the corresponding block (p1). Then, correction can be performed on the data of the corresponding block. Here, the correction can be limited to the inside of the corresponding block (d5), limited to the boundary of the block adjacent to the corresponding block (d1 to d9 excluding d5), or performed across the boundary between the inside and outside of the corresponding block (d1 to d9). That is, data of the corresponding block and its adjacent region can be used for correction. Here, the outer boundary to be corrected can be one or more (including all) of the upper, lower, left, right, upper left, upper right, lower left, and lower right directions (d1 to d9, respectively, excluding d5).

[0357] Then, data is obtained through the correction process of the corresponding block (p2), and the previously obtained correlation information is applied (p3) to obtain a predicted block (pred_t), which can be compensated for with the predicted block (pred_f) of the current block (p4).

[0358] 17 is an example diagram illustrating prediction in a color copy mode according to an embodiment of the present invention. Referring to FIG. 17, in the color copy mode, a predicted block can be generated and corrected using adjacent reference pixels of a corresponding block and a current block.

[0359] Specifically, correlation information can be obtained from the adjacent regions of the current block and the corresponding block (p1). Then, data is obtained from the corresponding block (p2), and the previously obtained correlation information is applied (p3) to obtain a predicted block (pred_t). This can be compensated for by the primary predicted block of the current block (p4), and correction can be performed on the primary predicted block (referred to as the predicted block). Here, correction can be limited to the inside of the predicted block (c5), or limited to the boundary between the predicted block and the adjacent block (c1 to c6 excluding c5), or performed across the inside and outside boundaries of the predicted block (c1 to c6). That is, the predicted block (i.e., data based on the corresponding block; here, "based" refers to the application of correlation information) and adjacent data of the predicted block (data in the adjacent region of the current block) can be used for correction. Here, the outer boundary to be corrected may be one or more (all included) of the upper, left, upper left, upper right, and lower left directions (c1 to c6, respectively, excluding c5).

[0360] The data obtained by the correction process of the predicted block can be compensated for with the (secondary or final) predicted block (pred_f) of the current block (p5).

[0361] 18 is a flowchart of a process for performing correction in a color copy mode according to an embodiment of the present invention, specifically, a process for selecting one of the correction processes described with reference to FIGS.

[0362] 18, correlation information can be obtained from adjacent regions of a current block and a corresponding block (S1800). Then, data of the corresponding block can be obtained (S1800). The following description assumes that it is implicitly or explicitly determined that compensation of the predicted block is to be performed. It can be determined whether compensation of the predicted block is to be performed in the current color space or in another color space (S1820).

[0363] If the predicted block is to be corrected in the current color space, correlation information is applied to the data of the corresponding block to generate the predicted block (S1830). This may be the same as the process in a general color copy mode. Then, the predicted block can be corrected using adjacent areas of the current block (S1840). Here, not only the adjacent areas of the current block but also the internal data of the obtained predicted block can be used.

[0364] If a prediction block is to be corrected in another color space, the correction of the corresponding block can be performed using adjacent regions of the corresponding block (S1850). Here, not only the adjacent regions of the corresponding block but also internal data of the corresponding block can be used. Then, a prediction block can be generated by applying correlation information to the corrected corresponding block (S1860).

[0365] The data obtained through the above process can be compensated for by the predicted block of the current block (S1870).

[0366] The classification by color space in the above example does not mean that the application examples of Figures 16 and 17 cannot be used together. That is, the application examples of Figures 16 and 17 can be combined. For example, as shown in Figure 16, a predicted block of a current block can be obtained by a correction process of a corresponding block, and as shown in Figure 17, a final predicted block can be obtained by a correction process of the obtained predicted block.

[0367] Performing correction in intra prediction of the present invention may mean applying filtering to a pixel to be corrected and other pixels (adjacent pixels in this example). Here, filtering may be performed using one pre-set filtering setting or by selecting one of multiple filtering settings. Here, the filtering setting may be the content included in the above-mentioned second reference pixel-related detailed setting.

[0368] The filtering setting may include whether filtering is applicable, the type of filter, filter coefficients, pixel positions used in the filter, etc. Here, the unit to which the filtering setting is applied may be a block or a pixel. Here, the encoding / decoding setting may be defined according to the image type, color components, color format (i.e., the composition ratio between color components), coding mode (Intra / Inter), block size / shape / position, block width-to-height ratio, pixel positions within the block, intra-frame prediction mode, constrained intra-frame prediction, etc., and may be involved in the filtering setting. Here, the term "block" is described assuming the current block, but it may also be understood as a concept including adjacent blocks of the current block or adjacent blocks of a corresponding block in a color copy mode. That is, this means that state information of the current block and other blocks can act as input variables for the filtering setting. In addition, information about the filtering setting may be explicitly included in units of a sequence, picture, slice, tile, block, etc.

[0369] Next, the types of filters in the filtering settings will be described. Filtering can be applied to adjacent pixels on a single line, such as a horizontal, vertical, or diagonal line, with the pixel to be corrected at the center (i.e., one-dimensional). Alternatively, filtering can be applied to spatially adjacent pixels to the left, right, upper, lower, upper left, upper right, lower left, or lower right, with the pixel to be corrected at the center (i.e., two-dimensional). In other words, filtering can be applied to adjacent pixels within an M×N area with the pixel to be corrected at the center. In the examples described below, we will assume that both M and N are 3 or less, but M or N can also have a value greater than 3. Generally, filtering can be applied to adjacent pixels symmetrically with respect to the pixel to be corrected, but an asymmetric configuration is also possible.

[0370] 19 is an exemplary diagram illustrating types of filters applied to a pixel to be corrected according to an embodiment of the present invention, specifically, an example in which filter application positions are determined symmetrically around a pixel to be corrected (thick line in the drawing).

[0371] Referring to Figure 19, Figures 19a and 19b represent horizontal and vertical 3-tap filters, Figures 19c and 19d represent diagonal 3-tap filters (tilted at angles of -45 and +45 degrees from the vertical), Figures 19e and 19f represent 5-tap filters with (+) or (x) shapes, and Figure 19g represents a square 9-tap filter.

[0372] As an example for applying filtering, a to g in FIG. 19 can be applied to the inside or outside (or boundary) of a block.

[0373] As another example, e to g in Fig. 19 can be applied to the interior of a block. Or a to d in Fig. 19 can be applied to the boundaries of the block. In particular, a in Fig. 19 can be applied to the left boundary of the block, and b in Fig. 19 can be applied to the top boundary of the block. c in Fig. 19 can be applied to the top left boundary of the block, and d in Fig. 19 can be applied to the top right and bottom left boundaries of the block.

[0374] The above examples are only some of the cases for filter selection based on the position of the pixel to be corrected, but are not limited thereto, and various application examples including the opposite case are possible.

[0375] 20 is an exemplary diagram illustrating types of filters applied to a pixel to be corrected according to an embodiment of the present invention, particularly illustrating a case where filter application positions are determined asymmetrically around the pixel to be corrected.

[0376] Referring to Figure 20, a and b in Figure 20 represent 2-tap filters that use only the left and upper pixels, respectively, and c in Figure 20 represents a 3-tap filter that uses only the left and upper pixels. d to f in Figure 20 represent 4-tap filters in the upper-left direction, upper direction, and left direction, respectively. g to j in Figure 20 represent 6-tap filters in the upper-left, upper-left, lower-left, and lower-right directions, respectively.

[0377] As an example for applying filtering, a to j in FIG. 20 can be applied inside or outside the block.

[0378] As another example, g to j in Fig. 20 can be applied to the interior of a block. Or a to f in Fig. 20 can be applied to the boundary of a block. In particular, a and f in Fig. 20 can be applied to the left boundary of a block, and b and e in Fig. 20 can be applied to the top boundary of a block. C and d in Fig. 20 can be applied to the top left boundary of a block.

[0379] The above examples are only some of the cases for filter selection based on the position of the pixel to be corrected, but are not limited thereto, and various application examples including the opposite case are possible.

[0380] 19 and 20 can be set in various ways. For example, in the case of a 2-tap filter, weights are set in a ratio of 1:1 or 1:3 (where 1 and 3 are weights of the pixel to be corrected), in the case of a 3-tap filter, weights are set in a ratio of 1:1:2 (where 2 is the weight of the pixel to be corrected), in the case of a 4-tap filter, weights are set in a ratio of 1:1:1:5 or 1:1:2:4 (where 4 and 5 are the weights of the pixel to be corrected), and in the case of a 5-tap filter, weights are set in a ratio of 1:1:2 (where 2 is the weight of the pixel to be corrected). Weights may be applied in a ratio of 1:1:1:4 (where 4 is the weight of the pixel to be corrected), in the case of a 6-tap filter, weights in a ratio of 1:1:1:1:2:2 (where 2 is the weight of the pixel to be corrected), and in the case of a 9-tap filter, weights in a ratio of 1:1:1:1:1:1:1:1:1:8 or 1:1:1:1:1:2:2:2:2:4 (where 8 and 4 are the weights of the pixel to be corrected). In the above examples, the pixel to which the next highest weight is applied after the weight of the pixel to be corrected may be a pixel that is close to the pixel to be corrected (vertically and horizontally adjacent) or a pixel located at the center of a symmetrical structure like the pixel to be corrected. The above examples are merely some examples of weight setting, and are not limited thereto, and various modifications are possible.

[0381] The filter used for compensation may be implicitly determined by the encoding / decoding setting or may be explicitly included in units of a sequence, picture, slice, tile, etc. Here, the description of defining the encoding / decoding setting may be derived from the various examples described above of the present invention.

[0382] Here, information indicating whether multiple filters are supported may be generated for each unit. If multiple filters are not supported, a pre-defined filter may be used, and if multiple filters are supported, additional filter selection information may be generated. Here, the filters may include the filters shown in Figures 19 and 20 or other filters in the candidate set.

[0383] Next, assume that compensation is performed in another color space as shown in Figure 16, and refer to Figure 10 (i.e., the block to be compensated is Ref_C). In the example described below, assume that the filter to be used for the block is determined by the previous process and that the filter is applied to all or most of the pixels in the block.

[0384] Although a collective filter is supported based on the state information of the neighboring regions of the correction target block, an adaptive filter based on the position of the correction target pixel can also be supported.

[0385] For example, a 5-tap filter (e in FIG. 19) can be applied to aa to pp of Ref_C centered on the pixel to be corrected. In this case, the same filter can be applied regardless of the position of the pixel to be corrected. Also, there can be no restrictions on applying filtering to adjacent areas of the block to be corrected.

[0386] Alternatively, a 5-tap filter may be applied to ff, gg, jj, and kk of Ref_C (i.e., inside the block) centered on the pixel to be corrected, and filtering may be applied to other pixels (block boundaries) based on the status of adjacent blocks. In particular, if the block to the left of Ref_C is on the outer edge of the picture or the coding mode is Inter (i.e., when a restricted intra-frame prediction setting is activated) and it is determined that the usability of the corresponding pixel is not possible (related explanations can be derived from the section confirming the usability of reference pixels of the present invention), a vertical filter (e.g., b in FIG. 19) may be applied to aa, ee, ii, and mm of Ref_C. Alternatively, if the block above Ref_C is not usable, a horizontal filter (e.g., a in FIG. 19) may be applied to aa to dd of Ref_C.

[0387] Although a collective filter is supported based on the intra-frame prediction mode, an adaptive filter based on the position of the pixel to be corrected can also be supported.

[0388] For example, in the color copy mode, in the case of a mode in which correlation information is obtained from the left and upper blocks, a 9-tap filter (g in FIG. 19) can be applied to aa to pp of Ref_C. In this case, the same filter can be applied regardless of the position of the pixel to be corrected.

[0389] Alternatively, in the color copy mode, in the mode in which correlation information is obtained from the left and lower left blocks, a 9-tap filter can be applied to ee to pp of Ref_C, and other pixels (upper boundary) can be filtered based on the prediction mode setting. In this example, since correlation information is obtained from the left and lower left blocks, it can be estimated that the correlation with the upper block is low. Therefore, a filter in the left and lower left direction (for example, i in FIG. 20) can be applied to aa to dd of Ref_C.

[0390] In the present invention, the color copy mode has been described mainly for the case of a color format of 4:4:4. The detailed configuration of the correction may differ depending on the color format. This example assumes that correction is performed in another color space.

[0391] In color copy mode, correlation information and predicted values ​​can be obtained by using the 4:4:4 color format, where one pixel in the current color space corresponds to one pixel in another color space, and the related data can be obtained directly from the corresponding pixel.

[0392] On the other hand, in the case of some color formats (4:2:0), one pixel in the current color space corresponds to one or more pixels (four in this example) in another color space. If a single predefined pixel is not selected and related data is not obtained from that pixel, a downsampling process may be required to obtain related data from multiple corresponding pixels.

[0393] Next, we will explain the prediction and correction process of the color copy mode for each color format. In the example below, we will assume that a downsampling process is performed for some formats (4:2:0). Also, the current block and the corresponding block are called blocks A and B, respectively.

[0394] <1> In-screen prediction in 4:4:4 format <1-1> Obtain pixel values ​​from adjacent areas of block A <1-2> Obtain pixel values ​​corresponding to the pixels in <1-1> from the adjacent area of ​​block B <1-3> Obtain correlation information based on pixel values ​​of adjacent areas in each color space <1-4> Extraction of pixels in block B and adjacent pixels Apply filtering to the pixels of <1-5> and <1-4> to correct the B block. <1-6> Obtain pixel values ​​in block B (M × N) corresponding to pixels in block A (M × N) <1-7> Apply correlation information to pixel values ​​in <1-6> to generate predicted pixels

[0395] <2> In-frame prediction in 4:2:0 format (1) <2-1> Obtain pixel values ​​from adjacent areas of block A <2-2> Extract pixels corresponding to the pixel in <2-1> and its neighboring pixels from the adjacent area of ​​block B <2-3> Apply downsampling to the pixels in <2-2> and obtain the pixel value corresponding to the pixel in <2-1> from the adjacent area of ​​block B. <2-4> Obtain correlation information based on pixel values ​​of adjacent areas in each color space <2-5> Extraction of pixels in block B and adjacent pixels Apply filtering to the pixels in <2-6> and <2-5> to correct the B block. <2-7> Extraction of pixels in block B (2M x 2N) corresponding to pixels in block A (M x N) and their adjacent pixels Apply downsampling to the pixels of <2-8> and <2-7> to obtain the pixel values ​​of block B. Apply correlation information to the pixel values ​​of <2-9> and <2-8> to generate predicted pixels.

[0396] <1> and <2> To explain the process, <1> applies one filtering in <1-5>, <2> It can be seen that multiple filtering processes are applied in <2-6> and <2-8>. <2-6> can be a process for correcting data used to obtain predicted pixels, and <2-8> can be a downsampling process for obtaining predicted pixels, and the filters in each process can be configured differently. Of course, each process can improve coding performance, but redundant filtering effects can occur. Also, the increased complexity due to the increased number of filtering processes may be unsuitable for some profiles. For this reason, in the case of some color formats, it may be necessary to support filtering that integrates the process.

[0397] <3> In-frame prediction in 4:2:0 format (2) <3-1> Obtain pixel values ​​from adjacent areas of block A <3-2> Extract pixels corresponding to the pixel in <3-1> and its neighboring pixels from the adjacent area of ​​block B Apply downsampling to the pixels in <3-3> and <3-2> to obtain pixel values ​​corresponding to the pixels in <3-1> in the adjacent area of ​​block B. <3-4> Obtain correlation information based on pixel values ​​of adjacent areas in each color space <3-5> Extraction of pixels in block B and adjacent pixels Apply filtering to the pixels in <3-6> and <3-5> to correct the B block. <3-7> Obtain pixel values ​​of block B (2M x 2N) corresponding to pixels of block A (M x N) <3-8> Generate predicted pixels by applying correlation information to the pixel values ​​of <3-7>

[0398] <4> In-frame prediction in 4:2:0 format (3) <4-1> Obtain pixel values ​​from adjacent areas of block A <4-2> Extract pixels corresponding to the pixel in <4-1> and its neighboring pixels from the adjacent area of ​​block B Apply downsampling to the pixels in <4-3> and <4-2> to obtain pixel values ​​corresponding to the pixels in <4-1> from the adjacent area of ​​block B. <4-4> Obtain correlation information based on pixel values ​​of adjacent areas in each color space <4-5> Extraction of pixels in block B (2M x 2N) corresponding to pixels in block A (M x N) and their adjacent pixels Apply downsampling to the pixels in <4-6> and <4-5> to obtain the pixel values ​​of block B. Apply correlation information to the pixel values ​​in <4-7> and <4-6> to generate predicted pixels.

[0399] <3> To explain the process, the downsampling process of block B is omitted and correlation information is applied to one pixel at a predetermined position. Instead, a correction process is performed, which can eliminate the drawbacks of the downsampling process.

[0400] on the other hand, <4> To explain this process, the compensation process is omitted and the downsampling process of block B is performed. Instead, a filter that can achieve the compensation effect can be used during the downsampling process. Although variations may occur due to the configuration in which the compensation process is included in the downsampling, the settings for the compensation process described above can be applied as is. That is, filter selection information for downsampling can be explicitly determined at a higher level or implicitly determined based on encoding / decoding settings. In addition, it can be implicitly or explicitly determined whether downsampling is performed using only internal data of block B, or whether downsampling is performed using external data in one or more directions, such as block B and left, right, upper, lower, upper left, upper right, lower left, or lower right.

[0401] The prediction mode determination unit performs a process for selecting an optimal mode from a group of multiple candidate prediction modes. Generally, the optimal mode can be determined in terms of coding cost using a rate-distortion technique that considers block distortion (e.g., distortion between a current block and a reconstructed block, SAD (Sum of Absolute Difference), SSD (Sum of Square Difference, etc.)) and the amount of bits generated by the corresponding mode. A prediction block generated based on the prediction mode determined by the above process can be transmitted to a subtraction unit and an addition unit.

[0402] To determine the optimal prediction mode, all prediction modes present in the prediction mode candidate group may be searched, or the optimal prediction mode may be selected through another determination process for the purpose of reducing the amount of calculation / complexity. For example, in a first step, some modes that exhibit good performance in terms of image quality degradation may be selected from all intra-frame prediction mode candidates, and in a second step, the optimal prediction mode may be selected from the modes selected in the first step, taking into consideration not only image quality degradation but also the amount of generated bits. In addition to the above methods, various methods may be applied in terms of reducing the amount of calculation / complexity.

[0403] In addition, the prediction mode determination unit may generally be a configuration that is included only in the encoder, but may also be a configuration that is included in the decoder depending on encoding / decoding settings. For example, if template matching is included as a prediction method or a method of guiding an intra-frame prediction mode in a neighboring region of a current block is included, in the latter case, it can be understood that a method of implicitly acquiring a prediction mode in a decoder is used.

[0404] The prediction mode encoding unit may encode the prediction mode selected by the prediction mode determination unit. The prediction mode encoding unit may encode index information corresponding to the prediction mode from a prediction mode candidate group, or may predict the prediction mode and encode information about the prediction mode. The former may be a method applied to a luminance component, and the latter may be a method applied to a chrominance component, but is not limited thereto.

[0405] When encoding by predicting a prediction mode, the predicted value (or prediction information) of the prediction mode can be called an MPM (Most Probable Mode). The MPM can be configured with one prediction mode or multiple prediction modes, and the number of MPMs (k, where k is an integer equal to or greater than 1, such as 1, 2, 3, 6, etc.) can be determined depending on the number of prediction mode candidate groups. When the MPM is configured with multiple prediction modes, it can be called an MPM candidate group.

[0406] MPM is a concept that supports efficient coding of prediction modes, and actually, a group of candidates can be formed based on the prediction mode of the current block, which has a high probability of occurrence.

[0407] For example, the MPM candidate group may be configured with preset prediction modes (or statistically frequently occurring prediction modes, such as DC, Planar, vertical, horizontal, and some diagonal modes), prediction modes of adjacent blocks (left, upper, upper left, upper right, lower left block, etc.), etc. Here, the prediction modes of adjacent blocks may be obtained from L0 to L3 (left block), T0 to T3 (upper block), TL (upper left block), R0 to R3 (upper right block), and B0 to B3 (lower left block) in FIG. 10.

[0408] When an MPM candidate group can be configured from two or more sub-block positions (e.g., L0, L2, etc.) in an adjacent block (e.g., a left block), the prediction modes of the corresponding block can be configured as a candidate group according to a predefined priority (e.g., L0-L1-L2, etc.). Alternatively, when an MPM candidate group cannot be configured from two or more sub-block positions, the prediction modes of sub-blocks corresponding to a predefined position (e.g., L0, etc.) can be configured as a candidate group. In particular, prediction modes at positions L3, T3, TL, R0, and B0 in the adjacent blocks can be selected as prediction modes of the corresponding adjacent blocks and included in the MPM candidate group. The above description is a partial example of configuring prediction modes of adjacent blocks as a candidate group, and is not limited to this example. In the example described below, a case where prediction modes at predefined positions are configured as a candidate group will be assumed.

[0409] In addition, when one or more prediction modes are configured in the MPM candidate group, modes derived from one or more prediction modes already included can also be configured as additional MPM candidate groups. In particular, when the kth mode (directional mode) is included in the MPM candidate group, modes derivable from the corresponding mode (modes spaced +a, -b based on k, where a and b are integers equal to or greater than 1, such as 1, 2, and 3) can be additionally included in the MPM candidate group.

[0410] There may be a priority for constructing MPM candidates, and the MPM candidates may be constructed in the order of prediction mode of adjacent blocks, preset prediction mode, induced prediction mode, etc. The process of constructing MPM candidates may be completed when the maximum number of MPM candidates is reached according to the priority. This process may include a redundancy confirmation process in which, if a prediction mode matches an already included prediction mode, the corresponding prediction mode is not configured in the candidate group and the procedure moves to the next priority candidate.

[0411] Next, it is assumed that the MPM candidate group is composed of six prediction modes.

[0412] For example, the candidate group may be configured in the order of LT-TL-TR-BL-Planar-DC-Vertical-Horizontal-Diagonal mode, etc. This may be the case where prediction modes of adjacent blocks are preferentially configured in the candidate group, and a previously set prediction mode is additionally configured.

[0413] Or LT-Planar-DC-<L+1> - <l-1> -<T+1>- <t-1>Candidates may be configured in the order of vertical-horizontal-diagonal mode, etc. Prediction modes of some adjacent blocks and some of the preset prediction modes may be configured preferentially, and an induced mode and some of the preset prediction modes may be additionally configured under the assumption that a prediction mode in a direction similar to the prediction mode of the adjacent block will occur.

[0414] The above examples are only a few examples of MPM candidate group configuration, and are not limited to these, and various modifications are possible.

[0415] MPM candidates can use binarization such as unary binarization or truncated rice binarization based on the index within the candidate set. That is, the mode bits can be represented by assigning short bits to candidates with small indices and long bits to candidates with large indices.

[0416] Modes that cannot be included in the MPM candidate group can be classified into the non-MPM candidate group. In addition, the non-MPM candidate group can be classified into two or more candidate groups depending on the encoding / decoding settings.

[0417] Next, it is assumed that there are 67 modes including directional modes and non-directional modes in the prediction mode candidate group, and six MPM candidates are supported, resulting in a non-MPM candidate group consisting of 61 prediction modes.

[0418] If the non-MPM candidate group consists of one candidate, the prediction modes that could not be included in the MPM candidate group composition process remain, so no additional candidate group composition process is required. Therefore, binarization such as fixed length binarization or truncated unary binarization can be used based on the index within the non-MPM candidate group.

[0419] Assuming that the non-MPM candidate group consists of two or more candidate groups, in this example, the non-MPM candidate group is classified into non-MPM_A (hereinafter, candidate group A) and non-MPM_B (hereinafter, candidate group B). Assume that candidate group A (p candidates, equal to or greater than the number of MPM candidate groups) is configured as a prediction mode that is more likely to occur in the prediction mode of the current block than candidate group B (q candidates, equal to or greater than the number of candidate groups A). Here, a candidate group A configuration process can be added.

[0420] For example, some prediction modes with equal intervals among directional modes (e.g., modes 2, 4, and 6) can be configured as candidate group A, or pre-defined prediction modes (e.g., modes derived from prediction modes included in the MPM candidate group) can be configured. The remaining prediction modes from the MPM candidate group configuration and candidate group A configuration can be configured as candidate group B, and no additional candidate group configuration process is required. Binary coding, such as fixed-length binarization or truncated unary binarization, can be used based on the indexes within candidate group A and candidate group B.

[0421] The above example is a partial example in which the non-MPM candidate group is composed of two or more non-MPM candidates, but is not limited to this and various modifications are possible.

[0422] Next, a process for predicting and encoding a prediction mode will be described.

[0423] Information (mpm_flag) on ​​whether the prediction mode of the current block matches the MPM (or some modes in the MPM candidate group) can be checked.

[0424] If there is a match with the MPM, the MPM index information (mpm_idx) can be additionally checked depending on the MPM configuration (one or more), and then the encoding process of the current block is completed.

[0425] If there is no match with the MPM, and if the non-MPM candidate set consists of one, the non-MPM index information (remaining_idx) can be checked, and then the encoding process of the current block is completed.

[0426] If there are multiple non-MPM candidate groups (two in this example), information (non_mpm_flag) can be checked to see whether the prediction mode of the current block matches some of the prediction modes in candidate group A.

[0427] If it matches candidate group A, it can check candidate group A index information (non_mpm_A_idx), and if it does not match candidate group A, it can check candidate group B index information (remaining_idx).Then, the encoding process of the current block is completed.

[0428] When the prediction mode candidate group configuration is fixed, the prediction mode supported by the current block, the prediction mode supported by the neighboring block, and the preset prediction mode may use the same prediction number index.

[0429] On the other hand, when the prediction mode candidate group configuration is adaptive, the prediction mode supported by the current block, the prediction mode supported by the neighboring block, and the preset prediction mode may use the same prediction number index or different prediction number indexes. For the following explanation, refer to FIG. 4.

[0430] In the prediction mode encoding process, a prediction mode candidate group unification (or adjustment) process for configuring an MPM candidate group may be performed. For example, the prediction mode of a current block may be one of a prediction mode candidate group of modes -5 to 61, and the prediction mode of an adjacent block may be one of a prediction mode candidate group of modes 2 to 66. In this case, some of the prediction modes of the adjacent block (mode 66) may be a mode not supported by the prediction mode of the current block, so a process of unifying them may be performed in the prediction mode encoding process. That is, this process may not be required when supporting a fixed intra-frame prediction mode candidate group configuration, but may be required when supporting an adaptive intra-frame prediction mode candidate group configuration, and detailed description thereof will be omitted.

[0431] Unlike the method using the MPM, the coding method can be performed by assigning an index to a prediction mode belonging to a prediction mode candidate group.

[0432] For example, a method of assigning an index to a prediction mode according to a predefined priority and encoding the corresponding index when a prediction mode of a current block is selected corresponds to this method. This means that a group of prediction mode candidates is fixedly configured and a fixed index is assigned to a prediction mode.

[0433] Alternatively, when a prediction mode candidate group is adaptively configured, the fixed index assignment method may not be suitable. To this end, a method can be applied in which indexes are assigned to prediction modes according to adaptive priority, and when a prediction mode of a current block is selected, the corresponding index is encoded. This allows prediction modes to be efficiently encoded by varying the indexes assigned to prediction modes according to the adaptive configuration of a prediction mode candidate group. That is, the adaptive priority is used to assign a candidate that is likely to be selected as a prediction mode of a current block to an index that generates a short mode bit.

[0434] Next, it is assumed that eight prediction modes including the preset prediction modes (directional mode and non-directional mode), color copy mode, and color mode (chrominance component) are supported in the prediction mode candidate group.

[0435] For example, assume that the pre-configured prediction modes support four of Planar, DC, Horizontal, Vertical, and Diagonal modes (Diagonal down left in this example), one color mode (C), and three color copy modes (CP1, CP2, CP3). The basic order of indexes assigned to prediction modes may be given as preset prediction mode-color copy mode-color mode, etc.

[0436] Here, the pre-set prediction modes, directional mode, non-directional mode, and color copy mode, can be easily classified into prediction modes with different prediction methods. However, the color mode may be a directional mode or a non-directional mode, which may overlap with the pre-set prediction modes. For example, if the color mode is a vertical mode, it may overlap with the vertical mode, which is one of the pre-set prediction modes.

[0437] When the number of prediction mode candidates is adaptively adjusted according to encoding / decoding settings, if the overlap occurs, the number of candidates can be adjusted (from 8 to 7). Alternatively, when the number of prediction mode candidates is maintained fixed, if the overlap occurs, other candidates can be added and considered and indexes can be assigned, and this setting will be assumed below. In addition, the adaptive prediction mode candidate group can be configured to be supported even when a variable mode such as a color mode is included. Therefore, adaptive index assignment can be considered as an example of an adaptive prediction mode candidate group configuration.

[0438] Next, we will explain how to assign indexes adaptively according to the color mode. It is assumed that the basic indexes are assigned in the following order: Planar(0)-Vertical(1)-Horizontal(2)-DC(3)-CP1(4)-CP2(5)-CP3(6)-C(7). It is also assumed that if the color mode does not match the preset prediction mode, the indexes are assigned in the above order.

[0439] For example, if the color mode matches one of the preset prediction modes (planar, vertical, horizontal, DC mode), the matching prediction mode is filled with the color mode index (7). The matching prediction mode index (one of 0 to 3) is filled with the preset prediction mode (diagoanal down left). In particular, if the color mode is horizontal mode, index assignment can be performed as follows: planar (0)-vertical (1)-diagoanal down left (2)-DC (3)-CP1 (4)-CP2 (5)-CP3 (6)-horizontal (7).

[0440] Alternatively, if the color mode matches one of the preset prediction modes, the prediction mode that matches the 0th index is filled. Then, the preset prediction mode (Diagonal down left) is filled in the color mode index (7). Here, if the filled prediction mode is not the existing 0th index (i.e., it is not the planar mode), the existing index configuration can be adjusted. In particular, if the color mode is the DC mode, index assignment can be performed as follows: DC(0)-Planar(1)-Vertical(2)-Horizontal(3)-CP1(4)-CP2(5)-CP3(6)-Diagonal down left(7).

[0441] The above examples are only some examples of adaptive index assignment, and various modifications are possible. Also, binarization such as fixed-length binarization, unary binarization, truncated unary binarization, and truncated rice binarization can be used based on the indexes in the candidate set.

[0442] Next, another example of performing encoding by assigning indexes to prediction modes belonging to a prediction mode candidate group will be described.

[0443] For example, a method of classifying prediction modes, prediction methods, etc. into multiple prediction mode candidate groups and assigning indices to prediction modes belonging to the corresponding candidate groups for encoding corresponds to this method. In this case, candidate group selection information encoding can be performed prior to the index encoding. As an example, a directional mode, a non-directional mode, and a color mode, which are prediction modes that perform prediction in the same color space, can belong to one candidate group (hereinafter, "S candidate group"), and a color copy mode, which is a prediction mode that performs prediction in a different color space, can belong to one candidate group (hereinafter, "D candidate group").

[0444] Next, it is assumed that nine prediction modes (color difference components) including a preset prediction mode, a color copy mode, and a color mode are supported in the prediction mode candidate group.

[0445] For example, assume that four preset prediction modes are supported among planar, DC, horizontal, vertical, and diagonal modes, one color mode (C), and four color copy modes (CP1, CP2, CP3, and CP4). The S candidate group may have five candidates consisting of the preset prediction modes and color modes, and the D candidate group may have four candidates consisting of color copy modes.

[0446] The S candidate set is an example of an adaptively configured prediction mode candidate set, and an example of adaptive index assignment has been described above, so a detailed description will be omitted. The D candidate set is an example of a fixedly configured prediction mode candidate set, so a fixed index assignment method can be used. For example, index assignment such as CP1(0)-CP2(1)-CP3(2)-CP4(3) can be performed.

[0447] Based on the index in the candidate group, binarization such as fixed length binarization, unary binarization, truncated unary binarization, truncated length binarization, etc. may be used. Also, various modifications are possible without being limited to the above examples.

[0448] The prediction-related information generated by the prediction mode encoder can be transmitted to the encoder and included in the bitstream.

[0449] In a video decoding method according to an embodiment of the present invention, intra prediction may be configured as follows. The intra prediction of the predictor may include a prediction mode decoding step, a reference pixel configuration step, and a prediction block generation step. In addition, the video decoding apparatus may be configured to include a prediction mode decoding unit, a reference pixel configuration unit, and a prediction block generation unit, which implement the prediction mode decoding step, the reference pixel configuration step, and the prediction block generation step. Some of the above-described steps may be omitted or other steps may be added, and the steps may be arranged in an order other than the above-described order.

[0450] The reference pixel construction unit and prediction block generation unit of the video decoding device perform the same functions as the corresponding components of the video encoding device, so detailed explanations will be omitted, and the prediction mode decoding unit can be performed using the same method used in the prediction mode encoding unit in reverse.

[0451] FIG. 21 is a diagram showing an inter prediction method according to an embodiment to which the present invention is applied.

[0452] Referring to FIG. 21, a candidate list for motion information prediction of a current block can be generated (S2100).

[0453] The candidate list may include one or more candidates based on an affine model (hereinafter referred to as "affine candidates"). An affine candidate may refer to a candidate having a control point vector. The control point vector refers to a motion vector of a control point for an affine model and may be defined with respect to a corner position of a block (e.g., at least one position of the upper left corner, the upper right corner, the lower left corner, or the lower right corner).

[0454] The affine candidates may include at least one of spatial candidates, temporal candidates, and structural candidates. Here, the spatial candidates may be derived from vectors of neighboring blocks spatially adjacent to the current block, and the temporal candidates may be derived from vectors of neighboring blocks temporally adjacent to the current block. Here, the neighboring blocks may refer to blocks coded using an affine model. The vectors may refer to motion vectors or control point vectors.

[0455] A method for deriving spatial / temporal candidates based on vectors of spatial / temporal surrounding blocks will be described in detail with reference to FIG.

[0456] Meanwhile, the configuration candidates can be derived based on a combination of motion vectors of spatial / temporal neighboring blocks of the current block, which will be described in detail with reference to FIG.

[0457] The plurality of affine candidates may be arranged in the candidate list based on a predetermined priority. For example, the plurality of affine candidates may be arranged in the candidate list in the order of spatial candidates, temporal candidates, and composition candidates. Alternatively, the plurality of affine candidates may be arranged in the candidate list in the order of temporal candidates, spatial candidates, and composition candidates. However, without being limited thereto, the temporal candidates may be arranged next to the composition candidates. Alternatively, some of the composition candidates may be arranged before the spatial candidates, and the remaining candidates may be arranged after the spatial candidates.

[0458] A control point vector of the current block can be derived based on the candidate list and the candidate index (S2110).

[0459] The candidate index may refer to an index coded to derive a control point vector of the current block. The candidate index may identify one of a plurality of affine candidates belonging to a candidate list. The control point vector of the current block may be derived using the control point vector of the affine candidate identified by the candidate index.

[0460] For example, assume that the type of the affine model of the current block is 4-parameter (i.e., the current block is determined to use two control point vectors). Here, if the affine candidate identified by the candidate index has three control point vectors, only two control point vectors (e.g., control point vectors with Idx=0, 1) can be selected from the three control point vectors and set as the control point vectors of the current block. Alternatively, the three control point vectors of the identified affine candidate can be set as the control point vectors of the current block. In this case, the type of the affine model of the current block can be updated to 6-parameter.

[0461] Conversely, assume that the type of the affine model of the current block is 6-parameter (i.e., when the current block is determined to use three control point vectors). Here, if the affine candidate specified by the candidate index has two control point vectors, one additional control point vector can be generated, and the two control point vectors of the affine candidate and the additional control point vector can be set as the control point vectors of the current block. The additional control point vector can be derived based on at least one of the two control point vectors of the affine candidate, the size or position information of the current / peripheral block. Alternatively, the two control point vectors of the specified affine candidate can be set as the control point vectors of the current block. In this case, the type of the affine model of the current block can be updated to 4-parameter.

[0462] The motion vector of the current block can be derived based on the control point vectors of the current block (S2120).

[0463] The motion vector can be derived in units of sub-blocks of the current block. Here, the N×M sub-blocks can be in the form of a rectangle (N>M or N<M) or a square (N = M). The values of N and M can be 4, 8, 16, 32 or more. The size / shape of the sub-block can be a fixed size / shape predefined in the decoder.

[0464] Alternatively, the size / shape of the sub-block can also be variably derived based on the attributes of the blocks described above. For example, if the size of the current block is greater than or equal to a predetermined threshold, the current block can be divided into units of the first sub-block (e.g., 8×8, 16×16), and if not, the current block can be divided into units of the second sub-block (e.g., 4×4). Alternatively, information about the size / shape of the sub-block can also be encoded and signaled by the encoder.

[0465] Inter prediction can be performed on the current block using the derived motion vector (S2130).

[0466] Specifically, a reference block can be identified using a motion vector of the current block. The reference block can be identified for each sub-block of the current block. The reference block for each sub-block can belong to one reference picture. That is, the sub-blocks belonging to the current block can share one reference picture. Alternatively, a reference picture index can be set independently for each sub-block of the current block.

[0467] The identified reference block can be set as a predicted block of the current block. The above-described embodiment can be applied equally / similarly not only to merge mode but also to general inter modes (e.g., AMVP mode). The above-described embodiment can be performed only when the size of the current block is greater than or equal to a predetermined threshold. Here, the threshold can be 8x8, 8x16, 16x8, 16x16, or more.

[0468] FIG. 22 is a diagram illustrating a method for deriving affine candidates from spatial / temporal neighboring blocks according to one embodiment of the present invention.

[0469] For convenience of explanation, this embodiment describes a method for deriving affine candidates from spatially surrounding blocks.

[0470] 5, the width and height of a current block 2200 are cbW and cbH, respectively, and the position of the current block is (xCb, yCb). The width and height of a spatially surrounding block 2210 are nbW and nbH, respectively, and the position of the spatially surrounding block is (xNb, yNb). While FIG. 22 illustrates the spatially surrounding block as the top left block of the current block, it is not limited thereto. That is, the spatial surrounding block may include at least one of the block to the left of the current block, the bottom left block, the top right block, the top block, or the top left block.

[0471] A spatial candidate may have n control point vectors (cpMVs), where the value of n may be an integer of 1, 2, 3, or more. The value of n may be determined based on at least one of information on whether to decode in subblock units, information on whether the block is coded in an affine model, or information on the type of affine model (4-parameter or 6-parameter).

[0472] The information may be coded and signaled by the coding device. Alternatively, all or part of the information may be derived by the decoding device based on block attributes. Here, the block may refer to the current block or the spatial / temporal neighboring blocks of the current block. The attributes may refer to size, shape, position, partition type, inter mode, parameters related to residual coefficients, etc. The inter mode is a mode predefined in the decoding device, and may refer to merge mode, skip mode, AMVP mode, affine model, intra / inter combined mode, current picture reference mode, etc. Alternatively, the n value may be derived by the decoding device based on the above-mentioned block attributes.

[0473] In this embodiment, the n control point vectors may be represented as a first control point vector (cpMV[0]), a second control point vector (cpMV[1]), a third control point vector (cpMV[2]), ..., an nth control point vector (cpMV[n-1]). For example, the first control point vector (cpMV[0]), the second control point vector (cpMV[1]), the third control point vector (cpMV[2]), and the fourth control point vector (cpMV[3]) may be vectors corresponding to the upper left sample, the upper right sample, the lower left sample, and the lower right sample of the block, respectively. Here, it is assumed that a spatial candidate has three control point vectors, and the three control point vectors may be any control point vectors selected from the first to nth control point vectors. However, without being limited thereto, the spatial candidate may have two control point vectors, and the two control point vectors may be any control point vectors selected from the first to nth control point vectors.

[0474] Meanwhile, the control point vectors of the spatial candidates can be derived differently depending on whether the boundary 2220 shown in FIG. 22 is a coding tree block boundary (CTU boundary).

[0475] 1. When the boundary 2220 of the current block does not touch the boundary of the coding tree block (CTU boundary)

[0476] The first control point vector can be derived based on at least one of the first control point vector of the spatially surrounding block, a predetermined difference value, position information (xCb, yCb) of the current block, or position information (xNb, yNb) of the spatially surrounding block.

[0477] The number of difference values ​​may be one, two, three, or more. The number of difference values ​​may be variably determined in consideration of the block attributes, or may be a fixed value predefined for the decoding device. The difference value may be defined as a difference value between any one of a plurality of control point vectors and another one of the plurality of control point vectors. For example, the difference value may include at least one of a first difference value between a second control point vector and a first control point vector, a second difference value between a third control point vector and a first control point vector, a third difference value between a fourth control point vector and a third control point vector, or a fourth difference value between a fourth control point vector and a second control point vector.

[0478] For example, the first control point vector can be derived by the following mathematical formula 1:

number

[0479] In Equation 1, the variables mvScaleHor and mvScaleVer may represent the first control point vector of the spatial surrounding block, or may represent values ​​derived by applying a k-shift operation to the first control point vector. Here, k may be an integer of 1, 2, 3, 4, 5, 6, 7, 8, 9, or greater. The variables dHorX and dVerX correspond to the x and y components, respectively, of the first difference value between the second control point vector and the first control point vector. The variables dHorY and dVerY correspond to the x and y components, respectively, of the second difference value between the third control point vector and the first control point vector. The above variables can be derived according to Equation 2 below.

[0480]

number

[0481] The second control point vector may be derived based on at least one of the first control point vector of the spatially surrounding block, a predetermined difference value, position information (xCb, yCb) of the current block, block size (width or height), or position information (xNb, yNb) of the spatially surrounding block. Here, the block size may refer to the size of the current block and / or the spatially surrounding block. The difference value is the same as that described for the first control point vector, so a detailed description will be omitted here. However, the range and / or number of difference values ​​used in the process of deriving the second control point vector may differ from that of the first control point vector.

[0482] For example, the second control point vector can be derived by the following mathematical formula 3:

[0483]

number

[0484] In Equation 3, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those described in Equation 1, so detailed description thereof will be omitted here.

[0485] The third control point vector may be derived based on at least one of the first control point vector of the spatially surrounding block, a predetermined difference value, position information (xCb, yCb) of the current block, block size (width or height), or position information (xNb, yNb) of the spatially surrounding block. Here, the block size may refer to the size of the current block and / or the spatially surrounding block. The difference value is the same as that described for the first control point vector, so a detailed description will be omitted here. However, the range and / or number of difference values ​​used in the process of deriving the third control point vector may be different from those of the first control point vector or the second control point vector.

[0486] For example, the third control point vector can be derived by the following mathematical formula 4:

number

[0487] In Equation 4, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those described in Equation 1, so detailed description will be omitted here. Meanwhile, the nth control point vector of the spatial candidate can be derived through the above-described process.

[0488] 2. When the current block boundary 2220 is adjacent to the coding tree block boundary (CTU boundary)

[0489] The first control point vector can be derived based on at least one of the motion vector (MV) of the spatially surrounding block, a predetermined difference value, position information (xCb, yCb) of the current block, or position information (xNb, yNb) of the spatially surrounding block.

[0490] The motion vector may be a motion vector of a sub-block located at the bottom of a spatially peripheral block. The sub-block may be located at the leftmost, center, or rightmost of a plurality of sub-blocks located at the bottom of the spatially peripheral block. Alternatively, the motion vector may represent an average value, maximum value, or minimum value of the motion vectors of the sub-blocks.

[0491] The number of difference values ​​may be one, two, three, or more. The number of difference values ​​may be variably determined in consideration of the attributes of the blocks, or may be a fixed value predefined for the decoding device. The difference value may be defined as a difference value between any one of a plurality of motion vectors stored in sub-block units in the spatially neighboring blocks. For example, the difference value may refer to a difference value between the motion vector of the lower right sub-block and the motion vector of the lower left sub-block of the spatially neighboring blocks.

[0492] For example, the first control point vector can be derived by the following mathematical formula 5.

number

[0493] In Equation 5, the variables mvScaleHor and mvScaleVer may represent the motion vectors (MVs) of the spatially surrounding blocks or values ​​derived by applying a k-shift operation to the motion vectors, where k may be an integer of 1, 2, 3, 4, 5, 6, 7, 8, 9, or more.

[0494] The variables dHorX and dVerX correspond to the x and y components of a predetermined difference value, respectively. Here, the difference value refers to the difference value between the motion vector of the bottom right sub-block and the motion vector of the bottom left sub-block in the spatially surrounding blocks. The variables dHorY and dVerY can be derived based on the variables dHorX and dVerX. The above variables can be derived by the following mathematical formula 6.

[0495]

number

[0496] The second control point vector may be derived based on at least one of the motion vectors (MV) of spatially surrounding blocks, a predetermined difference value, position information (xCb, yCb) of the current block, block size (width or height), or position information (xNb, yNb) of spatially surrounding blocks. Here, the block size may refer to the size of the current block and / or spatially surrounding blocks. The motion vectors and difference values ​​are the same as those described for the first control point vector, so a detailed description will be omitted here. However, the position of the motion vector, the range of difference values, and / or the number of difference values ​​used in the process of deriving the second control point vector may differ from those of the first control point vector.

[0497] For example, the second control point vector can be derived by the following mathematical formula 7.

number

[0498] In Equation 7, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those described in Equation 5, so detailed description thereof will be omitted here.

[0499] The third control point vector may be derived based on at least one of the motion vectors (MV) of the spatially surrounding blocks, a predetermined difference value, position information (xCb, yCb) of the current block, block size (width or height), or position information (xNb, yNb) of the spatially surrounding blocks. Here, the block size may refer to the size of the current block and / or the spatially surrounding blocks. The motion vectors and difference values ​​are the same as those described for the first control point vector, so a detailed description will be omitted here. However, the position, range, and / or number of the difference values ​​of the motion vectors used in the process of deriving the third control point vector may differ from those of the first control point vector or the second control point vector.

[0500] For example, the third control point vector can be derived by the following mathematical formula 8:

number

[0501] In Equation 8, the variables mvScaleHor, mvScaleVer, dHorX, dVerX, dHorY, and dVerY are the same as those described in Equation 5, so detailed description will be omitted here. Meanwhile, the nth control point vector of the spatial candidate can be derived through the above-described process.

[0502] The above-described affine candidate derivation process may be performed for each of predefined spatially neighboring blocks, which may include at least one of a block to the left of the current block, a bottom left block, a top right block, a top block, or a top left block.

[0503] Alternatively, the affine candidate derivation process may be performed for each group of the spatially neighboring blocks, where the spatially neighboring blocks may be classified into a first group including the left block and the bottom left block, and a second group including the top right block, the top block, and the top left block.

[0504] For example, one affine candidate can be derived from the spatially neighboring blocks belonging to the first group. The deriving can be performed according to a predetermined priority until a usable affine candidate is found. The priority can be from the left block to the bottom left block, or vice versa.

[0505] Similarly, one affine candidate can be derived from the spatially neighboring blocks belonging to the second group. The derivation can be performed until a usable affine candidate is found according to a predetermined priority. The priority can be in the order of the top right block → top block → top left block, or the reverse order.

[0506] The above-described embodiment can be applied equally / similarly to temporally neighboring blocks. Here, the temporally neighboring blocks may be blocks that belong to a different picture from the current block but are located at the same position as the current block. The block at the same position may be a block including the position of the top left sample of the current block, the center position, or the position of a sample adjacent to the bottom right sample of the current block.

[0507] Alternatively, the temporally neighboring block may refer to a block shifted by a predetermined displacement vector from the block at the same position, where the displacement vector may be determined based on the motion vector of any one of the spatially neighboring blocks of the current block.

[0508] FIG. 23 is a diagram illustrating a method for deriving configuration candidates based on a combination of motion vectors of spatial / temporal surrounding blocks according to an embodiment of the present invention.

[0509] The configuration candidates of the present invention can be derived based on a combination of at least two control point vectors (hereinafter referred to as control point vectors (cpMVCorner[n])) corresponding to each corner of the current block, where n can be 0, 1, 2, or 3.

[0510] The control point vector may be derived based on the motion vectors of spatially and / or temporally neighboring blocks. Here, the spatially neighboring blocks may include at least one of a first neighboring block (C, D, or E) adjacent to the upper left sample of the current block, a second neighboring block (F or G) adjacent to the upper right sample of the current block, or a third neighboring block (A or B) adjacent to the lower left sample of the current block. The temporal neighboring block may be a block belonging to a different picture from the current block, and may refer to a fourth neighboring block (Col) adjacent to the lower right sample of the current block.

[0511] The first neighboring block may refer to the neighboring block at the top left (D), top (E), or left (C) of the current block. It is determined whether the motion vectors of neighboring blocks C, D, and E are available according to a predetermined priority, and the control point vector is determined using the motion vectors of the available neighboring blocks. The availability determination may be performed until a neighboring block having an available motion vector is found. Here, the priority order may be D→E→C. However, it is not limited thereto, and may also be D→C→E, C→D→E, or E→D→C.

[0512] The second neighboring block may refer to the neighboring block at the top (F) or top right (G) of the current block. Similarly, it is possible to determine whether the motion vectors of neighboring blocks F and G are available according to a predetermined priority, and to determine the control point vector using the motion vector of the available neighboring block. The availability determination may be performed until a neighboring block with an available motion vector is found. Here, the priority may be F→G or G→F.

[0513] The third neighboring block may refer to a neighboring block to the left (B) or the bottom left (A) of the current block. Similarly, it is possible to determine whether the motion vectors of neighboring blocks are available according to a predetermined priority, and to determine the control point vector using the motion vectors of the available neighboring blocks. The availability determination may be performed until a neighboring block having an available motion vector is found. Here, the priority may be in the order of A → B or B → A.

[0514] For example, the first control point vector (cpMVCorner[0]) can be set to the motion vector of the first surrounding block, the second control point vector (cpMVCorner[1]) can be set to the motion vector of the second surrounding block, the third control point vector (cpMVCorner[2]) can be set to the motion vector of the third surrounding block, and the fourth control point vector (cpMVCorner[3]) can be set to the motion vector of the fourth surrounding block.

[0515] Alternatively, any one of the first to fourth control point vectors can be derived based on the other one. For example, the second control point vector can be derived by applying a predetermined offset vector to the first control point vector. The offset vector can be a difference vector between the third control point vector and the first control point vector, or can be derived by applying a predetermined scaling factor to the difference vector. The scaling factor can be determined based on at least one of the width and height of the current block and / or the surrounding blocks.

[0516] According to the present invention, K configuration candidates (ConstK) can be determined by combining at least two of the first to fourth control point vectors. The value of K can be an integer of 1, 2, 3, 4, 5, 6, 7, or more. The value of K can be derived based on information signaled by the encoding device, or can be a value already agreed upon with the decoding device. The information can include information indicating the maximum number of configuration candidates to be included in the candidate list.

[0517] Specifically, the first configuration candidate (Const1) can be derived by combining the first to third control point vectors. For example, the first configuration candidate (Const1) can have a control point vector as shown in Table 1 below. Meanwhile, it is also possible to restrict the configuration of the control point vector as shown in Table 1 only when the reference picture information of the first peripheral block is the same as the reference picture information of the second and third peripheral blocks. Here, the reference picture information may refer to a reference picture index indicating the position of the corresponding reference picture in a reference picture list, or may refer to a POC (picture order count) value indicating the output order.

[0518] [Table 1] The second configuration candidate (Const2) can be derived by combining the first, second, and fourth control point vectors. For example, the second configuration candidate (Const2) can have a control point vector as shown in Table 2 below. Meanwhile, the control point vector can be restricted to be constructed as shown in Table 2 only if the reference picture information of the first peripheral block is the same as the reference picture information of the second and fourth peripheral blocks. Here, the reference picture information is as described above.

[0519] [Table 2] The third configuration candidate (Const3) can be derived by combining the first, third, and fourth control point vectors. For example, the third configuration candidate (Const3) can have a control point vector as shown in Table 3 below. Meanwhile, the control point vector can be restricted to be constructed as shown in Table 2 only if the reference picture information of the first peripheral block is the same as the reference picture information of the third and fourth peripheral blocks. Here, the reference picture information is as described above.

[0520] [Table 3] The fourth configuration candidate (Const4) can be derived by combining the second, third, and fourth control point vectors. For example, the fourth configuration candidate (Const4) can have a control point vector as shown in Table 4 below. Meanwhile, it can be restricted to be configured as shown in Table 4 only if the reference picture information of the second peripheral block is the same as the reference picture information of the third and fourth peripheral blocks. Here, the reference picture information is as described above.

[0521] [Table 4] The fifth configuration candidate (Const5) can be derived by combining the first and second control point vectors. For example, the fifth configuration candidate (Const5) can have a control point vector as shown in Table 5 below. Meanwhile, the control point vector can be restricted to be constructed as shown in Table 5 only when the reference picture information of the first peripheral block is the same as the reference picture information of the second peripheral block. Here, the reference picture information is as described above.

[0522] [Table 5] The sixth configuration candidate (Const6) can be derived by combining the first and third control point vectors. For example, the sixth configuration candidate (Const6) can have a control point vector as shown in Table 6 below. Meanwhile, the control point vector can be restricted to be constructed as shown in Table 6 only when the reference picture information of the first peripheral block is the same as the reference picture information of the third peripheral block. Here, the reference picture information is as described above.

[0523] [Table 6] In Table 6, cpMvCorner[1] may be a second control point vector derived based on the first and third control point vectors. The second control point vector may be derived based on at least one of the first control point vector, a predetermined difference value, or the size of the current / neighboring block. For example, the second control point vector may be derived using the following mathematical formula 9.

[0524]

number

[0525] The first to sixth configuration candidates described above may all be included in the candidate list, or only some of them may be included in the candidate list.

[0526] The methods according to the present invention may be embodied in the form of program instructions executable by various computer means and recorded on a computer-readable medium. The computer-readable medium may include, alone or in combination with other program instructions, data files, data structures, and the like. The program instructions recorded on the computer-readable medium may be specially designed and constructed for the present invention, or they may be of the type well known and available to those skilled in the art of computer software.

[0527] Examples of computer-readable media include hardware devices specially configured to store and execute program instructions, such as read-only memory (ROM), RAM, flash memory, etc. Examples of program instructions include not only machine code, such as that produced by a compiler, but also high-level language code executable by a computer using an interpreter, etc. The above-mentioned hardware devices can be configured to operate as at least one software module to perform the operations of the present invention, and vice versa.

[0528] Furthermore, the above-described methods or apparatuses may be implemented in such a manner that all or part of their components or functions are combined or separated.

[0529] Although the present invention has been described above with reference to preferred embodiments, those skilled in the art will understand that various modifications and variations of the present invention can be made without departing from the spirit and scope of the present invention as set forth in the following claims. [Industrial Applicability]

[0530] The present invention can be used to encode / decode video signals. < / l-1> < / array> < / qt>

Claims

1. 1. A method of decoding video, comprising: determining a prediction mode of the current chrominance block as a mode for predicting the current chrominance block from a luminance block corresponding to the current chrominance block; obtaining a first predicted sample of the current chrominance block based on the determination; obtaining a second predicted sample of the current chrominance block based on the determination; and obtaining a final predicted sample of the current chrominance block based on the first predicted sample and the second predicted sample; generating residual samples of the current chrominance block; reconstructing the current chrominance block based on the final predicted samples and the residual samples; Equipped with the first predicted sample is predicted using a DC mode; the second predicted sample is obtained based on a luma block corresponding to the current chroma block; method.

2. the second predicted sample is further obtained based on correlation information between the current chrominance block and the corresponding luminance block. The method of claim 1.

3. the second predicted samples are obtained by applying the correlation information to sample values ​​of a luminance component calculated based on sample values ​​in the corresponding luminance block. The method of claim 2.

4. the correlation information is obtained from a bitstream. The method of claim 2.

5. the sample values ​​of the luminance component are calculated based on subsampling and averaging the sample values ​​within the corresponding luminance block; The method of claim 3.

6. 1. A method for encoding an image, comprising: determining a prediction mode of the current chrominance block as a mode for predicting the current chrominance block from a luminance block corresponding to the current chrominance block; obtaining a first predicted sample of the current chrominance block based on the determination; obtaining a second predicted sample of the current chrominance block based on the determination; and obtaining a final predicted sample of the current chrominance block based on the first predicted sample and the second predicted sample; Equipped with encoding the current chrominance block further includes generating residual samples of the current chrominance block based on the final predicted samples; and encoding the current chrominance block based on the residual samples; the first predicted sample is predicted using a DC mode; the second predicted sample is obtained based on a luma block corresponding to the current chroma block; method.

7. 1. A method for transmitting a bitstream, comprising: determining a prediction mode of the current chrominance block as a mode for predicting the current chrominance block from a luminance block corresponding to the current chrominance block; obtaining a first predicted sample of the current chrominance block based on the determination; obtaining a second predicted sample of the current chrominance block based on the determination; and obtaining a final predicted sample of the current chrominance block based on the first predicted sample and the second predicted sample; encoding the current chrominance block based on the final predicted samples into the bitstream; transmitting the bitstream; Equipped with encoding the current chrominance block further includes generating residual samples of the current chrominance block based on the final predicted samples; and encoding the current chrominance block based on the residual samples; the first predicted sample is predicted using a DC mode; the second predicted sample is obtained based on a luma block corresponding to the current chroma block; method.

8. 1. A non-transitory computer-readable storage medium storing a bitstream generated by a video encoding method, the video encoding method comprising: determining a prediction mode of the current chrominance block as a mode for predicting the current chrominance block from a luminance block corresponding to the current chrominance block; obtaining a first predicted sample of the current chrominance block based on the determination; obtaining a second predicted sample of the current chrominance block based on the determination; and obtaining a final predicted sample of the current chrominance block based on the first predicted sample and the second predicted sample; encoding the current chrominance block based on the final predicted samples; Equipped with encoding the current chrominance block further includes generating residual samples of the current chrominance block based on the final predicted samples; and encoding the current chrominance block based on the residual samples; the first predicted sample is predicted using a DC mode; the second predicted sample is obtained based on a luma block corresponding to the current chroma block; A non-transitory computer-readable storage medium.

Images