Image decoding and encoding methods and bitstream transmission methods

Abstract

The invention relates to a method of internal prediction and a means of reading by a computer. The internal prediction method under the invention includes a division step, based on the division information of the encoded block, dividing the encoded block into subblocks, the division information including at least one of the following: first division information indicating whether it divides into four subblocks, second division information indicating whether it divides into two subblocks, or third division information indicating the direction of division; determining the encoding order of the subblocks based on the division information; and generating a prediction block of each subblock based on the encoding order of the subblocks and the internal prediction mode of each subblock.

Description

Video encoding / decoding method and device

[0001] The present invention relates to image encoding and decoding technology, and more particularly, to an encoding / decoding method and device in intra-screen prediction.

[0002] With the proliferation of the Internet and mobile devices, and the advancement of information and communication technology, the use of multimedia data is rapidly increasing. Consequently, the need to improve the performance and efficiency of image processing systems to perform various services and tasks through image prediction in various systems is growing significantly. However, research and development results that can meet this demand are inadequate.

[0003] In this way, in conventional image encoding and decoding methods and devices, there is a demand for performance improvement in image processing, especially image encoding or image decoding.

[0004] The present invention aims to provide a method and device for intra-screen prediction. Furthermore, the present invention aims to provide a method and device for intra-screen prediction at the sub-block level. Furthermore, the present invention aims to provide a method and device for determining the division and encoding order at the sub-block level.

[0005] The video encoding / decoding method and device according to the present invention can configure a candidate group regarding the division shape of a current block, determine the division shape of the current block into sub-blocks based on the candidate group and candidate index, derive an intra-screen prediction mode for each current block, and perform intra-screen prediction of the current block based on the intra-screen prediction mode of the current block and the division shape of the sub-block.

[0006] According to the present invention, encoding / decoding performance can be improved through intra-screen prediction at the sub-block level. Furthermore, the present invention can enhance prediction accuracy by efficiently constructing a candidate set for sub-block-level segmentation patterns. Furthermore, the present invention can improve encoding / decoding efficiency of intra-screen prediction by adaptively applying the encoding order at the sub-block level.

[0007] FIG. 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0008] FIG. 2 is a block diagram of a video encoding device according to one embodiment of the present invention.

[0009] FIG. 3 is a block diagram of an image decoding device according to one embodiment of the present invention.

[0010] Figure 4 is an exemplary diagram showing various division forms that can be obtained from the block division unit of the present invention.

[0011] FIG. 5 is an exemplary diagram showing an on-screen prediction mode according to one embodiment of the present invention.

[0012] FIG. 6 is an exemplary diagram illustrating a reference pixel configuration used for in-screen prediction according to one embodiment of the present invention.

[0013] FIG. 7 is a conceptual diagram illustrating a target block and an adjacent block of an on-screen prediction according to one embodiment of the present invention.

[0014] Figure 8 shows various division forms of sub-blocks obtainable based on an encoding block.

[0015] FIG. 9 is an example diagram of a reference pixel area used based on an in-screen prediction mode according to one embodiment of the present invention.

[0016] FIG. 10 illustrates an example of an encoding order that can be had in a diagonal up right direction prediction mode according to one embodiment of the present invention.

[0017] FIG. 11 illustrates an example of an encoding order that may be present in a horizontal mode according to one embodiment of the present invention.

[0018] FIG. 12 illustrates an example of an encoding order that can be had in a diagonal down right direction prediction mode according to one embodiment of the present invention.

[0019] FIG. 13 illustrates an example of an encoding order that may be present in a vertical mode according to one embodiment of the present invention.

[0020] FIG. 14 shows an example of an encoding order that can be had in a diagonal down left direction mode according to one embodiment of the present invention.

[0021] FIG. 15 is an exemplary diagram of an encoding order considering an intra-screen prediction mode and a segmentation shape according to one embodiment of the present invention.

[0022] The video encoding / decoding method and device according to the present invention can configure a candidate group regarding the division shape of a current block, determine the division shape of the current block into sub-blocks based on the candidate group and candidate index, derive an intra-screen prediction mode for each current block, and perform intra-screen prediction of the current block based on the intra-screen prediction mode of the current block and the division shape of the sub-block.

[0023] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated and described in detail in the drawings. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0024] Terms such as first, second, A, and B may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component. The term "and / or" includes a combination of multiple related items described herein or any of multiple related items described herein.

[0025] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0026] The terminology used herein is merely used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In the present invention, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted consistent with their meaning in the context of the relevant technology, and unless explicitly defined herein, they shall not be construed in an idealized or overly formal sense.

[0028] Typically, it can be composed of one or more color spaces depending on the color format of the image. Depending on the color format, it can be composed of one or more pictures with a certain size or one or more pictures with different sizes. For example, in the YCbCr color configuration, color formats such as 4:4:4, 4:2:2, 4:2:0, and Monochrome (consisting of only Y) can be supported. For example, in the case of YCbCr 4:2:0, it can be composed of one luminance component (Y in this example) and two chrominance components (Cb / Cr in this example), and at this time, the composition ratio of the chrominance component and the luminance component can have a horizontal and vertical ratio of 1:2. For example, in the case of 4:4:4, it can have the same horizontal and vertical composition ratio. In the case of being composed of one or more color spaces as in the above example, the picture can be divided into each color space.

[0029] Images can be classified into I, P, B, etc. according to the image type (e.g., picture type, slice type, tile type, etc.). The I image type can mean an image that is encoded / decoded on its own without using a reference picture. The P image type can mean an image that is encoded / decoded using a reference picture but allows only forward prediction. The B image type can mean an image that is encoded / decoded using a reference picture and allows forward / backward prediction. However, depending on the encoding / decoding settings, some of the above types may be combined (combining P and B) or image types with different configurations may be supported.

[0030] The various sub / decoding information generated in the present invention can be processed explicitly or implicitly. Here, explicit processing can be understood as generating selection information indicating one candidate among multiple candidate groups for encoding information from a sequence, slice, tile, block, sub-block, etc., and including this in the bitstream, and then parsing the related information at the same level of unit as the encoder in the decoder to restore it as decoding information. Here, implicit processing can be understood as processing sub / decoding information using the same process, rules, etc. in the encoder and decoder.

[0031]

[0032] FIG. 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0033] Referring to FIG. 1, the image encoding device (105) and the decoding device (100) may be user terminals such as a personal computer (PC), a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a PlayStation Portable (PSP), a wireless communication terminal, a smart phone, or a TV, or server terminals such as an application server and a service server, and may include various devices including a communication device such as a communication modem for communicating with various devices or wired / wireless communication networks, a memory (memory, 120, 125) for storing various programs and data for inter- or intra-predicting for encoding or decoding images, or a processor (processor, 110, 115) for executing a program and performing calculations and controls.

[0034] In addition, the image encoded into a bitstream by the image encoding device (105) can be transmitted to the image decoding device (100) through a wired or wireless communication network such as the Internet, a local area network, a wireless LAN, a WiBro network, or a mobile communication network in real time or non-real time, or through various communication interfaces such as a cable or a universal serial bus (USB), and can be decoded in the image decoding device (100) to be restored into an image and played back. In addition, the image encoded into a bitstream by the image encoding device (105) can be transmitted from the image encoding device (105) to the image decoding device (100) through a computer-readable recording medium.

[0035] The aforementioned video encoding device and video decoding device may be separate devices, but may be implemented as a single video encoding / decoding device. In this case, some components of the video encoding device may be implemented to include at least the same structure or perform at least the same function as some components of the video decoding device, as substantially the same technical elements.

[0036] Therefore, in the detailed description of the technical elements and their operating principles below, overlapping descriptions of corresponding technical elements will be omitted. In addition, since the image decoding device corresponds to a computing device that applies the image encoding method performed in the image encoding device to decoding, the description below will focus on the image encoding device.

[0037] A computing device may include a memory storing a program or software module implementing an image encoding method and / or an image decoding method, and a processor connected to the memory to execute the program. Here, the image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder, respectively.

[0038]

[0039] FIG. 2 is a block diagram of a video encoding device according to one embodiment of the present invention.

[0040] Referring to FIG. 2, the image encoding device (20) may include a prediction unit (200), a subtraction unit (205), a transformation unit (210), a quantization unit (215), an inverse quantization unit (220), an inverse transformation unit (225), an addition unit (230), a filter unit (235), an encoded picture buffer (240), and an entropy encoding unit (245).

[0041] The prediction unit (200) can be implemented using a prediction module, which is a software module, and can generate a prediction block using an intra-prediction method or an inter-prediction method for a block to be encoded. The prediction unit (200) can generate a prediction block by predicting a current block to be encoded in an image. In other words, the prediction unit (200) can generate a prediction block having a predicted pixel value of each pixel by predicting the pixel value of each pixel of the current block to be encoded in an image according to intra-prediction or inter-prediction. In addition, the prediction unit (200) can transmit information necessary for generating a prediction block, such as information on a prediction mode such as an intra-prediction mode or an inter-prediction mode, to the encoding unit, so that the encoding unit can encode information on the prediction mode. At this time, the processing unit where the prediction is performed and the processing unit where the prediction method and specific contents are determined can be determined according to the encoding / decoding settings. For example, the prediction method, prediction mode, etc. can be determined by prediction unit, and prediction can be performed by transformation unit.

[0042] In the inter-frame prediction unit, the motion prediction method can be divided into a translational motion model and a non-translational motion model. In the case of the translational motion model, prediction is performed by considering only parallel translation, and in the case of the non-translational motion model, prediction can be performed by considering not only parallel translation but also rotation, perspective, zoom in / out, etc. Assuming unidirectional prediction, the translational motion model may require one motion vector, but the non-translational motion model may require more than one motion vector. In the case of the non-translational motion model, each motion vector may be information applied to a preset position of the current block, such as the upper left corner, the upper right corner, etc. of the current block, and the position of the region to be predicted in the current block can be obtained in units of pixels or sub-blocks through the motion vector. In the inter-frame prediction unit, depending on the motion model, some of the processes described below may be commonly applied, and some processes may be applied individually.

[0043] The inter-screen prediction unit may include a reference picture construction unit, a motion estimation unit, a motion compensation unit, a motion information determination unit, and a motion information encoding unit. The reference picture construction unit may include pictures encoded before or after the current picture in a reference picture list (L0, L1). A prediction block may be obtained from a reference picture included in the reference picture list, and depending on the encoding settings, the current image may also be configured as a reference picture and included in at least one of the reference picture lists.

[0044] In the inter-screen prediction unit, the reference picture configuration unit may include a reference picture interpolation unit, and may perform an interpolation process for decimal unit pixels according to the interpolation precision. For example, an interpolation filter based on an 8-tap DCT may be applied to the luminance component, and an interpolation filter based on a 4-tap DCT may be applied to the chrominance component.

[0045] In the inter-screen prediction unit, the motion estimation unit is a process of searching for a block with a high correlation with the current block through reference pictures, and various methods such as FBMA (Full search-based block matching algorithm) and TSS (Three step search) can be used, and the motion compensation unit refers to the process of obtaining a prediction block through the motion estimation process.

[0046] In the inter-screen prediction unit, a motion information determination unit may perform a process for selecting optimal motion information of the current block, and the motion information may be encoded by a motion information encoding mode such as Skip Mode, Merge Mode, or Competition Mode. The above modes may be configured by combining supported modes according to a motion model, and examples thereof may include Skip Mode (Movement), Skip Mode (Non-Movement), Merge Mode (Movement), Merge Mode (Non-Movement), Competition Mode (Movement), and Competition Mode (Non-Movement). Depending on the encoding settings, some of the above modes may be included in the candidate group.

[0047] The above motion information encoding mode can obtain a prediction value of motion information (motion vector, reference picture, prediction direction, etc.) of the current block from at least one candidate block, and when two or more candidate blocks are supported, optimal candidate selection information can be generated. Skip mode (no residual signal) and merge mode (residual signal presence) can use the prediction value as it is as motion information of the current block, and competitive mode can generate differential information between the motion information of the current block and the prediction value.

[0048] The candidate set for motion information prediction of the current block can have adaptive and diverse configurations depending on the motion information encoding mode. The candidate set may include motion information from blocks spatially adjacent to the current block (e.g., left, top, top-left, top-right, bottom-left blocks), motion information from temporally adjacent blocks, or mixed motion information from both spatial and temporal candidates.

[0049] The above temporally adjacent blocks include blocks in another image that correspond to (or are equivalent to) the current block, and may refer to blocks located to the left, right, above, below, upper left, upper right, lower left, lower right, etc., with the current block as the center. The above mixed motion information may refer to information obtained as an average, median, etc., through motion information of spatially adjacent blocks and motion information of temporally adjacent blocks.

[0050] There may be a priority order for configuring the motion information prediction value candidate set. The order of inclusion in the prediction value candidate set can be determined based on the priority order, and the candidate set can be completed once the number of candidates (determined according to the motion information encoding mode) is filled according to the priority order. At this time, the priority order may be determined in the following order: motion information of spatially adjacent blocks, motion information of temporally adjacent blocks, or mixed motion information of spatial and temporal candidates, but other variations are also possible.

[0051] For example, among spatially adjacent blocks, the candidate group can be included in the order of left - upper - upper right - lower left - upper left blocks, and among temporally adjacent blocks, the candidate group can be included in the order of lower right - middle - right - lower blocks, etc.

[0052] The subtraction unit (205) can generate a residual block by subtracting the predicted block from the current block. In other words, the subtraction unit (205) can generate a residual block, which is a block-type residual signal, by calculating the difference between the pixel value of each pixel of the current block to be encoded and the predicted pixel value of each pixel of the predicted block generated through the prediction unit. In addition, the subtraction unit (205) can also generate a residual block based on units other than the block units obtained through the block division unit described below.

[0053] The transformation unit (210) can transform a signal belonging to the spatial domain into a signal belonging to the frequency domain, and the signal obtained through the transformation process is called a transformation coefficient. For example, a transformation block having a transformation coefficient can be obtained by transforming a residual block having a residual signal received from a subtraction unit. The input signal is determined according to the encoding settings, and this is not limited to a residual signal.

[0054] The transformation unit can transform the residual block using a transformation technique such as Hadamard Transform, Discrete Sine Transform (DST Based-Transform), Discrete Cosine Transform (DCT Based-Transform), etc., and is not limited thereto, and various transformation techniques that are improved and modified thereof can be used.

[0055] At least one of the above transformation techniques may be supported, and at least one detailed transformation technique may be supported in each transformation technique. In this case, the detailed transformation technique may be a transformation technique in which a portion of the basis vector is configured differently in each transformation technique.

[0056] For example, in the case of DCT, one or more detailed transformation techniques from DCT-1 to DCT-8 may be supported, and in the case of DST, one or more detailed transformation techniques from DST-1 to DST-8 may be supported. A group of transformation technique candidates may be formed by configuring some of the detailed transformation techniques. For example, DCT-2, DCT-8, and DST-7 may be configured as a group of transformation technique candidates to perform transformation.

[0057] Transformations can be performed in both horizontal and vertical directions. For example, a one-dimensional transformation can be performed in the horizontal direction using the DCT-2 transform technique, and a one-dimensional transformation can be performed in the vertical direction using the DST-7 transform technique, thereby performing a total two-dimensional transformation to transform pixel values ​​in the spatial domain into the frequency domain.

[0058] Conversion can be performed using a fixed single transformation technique, or the transformation technique can be adaptively selected based on encoding / decoding settings. In this case, in the adaptive case, the transformation technique can be selected using an explicit or implicit method. In the explicit case, information on selecting each transformation technique applied in the horizontal and vertical directions or information on selecting a set of transformation techniques can be generated at the unit of a block, etc. In the implicit case, encoding settings can be defined based on the image type (I / P / B), color components, block size, shape, intra-screen prediction mode, etc., and a predefined transformation technique can be selected accordingly.

[0059] Additionally, depending on the encoding settings, some of the above transformations may be omitted. This means that one or more of the horizontal / vertical units may be omitted, either explicitly or implicitly.

[0060] In addition, the conversion unit can transmit information required to generate a conversion block to the encoding unit to encode it, and include the corresponding information in a bitstream and transmit it to the decoder, and the decoding unit of the decoder can parse the information and use it in the reverse conversion process.

[0061] The quantization unit (215) can quantize the input signal, and the signal obtained through the quantization process is called a quantized coefficient. For example, a residual block having a residual transform coefficient received from a transform unit can be quantized to obtain a quantized block having a quantized coefficient. The input signal is determined according to the encoding settings, and is not limited to the residual transform coefficient.

[0062] The quantization unit can quantize the transformed residual block using a quantization technique such as Dead Zone Uniform Threshold Quantization, Quantization Weighted Matrix, etc., but is not limited thereto, and various quantization techniques that are improved and modified therefrom can be used.

[0063] The quantization process may be omitted depending on the encoding settings. For example, the quantization process (including the inverse process) may be omitted depending on the encoding settings (e.g., the quantization parameter is 0, i.e., a lossless compression environment). As another example, if the compression performance through quantization is not achieved depending on the characteristics of the image, the quantization process may be omitted. In this case, the area where the quantization process is omitted among the quantization blocks (M x N) may be the entire area or a part of the area (M / 2 x N / 2, M x N / 2, M / 2 x N, etc.), and the quantization omission selection information may be implicitly or explicitly determined.

[0064] The quantization unit can transmit information necessary for generating a quantization block to the encoding unit to encode it, and can include the corresponding information in a bitstream and transmit it to the decoder, and the decoding unit of the decoder can parse the information and use it in the dequantization process.

[0065] Although the above example was explained under the assumption that the residual block is transformed and quantized through the transform unit and the quantization unit, the residual block may be transformed into a residual signal to generate a residual block having transform coefficients without performing the quantization process, and not only may the residual signal of the residual block be transformed into a transform coefficient and only the quantization process be performed, but neither the transform nor the quantization process may be performed. This may be determined according to the encoder settings.

[0066] The inverse quantization unit (220) inversely quantizes the residual block quantized by the quantization unit (215). That is, the inverse quantization unit (220) inversely quantizes the quantized frequency coefficient series to generate a residual block having frequency coefficients.

[0067] The inverse transform unit (225) inversely transforms the residual block inversely quantized by the inverse quantization unit (220). That is, the inverse transform unit (225) inversely transforms the frequency coefficients of the inversely quantized residual block to generate a residual block having pixel values, i.e., a restored residual block. Here, the inverse transform unit (225) can perform inverse transformation by inversely using the transformation method used in the transformation unit (210).

[0068] The addition unit (230) adds the prediction block predicted by the prediction unit (200) and the residual block restored by the inverse transformation unit (225) to restore the current block. The restored current block is stored as a reference picture (or reference block) in the encoded picture buffer (240) and can be used as a reference picture when encoding the next block of the current block or other blocks or other pictures in the future.

[0069] The filter unit (235) may include one or more post-processing filter processes such as a deblocking filter, a Sample Adaptive Offset (SAO), and an Adaptive Loop Filter (ALF). The deblocking filter may remove block distortion occurring at the boundary between blocks in a restored picture. The ALF may perform filtering based on a value obtained by comparing the restored image and the original image after the blocks are filtered through the deblocking filter. The SAO may restore the offset difference from the original image on a pixel basis for a residual block to which the deblocking filter is applied. Such post-processing filters may be applied to restored pictures or blocks.

[0070] The encoded picture buffer (240) can store a block or picture restored through the filter unit (235). The restored block or picture stored in the encoded picture buffer (240) can be provided to a prediction unit (200) that performs intra-screen prediction or inter-screen prediction.

[0071] The entropy encoding unit (245) scans the generated quantized frequency coefficient series according to various scan methods to generate a quantized coefficient series, and encodes it using an entropy encoding technique, etc., and outputs it. The scan pattern can be set to one of various patterns such as zigzag, diagonal, and raster. In addition, encoded data including encoded information transmitted from each component can be generated and output as a bitstream.

[0072]

[0073] FIG. 3 is a block diagram of an image decoding device according to one embodiment of the present invention.

[0074] Referring to FIG. 3, the image decoding device (30) may be configured to include an entropy decoding unit (305), a prediction unit (310), an inverse quantization unit (315), an inverse transformation unit (320), an adder / subtractor (325), a filter (330), and a decoding picture buffer (335).

[0075] Additionally, the prediction unit (310) may be configured to include an intra-screen prediction module and an inter-screen prediction module.

[0076] First, when the image bitstream transmitted from the image encoding device (20) is received, it can be transmitted to the entropy decoding unit (305).

[0077] The entropy decoding unit (305) can decode the bitstream and decode decoded data including quantized coefficients and decoding information transmitted to each component.

[0078] The prediction unit (310) can generate a prediction block based on data transmitted from the entropy decoding unit (305). At this time, a reference picture list using a default configuration technique can also be constructed based on a reference image stored in the decoded picture buffer (335).

[0079] The inter-screen prediction unit may include a reference picture composition unit, a motion compensation unit, and a motion information decoding unit, some of which may perform the same process as the encoder and some of which may perform the reverse derivation process.

[0080] The inverse quantization unit (315) can inverse quantize the quantized transform coefficients provided as a bitstream and decoded by the entropy decoding unit (305).

[0081] The inverse transform unit (320) can generate a residual block by applying inverse DCT, inverse integer transform, or similar inverse transform techniques to the transform coefficients.

[0082] At this time, the inverse quantization unit (315) and the inverse transformation unit (320) perform the reverse process of the process performed by the transformation unit (210) and the quantization unit (215) of the image encoding device (20) described above, and can be implemented in various ways. For example, the same process and inverse transformation shared with the transformation unit (210) and the quantization unit (215) may be used, or the transformation and quantization process may be performed in reverse using information about the transformation and quantization process from the image encoding device (20) (e.g., transformation size, transformation shape, quantization type, etc.).

[0083] The residual block that has undergone the inverse quantization and inverse transformation process can be added to the prediction block derived by the prediction unit (310) to generate a restored image block. This addition can be performed by an adder / subtractor (325).

[0084] The filter (330) may apply a deblocking filter to the restored image block to remove blocking effects as needed, and may additionally use other loop filters before and after the decoding process to improve video quality.

[0085] The restored and filtered image blocks can be stored in the decoded picture buffer (335).

[0086]

[0087] Although not shown in the drawing, the image encoding / decoding device may further include a picture division unit and a block division unit.

[0088] The picture partitioning unit can partition (or segment) the picture into at least one processing unit, such as a color space (e.g., YCbCr, RGB, or XYZ), a tile, a slice, a basic coding unit (or a maximum coding unit. Coding Tree Unit. CTU), and the block partitioning unit can partition the basic coding unit into at least one processing unit (e.g., encoding, prediction, transform, quantization, entropy, and in-loop filter units).

[0089] A basic coding unit can be obtained by dividing a picture into equal intervals in the horizontal and vertical directions. Based on this, segmentation such as tiles and slices can be performed, but is not limited thereto. The segmentation units such as tiles and slices can be composed of integer multiples of the basic coding block. However, exceptional cases may occur in segmentation units located at the image boundary. For this purpose, the size of the basic coding block may be adjusted.

[0090] For example, a picture may be divided into basic coding units and then split into said units, or a picture may be divided into said units and then split into basic coding units. In the present invention, the former case is assumed and described for each unit's division and division order, but the present invention is not limited thereto, and the latter case may also be possible depending on the encoding / decoding settings. In the latter case, the size of the basic coding unit may be adaptively modified depending on the division unit (tile, etc.). That is, it means that basic coding blocks having different sizes for each division unit may be supported.

[0091] In the present invention, the following examples will be described with the default setting of dividing a picture into basic encoding units. The default setting may mean that a picture is not divided into tiles or slices, or that a picture is one tile or one slice. However, it should be understood that the various embodiments described below can be applied identically or with modifications even in cases where each division unit (tile, slice, etc.) is first divided as described above and then divided into basic encoding units based on the obtained units (i.e., cases where each division unit is not an integer multiple of the basic encoding unit, etc.).

[0092] Among the above division units, a slice may be composed of a group of at least one consecutive block according to a scan pattern, a tile may be composed of a group of rectangular spatially adjacent blocks, and additional division units may be supported and configured according to their definitions. Slices and tiles may be division units supported for purposes such as parallel processing, and for this purpose, references between division units may be restricted (i.e., references may not be possible).

[0093] A slice can generate division information for each unit with information about the starting position of a continuous block, and for a tile, information about a horizontal or vertical division line can be generated, or information about the position of a tile (e.g., upper left, upper right, lower left, lower right) can be generated.

[0094] At this time, slices and tiles can be divided into multiple units depending on the encoding / decoding settings.

[0095] For example, some units may be a unit containing configuration information that affects the encoding / decoding process (i.e., including a tile header or slice header), and some units may be a unit that does not contain configuration information. Or, some units It may be a unit that cannot refer to other units during the encoding / decoding process, and some units may be a unit that can be referenced. Also, some units is a different unit It may be a hierarchical relationship including or some units is a different unit It can be an equal relationship.

[0096] Here, A and B can be a slice and a tile (or a tile and a slice). Alternatively, A and B can be composed of either a slice or a tile. For example, A can be composed of a slice / tile <type 1> and B can be composed of a slice / tile <type 2>.

[0097] Here, Type 1 and Type 2 can each be a single slice or tile. Alternatively, Type 1 can be a plurality of slices or tiles (a group of slices or a group of tiles) (including Type 2), and Type 2 can be a single slice or tile.

[0098] As previously described, the present invention assumes that a picture is composed of a single slice or tile. However, if two or more segmentation units occur, the above description can be understood and applied to the embodiments described below. In addition, A and B are examples of characteristics that segmentation units may have, and examples in which A and B of each example are mixed are also possible.

[0099] Meanwhile, the block segmentation unit can obtain information about the basic coding unit from the picture segmentation unit, and the basic coding unit can mean a basic (or starting) unit for prediction, transformation, quantization, etc. in the image encoding / decoding process. At this time, the basic coding unit can be composed of one luminance basic coding block (or maximum coding block. Coding Tree Block. CTB) and two basic chrominance coding blocks according to the color format (YCbCr in this example), and the size of each block can be determined according to the color format. In addition, a coding block (Coding Block. CB) can be obtained according to the segmentation process. The coding block can be understood as a unit that is not further divided into coding blocks according to certain restrictions, and can be set as the starting unit for division into lower units. In the present invention, the block is not limited to a rectangular shape, but can be understood as a broad concept including various shapes such as a triangle and a circle. For convenience of explanation, a rectangular shape is assumed and explained.

[0100] It should be understood that the following description targets one color component, but can be changed and applied to other color components in proportion to the ratio according to the color format (for example, in the case of YCbCr 4:2:0, the width and height ratio of the luminance component and the chrominance component is 2:1). It should also be understood that although block division dependent on other color components may be possible (for example, dependent on the block division result of Y in Cb / Cr), independent block division for each color component may be possible. It should also be understood that although a common block division setting (considering proportionality to the length ratio) may be used, it is also necessary to consider and understand that individual block division settings may be used depending on the color component.

[0101] In the block partitioning section, blocks can be expressed as M × N, and the maximum and minimum values ​​of each block can be obtained within a range. For example, if the maximum value of a block is set to 256 × 256 and the minimum value is set to 4 × 4, 2 m ×2 n A block of size 2m × 2m (in this example, m and n are integers from 2 to 8) or a block of size 2m × 2m (in this example, m and n are integers from 2 to 128) or a block of size m × m (in this example, m and n are integers from 4 to 256) can be obtained. Here, m and n may or may not be the same, and the ranges supported by the block, such as the maximum and minimum values, may occur more than once.

[0102] For example, information about the maximum and minimum sizes of blocks may be generated, and information about the maximum and minimum sizes of blocks in some segmentation settings may be generated. Here, in the former case, it may be range information about the maximum and minimum sizes that may occur in the image, and in the latter case, it may be information about the maximum and minimum sizes that may occur according to some segmentation settings. Here, the segmentation settings may be defined by the image type (I / P / B), color component (YCbCr, etc.), block type (encoding / prediction / conversion / quantization, etc.), segmentation type (Index or Type), segmentation method (QT, BT, TT, etc. in the Tree method, SI2, SI3, SI4, etc. in the Index method), etc.

[0103] In addition, there may be restrictions on the width / height ratio (block shape) that a block can have, and boundary value conditions may be set for this. At this time, only blocks less than / less than an arbitrary boundary value (k) may be supported, and k may be defined according to the width / height ratio such as A / B (A is the longer or equal value of width and height, B is the remaining value), and may be a real number greater than or equal to 1, such as 1.5, 2, 3, 4, etc. As in the above example, restrictions on the shape of a block in an image may be supported, or one or more restrictions may be supported depending on the segmentation settings.

[0104] In summary, whether block splitting is supported can be determined by the scope and conditions described above, as well as the split settings described below. For example, if a candidate (child block) for splitting a block (parent block) satisfies the supported block conditions, the splitting may be supported. If the candidate (child block) does not satisfy the supported block conditions, the splitting may not be supported.

[0105] A block division unit can be set in relation to each component of a video encoding device and a decoding device, and through this process, the size and shape of the block can be determined. At this time, the set block can be defined differently depending on the component, and can correspond to a prediction block in the case of a prediction unit, a transform block in the case of a transform unit, and a quantization block in the case of a quantization unit. However, the present invention is not limited to this, and block units according to other components can be additionally defined. In the present invention, the case where the input and output of each component are rectangular is described, but some components may allow input / output of other shapes (e.g., triangle, etc.).

[0106] The size and shape of the initial (or starting) block of the block division unit can be determined from the upper unit. The initial block can be divided into blocks of smaller sizes, and when the optimal size and shape according to the block division are determined, the block can be determined as the initial block of the lower unit. Here, the upper unit can be an encoding block, and the lower unit can be a prediction block or a transform block, but the present invention is not limited thereto, and various examples of modifications may be possible. When the initial block of the lower unit is determined as in the above example, a division process can be performed to find a block of the optimal size and shape, like the upper unit.

[0107] In summary, the block splitting unit can split the basic coding block (or maximum coding block) into at least one coding block, and the coding block can be split into at least one prediction block / transform block / quantization block. In addition, the prediction block can be split into at least one transform block / quantization block, and the transform block can be split into at least one quantization block. Here, some blocks may have a dependent relationship with other blocks (i.e., defined by upper units and lower units) or may have an independent relationship. For example, the prediction block may be a higher unit of the transform block or may be an independent unit from the transform block, and various relationship settings may be possible depending on the type of block.

[0108] Depending on the sub / decoding settings, the combination of upper and lower units may be determined. Here, combining between units means that, rather than performing division from the upper unit to the lower unit, the sub / decoding process (e.g., prediction unit, transformation unit, inverse transformation unit, etc.) of the lower unit is performed with the block (size and shape) of the upper unit. In other words, the division process across multiple units may be shared, and the division information may be generated from one of the units (e.g., the upper unit).

[0109] For example, prediction, transformation and inverse transformation processes can be performed in the coding block (when the coding block is combined with a prediction block and a transformation block).

[0110] For example, the prediction process can be performed in the coding block (when the coding block is combined with the prediction block), and the transformation and inverse transformation processes can be performed in a transform block that is the same as or smaller than the coding block.

[0111] For example, the prediction process can be performed on a prediction block that is the same as or smaller than the encoding block (when the encoding block is combined with a transform block), and the transformation and inverse transformation processes can be performed on the encoding block.

[0112] For example, the prediction process can be performed on a prediction block that is the same as or smaller than the encoding block (when the prediction block is combined with a transform block), and the transformation and inverse transformation processes can be performed on the prediction block.

[0113] For example, the prediction process can be performed in a prediction block that is the same as or smaller than the encoding block (if not combined in any block), and the transformation and inverse transformation processes can be performed in a transform block that is the same as or smaller than the encoding block.

[0114] Although the above examples describe various cases regarding encoding, prediction, and transformation blocks, they are not limited thereto.

[0115] The combination between the above units may be supported by a fixed setting in the image, or may be supported by an adaptive setting considering various sub / decoding elements. Here, the sub / decoding elements may include an image type, a color component, an encoding mode (Intra / Inter), a split setting, a block size / shape / position, a width / height ratio, prediction-related information (e.g., an intra-screen prediction mode, an inter-screen prediction mode, etc.), transformation-related information (e.g., transformation technique selection information, etc.), quantization-related information (e.g., quantization region selection information, quantized transform coefficient encoding information, etc.), etc.

[0116] As described above, when a block of optimal size and shape is found, mode information (e.g., segmentation information, etc.) for the block can be generated. The mode information, together with information generated from the component to which the block belongs (e.g., prediction-related information and transformation-related information, etc.), can be included in the bitstream and transmitted to the decoder, where it can be parsed into units of the same level and used in the image decoding process.

[0117] Below, the division method will be explained, and for the convenience of explanation, it is assumed that the initial block is square in shape. However, it is not limited to this, as the same or similar method can be applied even if the initial block is rectangular in shape.

[0118] The block partitioning unit can support various types of partitioning. For example, it can support tree-based partitioning, index-based partitioning, and other methods. Tree-based partitioning can determine the partitioning type based on various types of information (e.g., whether a partition is present, tree type, partitioning direction, etc.), while index-based partitioning can determine the partitioning type based on specific index information.

[0119]

[0120] Figure 4 is an exemplary diagram illustrating various partitioning configurations that can be achieved by the block partitioning unit of the present invention. While this example assumes that the partitioning configuration shown in Figure 4 is achieved through a single partitioning operation (or process), this is not limited to this scenario and may also be achieved through multiple partitioning operations. Furthermore, additional partitioning configurations not illustrated in Figure 4 may be possible.

[0121] (tree-based partitioning)

[0122] In the tree-based partitioning of the present invention, quad trees (QT), binary trees (BT), ternary trees (TT), etc. may be supported. When one tree method is supported, it may be referred to as single-tree partitioning, and when two or more tree methods are supported, it may be referred to as multi-tree partitioning.

[0123] For QT, it means the method (n) in which the block is divided into two (i.e., four) in the horizontal and vertical directions, for BT, it means the method (b to g) in which the block is divided into two in either the horizontal or vertical direction, and for TT, it means the method (h to m) in which the block is divided into three in either the horizontal or vertical direction.

[0124] Here, for QT, it is possible to support a method (o, p) that divides in four directions, limiting the division direction to either horizontal or vertical. In addition, for BT, it is possible to support only a method with equal sizes (b, c), a method with unequal sizes (d to g), or a mixture of the two. In addition, for TT, it is possible to support only a method (h, j, k, m) that has a division arrangement that is biased to a specific direction (such as 1:1:2, 2:1:1 in the left-to-right or top-to-bottom direction), a method (i, l) that is centered (such as 1:2:1), or a mixture of the two. In addition, a method (q) that divides in four directions each in the horizontal and vertical directions (i.e., 16 divisions) can also be supported.

[0125] And among the tree methods, the z-division method (b, d, e, h, i, j, o) may be supported only in the horizontal division direction, or the z-division method (c, f, g, k, l, m, p) may be supported only in the vertical division direction, or a combination of the two methods may be supported. Here, z may be an integer greater than or equal to 2, such as 2, 3, or 4.

[0126] In the present invention, it is explained assuming that QT supports n, BT supports b and c, and TT supports i and l.

[0127] Depending on the encoding / decoding settings, one or more of the above tree splitting methods may be supported. For example, QT may be supported, QT / BT may be supported, or QT / BT / TT may be supported.

[0128] The above example is an example of a case where the basic tree split is QT and BT and TT are included in the additional splitting method depending on whether other trees are supported, but various variations are possible. In this case, information on whether other trees are supported (such as bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, which can have a value of 0 or 1, and 0 means not supported and 1 means supported) can be implicitly determined according to the encoding / decoding settings or can be explicitly determined in units such as sequences, pictures, slices, and tiles.

[0129] The split information may include information on whether or not to split (tree_part_flag. Or, qt_part_flag, bt_part_flag, tt_part_flag, bt_tt_part_flag. Can have a value of 0 or 1, 0 means not split and 1 means split). In addition, depending on the split method (BT and TT), information on the split direction (dir_part_flag. Or, bt_dir_part_flag, tt_dir_part_flag, bt_tt_dir_part_flag. Can have a value of 0 or 1, 0 means <horizontal / horizontal> and 1 means <vertical / vertical>) may be added, which may be information that can occur when splitting is performed.

[0130] When multiple tree partitions are supported, various partitioning configurations are possible. The following example assumes that the partitioning information is structured at a single depth level (i.e., recursive partitioning is possible if the supported partitioning depth is set to more than one, but this is for convenience of explanation).

[0131] For example (1), information on whether or not a partition has been performed is checked. If a partition is not performed, the partition is terminated.

[0132] If a split is performed, the selection information for the split type is checked (e.g., tree_idx. QT if 0, BT if 1, TT if 2). At this time, the split direction information is additionally checked according to the selected split type, and the next step is moved on (if additional splitting is possible due to reasons such as the split depth not reaching the maximum, start over from the beginning; if splitting is not possible, the splitting is terminated).

[0133] For example (2), information regarding whether a certain tree method (QT) is split is checked and the next step is taken. If no split is performed, information regarding whether a certain tree method (BT) is split is checked. If no split is performed, information regarding whether a certain tree method (TT) is split is checked. If no split is performed, the splitting process is terminated.

[0134] If some tree-based (QT) splitting is performed, proceed to the next step. Furthermore, if some tree-based (BT) splitting is performed, the split direction information is checked and the process proceeds to the next step. Furthermore, if some tree-based (TT) splitting is performed, the process proceeds to the next step.

[0135] For example (3), information regarding whether a certain tree method (QT) is split is checked. If splitting is not performed, information regarding whether a certain tree method (BT and TT) is split is checked. If splitting is not performed, the splitting is terminated.

[0136] If some tree-based (QT) splitting is performed, proceed to the next step. Also, if some tree-based (BT and TT) splitting is performed, check the splitting direction information and proceed to the next step.

[0137] The above examples illustrate the existence of tree splitting priorities (examples 2 and 3) or the absence of such priorities (example 1), but various variations are possible. Furthermore, while the above examples illustrate the case where the splitting of the current step is independent of the splitting results of the previous step, it is also possible to configure the splitting of the current step to depend on the splitting results of the previous step.

[0138] For example, in the case of examples 1 to 3, if some tree-style partitioning (QT) was performed in the previous stage and then moved to the current stage, the same tree-style partitioning (QT) can be supported in the current stage as well.

[0139] On the other hand, if some tree-style splits (QT) were not performed in the previous stage and other tree-style splits (BT or TT) were performed and then passed to the current stage, it may also be possible to set up that some tree-style splits (BT and TT) are supported in subsequent stages including the current stage, excluding some tree-style splits (QT).

[0140] In the above case, it means that the tree configuration supported for block partitioning can be adaptive, which means that the partitioning information configuration described above can also be configured differently. (The example described below assumes example 3.) That is, in the above example, if the partitioning of some tree methods (QT) was not performed in the previous step, the partitioning process can be performed without considering some tree methods (QT) in the current step. In addition, the partitioning information regarding the related tree method (e.g., information on whether to partition, information on the direction of partitioning, etc.) In this example <qt>In , information about whether to split can be removed and configured.

[0141] The above example is about adaptive partition information configuration for cases where block partitioning is allowed (e.g., block size exists within a range between maximum and minimum values, partition depth of each tree method does not reach maximum depth <allowed depth>, etc.), and adaptive partition information configuration may also be possible for cases where block partitioning is restricted (e.g., block size does not exist within a range between maximum and minimum values, partition depth of each tree method reaches maximum depth, etc.).

[0142] As previously mentioned, in the present invention, tree-based partitioning can be performed using a recursive method. For example, if the partition flag of an encoding block having a partition depth of k is 0, encoding of the encoding block is performed in the encoding block having a partition depth of k, and if the partition flag of the encoding block having a partition depth of k is 1, encoding of the encoding block is performed in N sub-encoding blocks having a partition depth of k+1 (where N is an integer greater than or equal to 2, such as 2, 3, or 4) according to the partitioning method.

[0143] The above sub-coding block can be set as an encoding block (k+1) again and divided into sub-coding blocks (k+2) through the above process, and this hierarchical division method can be determined according to division settings such as the division range and the division allowable depth.

[0144] At this time, the bitstream structure for expressing the segmentation information can be selected from one or more scanning methods. For example, the bitstream of the segmentation information can be configured based on the segmentation depth order, or the bitstream of the segmentation information can be configured based on whether or not the segmentation has occurred.

[0145] For example, in the case of the split depth order criterion, it is a method of acquiring split information at the current level of depth based on the first block and then acquiring split information at the next level of depth, and in the case of the splitting status criterion, it means a method of preferentially acquiring additional split information from a block split based on the first block, and other additional scanning methods may be considered.

[0146]

[0147] (Index-based partitioning)

[0148] In the index-based partitioning of the present invention, the CSI (Constant Split Index) method and the VSI (Variable Split Index) method can be supported.

[0149] The CSI method may be a method in which k sub-blocks are obtained through division in a predetermined direction, and k may be an integer greater than or equal to 2, such as 2, 3, or 4. Specifically, it may be a division method in which the size and shape of the sub-block are determined based on the k value regardless of the size and shape of the block. Here, the predetermined direction may be one or a combination of two or more directions among horizontal, vertical, and diagonal (upper left to lower right, or lower left to upper right, etc.).

[0150] The index-based CSI partitioning method of the present invention may include candidates that are partitioned into z in one of the horizontal or vertical directions. In this case, z may be an integer greater than or equal to 2, such as 2, 3, or 4, and one of the horizontal or vertical lengths of each sub-block may be the same and the other may be the same or different. The ratio of the horizontal or vertical lengths of the sub-blocks is A1:A2:...:A Z A1 to A Z can be an integer greater than or equal to 1, such as 1, 2, or 3.

[0151] In addition, it can include candidates that are divided into x and y in the horizontal and vertical directions, respectively. At this time, x and y can be integers greater than 1, such as 1, 2, 3, and 4, but the case where x and y are 1 at the same time (because a already exists) can be restricted. In the case of Fig. 4, the case where the ratio of the horizontal or vertical length of each sub-block is the same is illustrated, but it can also include candidates that include different cases.

[0152] Additionally, it may include candidates that are split into w in one of the diagonal directions (top left -> bottom right) or in one of the diagonal directions (bottom left -> top right), and w may be an integer greater than or equal to 2, such as 2 or 3.

[0153] Referring to Fig. 4, the length ratio of each sub-block can be divided into a symmetrical split shape (b) and an asymmetrical split shape (d, e), and a split shape that is oriented in a specific direction (k, m) and a split shape that is positioned in the center (k). The split shape can be defined by various encoding / decoding factors including not only the length ratio of the sub-block but also the shape of the sub-block, and the supported split shape can be implicitly or explicitly determined according to the encoding / decoding settings. Thus, a candidate group in an index-based splitting method can be determined based on the supported split shape.

[0154] Meanwhile, the VSI method may be a method in which one or more sub-blocks are obtained through division in a predetermined direction while the width (w) or height (h) of the sub-block is fixed, and w and h may be integers greater than or equal to 1, such as 1, 2, 4, or 8. Specifically, it may be a division method in which the number of sub-blocks is determined based on the size and shape of the block and the w or n value.

[0155] The index-based VSI partitioning method of the present invention may include candidates for partitioning by fixing either the horizontal or vertical length of a sub-block. Alternatively, it may include candidates for partitioning by fixing the horizontal and vertical lengths of a sub-block. Since the horizontal or vertical length of a sub-block is fixed, it may have the characteristic of allowing for equal partitioning in the horizontal or vertical direction, but is not limited thereto.

[0156] If the block before splitting is M x N and the horizontal length of the sub-block is fixed (w), or the vertical length is fixed (h), or the horizontal and vertical lengths are fixed (w, h), the number of sub-blocks obtained can be (M*N) / w, (M*N) / h, and (M*N) / w / h, respectively.

[0157] Depending on the encoding / decoding settings, only the CSI method, only the VSI method, or both methods may be supported, and information about the supported methods may be implicitly or explicitly specified.

[0158] In the present invention, a description is given assuming that the CSI method is supported.

[0159] Depending on the encryption / decryption settings, a candidate group can be formed by including two or more candidates among the above index splits.

[0160] For example, candidate groups such as {a, b, c}, {a, b, c, n}, {a to g, n} can be formed, and an example of forming a candidate group of a block shape that is predicted to occur frequently based on general statistical characteristics, such as a block shape that is divided into two in the horizontal or vertical direction or divided into two in the horizontal and vertical directions respectively, can be formed.

[0161] Alternatively, a candidate set such as {a, b}, {a, o}, {a, b, o} or {a, c}, {a, p}, {a, c, p} can be constructed, which includes candidates that are divided into 2 and 4 in the horizontal and vertical directions, respectively. This can be an example of constructing a candidate set of block shapes that are predicted to have many divisions in a specific direction.

[0162] Alternatively, a candidate group such as {a, o, p} or {a, n, q} can be formed, which may be an example of forming a candidate group of block shapes in which it is predicted that many splits having a size smaller than the block before splitting will occur.

[0163] Alternatively, a candidate group such as {a, r, s} can be formed, which may be an example of forming a candidate group of non-rectangular partition shapes by judging that the optimal partition result that can be obtained in a rectangular shape through another method (tree method) in the block before partitioning has been obtained.

[0164] As in the above example, various candidate configurations may be possible, and one or more candidate configurations may be supported by considering various encoding / decoding elements.

[0165] Once the above candidate group configuration is complete, various segmentation information configurations may be possible.

[0166] For example, index selection information may be generated from a candidate set that includes non-split candidates (a) and split candidates (b to s).

[0167] Alternatively, information indicating whether a split is performed (whether the split shape is a or not) may be generated, and index selection information may be generated from a candidate group consisting of candidates (b to s) to be split when a split is performed (if not a).

[0168] In addition to the above description, various methods of configuring partition information may be possible, and, excluding the information indicating whether or not to partition, binary bits may be assigned to the index of each candidate in the candidate group through various methods such as fixed-length binarization and variable-length binarization. If the number of candidates is two, one bit may be assigned to the index selection information, and if there are three or more, more than one bit may be assigned to the index selection information.

[0169] Unlike tree-based partitioning, index-based partitioning can be a method of selectively organizing candidate groups into partition types that are expected to occur frequently.

[0170] And since the amount of bits required to express index information may increase depending on the number of supported candidates, it may be a method suitable for single-layer partitioning (e.g., limiting the partition depth to 0) rather than a tree-based hierarchical partitioning (recursive partitioning). In other words, it may be a method that supports a single partitioning operation, and the sub-blocks obtained through index-based partitioning may not be further partitioned.

[0171] At this time, it may mean that further division into blocks of the same type with smaller sizes is impossible (for example, an encoding block obtained through an index division method cannot be further divided into encoding blocks), but a setting in which further division into blocks of a different type is impossible (for example, a encoding block cannot be divided into not only encoding blocks but also prediction blocks). Of course, it is not limited to the above example, and other examples of modifications may be possible.

[0172]

[0173] Next, we will look at the case where the block splitting settings are determined based on the type of block among the encryption / decryption elements.

[0174] First, an encoding block can be obtained through a splitting process. Here, a tree-based splitting method can be used for the splitting process, and depending on the tree type, splitting forms such as a (no split), n (QT), b, c (BT), i, l (TT) in Fig. 4 can be produced. Depending on the encoding / decoding settings, various combinations of each tree type, such as QT / QT+BT / QT+BT+TT, can be possible.

[0175] The example described below shows a process in which prediction blocks and transformation blocks are finally segmented based on the encoding blocks obtained through the above process, and assumes that prediction, transformation, and inverse transformation processes are performed based on each segmented size.

[0176] As an example (1), a prediction block may be set to the same size as the encoding block to perform the prediction process, and a transformation block may be set to the same size as the encoding block (or prediction block) to perform the transformation and inverse transformation processes. Since the prediction block and transformation block are set based on the encoding block, there is no separate segmentation information.

[0177] For example (2), a prediction block may be set to the same size as the encoding block to perform the prediction process. In the case of a transformation block, a transformation block may be obtained through a division process based on the encoding block (or prediction block), and transformation and inverse transformation processes may be performed based on the obtained size.

[0178] Here, the splitting process can use a tree-based splitting method, and depending on the tree type, the splitting results can be a (no split), b, c (BT), i, l (TT), n (QT) of Fig. 4, etc. Depending on the encoding / decoding settings, various combinations of each tree type, such as QT / BT / QT+BT / QT+BT+TT, etc., can be possible.

[0179] Here, the splitting process can use an index-based splitting method, and depending on the index type, splitting results such as a (no split), b, c, d, etc. in Fig. 4 can be produced. Depending on the encoding / decoding settings, various candidate group configurations such as {a, b, c}, {a, b, c, d}, etc. can be possible.

[0180] As an example (3), in the case of a prediction block, a segmentation process can be performed based on an encoding block to obtain a prediction block, and a prediction process can be performed based on the obtained size. In the case of a transformation block, the size of the encoding block can be set as is, and the transformation and inverse transformation processes can be performed. This example may correspond to a case where the prediction block and the transformation block have an independent relationship with each other.

[0181] Here, the splitting process can use an index-based splitting method, and depending on the index type, the splitting results such as a (no split), b to g, n, r, s, etc. in Fig. 4 can be produced. Depending on the encoding / decoding settings, various candidate group configurations such as {a, b, c, n}, {a to g, n}, {a, r, s}, etc. can be possible.

[0182] As an example (4), in the case of a prediction block, a prediction block can be obtained by performing a splitting process based on an encoding block, and a prediction process can be performed based on the obtained size. In the case of a transformation block, the size of the prediction block can be set as is, and the transformation and inverse transformation processes can be performed. This example may be a case where the transformation block is set with the same size as the obtained prediction block, or vice versa (the prediction block is set with the same size as the transformation block).

[0183] Here, the splitting process can use a tree-based splitting method, and depending on the tree type, splitting forms such as a (no split), b, c (BT), n (QT) in Fig. 4 can be produced. Depending on the encoding / decoding settings, various combinations of each tree type such as QT / BT / QT+BT can be possible.

[0184] Here, the splitting process can use an index-based splitting method, and depending on the index type, split forms such as a (no split), b, c, n, o, p of Fig. 4 can be produced. Depending on the encoding / decoding settings, various candidate group configurations such as {a, b}, {a, c}, {a, n}, {a, o}, {a, p}, {a, b, c}, {a, o, p}, {a, b, c, n}, {a, b, c, n, p} can be possible. In addition, among the index-based splitting methods, the VSI method can be used alone or mixed with the CSI method to form a candidate group.

[0185] As an example (5), in the case of a prediction block, a splitting process may be performed based on an encoding block to obtain a prediction block, and a prediction process may be performed based on the obtained size. In addition, in the case of a transformation block, a splitting process may be performed based on an encoding block to obtain a prediction block, and a transformation and inverse transformation process may be performed based on the obtained size. This example may be a case where splitting of each of a prediction block and a transformation block is performed based on an encoding block.

[0186] Here, the partitioning process can use a tree-based partitioning method or an index-based partitioning method, and the candidate group can be configured in the same or similar way as in example 4.

[0187] The above examples illustrate some possible scenarios, such as whether the partitioning process for each type of block is shared. However, this is not a limitation, and various variations are possible. Furthermore, block partitioning settings can be determined by considering not only the block type but also various encoding / decoding factors.

[0188] At this time, the encoding / decoding elements may include image type (I / P / B), color component (YCbCr), block size / shape / position, block width / height ratio, block type (encoding block, prediction block, transform block, quantization block, etc.), segmentation status, encoding mode (Intra / Inter), prediction-related information (intra-screen prediction mode, inter-screen prediction mode, etc.), transform-related information (transformation technique selection information, etc.), quantization-related information (quantization area selection information, quantized transform coefficient encoding information, etc.).

[0189]

[0190] In an image encoding method according to an embodiment of the present invention, intra-screen prediction may be configured as follows. The intra-screen prediction of the prediction unit may include a reference pixel configuration step, a prediction block generation step, a prediction mode determination step, and a prediction mode encoding step. In addition, the image encoding apparatus may be configured to include a reference pixel configuration unit, a prediction block generation unit, and a prediction mode encoding unit that implement the reference pixel configuration step, the prediction block generation step, the prediction mode determination step, and the prediction mode encoding step. Some of the above-described processes may be omitted, other processes may be added, and the order may be changed to a different order than the order described above.

[0191] FIG. 5 is an exemplary diagram showing an on-screen prediction mode according to one embodiment of the present invention.

[0192] Referring to Fig. 5, 67 prediction modes are configured as prediction mode candidates for within-screen prediction, and it is assumed that 65 of them are directional modes and 2 are non-directional modes (DC, Planar), but this is not limited to the case and various configurations may be possible. In this case, the directional modes may be distinguished by inclination (e.g., dy / dx) or angle information (Degree). In addition, all or part of the above prediction modes may be included in the prediction mode candidates of the luminance component or the chrominance component, and other additional modes may be included in the prediction mode candidates.

[0193] In the present invention, the direction of the directional mode may mean a straight line, and a curved directional mode may also be additionally configured as a prediction mode. In addition, in the case of the non-directional mode, it may include a DC mode that obtains a prediction block by averaging (or weighted average, etc.) the pixels of adjacent neighboring blocks of the current block (e.g., left, top left, top right, bottom left blocks, etc.), and a Planar mode that obtains a prediction block by linear interpolation of pixels of neighboring blocks, etc.

[0194] Here, in the case of DC mode, the reference pixels used for generating prediction blocks can be obtained from blocks grouped in various combinations such as left, top, left+top, left+bottom left, top+top right, left+top+bottom left+top right, etc., and the reference pixel acquisition block location can be determined according to the encoding / decoding settings defined by the image type, color component, block size / shape / location, etc.

[0195] Here, in the case of the Planar mode, the pixels used for generating the prediction block can be obtained from an area composed of reference pixels (e.g., left, top, top left, top right, bottom left, etc.) and an area not composed of reference pixels (e.g., right, bottom, bottom right, etc.). In the case of an area not composed of reference pixels (i.e., not encoded), one or more pixels in an area composed of reference pixels can be implicitly obtained (e.g., copied as is, weighted average, etc.), or information on at least one pixel in an area not composed of reference pixels can be explicitly generated. Therefore, a prediction block can be generated using an area composed of reference pixels and an area not composed of reference pixels as described above.

[0196] Additional non-directional modes other than those described above may be included, and although the present invention focuses on the linear directional mode and the non-directional modes of DC and Planar, changes to other cases may also be applied.

[0197] Fig. 5 may be a prediction mode that is fixedly supported regardless of the block size. Additionally, the prediction mode supported depending on the block size may differ from Fig. 5.

[0198] For example, the number of prediction mode candidates can be adaptive (e.g., the angles between prediction modes are equally spaced, but the angles are set differently, e.g., 9, 17, 33, 65, 129 based on directional modes), or the number of prediction mode candidates can be fixed but have different configurations (e.g., directional mode angles, non-directional types, etc.).

[0199] Additionally, Fig. 5 may be a prediction mode that is fixedly supported regardless of the shape of the block. Additionally, the prediction mode supported depending on the shape of the block may differ from Fig. 5.

[0200] For example, the number of prediction mode candidates may be adaptive (e.g., fewer or more prediction modes derived from horizontal or vertical directions depending on the aspect ratio of the block), or the number of prediction mode candidates may be fixed but configured differently (e.g., more finely tuned prediction modes derived from horizontal or vertical directions depending on the aspect ratio of the block).

[0201] Alternatively, the prediction mode on the side with a longer block length may support a larger number of prediction modes, and the prediction mode on the side with a shorter block length may support a smaller number of prediction modes. In the case of the side with a longer block length, the prediction mode interval may be a mode located to the right of mode 66 in FIG. 5 (e.g., a mode with an angle greater than +45 degrees with respect to mode 50, i.e., a mode numbered 67 to 80, etc.) or a mode located to the left of mode 2 (e.g., a mode with an angle greater than -45 degrees with respect to mode 18, i.e., a mode numbered -1 to -14, etc.). This may be determined by the ratio of the width and height of the block, and the opposite situation may also be possible.

[0202] In the present invention, the description is centered on the case where the prediction mode is a fixedly supported prediction mode (regardless of any encoding / decoding element) as shown in FIG. 5, but it may also be possible to set a prediction mode that is adaptively supported depending on the encoding settings.

[0203] Additionally, when classifying the prediction mode, horizontal and vertical modes (modes 18 and 50), some diagonal modes (Diagonal up right <2>, Diagonal down right <34>, Diagonal down left <66>, etc.) can be used as criteria, which can be a classification method performed based on some directionality (or angle. 45 degrees, 90 degrees, etc.).

[0204] And some modes located at both ends of the directional modes (modes 2 and 66) may be the modes that serve as the basis for the prediction mode classification, which is a possible example when the prediction mode configuration within the screen is as shown in Fig. 5. That is, when the prediction mode configuration is adaptive, it may also be possible for the above-mentioned reference modes to change. For example, it may be possible for mode 2 to be replaced by a mode having a number less than or greater than 2 (such as -2, -1, 3, 4), or for mode 66 to be replaced by a mode having a number less than or greater than 66 (such as 64, 66, 67, 68).

[0205] Additionally, additional prediction modes (color copy mode, color mode) regarding color components may be included in the prediction mode candidate set. Here, the color copy mode may be a prediction mode related to a method of obtaining data for generating a prediction block from an area located in another color space, and the color mode may be a prediction mode related to a method of obtaining a prediction mode from an area located in another color space.

[0206]

[0207] Fig. 6 is an exemplary diagram illustrating a reference pixel configuration used for intra-screen prediction according to one embodiment of the present invention. The size and shape (M x N) of a prediction block can be obtained through a block division unit.

[0208] In-screen prediction may be generally performed in units of prediction blocks, but may also be performed in units of encoding blocks, transformation blocks, etc., depending on the settings of the block division unit. After checking the block information, the reference pixel configuration unit can configure the reference pixels used for prediction of the current block. At this time, the reference pixels are stored in temporary memory (e.g., array <array>. It can be managed through a 1st, 2nd array, etc., and is created and removed for each prediction process within the screen of the block, and the size of the temporary memory can be determined according to the configuration of the reference pixel.

[0209] In this example, we assume that the left, top, top-left, top-right, and bottom-left blocks centered on the current block are used for prediction of the current block. However, this is not limited to this case, and a block candidate group with a different configuration may be used for prediction of the current block. For example, the candidate group of neighboring blocks for the above-mentioned reference pixel may be an example of a case where it follows a raster or Z scan, and some of the candidates may be removed depending on the scan order, or may be configured by including other block candidates (for example, a right, bottom, and bottom-right block may be additionally configured).

[0210] Additionally, if some prediction modes (color copy mode) are supported, some areas in other color spaces can be used for prediction of the current block, so these can also be considered as reference pixels.

[0211]

[0212] Figure 7 is a conceptual diagram illustrating a target block and adjacent blocks for intra-screen prediction according to one embodiment of the present invention. Specifically, the left side of Figure 7 represents a block adjacent to the current block in the current color space, and the right side represents a corresponding block in another color space. For convenience of explanation, the following description assumes that the current block and adjacent blocks in the current color space are the basic reference pixel configuration.

[0213] As shown in Fig. 6, the reference pixels used for prediction of the current block can be composed of adjacent pixels of the left, upper, upper-left, upper-right, and lower-left blocks (Ref_L, Ref_T, Ref_TL, Ref_TR, and Ref_BL in Fig. 6). At this time, the reference pixels are generally composed of pixels of the neighboring block closest to the current block (a in Fig. 6, expressed as a reference pixel line), but it may also be possible to compose other pixels (b in Fig. 6 and pixels of other outer lines) as reference pixels.

[0214] The pixels adjacent to the current block can be classified into at least one reference pixel line, and the pixel closest to the current block can be divided into ref_0 {for example, pixels whose inter-pixel distance is 1 from the boundary pixels of the current block, p(-1,-1) ~ p(2m-1,-1), p(-1,0) ~ p(-1,2n-1)}, the next adjacent pixel {for example, pixels whose inter-pixel distance from the boundary pixels of the current block is 2, p(-2,-2) ~ p(2m,-2), p(-2,-1) ~ p(-2,2n)} can be divided into ref_1, the next adjacent pixel {for example, pixels whose inter-pixel distance from the boundary pixels of the current block is 3, p(-3,-3) ~ p(2m+1, -3), p(-3,-2) ~ p(-3,2n+1)} can be divided into ref_2, and so on. In other words, the pixels can be classified into reference pixel lines according to their inter-pixel distances from the boundary pixels of the current block.

[0215] The number of reference pixel lines supported here can be N or more, and N can be an integer greater than or equal to 1, such as 1 to 5. In this case, it is common for the reference pixel line candidates to be sequentially included starting from the reference pixel line closest to the current block, but this is not limited thereto. For example, if N is 3<ref_0, ref_1, ref_2> The above candidates can be sequentially composed as follows, or<ref_0, ref_1, ref_3> ,<ref_0, ref_2, ref_3> ,<ref_1, ref_2, ref_3> It may also be possible for the candidate set to be configured such that it excludes non-sequential or closest reference pixel lines.

[0216] Prediction can be performed using all reference pixel lines within the candidate set, or using some reference pixel lines (one or more).

[0217] For example, depending on the encoding / decoding settings, one of a plurality of reference pixel lines may be selected and intra-screen prediction may be performed using that reference pixel line. Alternatively, two or more of a plurality of reference pixel lines may be selected and intra-screen prediction may be performed using that reference pixel line (e.g., by applying a weighted average, etc. to the data of each reference pixel line).

[0218] Here, reference pixel line selection can be implicitly or explicitly determined. For example, implicitly means that it is determined by the encoding / decoding settings, which are defined by one or more combinations of factors such as image type, color components, and block size / shape / location. Explicitly means that reference pixel line selection information can be generated at the block level or other units.

[0219] Although the present invention is described primarily with respect to a case in which intra-screen prediction is performed using the nearest reference pixel line, it should be understood that the various embodiments described below can be applied in the same or similar manner even when multiple reference pixel lines are used.

[0220] For example, in the sub-block unit intra-screen prediction described below, a setting may be supported in which the above information is implicitly determined only by considering the case where the nearest reference pixel line is used. That is, the sub-block unit intra-screen prediction may be performed using a preset reference pixel line, and the reference pixel line may be selected through implicit processing. Here, the reference pixel line may mean the nearest reference pixel line, but is not limited thereto.

[0221] Alternatively, a reference pixel line can be adaptively selected for intra-screen prediction at the sub-block level, and various reference pixel lines, including the nearest reference pixel, can be selected to perform intra-screen prediction at the sub-block level. That is, intra-screen prediction at the sub-block level can be performed using reference pixel lines determined by considering various encoding / decoding factors, and the reference pixel line can be selected through implicit or explicit processing.

[0222] The reference pixel configuration unit of the screen prediction of the present invention may include a reference pixel generation unit, a reference pixel interpolation unit, a reference pixel filter unit, etc., and may be configured to include all or part of the above configurations.

[0223] In the reference pixel configuration section, the availability of reference pixels can be checked to classify available and unavailable reference pixels. Here, the availability of the reference pixel is determined to be unavailable if at least one of the following conditions is satisfied.

[0224] For example, if it is located outside the picture boundary, if it does not belong to the same partitioning unit as the current block (e.g., units that cannot be referenced to each other, such as slices or tiles. However, if units such as slices or tiles have the characteristic of being referenced to each other, an exception is made even if they are not the same partitioning unit), or if the encoding / decoding is not completed, it can be judged as unusable if any of the above conditions are not met. In other words, if none of the above conditions are met, it can be judged as usable.

[0225] Additionally, the use of reference pixels can be restricted by the sub / decoding settings. For example, even if the above conditions are met, the use of reference pixels can be restricted depending on whether constrained intra-picture prediction (e.g., constrained_intra_pred_flag) is performed. Constrained intra-picture prediction can be performed when error-resilient sub / decoding is desired due to external factors such as the communication environment, and when the use of blocks reconstructed by referencing other images as reference pixels is prohibited.

[0226] If constrained intra-picture prediction is disabled (e.g., I-picture type, or constrained_intra_pred_flag = 0 in P or B-picture types), all reference pixel candidate blocks may be available.

[0227] Alternatively, when constrained intra-picture prediction is enabled (e.g., constrained_intra_pred_flag = 1 in P or B picture types), the reference pixel candidate block will be assumed to be a condition for determining whether to use it or not depending on the encoding mode (Intra or Inter), but the condition may also be determined based on various other encoding / decoding factors.

[0228] Since a reference pixel is composed of one or more blocks, after checking the possibility of the above reference pixels, they can be classified into three cases: <All available>, <Partially available>, and <All unavailable>. In the remaining cases except for the case where all are available, reference pixels at the locations of unavailable candidate blocks can be filled in or created.

[0229] If a reference pixel candidate block is available, the pixel at that location can be included in the reference pixel memory of the current block. At this time, the pixel data can be copied as is or included in the reference pixel memory through processes such as reference pixel filtering and reference pixel interpolation. In addition, if the reference pixel candidate block is unavailable, the pixel obtained through the reference pixel generation process can be included in the reference pixel memory of the current block.

[0230] The following examples illustrate how to generate reference pixels for unusable block locations using various methods.

[0231] For example, a reference pixel can be generated using an arbitrary pixel value. Here, the arbitrary pixel value can be a pixel value (e.g., a minimum value, maximum value, median value, etc.) belonging to a pixel value range (e.g., a pixel value range based on bit depth or a pixel value range according to pixel distribution within the corresponding image). Specifically, this can be an example applied when all reference pixel candidate blocks are unavailable.

[0232] Alternatively, reference pixels can be generated from areas of the image where decoding / encoding has been completed. Specifically, reference pixels can be generated from at least one available block adjacent to an unusable block. At this time, at least one of the following methods can be used: extrapolation, interpolation, and copying.

[0233] After completing the above-described reference pixel configuration in the reference pixel interpolation unit, a decimal unit of reference pixels can be generated through linear interpolation of the reference pixels. Alternatively, the reference pixel interpolation process can be performed after performing the reference pixel filter process described below.

[0234] At this time, the interpolation process is not performed in the case of horizontal, vertical, some diagonal modes (e.g., modes with a 45 degree difference in vertical / horizontal such as Diagonal up right, Diagonal down right, and Diagonal down left, corresponding to modes 2, 34, and 66 in Fig. 5), non-directional modes, color copy modes, etc., and the interpolation process can be performed in the case of other modes (other diagonal modes).

[0235] The pixel positions where interpolation is performed (i.e., which decimal unit is interpolated, determined from 1 / 2 to 1 / 64, etc.) can be determined depending on the prediction mode (e.g., directionality of the prediction mode, dy / dx, etc.) and the positions of the reference and predicted pixels. At this time, regardless of the precision of the decimal unit, one filter (e.g., assuming the same filter in which the mathematical formula used to determine the filter coefficients or the length of the filter taps, but assuming a filter in which only the coefficients are adjusted according to the precision of the decimal unit <e.g., 1 / 32, 7 / 32, 19 / 32>) can be applied, or one of a plurality of filters (e.g., assuming a filter in which the mathematical formula used to determine the filter coefficients or the length of the filter taps is distinguished) can be selected and applied depending on the decimal unit.

[0236] In the former case, it may be an example of using integer unit pixels as input for interpolation of decimal unit pixels, and in the latter case, it may be an example of changing the input pixels step by step (for example, using integer pixels in the case of 1 / 2 units, using integer and 1 / 2 unit pixels in the case of 1 / 4 units, etc.), but it is not limited thereto, and the present invention will be described mainly with the former case in mind.

[0237] For reference pixel interpolation, fixed filtering or adaptive filtering can be performed, which can be determined by one or more combinations of encoding / decoding settings (e.g., image type, color components, block position / size / shape, block width / height ratio, prediction mode, etc.).

[0238] Fixed filtering can perform reference pixel interpolation using one filter, and adaptive filtering can perform reference pixel interpolation using one of multiple filters.

[0239] At this time, in the case of adaptive filtering, one of the multiple filters may be implicitly determined or explicitly determined depending on the encoding / decoding settings. At this time, the type of filter may be composed of a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, an 8-tap Kalman filter, etc., and it may also be possible for the supported filter candidates to be defined differently depending on the color component (for example, some of the types of filters may be the same or different, and the length of the filter tap may be short or long, etc.).

[0240] In order to improve the accuracy of prediction by reducing the remaining deterioration through the encoding / decoding process in the reference pixel filter unit, filtering can be performed on the reference pixel. The filter used at this time may be, but is not limited to, a low-pass filter. Whether or not filtering is applied can be determined depending on the encoding / decoding settings (which can be derived from the above description). In addition, when filtering is applied, either fixed filtering or adaptive filtering can be applied.

[0241] Static filtering means that reference pixel filtering is not performed or is applied using a single filter. Adaptive filtering means that whether filtering is applied is determined by the encoding / decoding settings, and if more than one filter type is supported, one of them can be selected.

[0242] At this time, multiple filters can be supported, distinguished by various filter coefficients, filter tap lengths, etc., such as a 3-tap filter such as [1, 2, 1] / 4, a 5-tap filter such as [2, 3, 6, 3, 2] / 16, etc.

[0243] The reference pixel interpolation unit and reference pixel filter unit introduced in the above reference pixel configuration step may be necessary components for improving prediction accuracy. The above two processes may be performed independently, but a configuration that combines the two processes (i.e., processing them as a single filter) may also be possible.

[0244] A prediction block generation unit may generate a prediction block according to at least one prediction mode, and may use a reference pixel based on the prediction mode. At this time, the reference pixel may be used for a method such as extrapolation (directional mode) depending on the prediction mode, and may be used for a method such as interpolation or averaging (DC) or copying (non-directional mode).

[0245] The prediction mode decision unit performs a process to select the optimal mode from among multiple prediction mode candidates. Typically, the optimal mode in terms of encoding cost can be determined by using a rate-distortion technique that considers block distortion (e.g., distortion between the current block and the restored block, SAD (sum of absolute difference), SSD (sum of square difference), etc.) and the amount of bits generated according to the corresponding mode. The prediction block generated based on the prediction mode determined through the above process can be transmitted to the subtraction unit and the addition unit.

[0246] To determine the optimal prediction mode, all prediction modes within the candidate prediction mode pool can be explored, or the optimal prediction mode can be selected through another decision process aimed at reducing computational complexity. For example, in the first step, some modes that show good performance in terms of image quality degradation can be selected from all within-screen prediction mode candidates. In the second step, the optimal prediction mode can be selected by considering not only image quality degradation but also the generated bit rate for the modes selected in the first step. In addition to the above method, various methods for reducing computational complexity can be applied.

[0247] Additionally, the prediction mode decision unit may typically be a component included only in the encoder, but may also be included in the decoder, depending on the encoding / decoding settings. For example, if template matching is included as a prediction method, or if a method is included that derives the prediction mode within the frame from adjacent regions of the current block. In the latter case, it can be understood that a method is used to implicitly obtain the prediction mode in the decoder.

[0248] The prediction mode encoding unit can encode the prediction mode selected through the prediction mode determination unit. Index information corresponding to the prediction mode can be encoded from the group of prediction mode candidates, or the prediction mode can be predicted and information about it can be encoded. The former may be a method applied to the luminance component, and the latter may be a method applied to the chrominance component, but is not limited thereto.

[0249] When predicting and encoding a prediction mode, the predicted value (or prediction information) of the prediction mode can be referred to as an MPM (Most Probable Mode). An MPM can be composed of one prediction mode or multiple prediction modes, and the number of MPMs (k, where k is an integer greater than or equal to 1, such as 1, 2, 3, or 6) can be determined depending on the number of prediction mode candidates. When an MPM is composed of multiple prediction modes, it can be referred to as an MPM candidate group.

[0250] MPM candidates can be supported under fixed settings or adaptively configured based on various encoding / decoding factors. For example, the candidate set configuration can be determined based on which reference pixel layer is used among multiple reference pixel layers, or whether intra-picture prediction is performed at the block level or sub-block level. For convenience of explanation, we assume that MPM candidates are configured under a single setting. It should be noted that candidate sets for other intra-picture prediction modes, as well as MPM candidates, can also be adaptive.

[0251] MPM is a concept supported for efficiently encoding prediction modes, and can actually form a group of candidates as prediction modes with a high probability of occurring as the prediction mode of the current block.

[0252] For example, the MPM candidate set can be composed of a preset prediction mode (or a statistically frequently occurring prediction mode, such as DC, Planar, vertical, horizontal, or some diagonal modes), a prediction mode of adjacent blocks (such as left, top, top-left, top-right, and bottom-left blocks), etc. In this case, the prediction mode of adjacent blocks can be obtained from L0 to L3 (left blocks), T0 to T3 (top blocks), TL (top-left blocks), R0 to R3 (top-right blocks), and B0 to B3 (bottom-left blocks) in Fig. 7.

[0253] If it is possible to construct an MPM candidate group from two or more sub-block positions (e.g., L0, L2, etc.) in an adjacent block (e.g., a left block), the prediction mode of the corresponding block can be constructed in the candidate group according to a predefined priority order (e.g., L0 - L1 - L2, etc.). Alternatively, if it is impossible to construct an MPM candidate group from two or more sub-block positions, the prediction mode of the sub-block corresponding to the predefined position (e.g., L0, etc.) can be constructed in the candidate group. Specifically, the prediction modes of positions L3, T3, TL, R0, and B0 among the adjacent blocks can be selected as the prediction modes of the corresponding adjacent blocks and included in the MPM candidate group. The above description is some cases of constructing the prediction modes of the adjacent blocks in the candidate group, and is not limited thereto. In the example described below, it is assumed that the prediction modes of the predefined positions are constructed in the candidate group.

[0254] In addition, if one or more prediction modes are configured as MPM candidates, modes derived from one or more of the included prediction modes can also be additionally configured as MPM candidates. Specifically, if the kth mode (directional mode) is included in the MPM candidate group, modes derivable from that mode (modes with an interval of +a, -b based on k, where a and are integers greater than or equal to 1, such as 1, 2, or 3) can be additionally included in the MPM candidate group.

[0255] There may be a priority order for configuring the MPM candidate set, and the MPM candidate set can be configured in the following order: prediction mode of adjacent blocks, preset prediction mode, and derived prediction mode. The process of configuring the MPM candidate set can be completed by filling the maximum number of MPM candidates according to the priority order. In the above process, a redundancy check process may be included, where if the prediction mode matches an already included prediction mode, the prediction mode is not configured in the candidate set and the order is passed to the candidate with the next highest priority.

[0256] The following assumes that the MPM candidate set consists of six prediction modes.

[0257] For example, the candidate set can be configured in the order of L - T - TL - TR - BL - Planar - DC - Vertical - Horizontal - Diagonal mode. The prediction mode of the adjacent block is configured as a priority in the candidate set, and there may be cases where a preset prediction mode is additionally configured.

[0258] Or, L - T - Planar - DC -<L+1> - <l-1> - <T+1> - <t-1>- The candidate group can be configured in the order of vertical - horizontal - diagonal mode, etc. Some of the prediction modes of some adjacent blocks and some of the preset prediction modes are configured first, and there may be cases where a mode derived under the assumption that a prediction mode in a direction similar to the prediction mode of the adjacent block will occur and some of the preset prediction modes are additionally configured.

[0259] The above examples are some cases regarding the composition of MPM candidates, and are not limited thereto, and various modified examples may be possible.

[0260] MPM candidates can be binarized using methods such as unary binarization and truncated Rice binarization based on the indices within the candidate set. That is, short bits can be assigned to candidates with small indices, and long bits can be assigned to candidates with large indices to represent mode bits.

[0261] Modes not included in the MPM candidate set may be classified as non-MPM candidates. Furthermore, depending on the encoding / decoding settings, non-MPM candidates may be classified into two or more candidate sets.

[0262] The following assumes that there are 67 modes including directional and non-directional modes in the prediction mode candidate set, and that 6 MPM candidates are supported, resulting in 61 prediction modes in the non-MPM candidate set.

[0263] When a non-MPM candidate set consists of a single candidate, no additional candidate set construction is required, as the remaining prediction modes are not included in the MPM candidate set construction process. Therefore, binarization methods such as fixed-length binarization and truncated unary binarization can be used based on the indices within the non-MPM candidate set.

[0264] Assuming that the non-MPM candidate set consists of two or more candidate sets, in this example, the non-MPM candidates are classified into non-MPM_A (candidates A or more) and non-MPM_B (candidates B or more). It is assumed that candidate A (p or more, the number of MPM candidates) is configured as a prediction mode with a higher probability of occurring as the prediction mode of the current block than candidate B (q or more, the number of A candidates). In this case, the process of configuring candidate A may be added.

[0265] For example, some prediction modes with even spacing among directional modes (e.g., modes 2, 4, 6, etc.) can be configured in candidate A, or a preset prediction mode (e.g., a mode derived from a prediction mode included in the MPM candidate group, etc.) can be configured. Through the configuration of the MPM candidate group and the configuration of candidate A, the remaining prediction modes can be configured as candidate B, and no additional candidate group configuration process is required. Binarization such as fixed-length binarization or truncated unary binarization can be used based on the indices in candidate A and candidate B.

[0266] The above examples are some cases where non-MPM candidates are composed of two or more, and are not limited thereto, and various modified examples may be possible.

[0267] The following shows the process for predicting and encoding the prediction mode.

[0268] You can check information (mpm_flag) about whether the prediction mode of the current block matches the MPM (or some mode in the MPM candidate set).

[0269] If there is a match with the MPM, the MPM index information (mpm_idx) can be additionally checked depending on the MPM configuration (one or more). After that, the encoding process of the current block is completed.

[0270] If there is only one non-MPM candidate in the MPM, the non-MPM index information (remaining_idx) can be checked. The encoding process for the current block is then completed.

[0271] If the non-MPM candidate set consists of multiple (two in this example), information (non_mpm_flag) can be checked on whether the prediction mode of the current block matches some prediction mode in the A candidate set.

[0272] If it matches candidate A, you can check candidate A index information (non_mpm_A_idx), and if it does not match candidate A, you can check candidate B index information (remaining_idx). After that, the encoding process of the current block is completed.

[0273] When the configuration of the prediction mode candidate set is fixed, the prediction mode supported in the current block, the prediction mode supported in the adjacent block, and the preset prediction mode can use the same prediction number index.

[0274] Meanwhile, when the prediction mode candidate set is adaptive, the prediction mode supported by the current block, the prediction mode supported by the adjacent block, and the preset prediction mode may use the same prediction number index or different prediction number indices. Refer to FIG. 5 for the following explanation.

[0275] In the above prediction mode encoding process, a prediction mode candidate unification (or adjustment) process may be performed for configuring MPM candidates, etc. For example, the prediction mode of the current block may be one of the prediction mode candidates from modes -5 to 61, and the prediction mode of the adjacent block may be one of the prediction mode candidates from modes 2 to 66. In this case, since some of the prediction modes of the adjacent block (mode 66) may be modes not supported by the prediction mode of the current block, a process for unifying them may be performed in the prediction mode encoding process. That is, this process may not be required when a fixed configuration of the intra-picture prediction mode candidate group is supported, and may be required when an adaptive configuration of the intra-picture prediction mode candidate group is supported, and a detailed description thereof will be omitted.

[0276] Unlike the method using the above MPM, encoding can be performed by assigning an index to a prediction mode belonging to a prediction mode candidate group.

[0277] For example, a method of assigning indices to prediction modes based on a predefined priority and encoding the corresponding indices once the prediction mode of the current block is selected is applicable. This means that a set of prediction mode candidates is fixedly configured and a fixed index is assigned to the prediction mode.

[0278] Alternatively, if the prediction mode candidate set is adaptively configured, the fixed index assignment method described above may not be suitable. To address this, a method may be employed in which indices are assigned to prediction modes according to adaptive priorities, and the corresponding indices are encoded once the prediction mode of the current block is selected. This allows for efficient encoding of prediction modes by varying the indices assigned to the prediction modes due to the adaptive configuration of the prediction mode candidate set. In other words, the adaptive priorities may be intended to assign candidates with a high probability of being selected as the prediction mode of the current block to indices that generate short mode bits.

[0279] The following assumes that eight prediction modes are supported, including the preset prediction modes (directional and non-directional modes), color copy mode, and color mode (chrominance component).

[0280] For example, let's assume that the preset prediction modes support four of Planar, DC, Horizontal, Vertical, and Diagonal modes (Diagonal down left in this example), one color mode (C), and three color copy modes (CP1, CP2, CP3). The default order of the indices assigned to the prediction modes can be given as preset prediction mode - color copy mode - color mode, etc.

[0281] At this time, the preset prediction modes, directional mode, non-directional mode, and color copy mode, can be easily distinguished as prediction modes with distinct prediction methods. However, color mode can be either directional or non-directional, and there is a possibility that it overlaps with the preset prediction modes. For example, if the color mode is vertical mode, there may be a case where it overlaps with vertical mode, which is one of the preset prediction modes.

[0282] When adaptively adjusting the number of prediction mode candidates according to the encoding / decoding settings, if the above-described overlapping case occurs, the number of candidates can be adjusted (from 8 to 7). Alternatively, if the number of prediction mode candidates is maintained fixed, if the above-described overlapping case occurs, other candidates can be added and considered for index allocation, and this setting will be described below. In addition, the adaptive prediction mode candidate set can be a configuration that is supported even when including a variable mode such as a color mode. Therefore, a case where adaptive index allocation is performed can be considered an example of an adaptive prediction mode candidate set configuration.

[0283] The following describes cases where adaptive index allocation is performed depending on the color mode. It is assumed that the basic indices are allocated in the following order: Planar(0) - Vertical(1) - Horizontal(2) - DC(3) - CP1(4) - CP2(5) - CP3(6) - C(7). It is also assumed that index allocation is performed in the above order when the color mode does not match the preset prediction mode.

[0284] For example, if the color mode matches one of the preset prediction modes (Planar, Vertical, Horizontal, DC mode), the index (7) of the color mode is filled with the matching prediction mode. The index (one of 0 to 3) of the matching prediction mode is filled with the preset prediction mode (Diagoanl down left). Specifically, if the color mode is Horizontal mode, the index assignment can be performed as Planar(0) - Vertical(1) - Diagoanal down left(2) - DC(3) - CP1(4) - CP2(5) - CP3(6) - Horizontal(7).

[0285] Alternatively, if the color mode matches one of the preset prediction modes, the matching prediction mode is filled in at index 0. Then, the preset prediction mode (Diagoanal down left) is filled in at the index (7) of the color mode. At this time, if the filled prediction mode is not the existing index 0 (i.e., not the Planar mode), the existing index configuration may be adjusted. Specifically, if the color mode is the DC mode, the index assignment may be performed as follows: DC (0) - Planar (1) - Vertical (2) - Horizontal (3) - CP1 (4) - CP2 (5) - CP3 (6) - Diagonal down left (7).

[0286] The above examples illustrate some cases of adaptive index allocation, and various variations are possible. Furthermore, binarization methods such as fixed-length binarization, unary binarization, truncated unary binarization, and truncated Rice binarization may be used based on the indices within the candidate set.

[0287] The following is another example of performing encoding by assigning indices to prediction modes belonging to the prediction mode candidate set.

[0288] For example, a method of classifying prediction mode candidates into multiple groups by dividing them into prediction modes, prediction methods, etc., and encoding them by assigning indices to prediction modes belonging to the candidate groups is applicable. In this case, encoding candidate group selection information may precede the index encoding. For example, directional mode, non-directional mode, and color mode, which are prediction modes that perform prediction in the same color space, may belong to one candidate group (above S candidate group), and color copy mode, which is a prediction mode that performs prediction in a different color space, may belong to one candidate group (above D candidate group).

[0289] The following assumes that nine prediction modes are supported (chrominance components), including the preset prediction mode, color copy mode, and color mode in the prediction mode candidate set.

[0290] For example, let's assume that the preset prediction modes support four of the Planar, DC, Horizontal, Vertical, and Diagonal modes, and one color mode (C) and four color copy modes (CP1, CP2, CP3, CP4). The S candidate group can have five candidates composed of the preset prediction modes and color modes, and the D candidate group can have four candidates composed of color copy modes.

[0291] The S candidate group is an example of an adaptively configured prediction mode candidate group, and an example of adaptive index allocation has been described above, so a detailed description is omitted. The D candidate group is an example of a fixedly configured prediction mode candidate group, and thus a fixed index allocation method can be used. For example, an index allocation such as CP1(0) - CP2(1) - CP3(2) - CP4(3) can be performed.

[0292] Binarizations such as fixed-length binarization, unary binarization, truncated unary binarization, and truncated Rice binarization can be used based on the indices within the above candidate set. Furthermore, various modified examples are possible, not limited to the above examples.

[0293] The candidate set configuration, such as MPM for predictive mode encoding, can be performed on a block-by-block basis. Alternatively, the candidate set configuration process can be omitted, and a predetermined candidate set or a candidate set obtained through various methods can be used. This configuration can be supported for purposes such as complexity reduction.

[0294] For example (1), one predefined candidate group can be used, or one of multiple predefined candidates can be used depending on the encoding / decoding settings. For example, in the case of MPM candidates, predefined candidates such as {Planar - DC - Vertical - Horizontal - Diagonal down left <No. 66 in FIG. 5> - Diagonal down right <No. 34 in FIG. 5>} can be used. Alternatively, it may be possible to apply to this example a candidate group that is formed when all adjacent blocks are unusable during the MPM candidate group formation.

[0295] As an example (2), a candidate group of blocks for which encoding is complete can be used. At this time, the blocks for which encoding is complete can be selected based on an encoding order (a predetermined scan method, for example, z-scan, vertical scan, horizontal scan, etc.) or can be selected from adjacent blocks such as the left, top, top-left, top-right, and bottom-left of the current block. However, the adjacent blocks can be limited to positions that belong to a partition unit that can refer to each other with the current block (for example, when each block has a referential property even if the slice or tile to which it belongs is different. For example, when it belongs to the same tile group but is a different tile), and when it belongs to a partition unit that cannot be referred to (for example, when each block has a different slice or tile to which it belongs but has a non-referential property. For example, when it belongs to a different tile group), the block at that position can be excluded from the candidates.

[0296] Here, adjacent blocks can be determined based on the state of the current block. For example, if the current block is square, a group of available block candidates from among the blocks located according to a predetermined first priority can be borrowed (or shared). Alternatively, if the current block is rectangular, a group of available block candidates from among the blocks located according to a second priority can be borrowed. At this time, the second or third priority can be supported depending on the length / width ratio of the block. The priority for selecting a candidate block to be borrowed can have various configurations, such as left - top - top right - bottom left - top left, or top - left - top left - top right - bottom left. At this time, the first to third priorities may all have the same configuration, all have different configurations, or some configurations may have the same configuration.

[0297] Borrowing from adjacent blocks may be performed only when the candidate group of the current block is greater than / exceeds a predetermined threshold value. Alternatively, borrowing from adjacent blocks may be performed only when the candidate group is less than / exceeds a predetermined threshold value. Here, the threshold value may be defined as the minimum or maximum size of a block that allows borrowing of the candidate group. The threshold value may be expressed as the horizontal length (W) or vertical length (H) of the block, such as W x H, W * H, etc., and W and H may be integers of 4, 8, 16, 32, or higher.

[0298] For example (3), a common candidate set can be formed in an upper block composed of a set of blocks, and lower blocks within the upper block can use the candidate set. In this case, the number of lower blocks can be an integer greater than or equal to 1, such as 1, 2, 3, or 4.

[0299] Here, the upper block may be an ancestor block of the lower block (including the parent block) or a block composed of any arbitrary group. In this case, the ancestor block may refer to a block before division in the previous step (where the difference in division depth is 1 or more) during the division process for obtaining the lower block. For example, the parent block of the 0th and 1st sub-blocks of 4N x 2N in b of Fig. 4 may refer to 4N x 4N in a of Fig. 4.

[0300] Borrowing (or sharing) from a lower block may be performed only when the candidate for the upper block is greater than or equal to a predetermined first boundary value. Alternatively, borrowing from a lower block may be performed only when the candidate for the upper block is less than or equal to a predetermined second boundary value.

[0301] Here, the above boundary value can be defined as the minimum or maximum size of the block for which candidate borrowing is allowed. Only one of the above boundary values ​​or both can be supported, and the boundary value can be expressed as the width (W) of the block, the height (H), W x H, W * H, etc., and the W and H can be integers of 8, 16, 32, 64, or higher.

[0302] On the other hand, borrowing from the upper block can be performed only when the candidate of the lower block is greater than / exceeds a predetermined third boundary value. Alternatively, borrowing from the upper block can be performed only when the candidate is less than / exceeds a predetermined fourth boundary value.

[0303] Here, the above boundary value can be defined as the minimum or maximum size of a block that allows borrowing of a candidate group. Only one of the above boundary values ​​or both can be supported, and the boundary value can be expressed as the width (W) of the block, the height (H), W x H, W * H, etc., and the W and H can be integers of 4, 8, 16, 32, or higher.

[0304] At this time, the first boundary value (or second boundary value) may be greater than or equal to the third boundary value (or fourth boundary value).

[0305] Candidate borrowing (or sharing) may be selectively utilized based on any one of the aforementioned embodiments, and may also be selectively utilized based on a combination of at least two of Embodiments 1 to 3. Furthermore, candidate borrowing may also be selectively utilized based on any one of the detailed configurations of each embodiment, and may also be selectively utilized based on a combination of one or more detailed configurations, etc.

[0306] Additionally, information regarding whether candidate groups are borrowed, the attributes of the blocks involved in candidate group borrowing (size / shape / location / length ratio), and the segmentation status (segmentation method, segmentation type, segmentation depth, etc.) can be explicitly processed. Furthermore, encoding elements such as image type and color components can act as input variables in candidate group borrowing settings. Candidate group borrowing can be performed based on the above information and encoding / decoding settings.

[0307] Prediction-related information generated through the prediction mode encoding unit can be transmitted to the encoding unit and included in the bitstream.

[0308]

[0309] In an image decoding method according to an embodiment of the present invention, intra-screen prediction may be configured as follows. The intra-screen prediction of the prediction unit may include a prediction mode decoding step, a reference pixel configuration step, and a prediction block generation step. In addition, the image decoding device may be configured to include a prediction mode decoding unit, a reference pixel configuration unit, and a prediction block generation unit that implement the prediction mode decoding step, the reference pixel configuration step, and the prediction block generation step. Some of the above-described processes may be omitted, other processes may be added, and the order may be changed to a different order than the order described above.

[0310] The reference pixel configuration unit and prediction block generation unit of the image decoding device perform the same role as the corresponding configuration of the image encoding device, so a detailed description is omitted, and the prediction mode decoding unit can be performed using the reverse method used in the prediction mode encoding unit.

[0311]

[0312] (Sub-block unit on-screen prediction)

[0313] Figure 8 illustrates various sub-block division forms obtainable based on an encoding block. Here, the encoding block is referred to as a parent block, and the sub-block may be a child block. Here, the sub-block may be a unit where prediction is performed or a unit where transformation is performed. This example assumes that a single prediction information is shared across sub-blocks. This means that a single prediction mode is generated and used by each sub-block.

[0314] Referring to FIG. 8, the encoding order of the sub-blocks can be determined according to various order combinations of a to p of FIG. 8. For example, one of the following orders can be followed: z-scan (left -> right, top -> bottom), vertical scan (top -> bottom), horizontal scan (left -> right), inverse vertical scan (bottom -> top), inverse horizontal scan (right -> left).

[0315] The above encoding order may be a pre-promised order in the video encoder / decoder. Alternatively, the encoding order of the sub-block may be determined by considering the division direction of the parent block. For example, if the parent block is divided horizontally, the encoding order of the sub-block may be determined by vertical scan. If the parent block is divided vertically, the encoding order of the sub-block may be determined by horizontal scan.

[0316] When encoding is performed on a sub-block basis, the reference data used for prediction can be obtained from a closer location, and it can be efficient because only one prediction mode is generated and shared across sub-blocks.

[0317] For example, referring to (b) of Fig. 8, when encoding is performed in parent block units, the lower right sub-block can perform prediction using adjacent pixels of the parent block. On the other hand, when encoding is performed in sub-block units, the lower right sub-block can perform prediction using pixels that are more adjacent than the parent block because there are upper left, upper right, and lower left sub-blocks that are restored according to a predetermined encoding order (z-scan in this example).

[0318] For intra-screen prediction based on optimal segmentation considering image characteristics, a candidate group can be formed based on one or more segmentation forms with a high probability of occurrence.

[0319] This example assumes that segmentation information is generated using an index-based segmentation method. The candidate groups can be organized into various segmentation types, as follows:

[0320] Specifically, a candidate group consisting of N partition shapes can be formed, where N can be an integer greater than or equal to 2. The candidate group can include a combination of at least two of the seven partition shapes illustrated in FIG. 8.

[0321] A parent block can be split into predetermined sub-blocks by selectively using any one of a plurality of split shapes belonging to the candidate group. The selection can be performed based on an index signaled by an image encoding device. The index can mean information specifying a split shape of the parent block. Alternatively, the selection can be performed by an image decoding device in consideration of the attributes of the parent block. Here, the attributes can mean the position, size, shape, area, width / height ratio, length of any one of width / height, split depth, image type (I / P / B), color component (e.g., luminance, chrominance), value of an intra-picture prediction mode, whether the intra-picture prediction mode is a non-directional mode, angle of the intra-picture prediction mode, position of a reference pixel, etc. The block can mean an encoding block or a prediction block and / or a transform block corresponding to the encoding block. The position of the block can mean whether it is located at the boundary of a predetermined image (or fragment image) of the parent block. Here, the image (or fragment image) may mean at least one of a picture, slice group, tile group, slice, tile, CTU row, or CTU to which the parent block belongs.

[0322] For example (1), candidate groups such as {a ~ d}, {a ~ g} in Fig. 8 can be configured, which can be candidate group configurations that take into account various partitioning forms. Assuming {a ~ d}, various binary bits can be assigned to each index (assuming that the indexes are assigned in alphabetical order. The same assumption applies to the following example).

[0323] idxbin type 1bin type 2000010110210110311111

[0324] In the above Table 1, bin type 1 may be an example of binarization that considers all possible division forms, and bin type 2 may be an example of binarization in which a bit indicating whether or not division occurs (the first bit) is first allocated, and in the case of division (the first bit is 1), only candidates that are not divided are excluded from the possible division forms.

[0325] As an example (2), candidate groups such as {a, c, d}, {a, f, g} of Fig. 8 can be configured, which can be candidate group configurations that take into account division in a specific direction (horizontal or vertical). Assuming the case of {a, f, g}, various binary bits can be assigned to each index.

[0326] idxbin type 1bin type 2bin type 3001010110011211110

[0327] Table 2 above is an example of binarization allocated based on the properties of a block, and may be an example of a case where the shape of the block is taken into account. In bin type 1, an example may be that 1 bit is allocated when the parent block is square and not split, and 2 bits are allocated when split in a horizontal or vertical direction.

[0328] For bin type 2, an example could be that when the parent block is rectangular in shape and long horizontally, 1 bit is allocated if it is split horizontally, and 2 bits are allocated otherwise. For bin type 3, an example could be that when the parent block is rectangular in shape and long vertically, 1 bit is allocated if it is split vertically, and 2 bits are allocated otherwise. This could be an example of allocating shorter bits if it is determined that a split similar to the shape of the parent block is more likely to occur, but is not limited thereto and may include examples of variations including the opposite case.

[0329] As an example (3), a candidate group such as {a, c, d, f, g} of Fig. 8 can be formed, which may be another example of a candidate group formation that takes into account division in a specific direction.

[0330] idxbin type 1bin type 20001100100210111031101014111111

[0331] In the above Table 3, bin type 1 is first assigned a flag (the first bit) indicating whether to split, followed by a flag distinguishing the number of splits. Here, if the flag distinguishing the number of splits (the second bit) is 0, the number of splits may be 2, and if it is 1, the number of splits may be 4. In addition, the subsequent flag may be a flag indicating the split direction, and if it is 0, it may be a horizontal split, and if it is 1, it may be a vertical split. The above description is only one example and is not limited thereto, and various modifications, including opposite configurations, are possible.

[0332] The above segmentation information may be a configuration supported under general circumstances, but may be transformed into other settings depending on the encoding / decoding environment, etc. In other words, it may be possible to support exceptional configurations regarding the segmentation information, or to replace the segmentation form expressed by the segmentation information with another segmentation form.

[0333] For example, there may be unsupported segmentation patterns depending on the block's properties. Since the block's properties have been discussed in the previous example, a detailed description thereof will be omitted. Here, a block may refer to at least one of a parent block or a sub-block.

[0334] For example, let's assume that the supported segmentation shapes are {a, f, g} in Fig. 8 and that the size of the parent block is 4M x 4N. If some segmentation shapes are not supported due to the minimum value condition of the block in the image or the location on the boundary of a certain image (or fragment image) of the parent block, various processing may be possible. For the following example, let's assume that the minimum value of the horizontal length of a block in the image is 2M and the minimum value of the block area is 4*M*N.

[0335] First, the candidate set can be reconstructed by excluding the unattainable partition shape. The possible candidates in the existing candidate set are 4M x 4N, 4M x N, and M x 4N. The reconstructed candidate set, excluding the unattainable candidate (4M x N), can be 4M x 4N, 4M x N. In this case, binarization can be performed again on the candidates in the reconstructed candidate set. In this example, 4M x 4N, 4M x N can be selected with a single flag (1 bit).

[0336] As another example, the candidate set can be reconstructed by replacing an unattainable partition shape. In the above example, the unattainable partition shape may be a vertical partition shape (e.g., a 4-partition). However, the candidate set can be reconstructed by replacing it with another partition shape (e.g., a 2M x 4N) that maintains the vertical partition shape. This can maintain the flag configuration based on the existing partition information.

[0337] As in the above example, the candidate set can be reconstructed by adjusting the number of candidates or replacing existing candidates. The above example only includes descriptions of some cases, and various modified examples may be possible.

[0338]

[0339] (Encoding order of sub-blocks)

[0340] It may be possible to set the encoding order of various sub-blocks, such as in Fig. 8. The encoding order may be implicitly determined according to the sub / decoding settings. At this time, the segmentation shape, image type, color components, size / shape / position of the parent block, block width / height ratio, prediction mode-related information (e.g., intra-screen prediction mode, used reference pixel positions, etc.), segmentation status, etc. may be included in the sub / decoding elements.

[0341] Alternatively, explicit handling of the encoding order of sub-blocks may be possible. Specifically, a candidate set with a high probability of occurrence can be constructed based on the partitioning pattern, and selection information for one of the candidates can be generated. Thus, the supported encoding order candidates can be adaptively constructed based on the partitioning pattern.

[0342] Although encoding of sub-blocks can be performed using a single fixed encoding order as in the above example, a method of applying an adaptive encoding order may also be possible.

[0343] Referring to Fig. 8, many encoding orders can be obtained according to the various order assignments of a to p described in the drawing. Since the positions and numbers of sub-blocks obtained may vary depending on each partitioning type, the configuration of an encoding order specialized for the partitioning type may be important. In addition, a partitioning type such as Fig. 8 (a) in which partitioning into sub-blocks is not performed does not require processing of the encoding order. Therefore, when explicitly processing information about the encoding order, the partitioning information can be confirmed first, and then information about the encoding order of the sub-blocks can be generated based on the selected partitioning information.

[0344] Referring to (c) of Fig. 8, a vertical scan in which 0 and 1 are applied to a and b, and an inverse scan in which 1 and 0 are applied to a and b, respectively, can be supported as candidates, and a 1-bit flag for selecting one of them can be generated.

[0345] Alternatively, referring to (b) of FIG. 8, a z-scan with 0 to 3 applied to a to d, a z-scan rotated 90 degrees to the left with 1, 3, 0, 2 applied respectively, an inverse z-scan with 3, 2, 1, 0 applied respectively, a z-scan rotated 90 degrees to the right with 2, 0, 3, 1 applied respectively may be supported, and a flag of one or more bits may be generated to select one of these.

[0346] The following describes a case where the encoding order of sub-blocks is implicitly determined. For convenience of explanation, the encoding order is shown for the case where the segmentation forms (c) {or (f)} and (d) {or (g)} of Fig. 8 are obtained (refer to Fig. 5 for the prediction mode).

[0347] For example, when the on-screen prediction mode is vertical mode, horizontal mode, diagonal down right mode (modes 19 to 49), diagonal down left mode (modes 51 and above), or diagonal up right mode (modes 17 and below), vertical scan and horizontal scan can be determined.

[0348] Alternatively, when the on-screen prediction mode is a diagonal down left mode (51 or higher), vertical scan or inverse horizontal scan can be determined.

[0349] Alternatively, when the on-screen prediction mode is a Diagonal up right direction mode (17 or lower), inverse vertical scan and horizontal scan can be determined.

[0350] The above example may be an encoding order based on a predetermined scan order. In this case, the predetermined scan order may be one of z-scan, vertical scan, or horizontal scan. Alternatively, the scan order may be determined based on the location and distance of pixels referenced during prediction within the screen. For this purpose, an inverse scan may be additionally considered.

[0351]

[0352] FIG. 9 is an exemplary diagram of a reference pixel area used based on an on-screen prediction mode according to one embodiment of the present invention. Referring to FIG. 9, it can be seen that the area referenced according to the direction of the prediction mode is shaded.

[0353] Figure 9 (a) shows an example in which adjacent areas of a parent block are divided into left, upper, upper left, upper right, and lower left areas. Figure 9 (b) shows the left and lower left areas referenced in the Diagonal up right direction mode, Figure 9 (c) shows the left area referenced in the horizontal mode, Figure 9 (d) shows the left, upper, and upper left areas referenced in the Diagonal down right direction mode, Figure 9 (e) shows the upper area referenced in the vertical mode, and Figure 9 (f) shows the upper and upper right areas referenced in the Diagonal down left direction mode.

[0354] When performing prediction using adjacent reference pixels (or reconstructed pixels of sub-blocks), pre-defining the encoding order of sub-blocks can have the advantage of eliminating the need to separately signal related information. Various segmentation patterns can be adopted, and thus, based on the referenced region (or prediction mode), examples can be provided, such as the following:

[0355] FIG. 10 illustrates an example of an encoding order that may be available in a diagonal up-right direction prediction mode according to one embodiment of the present invention. Examples of assigning a higher priority to a sub-block adjacent to the lower left can be seen through (a) to (g) of FIG. 10.

[0356] FIG. 11 illustrates an example of an encoding order that may be present in horizontal mode according to one embodiment of the present invention. An example in which a sub-block adjacent to the left is assigned a higher priority can be seen through FIG. 11 (a) to (g).

[0357] Figure 12 illustrates an example of an encoding order that may be present in a diagonal down-right prediction mode according to one embodiment of the present invention. Examples of assigning a higher priority to a sub-block adjacent to the upper left can be seen in Figures 12 (a) to (g).

[0358] FIG. 13 illustrates an example of an encoding order that may be present in vertical mode according to one embodiment of the present invention. Examples of assigning a higher priority to a sub-block adjacent to the top can be seen through FIG. 13 (a) to (g).

[0359] Figure 14 illustrates an example of an encoding order that may be present in a diagonal down-left direction mode according to one embodiment of the present invention. An example in which a sub-block adjacent to the upper right direction is assigned a higher priority can be seen through Figures 14 (a) to (g).

[0360] The above example is one example of defining the encoding order from adjacent sub- / decoding regions. Other variations are possible. Furthermore, various configurations are possible in which the encoding order is defined based on other sub- / decoding elements.

[0361]

[0362] Figure 15 is an exemplary diagram illustrating an encoding order considering an intra-screen prediction mode and a partitioning type according to one embodiment of the present invention. Specifically, the encoding order of sub-blocks can be implicitly determined for each partitioning type depending on the intra-screen prediction mode. Furthermore, an example of reconstructing a candidate group by replacing an unobtainable partitioning type with another partitioning type is described. For convenience of explanation, it is assumed that the parent block is 4M x 4N.

[0363] Referring to (a) of Fig. 15, it can be divided into a 4M x N shape for intra-screen prediction in sub-block units. In this example, the inverse vertical scan order can be followed. If the 4M x N shape of the division cannot be obtained, the division into a 2M x 2N shape according to a predetermined priority can be performed. The encoding order for this case is illustrated in the drawing, so a detailed description is omitted. If the 2M x 2N shape cannot be obtained, the division into a 4M x 2N shape of the next priority can be performed. The division shapes can be supported according to such predetermined priorities, and the division in sub-block units can be performed based on this. If all the predefined division shapes cannot be obtained, it can be seen that the division into sub-blocks of the corresponding parent block is impossible, and encoding is performed for the parent block.

[0364] Referring to (b) of Fig. 15, it can be divided into M x 4N shapes for intra-screen prediction in sub-block units. As shown in (a) of Fig. 15, it is assumed that the encoding order is predetermined based on each division shape. In this example, it can be understood as a case where it is replaced according to the order of 2M x 2N, 2M x 4N.

[0365] Referring to (c) of Fig. 15, it can be divided into 2M x 2N shapes for intra-screen prediction in sub-block units. In this example, it can be understood as a case where it is replaced in the order of 4M x 2N, 2M x 4N.

[0366] Referring to (d) of Fig. 15, it can be divided into a 4M x N shape for intra-screen prediction in sub-block units. In this example, it can be understood as a case where it is replaced in the order of 2M x 2N, 4M x 2N.

[0367] Referring to (e) of Fig. 15, it can be divided into M x 4N shapes for intra-screen prediction in sub-block units. In this example, it can be understood as a case where it is replaced in the order of 2M x 2N, 2M x 4N.

[0368] The above example is one example of supporting division into other shapes according to a given order when an unobtainable division shape exists, and various examples of variations may be possible.

[0369] For example, if the partition shapes of {4M x 4N, M x 4N, 4M x N} are supported, it may be possible that the M x 4N or 4M x N shape in which the partition is performed is replaced with the 2M x 4N or 4M x 2N shape when it is unobtainable.

[0370] As in the above example, the sub-block division settings can be determined according to various sub- / decoding elements, and since the sub- / decoding elements can be derived from the description of the sub-block division already described above, a detailed description is omitted.

[0371] Additionally, if the sub-block division is determined, prediction and transformation can be performed as is. As previously described, the intra-screen prediction mode is determined for each parent block, and prediction can be performed accordingly.

[0372] Additionally, transformation and inverse transformation may be performed based on the relevant settings determined for the parent block, and transformation and inverse transformation may be performed for each sub-block based on the settings determined for the parent block. Alternatively, relevant settings may be determined based on each sub-block, and transformation and inverse transformation may be performed for each sub-block based on the relevant settings. Transformation and inverse transformation may be performed based on one of the above settings.

[0373]

[0374] The methods according to the present invention may be implemented in the form of program instructions that can be executed by various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either singly or in combination. The program instructions recorded on the computer-readable medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the computer software art.

[0375] Examples of computer-readable media may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions may include not only machine language codes generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc. The hardware devices described above may be configured to operate with at least one software module to perform the operations of the present invention, and vice versa.

[0376] Additionally, the above-described method or device may be implemented by combining all or part of its configuration or function, or may be implemented separately.

[0377] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

[0378] The present invention can be used to encode / decode images. < / l-1> < / array> < / qt>

Claims

1. A step of deriving an on-screen prediction mode of the current block; A step of checking whether the current block is divided and performing on-screen prediction in sub-block units; A step of determining the encoding order of the sub-block based on the prediction mode within the screen; and An intra-screen prediction method for generating a prediction block based on the above sub-block encoding order and the intra-screen prediction mode.

Images