Method and apparatus for encoding / decoding image

Abstract

An image decoding method performed by an image decoding apparatus includes: determining a reference pixel line of a current block from a reference pixel line candidate set; determining an intra-frame prediction mode of the current block from an intra-frame prediction mode candidate set; and predicting the current block based on the intra prediction mode of the current block and the reference pixel line.

Description

[0001] This application is a divisional application of an invention patent application with an application date of June 25, 2019, application number 201980042714.X, and invention name “Method and device for encoding / decoding images”. Technical Field

[0002] The present disclosure relates to a technique for encoding and decoding an image, and more particularly, to a method and apparatus for performing encoding / decoding in intra prediction. Background Art

[0003] With the widespread use of the Internet and portable terminals and the development of information and communication technology, multimedia data is increasingly used. Therefore, in order to provide various services or perform various tasks through image prediction in various systems, it is urgent to improve the performance and efficiency of image processing systems. However, research and development results have not kept up with this trend.

[0004] Therefore, existing methods and apparatuses for encoding / decoding images need to improve performance in image processing, particularly in image encoding or image decoding. Summary of the invention

[0005] [Technical issues]

[0006] The purpose of the present invention is to provide an intra-frame prediction method.

[0007] In addition, the present invention provides a method and apparatus for constructing reference pixels for intra prediction.

[0008] Another object of the present invention is to provide a method and apparatus for performing reference pixel filtering and interpolation.

[0009] In addition, the present invention provides a method and apparatus for constructing an intra-frame prediction mode candidate group.

[0010] In addition, the present invention provides a method and apparatus for intra-frame prediction in sub-block units.

[0011] [Technical solution]

[0012] An image encoding / decoding method and apparatus according to the present invention constructs an intra-frame prediction mode candidate group of a target block, determines reference pixels for the intra-frame prediction mode of the target block, performs at least one of filtering or interpolation on the reference pixels, and performs intra-frame prediction of the target block based on the intra-frame prediction mode candidate group and the reference pixels.

[0013] In the image encoding / decoding method and apparatus according to the present invention, filtering can be selectively performed based on at least one of the following: size, shape, position, color component, reference pixel line, whether intra-frame prediction based on sub-block units is applied, or the intra-frame prediction mode of the target block.

[0014] In the image encoding / decoding method and apparatus according to the present invention, when the intra prediction mode of the target block is the diagonal mode, filtering may be performed, otherwise, when the intra prediction mode of the target block is not the diagonal mode, filtering may not be performed.

[0015] In the image encoding / decoding method and apparatus according to the present invention, the diagonal line mode may include at least one of a first mode referring to only pixels of integer units or a second mode referring to pixels of fractional units.

[0016] In the image encoding / decoding method and apparatus according to the present invention, when the intra-frame prediction mode of the target block is a first mode that refers only to pixels of integer units, interpolation may not be performed on the reference pixels, and when the intra-frame prediction mode of the target block is a second mode that refers to pixels of fractional units, interpolation may be performed on the reference pixels.

[0017] In the image encoding / decoding method and apparatus according to the present invention, the reference pixel may be located on at least one of a plurality of reference pixel lines adjacent to the target block.

[0018] In the image encoding / decoding method and apparatus according to the present invention, the intra prediction mode candidate group may include at least one priority allocation mode or a predetermined Most Probable Mode (MPM).

[0019] In the image encoding / decoding method and apparatus according to the present invention, the priority allocation mode may include at least one of a planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0020] In the image encoding / decoding method and apparatus according to the present invention, intra prediction may be performed in units of subblocks constituting a target block.

[0021] An image decoding method performed by an image decoding device according to the present invention includes: determining a reference pixel line of a current block from a reference pixel line candidate set; determining an intra-frame prediction mode of the current block from an intra-frame prediction mode candidate set; and predicting the current block based on the intra-frame prediction mode of the current block and the reference pixel line.

[0022] An image encoding method performed by an image encoding device according to the present invention includes: determining a reference pixel line of a current block from a reference pixel line candidate set; determining an intra-frame prediction mode of the current block from an intra-frame prediction mode candidate set; and predicting the current block based on the intra-frame prediction mode of the current block and the reference pixel line.

[0023] A bit stream sending method according to the present invention comprises: generating a bit stream to be decoded by an image decoding method; and sending the bit stream, wherein the image decoding method comprises: determining a reference pixel line of a current block from a reference pixel line candidate set; determining an intra-frame prediction mode of the current block from an intra-frame prediction mode candidate set; and predicting the current block based on the intra-frame prediction mode of the current block and the reference pixel line.

[0024] [Beneficial Effects]

[0025] In case of using the intra prediction method according to the present invention as described above, the intra prediction mode candidate group can be effectively constructed to cause a reduction in bits representing the prediction mode of the target block, thereby improving encoding performance.

[0026] According to the present invention, encoding performance can be improved by configuring filtered or interpolated reference pixels.

[0027] According to the present invention, encoding performance can be improved by intra-frame prediction in sub-block units. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a conceptual diagram illustrating an image encoding and decoding system according to an embodiment of the present disclosure.

[0029] Figure 2 : is a block diagram showing an image encoding device according to an embodiment of the present disclosure.

[0030] Figure 3 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0031] Figure 4 is an exemplary diagram showing various types of divisions that can be obtained in the block segmentation unit of the present disclosure.

[0032] Figure 5 An example of a family tree illustrating the members of a family and the genetic characteristics of people related to the family is shown.

[0033] Figure 6 Examples of various arrangements of related blocks having a horizontal relationship with a target block are shown.

[0034] Figure 7Examples of various arrangements of related blocks having a perpendicular relationship to the target block are shown.

[0035] Figure 8 Various arrangement examples of related blocks having a vertical relationship and a horizontal relationship with the target block are shown.

[0036] Fig. 9 is an example graph of the block partitioning obtained according to the tree type.

[0037] Fig.10 An example diagram of segmentation obtained based on QT, BT, and TT.

[0038] Fig.11 is an example graph for examining the correlation between blocks based on the segmentation method and segmentation settings.

[0039] Fig.12 is a diagram showing examples of intra prediction modes supported by an image encoding / decoding apparatus.

[0040] Fig.13 is an exemplary diagram showing a configuration of reference pixels used for intra prediction according to an embodiment of the present disclosure.

[0041] Fig.14 is a diagram showing an example of a configuration of reference pixels that can be obtained according to a basic block.

[0042] Fig.15 is a conceptual diagram showing a target block for intra prediction and blocks adjacent to the target block. DETAILED DESCRIPTION

[0043] According to the image encoding / decoding method and apparatus of the present invention, an intra-frame prediction mode candidate group of a target block is configured, reference pixels of the intra-frame prediction mode for the target block are determined, at least one of filtering or interpolation of the reference pixels is performed, and intra-frame prediction of the target block is performed based on the intra-frame prediction mode candidate group and the reference pixels.

[0044] In the image encoding / decoding method and apparatus according to the present invention, filtering can be selectively performed based on at least one of the following: size, shape, position, color component, reference pixel line, whether intra-frame prediction based on sub-block units is applied, or the intra-frame prediction mode of the target block.

[0045] In the image encoding / decoding method and apparatus according to the present invention, when the intra prediction mode of the target block is the diagonal mode, filtering may be performed, otherwise, when the intra prediction mode of the target block is not the diagonal mode, filtering may not be performed.

[0046] In the image encoding / decoding method and apparatus according to the present invention, the diagonal line mode may include at least one of a first mode referring to only pixels of integer units or a second mode referring to pixels of fractional units.

[0047] In the image encoding / decoding method and apparatus according to the present invention, when the intra-frame prediction mode of the target block is a first mode that refers only to pixels of integer units, interpolation may not be performed on the reference pixels, and when the intra-frame prediction mode of the target block is a second mode that refers to pixels of fractional units, interpolation may be performed on the reference pixels.

[0048] In the image encoding / decoding method and apparatus according to the present invention, the reference pixel may be located on at least one of a plurality of reference pixel lines adjacent to the target block.

[0049] In the image encoding / decoding method and apparatus according to the present invention, the intra prediction mode candidate group may include at least one priority allocation mode or a predetermined Most Probable Mode (MPM).

[0050] In the image encoding / decoding method and apparatus according to the present invention, the priority allocation mode may include at least one of a planar mode, a DC mode, a vertical mode, or a horizontal mode.

[0051] In the image encoding / decoding method and apparatus according to the present invention, intra prediction may be performed in units of subblocks constituting a target block.

[0052] Mode for the Invention

[0053] The present disclosure may be modified in various ways and have various embodiments. A specific embodiment of the present disclosure will be described with reference to the accompanying drawings. However, the embodiment is not intended to limit the technical scope of the present disclosure, and it should be understood that the present disclosure encompasses various modifications, equivalents, and alternatives within the scope and concept of the present disclosure.

[0054] The terms first, second, A and B used in this disclosure may be used to describe various components, rather than to limit components. These expressions are only used to distinguish one component from another component. For example, without departing from the scope of this disclosure, a first component may be referred to as a second component, and a second component may be referred to as a first component. The term "and / or" covers a combination of multiple related items or any one of multiple related items.

[0055] When it is mentioned that a component is “connected to” another component or “coupled to / coupled to another component”, it should be understood that the component is directly connected to the other component or connected to the other component through any other component. On the other hand, when it is mentioned that a component is “directly connected to” or “directly coupled to” another component, it should be understood that there are no other components between the components.

[0056] The terms used in the present disclosure are provided only for describing specific embodiments and are not intended to limit the present disclosure. Unless the context clearly dictates otherwise, the singular includes plural references. In the present disclosure, the term "including" or "having" indicates the presence of a feature, number, step, operation, component, part, or a combination thereof, but does not exclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or a combination thereof.

[0057] Unless otherwise defined, the terms used in this disclosure, including technical or scientific terms, may have the same meanings as those generally understood by those skilled in the art. Terms generally defined in dictionaries may be interpreted as having the same or similar meanings as the contextual meanings of the relevant technology. Unless otherwise defined, these terms should not be interpreted as ideal or excessive forms of meaning.

[0058] Typically, an image may include one or more color spaces depending on its color format. The image may include one or more pictures of the same size or one or more pictures of different sizes. For example, the YCbCr color configuration may support color formats such as 4:4:4, 4:2:2, 4:2:0, and monochrome (consisting only of Y). For example, YCbCr 4:2:0 may consist of a luminance component (Y in this example) and two chrominance components (Cb and Cr in this example). In this case, the configuration ratio of the chrominance component and the luminance component may have a width-height of 1:2. For example, in the case of 4:4:4, it may have the same configuration ratio in terms of width and height. As in the above example, when a picture includes one or more color spaces, the picture may be segmented into color spaces.

[0059] Images may be classified into I, P, and B according to their image types (e.g., pictures, slices, tiles, etc.). An I picture may be an image that is encoded / decoded without a reference picture. A P picture may be an image that is encoded / decoded using a reference picture but only allows forward prediction. A B picture may be an image that is encoded / decoded using a reference picture and allows bidirectional prediction. However, depending on the encoding / decoding configuration, some types (P and B) may be combined or image types with different compositions may be supported.

[0060] Various encoding / decoding information generated in the present disclosure may be processed explicitly or implicitly. Explicit processing may be understood as processing in which selection information indicating one of a plurality of candidates in a candidate group related to encoding information in a sequence, slice, tile, block, or sub-block is generated by an encoder, and the selection information is included in a bitstream, and the decoder parses the relevant information at the same unit level as in the encoder to reconstruct the relevant information into decoding information. Implicit processing may be understood as processing encoding / decoding information with the same process, rules, etc. at both the encoder and the decoder.

[0061] Figure 1 is a conceptual diagram illustrating an image encoding and decoding system according to an embodiment of the present disclosure.

[0062] Reference Figure 1 , each of the image encoding device 105 and the image decoding device 100 may be a user terminal, for example, a personal computer (PC), a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a portable game console (PSP), a wireless communication terminal, a smart phone or a television (TV), or a server terminal (for example, an application server or a service server). Each of the image encoding device 105 and the image decoding device 100 may be any of various devices, each device including: a communication device such as a communication modem for communicating with various devices or a wired / wireless communication network; a memory 120 or 125 storing various programs and data for inter-frame prediction or intra-frame prediction to encode or decode an image; or a processor 110 or 115 for performing calculation and control operations by executing a program.

[0063] In addition, the image encoding device 105 can transmit the image encoded as a bit stream to the image decoding device 100 in real time or non-real time through a wired / wireless communication network such as the Internet, a short-range wireless communication network, a wireless local area network (WLAN), a wireless broadband (Wi-Bro) network, or a mobile communication network, or via various communication interfaces such as a cable or a universal serial bus (USB), and the image decoding device 100 can reconstruct the received bit stream into an image by decoding the bit stream and reproduce the image. In addition, the image encoding device 105 can transmit the image encoded as a bit stream to the image decoding device 100 through a computer-readable recording medium.

[0064] Although the above-mentioned image encoding device and image decoding device may be separate devices, they may be incorporated into a single image encoding / decoding device according to the implementation. In this case, some components of the image encoding device may be substantially the same as their corresponding components of the image decoding device. Therefore, these components may be configured to include the same structure or perform at least the same function.

[0065] Therefore, in the following detailed description of the technical components and their operating principles, redundant descriptions of the corresponding technical components will be omitted. In addition, since the image decoding device is a computing device that applies the image encoding method performed in the image encoding device to decoding, the following description will focus on the image encoding device.

[0066] The computing device may include: a memory storing a program or software module for executing the image encoding method and / or the image decoding method; and a processor connected to the memory and executing the program. The image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder.

[0067] Figure 2 : is a block diagram showing an image encoding device according to an embodiment of the present disclosure.

[0068] Reference Figure 2 , the image encoding device 20 may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, a coded picture buffer 240 and an entropy encoding unit 245.

[0069] The prediction unit 200 can be implemented using a prediction module as a software module, and generates a prediction block by using intra prediction or inter prediction for a block to be encoded. The prediction unit 200 can generate a prediction block by predicting a target block to be encoded currently in an image. In other words, the prediction unit 200 can generate a prediction block having a predicted pixel value for each pixel by predicting the pixel value of a pixel in a target block according to inter prediction or intra prediction. In addition, the prediction unit 200 can provide information required to generate a prediction block (e.g., information about a prediction mode such as an intra prediction mode or an inter prediction mode) to the coding unit so that the coding unit can encode information about the prediction mode. The processing unit, the prediction method, and specific details about the processing unit that are subject to prediction can be configured according to the encoding / decoding configuration. For example, the prediction method and the prediction mode can be determined based on the prediction unit, and the prediction can be performed based on the transform unit.

[0070] The inter-frame prediction unit can be divided into temporal prediction and spatial prediction based on the reference picture. In the case of temporal prediction, it can be a prediction method for finding motion in a picture that is different from the current picture in time, and in the case of spatial prediction, it can be a prediction method for finding motion in the current picture that is the same as the current picture in time (an area where encoding has been completed, a preset area adjacent to the target block). This can be integrated and managed through a reference picture list, or it can be managed by segmentation into coding modes. For example, it can be configured by mixing the current picture and the pictures that have been encoded before or after the current picture in the reference picture list. The coding mode is divided into Mode_Intra, Mode_InterD, and Mode_InterC. Among them, it can be supported by being divided into Mode_InterD (referring to previous and subsequent pictures) and Mode_InterC (referring to the current picture).

[0071] In addition, the inter-frame prediction unit can distinguish the translation motion model and the non-translation motion model from each other according to the motion prediction method. For the translation motion model, only parallel translation is considered to perform prediction, while for the non-translation motion model, in addition to parallel translation, motions such as rotation, perspective, and zooming in / out can also be considered to perform prediction. Under the assumption of unidirectional prediction, the translation motion model may require a motion vector, while the non-translation motion model may require one or more motion information (e.g., one motion vector + rotation angle / scale factor, more than one motion vector, etc. The following assumes that more than one motion vector is used). In the case of a non-translation motion model, each motion vector may be information applied to a preset position in the target block (e.g., the upper left vertex, the upper right vertex, the lower left vertex of the target block), and the position of the area to be predicted in the target block based on the corresponding motion vector can be obtained in pixel units or sub-block units (an integer greater than or equal to 2, such as 4×4 or 8×8). The inter-frame prediction unit can apply a portion of the subsequent processing together according to the motion model, and apply another portion of the subsequent processing separately.

[0072] The inter prediction unit may include a reference picture construction unit, a motion estimation unit, a motion compensator, a motion information decision unit, and a motion information encoder. The reference picture construction unit may include an encoded picture before or after the current picture in the reference picture list L0 or L1. The prediction block may be obtained from the reference picture included in the reference picture list, and according to the encoding setting, the current image may also be configured as a reference picture and included in at least one reference picture list.

[0073] The reference picture construction unit of the inter-frame prediction unit may include a reference picture interpolator. The reference picture interpolator may perform interpolation on fractional pixels according to interpolation precision. For example, an 8-tap discrete cosine transform (DCT)-based interpolation filter may be applied to the luminance component, and a 4-tap DCT-based interpolation filter may be applied to the chrominance component.

[0074] The motion estimation unit of the inter prediction unit can use the reference picture to detect a block with high correlation with the target block. To this end, various methods can be used, such as full search based block matching algorithm (FBMA), three-step search (TSS), etc. The motion compensator can obtain the prediction block in the motion estimation process.

[0075] The motion information decision unit of the inter-frame prediction unit can perform the process of selecting the best motion information for the target block. The motion information can be encoded in a motion information encoding mode such as a skip mode, a merge mode, and a competition mode. The motion information encoding mode can be configured by combining the supported modes according to the motion model. Such examples may include (translation) skip mode, (non-translation) skip mode, (translation) merge mode, (non-translation) merge mode, (translation) competition mode, and (non-translation) competition mode. Depending on the encoding settings, some of these modes may be included in the candidate group.

[0076] In the motion information coding mode, the prediction value (motion vector, reference picture, prediction direction, etc.) of the motion information of the target block can be obtained from at least one candidate block. When two or more candidate blocks are supported, the best candidate selection information can be generated. In the skip mode (no residual signal) and the merge mode (there is a residual signal), the prediction value can be used as the motion information about the target block as it is, and in the competitive mode, the difference information between the motion information about the target block and the prediction value can be generated.

[0077] The candidate group for the motion information prediction value of the target block can be adaptive and can have various structures according to the motion information encoding mode. The motion information of the blocks spatially adjacent to the target block (e.g., left, top, upper left, upper right, lower left, etc.) can be included in the candidate group, and the motion information of the blocks temporally adjacent (e.g., left block, right block, top block, bottom block, upper left block, upper right block, lower left block, lower right block, etc., including the block <center> in a different image corresponding to the target block) can be included in the candidate group. Mixed motion information of spatial candidates and temporal candidates (e.g., information obtained as an average value, median, etc. of two or more candidates based on motion information of spatially adjacent blocks and motion information of temporally adjacent blocks. The motion information can be obtained in units of target blocks or sub-blocks of target blocks.) can be included in the candidate group.

[0078] The motion information may be prioritized to configure a motion information predictor candidate group. The order of the motion information to be included in the predictor candidate group may be set according to the priority. When the candidate group is filled with as many pieces of motion information as the number of candidates (determined according to the motion information encoding mode) in the candidate group according to the priority, the candidate group may be fully constructed. The motion information may be prioritized in the order of motion information about spatially adjacent blocks, motion information about temporally adjacent blocks, and mixed motion information about spatially adjacent blocks and temporally adjacent blocks. However, the priority may also be modified.

[0079] For example, motion information about spatially adjacent blocks may be included in the candidate group in the order of left block, upper block, upper right block, lower left block, and upper left block, and motion information about temporally adjacent blocks may be included in the candidate group in the order of lower right block, middle block, right block, and lower block.

[0080] The subtraction unit 205 can generate a residual block by subtracting the prediction block from the target block. In other words, the subtraction unit 205 can calculate the difference between the pixel value of each pixel in the target block to be encoded and the predicted pixel value of the corresponding pixel in the prediction block generated by the prediction unit to generate a residual signal in the form of a block, that is, a residual block. In addition, the subtraction unit 205 can generate a residual block in units other than blocks obtained by the block division unit described later.

[0081] The transform unit 210 may transform the spatial signal into a frequency signal. The signal obtained by the transform process is called a transform coefficient. For example, a residual block having a residual signal received from a subtraction unit may be transformed into a transform block having a transform coefficient, and an input signal is determined according to an encoding setting, which is not limited to a residual signal.

[0082] The transform unit may transform the residual block by a transform scheme such as, but not limited to, Hadamard transform, discrete sine transform (DST)-based transform, or DCT-based transform. These transform schemes may be changed and modified in various ways.

[0083] At least one of the transform schemes may be supported, and at least one sub-transform scheme of each transform scheme may be supported. The sub-transform scheme may be obtained by modifying a portion of a basis vector in the transform scheme.

[0084] For example, in the case of DCT, one or more of the sub-transform schemes DCT-1 to DCT-8 may be supported, and in the case of DST, one or more of the sub-transform schemes DST-1 to DST-8 may be supported. The transform scheme candidate group may be constructed with a portion of the sub-transform scheme. For example, DCT-2, DCT-8, and DST-7 may be grouped as a candidate group for transform.

[0085] Transformation can be performed in the horizontal direction / vertical direction. For example, a one-dimensional transform can be performed in the horizontal direction by DCT-2, and a one-dimensional transform can be performed in the vertical direction by DST-7. Using a two-dimensional transform, pixel values ​​can be transformed from the spatial domain to the frequency domain.

[0086] A fixed transform scheme may be adopted, or a transform scheme may be adaptively selected according to the encoding / decoding configuration. In the latter case, the transform scheme may be selected explicitly or implicitly. When the transform scheme is selected explicitly, information about the transform scheme or transform scheme set applied in each of the horizontal direction and the vertical direction may be generated, for example, at the block level. When the transform scheme is selected implicitly, the encoding settings may be defined according to the image type (I / P / B), color component, block size, block shape, block position, intra-frame prediction mode, etc., and a predetermined transform scheme may be selected according to the encoding settings.

[0087] In addition, some transforms may be skipped according to the encoding settings. That is, one or more of the horizontal unit and the vertical unit may be omitted explicitly or implicitly.

[0088] In addition, the transform unit can send information required to generate the transform block to the encoding unit, so that the encoding unit encodes the information, includes the encoded information in a bitstream, and sends the bitstream to the decoder. Therefore, the decoding unit of the decoder can parse the information from the bitstream for inverse transformation.

[0089] The quantization unit 215 may quantize the input signal. The signal obtained from the quantization is referred to as a quantization coefficient. For example, the quantization unit 215 may obtain a quantization block having a quantization coefficient by quantizing a residual block having a residual transform coefficient received from the transform unit, and the input signal may be determined according to an encoding setting, which is not limited to the residual transform coefficient.

[0090] The quantization unit may quantize the transformed residual block through, but not limited to, a quantization scheme such as dead zone uniform boundary value quantization, a quantization weight matrix, etc. The above quantization scheme may be changed and modified in various ways.

[0091] Quantization can be skipped according to the encoding settings. For example, quantization (and inverse quantization) can be skipped according to the encoding settings (for example, the quantization parameter is 0, that is, a lossless compression environment). In another example, when the compression performance based on quantization is not performed in consideration of the characteristics of the image, the quantization process can be omitted. Quantization can be skipped in the entire or partial area (M / 2×N / 2, M×N / 2, or M / 2×N) of the quantization block (M×N), and the quantization skip selection information can be set explicitly or implicitly.

[0092] The quantization unit may send information required to generate the quantized block to the encoding unit, so that the encoding unit encodes the information, includes the encoded information on the bitstream, and sends the bitstream to the decoder. Thus, the decoding unit of the decoder may parse the information from the bitstream for inverse quantization.

[0093] Although the above example is described under the assumption that the residual block is transformed and quantized by the transform unit and the quantization unit, the residual block having the transform coefficient can be generated by transforming the residual signal, and it may not be quantized. The residual block may be subjected to quantization only without being transformed. In addition, the residual block may be subjected to both transformation and quantization. These operations may be determined according to the encoding setting.

[0094] The inverse quantization unit 220 inversely quantizes the residual block quantized by the quantization unit 215. That is, the inverse quantization unit 220 generates a residual block having frequency coefficients by inversely quantizing the quantized frequency coefficient sequence.

[0095] The inverse transform unit 225 inversely transforms the residual block inversely quantized by the inverse quantization unit 220. That is, the inverse transform unit 225 inversely transforms the frequency coefficients of the inversely quantized residual block to generate a residual block having a pixel value, that is, a reconstructed residual block. The inverse transform unit 225 can perform inverse transformation by inversely performing the transformation scheme used by the transform unit 210.

[0096] The adding unit 230 reconstructs the target block by adding the prediction block predicted by the prediction unit 200 and the residual block restored by the inverse transform unit 225. The reconstructed target block is stored in the coded picture buffer 240 as a reference picture (or reference block) to be used as a reference picture when the next block, another block, or another picture of the target block is subsequently encoded.

[0097] The filter unit 235 may include one or more post-processing filters, such as a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF). The deblocking filter may remove block distortion occurring at the boundary between blocks in the reconstructed picture. The ALF may perform filtering based on a value obtained by comparing the reconstructed image after filtering the block by the deblocking filter with the original image. The SAO may reconstruct the offset difference at the pixel level between the original image and the residual block to which the deblocking filter is applied. These post-processing filters may be applied to the reconstructed picture or block.

[0098] The coded picture buffer 240 may store the block or picture reconstructed by the filter unit 235. The reconstructed block or picture stored in the coded picture buffer 240 may be provided to the prediction unit 200 which performs intra prediction or inter prediction.

[0099] The entropy coding unit 245 scans the generated quantized frequency coefficient sequence in various scanning methods to generate a quantized coefficient sequence, encodes the quantized coefficient sequence by entropy coding, and outputs the entropy coded coefficient sequence. The scanning pattern can be configured as one of various patterns (e.g., zigzag, diagonal, and raster). In addition, coded data including the coding information received from each component can be generated and output in a bit stream.

[0100] Figure 3 is a block diagram showing an image decoding apparatus according to an embodiment of the present disclosure.

[0101] Reference Figure 3 , the image decoding device 30 may be configured to include an entropy decoder 305 , a prediction unit 310 , an inverse quantization unit 315 , an inverse transform unit 320 , an addition unit / subtraction unit 325 , a filter 330 , and a decoded picture buffer 335 .

[0102] Furthermore, the prediction unit 310 may be configured to include an intra prediction module and an inter prediction module.

[0103] When the image bitstream is received from the image encoding device 20 , the image bitstream may be transmitted to the entropy decoder 305 .

[0104] The entropy decoder 305 may decode the bitstream into decoded data including quantized coefficients and decoded information to be transmitted to each component.

[0105] The prediction unit 310 may generate a prediction block based on data received from the entropy decoder 305. Based on the reference images stored in the decoded picture buffer 335, a reference picture list may be generated using a default configuration scheme.

[0106] The inter prediction unit may include a reference picture construction unit, a motion compensator, and a motion information decoder.Some of the components may perform the same process as in the encoder, and other components may perform the process of the encoder in reverse.

[0107] The inverse quantization unit 315 may inverse quantize the quantized transform coefficients provided in the bitstream and decoded by the entropy decoder 305 .

[0108] The inverse transform unit 320 may generate a residual block by applying an inverse DCT, an inverse integer transform, or a similar inverse transform technique to the transform coefficients.

[0109] The inverse quantization unit 315 and the inverse transform unit 320 may perform the processes of the transform unit 210 and the quantization unit 215 of the above-mentioned image encoding device 20 in reverse, and may be implemented in various ways. For example, the inverse quantization unit 315 and the inverse transform unit 320 may use the same processes and inverse transforms shared with the transform unit 210 and the quantization unit 215, and may use information about the transform and quantization processes (e.g., transform size, transform shape, quantization type, etc.) received from the image encoding device 20 to perform transform and quantization in reverse.

[0110] The residual block that has been inverse quantized and inverse transformed may be added to the prediction block obtained by the prediction unit 310 , thereby generating a reconstructed image block. The addition may be performed by the addition unit / subtraction unit 325 .

[0111] Regarding the filter 330, when necessary, a deblocking filter may be applied to remove blocking artifacts from the reconstructed image blocks. To improve the video quality before and after the decoding process, other loop filters may be additionally used.

[0112] The reconstructed and filtered image blocks may be stored in the decoded picture buffer 335 .

[0113] Although not shown in the drawings, the image encoding / decoding apparatus may further include a block dividing unit.

[0114] The block partition unit can be divided into blocks of various units and sizes. The basic coding unit (or maximum coding unit. Coding tree unit. CTU) can refer to the basic (or starting) unit for prediction, transformation, quantization, etc. in the image encoding / decoding process. In this case, the basic coding unit can be composed of a luminance basic coding block (maximum coding block or CTB) and two basic chrominance coding blocks according to the color format (YCbCr in this example), and the size of each block can be determined according to the color format. A coding block (CB) can be obtained according to the division process. A CB can be understood as a unit that is no longer further subdivided due to certain restrictions, and can be set as a starting unit for division into sub-units. In the present disclosure, blocks conceptually include various shapes such as triangles, circles, etc., without being limited to squares.

[0115] Although the following description is given in the context of one color component, it is also applicable to other color components with some modifications in proportion to the ratio according to the color format (for example, in the case of YCbCr 4:2:0, the width-to-height length ratio of the luminance component to the chrominance component is 2:1). In addition, although block division depending on other color components (for example, depending on the block division result of Y in Cb / Cr) is possible, it should be understood that block independent division of each color component is also possible. In addition, although a common block division configuration can be used (considering proportionality to the length ratio), it is also necessary to consider and understand the use of separate block division configurations according to color components.

[0116] In the block segmentation unit, a block can be represented as M×N, and the maximum and minimum values ​​of each block can be obtained within the range. For example, if the maximum and minimum values ​​of the block are 256×256 and 4×4 respectively, a size of 2 can be obtained. m ×2 n The blocks may be blocks of size 2m×2n (in this example, m and n are integers from 2 to 8), blocks of size 2m×2n (in this example, m and n are integers from 2 to 128), or blocks of size m×m (in this example, m and n are integers from 4 to 256). In this context, m and n may be equal or different, and one or more ranges such as maximum and minimum values ​​of the support blocks may be generated.

[0117] For example, information about the maximum size and the minimum size of a block may be generated, and information about the maximum size and the minimum size of a block may be generated in some partition configurations. In the former case, the information may be range information about the maximum size and the minimum size that can be generated in an image, and in the latter case, the information may be information about the maximum size and the minimum size that can be generated according to some partition configurations. The partition configuration may be defined by image type (I / P / B), color component (YCbCr, etc.), block type (coding / prediction / transformation / quantization), partition type (index or type), and partition scheme (quadtree (QT), binary tree (BT), and ternary tree (TT) as tree methods, and SI2, SI3, and SI4 as type methods).

[0118] In addition, there may be constraints on the aspect ratio (block shape) that can be used for blocks, and at this point, a boundary value may be set. Only blocks that are less than or equal to / less than a boundary value k may be supported, where k may be defined based on the aspect ratio A / B (A is the longer or equal value between width and height, and B is another value). k may be a real number equal to or greater than 1, for example, 1.5, 2, 3, 4, etc. As in the above example, constraints on the shape of a block in an image may be supported, or one or more constraints may be supported based on a partitioning configuration.

[0119] In summary, whether block partitioning is supported can be determined based on the above ranges and constraints and the partitioning configuration described later. For example, when the candidate (child block) split from the block (parent block) meets the supporting block condition, the partitioning can be supported, otherwise, the partitioning may not be supported.

[0120] The block segmentation unit can be configured with respect to each component of the image encoding device and the image decoding device, and the size and shape of the block can be determined in the process. Different blocks can be configured according to the components. The block may include a prediction block for a prediction unit, a transform block for a transform unit, and a quantization block for a quantization unit. However, the present disclosure is not limited to this, and block units may be additionally defined for other components. Although the shape of each of the input and output in each component is described as a rectangle in the present disclosure, the input and output of some components may have any other shape (e.g., a triangle).

[0121] The size and shape of the initial (or starting) block in the block partition unit can be determined from the higher unit. The initial block can be divided into smaller blocks. Once the optimal size and shape are determined based on the block partition, the block can be determined as the initial block of the lower unit. The higher unit can be a coding block, and the lower unit can be a prediction block or a transform block, but the present disclosure is not limited to this. On the contrary, various modifications are possible. Once the initial block of the lower unit is determined as in the above example, the partitioning process can be performed to detect the optimal size and shape block like the higher unit.

[0122] In summary, the block segmentation unit can segment the basic coding block (or the maximum coding block) into at least one coding block, and the coding block can be segmented into at least one prediction block / transform block / quantization block. In addition, the prediction block can be segmented into at least one transform block / quantization block, and the transform block can be segmented into at least one quantization block. Some blocks may have a dependency relationship with other blocks (i.e., defined by a higher unit and a lower unit), or may have an independent relationship with other blocks. For example, the prediction block may be a higher unit above the transform block, or may be a unit independent of the transform block. Various relationships can be established according to the type of block.

[0123] Depending on the encoding / decoding configuration, it may be determined whether to combine a higher unit and a lower unit. Combination between units means that a block of a higher unit is subjected to encoding / decoding processing of a lower unit (e.g., in a prediction unit, a transform unit, an inverse transform unit, etc.) without being divided into lower units. That is, this may mean that a division process is shared between multiple units, and division information is generated in one of the units (e.g., a higher unit).

[0124] For example, a coding block may be subjected to prediction, transformation, and inverse transformation (when a coding block is combined with a prediction block or a transform block).

[0125] For example, (when a coding block is combined with a prediction block), the coding block may be subjected to prediction, and a transform block equal to or smaller in size than the coding block may be subjected to transform and inverse transform.

[0126] For example, (when a coding block is combined with a transform block), a prediction block that is equal to or smaller in size than the coding block may be subjected to prediction, and the coding block may be subjected to transform and inverse transform.

[0127] For example, a prediction block that is equal to or smaller in size than a coding block (when a prediction block is combined with a transform block) may be subjected to prediction, transform, and inverse transform.

[0128] For example, (when there is no block combination), a prediction block equal to or smaller in size than a coding block may be subject to prediction, and a transform block equal to or smaller in size than a coding block may be subject to transform and inverse transform.

[0129] Although various cases of coding blocks, prediction blocks, and transform blocks have been described in the above examples, the present disclosure is not limited thereto.

[0130] For the combination between units, a fixed configuration can be supported in the image, or an adaptive configuration can be supported considering various encoding / decoding factors. The encoding / decoding factors include image type, color component, encoding mode (intra / inter), partition configuration, block size / shape / position, aspect ratio, prediction related information (e.g., intra prediction mode, inter prediction mode, etc.), transform related information (e.g., transform scheme selection information, etc.), quantization related information (e.g., quantization region selection information and quantization transform coefficient encoding information), etc.

[0131] When a block of optimal size and shape has been detected as described above, mode information (e.g., partition information) of the block may be generated. The mode information may be included in a bitstream together with information generated from a component to which the block belongs (e.g., prediction-related information and transform-related information) and transmitted to a decoder, and may be parsed by the decoder at the same unit level for use in a video decoding process.

[0132] Now, the partitioning scheme will be described. Although it is assumed that the initial block is shaped like a square for ease of description, the present disclosure is not limited thereto, and the description can be applied in the same or similar manner to a case where the initial block is rectangular.

[0133] The block segmentation unit may support various types of partitioning. For example, tree-based partitioning or index-based partitioning may be supported, and other methods may also be supported. In tree-based partitioning, the partitioning type may be determined based on various types of information (e.g., information indicating whether to perform partitioning, tree type, partitioning direction, etc.), while in index-based partitioning, the partitioning type may be determined using specific index information.

[0134] Figure 4 is an example diagram showing various types of partitioning that can be obtained in the block partitioner of the present disclosure. In this example, assume that Figure 4 The partition types shown in are obtained through one partition operation (or process), which should not be understood as limiting the present disclosure. The partition types can also be obtained in multiple partition operations. In addition, Figure 4 Other partitioning types not shown are also available.

[0135] (Tree-based partitioning)

[0136] In the tree-based partitioning of the present disclosure, QT, BT, and TT can be supported. If one tree method is supported, this can be called single-tree partitioning, and if two or more tree methods are supported, this can be called multi-tree partitioning.

[0137] In QT, a block is split into two partitions (n) in each of the horizontal and vertical directions, while in BT, a block is split into two partitions (b to g) in the horizontal or vertical direction. In TT, a block is split into three partitions (h to m) in the horizontal or vertical direction.

[0138] In QT, a block can be partitioned into four partitions (o and p) by limiting the partition direction to one of the horizontal and vertical directions. In addition, in BT, it is possible to support only partitions of equal size (b and c), only partitions of different sizes (d to g), or both types of partitions. In addition, in TT, it is possible to support partitions (h, j, k, and m) concentrated only in a specific direction (1:1:2 or 2:1:1 in the left->right or up->down directions), partitions (i and l) concentrated in the center (1:2:1), or both types of partitions. In addition, it is possible to support partitioning blocks into four partitions in each of the horizontal and vertical directions (i.e., a total of 16 partitions) (q).

[0139] In the tree method, it is possible to support splitting a block into z partitions (b, d, e, h, i, j, o) in the horizontal direction only, splitting a block into z partitions (c, f, g, k, l, m, p) in the vertical direction only, or both types of partitions. In this article, z can be an integer equal to or greater than 2, for example, 2, 3 or 4.

[0140] In the present disclosure, it is assumed that partition type n is supported as QT, partition types b and c are supported as BT, and partition types i and l are supported as TT.

[0141] Depending on the encoding / decoding settings, one or more of the tree partitioning schemes may be supported. For example, QT, QT / BT, or QT / BT / TT may be supported.

[0142] In the above example, the basic tree partitioning scheme is QT, and BT and TT are included as additional partitioning schemes depending on whether other trees are supported. However, various modifications may be made. Information indicating whether other trees are supported (bt_enabled_flag, tt_enabled_flag, and bt_tt_enabled_flag, where 0 indicates non-support and 1 indicates support) may be implicitly determined according to encoding / decoding settings or explicitly determined in units such as sequences, pictures, slices, or tiles.

[0143] The split information may include information indicating whether splitting is performed (tree_part_flag or qt_part_flag, bt_part_flag, tt_part_flag, and bt_tt_part_flag, which may have a value of 0 or 1, where 0 indicates no splitting and 1 indicates splitting). In addition, according to the splitting scheme (BT and TT), information about the splitting direction (dir_part_flag, or bt_dir_part_flag, tt_dir_part_flag, and bt_tt_dir_part_flag, which has a value of 0 or 1, where 0 indicates <width / horizontal> and 1 indicates <height / vertical>) may be added. This may be information generated when splitting is performed.

[0144] When supporting multi-tree partitioning, various partitioning information can be configured. For ease of description, the following description gives an example of how to configure the partitioning information at one depth level (that is, recursive partitioning can be performed by setting one or more supported partitioning depths).

[0145] In Example 1, information indicating whether the division is performed is checked. If the division is not performed, the division ends.

[0146] If partitioning is performed, selection information about the partitioning type is checked (e.g., tree_idx. 0 for QT, 1 for BT, and 2 for TT). The partitioning direction information is additionally checked according to the selected partitioning type, and the process proceeds to the next step (if additional partitioning is possible due to a reason such as when the partitioning depth has not reached the maximum value, the process starts again from the beginning, and if additional partitioning is not possible, the partitioning process ends).

[0147] In Example 2, information indicating whether the partition is performed with a specific tree scheme (QT) is checked, and the process proceeds to the next step. If the partition is not performed with the tree scheme (QT), information indicating whether the partition is performed with another tree scheme (BT) is checked. In this case, if the partition is not performed with the tree scheme, information indicating whether the partition is performed with a third tree scheme (TT) is checked. If the partition is not performed with the third tree scheme (TT), the partition process ends.

[0148] If the partitioning is performed with the tree scheme (QT), the process proceeds to the next step. In addition, the partitioning is performed with the second tree scheme (BT), the partitioning direction information is checked, and the process proceeds to the next step. If the partitioning is performed with the third tree scheme (TT), the partitioning direction information is checked, and the process proceeds to the next step.

[0149] In Example 3, information indicating whether the partitioning is performed in the tree scheme (QT) is checked. If the partitioning is not performed in the tree scheme (QT), information indicating whether the partitioning is performed in other tree schemes (BT and TT) is checked. If the partitioning is not performed, the partitioning process ends.

[0150] If the partitioning is performed in the tree scheme (QT), the process proceeds to the next step. In addition, the partitioning is performed in other tree schemes (BT and TT), the partitioning direction information is checked, and the process proceeds to the next step.

[0151] Although in the above examples, the tree partitioning schemes are prioritized (Examples 2 and 3) or are not prioritized (Example 1), various modifications are also available. In addition, in the above examples, the partitioning in the current step is independent of the partitioning result of the previous step. However, the partitioning in the current step may depend on the partitioning result of the previous step.

[0152] In Examples 1 to 3, if some tree partitioning schemes (QT) are performed in the previous step and thus the process proceeds to the current step, the same tree partitioning schemes (QT) may also be supported in the current step.

[0153] On the other hand, if a certain tree partitioning scheme (QT) is not executed in the previous step and another tree partitioning scheme (BT or TT) is executed, other tree partitioning schemes (BT and TT) except the certain tree partitioning scheme (QT) may be supported in the current step and subsequent steps.

[0154] In the above case, the tree configuration supporting block partitioning can be adaptive, and thus the above partitioning information can also be configured differently. (Assume that the example to be described later is Example 3). That is, if the partitioning is not performed with a certain tree scheme (QT) in the previous step, the partitioning process can be performed without considering the tree scheme (QT) in the current step. In addition, the partitioning information related to a certain tree scheme (for example, information indicating whether to perform the partitioning, information about the partitioning direction, etc.) can be removed. <qt>In the information indicating whether to perform the division).

[0155] The above example relates to adaptive partition information configuration for a case where block partitioning is allowed (e.g., the block size is within the range between the maximum value and the minimum value, the partition depth of each tree scheme has not reached the maximum depth (allowed depth), etc.). Even when block partitioning is restricted (e.g., the block size does not exist in the range between the maximum value and the minimum value, the partition depth of each tree scheme has reached the maximum depth, etc.), the partition information can be adaptively configured.

[0156] As already mentioned, in the present disclosure, tree-based partitioning may be performed recursively. For example, if the partition flag of a coding block having a partition depth k is set to 0, coding block encoding is performed in the coding block having a partition depth k. If the partition flag of a coding block having a partition depth k is set to 1, coding block encoding is performed in N sub-coding blocks having a partition depth k+1 according to a partitioning scheme (where N is an integer equal to or greater than 2, such as 2, 3, and 4).

[0157] In the above process, the sub coding block may be set as the coding block (k+1) and divided into the sub coding block (k+2). The hierarchical division scheme may be determined according to division settings such as a division range and an allowed division depth.

[0158] In this case, a bitstream structure representing the split information may be selected from one or more scanning methods. For example, the bitstream of the split information may be configured based on the order of split depths or based on whether splitting is performed.

[0159] For example, in the case of splitting depth order, splitting information is obtained at the current depth level based on the initial block, and then the splitting information is obtained at the next depth level. In the case of whether to perform splitting, additional splitting information is first obtained in the block segmented from the initial block, and other additional scanning methods can be considered.

[0160] The maximum block size and the minimum block size may have a common setting regardless of the tree type (or all trees), or may have separate settings for each tree, or may have a common setting for two or more trees. In this case, the maximum block size may be set to be equal to or less than the maximum coding block. If the maximum block size according to the predetermined first tree is different from the maximum coding block, the split is implicitly performed using the predetermined second tree method until the maximum block size of the first tree is reached.

[0161] In addition, regardless of the tree type, a common split depth may be supported, a separate split depth may be supported per tree, or a common split depth may be supported for two or more trees. Alternatively, a split depth may be supported for some trees, while not supported for other trees.

[0162] Explicit syntax elements for setting information may be supported, and some setting information may be determined implicitly.

[0163] (Index-based partitioning)

[0164] In the index-based partitioning of the present disclosure, a constant partition index (CSI) scheme and a variable partition index (VSI) scheme may be supported.

[0165] In the CSI scheme, k sub-blocks may be obtained by division in a predetermined direction, and k may be an integer equal to or greater than 2, such as 2, 3, or 4. Specifically, the size and shape of the sub-block may be determined based on k, regardless of the size and shape of the block. The predetermined direction may be one or a combination of two or more of a horizontal direction, a vertical direction, and a diagonal direction (upper left->lower right direction or lower left->upper right direction).

[0166] In the index-based CSI partitioning scheme of the present disclosure, z candidates may be obtained by partitioning in the horizontal direction or the vertical direction. In this case, z may be an integer equal to or greater than 2, such as 2, 3, or 4, and the sub-blocks may be equal in one of the width and height, and may be equal or different in the other of the width and height. The width or height length ratio of the sub-block is A1:A2:...:AZ, and each of A1 to AZ may be an integer equal to or greater than 1, such as 1, 2, or 3.

[0167] In addition, candidates may be obtained by dividing into x partitions and y partitions along the horizontal direction and the vertical direction, respectively. Each of x and y may be an integer equal to or greater than 1, such as 1, 2, 3, or 4. However, candidates in which both x and y are 1 may be limited (because a already exists). Although Figure 4 A case in which sub-blocks have the same width ratio or height ratio is shown, but candidates having different width ratios or height ratios may also be included.

[0168] In addition, the candidate may be partitioned into w partitions in one of the diagonal directions: upper left->lower right and lower left->upper right. Herein, w may be an integer equal to or greater than 2, such as 2 or 3.

[0169] Reference Figure 4 , according to the length ratio of each sub-block, the partition type can be classified into a symmetric partition type (b) and an asymmetric partition type (d and e). In addition, the partition type can be classified into a partition type (k and m) centered in a specific direction and a center partition type (k). The partition type can be defined by various encoding / decoding factors including the sub-block shape and the sub-block length ratio, and the supported partition types can be implicitly or explicitly determined according to the encoding / decoding settings. Therefore, the candidate group can be determined based on the partition types supported in the index-based partitioning scheme.

[0170] In the VSI scheme, when the width w or height h of each sub-block is fixed, one or more sub-blocks can be obtained by division in a predetermined direction. Herein, each of w and h can be an integer equal to or greater than 1, for example, 1, 2, 4, or 8. Specifically, the number of sub-blocks can be determined based on the size and shape of the block and the w or h value.

[0171] In the index-based VSI partitioning scheme of the present disclosure, the candidate may be divided into sub-blocks, each of which is fixed in one of width and length. Alternatively, the candidate may be divided into sub-blocks, each of which is fixed in both width and length. Since the width or height of the sub-block is fixed, equal division in the horizontal direction or the vertical direction may be allowed. However, the present disclosure is not limited thereto.

[0172] In a case where the size of the block before division is M×N, if the width w of each sub-block is fixed, the height h of each sub-block is fixed, or both the width w and the height h of each sub-block are fixed, the number of obtained sub-blocks may be (M*N) / w, (M*N) / h, or (M*N) / w / h.

[0173] According to encoding / decoding settings, only one or both of the CSI scheme and the VSI scheme may be supported, and information about the supported scheme may be determined implicitly or explicitly.

[0174] The present disclosure will be described in the context of supported CSI schemes.

[0175] The candidate group may be constructed to include two or more candidates in an index-based partitioning scheme according to encoding / decoding settings.

[0176] For example, a candidate group such as {a, b, c}, {a, b, c, n}, or {a to g and n} may be formed. The candidate group may include examples of block types that are predicted to appear multiple times based on general statistical features, such as a block that is partitioned into two partitions in the horizontal direction or in the vertical direction, or in each of the horizontal and vertical directions.

[0177] Alternatively, a candidate group such as {a, b}, {a, o} or {a, b, o}, or a candidate group such as {a, c}, {a, p} or {a, c, p} may be constructed. The candidate group may be an example including candidates each of which is divided into two partitions and four partitions in the horizontal direction and the vertical direction, respectively. This may be an example of configuring a block type predicted to be mainly divided in a specific direction as a candidate group.

[0178] Alternatively, a candidate group such as {a, o, p} or {a, n, q} may be constructed. This may be an example of constructing a candidate group to include a block type predicted to be divided into many partitions smaller than the block before division.

[0179] Alternatively, a candidate group such as {a, r, s} may be constructed, and it may be an example of determining the best partitioning result of a rectangular shape that can be obtained from a block before segmentation by other methods (tree method), and constructing a non-rectangular shape as a candidate group.

[0180] As noted from the above examples, various candidate group configurations may be available, and one or more may be supported taking into account various encoding / decoding factors.

[0181] Once the candidate set is fully constructed, various partition information configurations may be available.

[0182] For example, regarding a candidate group including the undivided candidate a and the divided candidates b to s, index selection information may be generated.

[0183] Alternatively, information indicating whether the split is performed (information indicating whether the split type is a) may be generated. If the split is performed (if the split type is not a), index selection information on a candidate group including the split candidates b to s may be generated.

[0184] The partition information may be configured in many other ways than those described above. In addition to the information indicating whether the partition is performed, binary bits may be allocated to the index of each candidate in the candidate group in various ways such as fixed-length binarization, variable-length binarization, etc. If the number of candidates is 2, 1 bit may be allocated to the index selection information, and if the number of candidates is 3, one or more bits may be allocated to the index selection information.

[0185] Compared to the tree-based partitioning scheme, a partitioning type predicted to occur multiple times may be included in a candidate group in the index-based partitioning scheme.

[0186] Since the number of bits used to represent index information can be increased according to the number of supported candidate groups, this scheme can be suitable for single-layer partitioning (e.g., the partition depth is limited to 0) rather than tree-based hierarchical partitioning (recursive partitioning). That is, a single partitioning operation can be supported, and the sub-blocks obtained by index-based partitioning may not be further split.

[0187] This may mean that further division into smaller blocks of the same type is impossible (for example, a coding block obtained by index-based division cannot be further split into coding blocks), and may also mean that further division into blocks of different types is impossible (for example, it is impossible to divide a coding block into a prediction block and a coding block). Obviously, the present disclosure is not limited to the above examples, and other modifications are also available.

[0188] Now, a description will be given of determining the block division setting mainly based on the block type among the encoding / decoding factors.

[0189] First, the coding blocks can be obtained in the partitioning process. A tree-based partitioning scheme can be used for the partitioning process, and a tree-based partitioning scheme can be generated such as Figure 4 The partition type of the tree can be a (no partition), n (QT), b, c (BT), i, or l (TT). Depending on the encoding / decoding settings, various combinations of tree types (e.g., QT / QT+BT / QT+BT+TT) may be available.

[0190] The following example is a process of finally partitioning the coding block obtained in the above process into a prediction block and a transform block. It is assumed that prediction, transform, and inverse transform are performed based on the size of each partition.

[0191] In Example 1, prediction may be performed by setting the size of the prediction block equal to the size of the coding block, and transform and inverse transform may be performed by setting the size of the transform block equal to the size of the coding block (or prediction block).

[0192] In Example 2, prediction may be performed by setting the size of the prediction block equal to the size of the coding block. A transform block may be obtained by dividing the coding block (or prediction block), and transform and inverse transform may be performed based on the size of the obtained transform block.

[0193] Here, a tree-based partitioning scheme can be used for the partitioning process, and according to the type of tree, such as Figure 4 The partition type of the tree can be a (no partition), n (QT), b, c (BT), i or l (TT). Depending on the encoding / decoding settings, various combinations of tree types (e.g., QT / QT+BT / QT+BT+TT) may be available.

[0194] Here, the partitioning process may be an index-based partitioning scheme. The partitioning type may be obtained according to the index type, for example, Figure 4 a (no segmentation), b, c, or d. Depending on the encoding / decoding settings, various candidate groups such as {a, b, c} and {a, b, c, d} can be constructed.

[0195] In Example 3, a prediction block can be obtained by dividing a coding block, and the prediction block is subjected to prediction based on the size of the obtained prediction block. For a transform block, its size is set to the size of the coding block, and transform and inverse transform can be performed on the transform block. In this example, the prediction block can be independent of the transform block.

[0196] Index-based partitioning schemes can be used for the partitioning process, and the partition type can be obtained based on the index type (e.g. Figure 4 Various candidate groups may be constructed according to encoding / decoding settings, for example, {a, b, c, n}, {a to g, n}, and {a, r, s}.

[0197] In Example 4, a prediction block may be obtained by dividing a coding block, and the prediction block is subjected to prediction based on the size of the obtained prediction block. For a transform block, its size is set to the size of the prediction block, and transform and inverse transform may be performed on the transform block. In this example, the transform block may have a size equal to the size of the obtained prediction block, or the obtained prediction block may have a size equal to the size of the transform block (the size of the transform block is set to the size of the prediction block).

[0198] Tree-based partitioning schemes can be used for the partitioning process, and partition types can be generated based on the tree type (e.g. Figure 4 Depending on the encoding / decoding settings, various combinations of tree types (e.g., QT / BT / QT+BT) may be available.

[0199] Here, the index-based partitioning scheme can be used for the partitioning process, and the partitioning type can be obtained according to the index type (e.g. Figure 4 A candidate group may be constructed according to encoding / decoding settings, such as {a, b}, {a, c}, {a, n}, {a, o}, {a, p}, {a, b, c}, {a, o, p}, {a, b, c, n}, and {a, b, c, n, p}. In addition, a candidate group may be constructed in a separate VSI scheme or in a combined CSI scheme and VSI scheme as an index-based partitioning scheme.

[0200] In Example 5, a prediction block may be obtained by dividing a coding block, and the prediction block is subjected to prediction based on the size of the obtained prediction block. A transform block may also be obtained by dividing a coding block, and the transform block is subjected to transformation and inverse transformation based on the size of the obtained transform block. In this example, each of the prediction block and the transform block may be obtained by dividing the coding block.

[0201] Here, a tree-based partitioning scheme and an index-based partitioning scheme may be used for the partitioning process, and a candidate group may be constructed in the same manner as Example 4 or in a similar manner.

[0202] In this case, the above examples are situations that may occur depending on whether the process of dividing each block type is shared, which should not be understood as limiting the present disclosure. Various modifications may also be available. In addition, various encoding / decoding factors and block types may be considered to determine the block division settings.

[0203] The encoding / decoding factors may include image type (I / P / B), color component (YCbCr), block size / shape / position, block aspect ratio, block type (coding block, prediction block, transform block, or quantization block), partition state, coding mode (intra / inter), prediction-related information (intra prediction mode or inter prediction mode), transform-related information (transform scheme selection information), and quantization-related information (quantization region selection information and quantization transform coefficient coding information).

[0204] (Description of the relationship between blocks)

[0205] Figure 5 An example of a family tree illustrating the genetic characteristics of family members and people related to the family is shown. For ease of illustration, horizontal and vertical relationships according to a specific gender (male) will be shown.

[0206] Reference Figure 5 , the target person (self) can have a horizontal relationship with the elder brother and younger brother (a), and the target person (self) can have a vertical relationship with the grandfather (grandfather), father (father, ideal ancestor), child (son, ideal offspring) and grandson (b). At this time, there may be a situation where the characters in the horizontal relationship have similar genetic factors, such as appearance, physique and personality. Alternatively, some elements may be similar, while some elements may be dissimilar. Whether the above genetic factors are all or partially similar may be determined by various environmental factors (including the mother).

[0207] The above description can be applied to the same or similar vertical relationships. For example, there may be a case where the subject person's genetic factors (appearance, physique, personality) are similar to those of the father. Alternatively, some of the subject person's genetic factors (appearance, physique) may be similar to those of the father, but some genetic factors (personality) may be different from those of the father (similar to the mother).

[0208] As another example, the target person may be genetically similar to the grandfather (or grandson), and the grandfather may be genetically similar to the grandson, but the similarity between each person can be determined based on the relationship between each person. In other words, the similarity between the grandfather and the target person (two major differences) may be high, while the similarity between the grandfather and the grandson (four major differences) may be low.

[0209] Normally, direct analysis may be the top priority for grasping the characteristics of the target person, but when it appears that the target person does not appear, direct analysis is impossible. In this case, the characteristics of the target person can be roughly grasped by conducting indirect analysis of people in various relationships as in the above example. Of course, it may be necessary to analyze people who have a high similarity with the target person.

[0210] The relationship between people based on various blood relationships has been described through the above examples, but the same or similar application can be applied to encoding for image compression. Here, block-based encoding will be taken as an example. In order to encode a predetermined block (target block), information of blocks (related blocks) placed in various relationships with the target block can be used / referenced.

[0211] Here, the information of the relevant block may be data based on a pixel value, data based on pattern information used in the encoding process, or data based on setting information used in the encoding process. For example, it may be a pixel value in the spatial domain or a coefficient value (or quantization coefficient) in the frequency domain of the relevant block. Alternatively, it may be pattern information generated in the process of encoding the relevant block. Alternatively, it may be information about a reference setting used in the encoding process of the relevant block (e.g., a reference candidate group). Here, the data based on a pixel value or the data based on pattern information may be information for configuring the reference setting.

[0212] In the present invention, the relationship between blocks (target block and related blocks, described with reference to the target block) can be defined as follows.

[0213] -Horizontal relationship: when the target block has no overlapping area with the related blocks (independent relationship between blocks)

[0214] -Vertical relationship: In the case where the target block is larger than the relevant block and contains the relevant block. Or, in the case where the target block is smaller than the relevant block and is contained in the relevant block (dependency between blocks)

[0215] Here, in the case of having a horizontal relationship, the related block can be located regardless of the space to which the target block belongs. That is, the related block may belong to the same space as the target block in time, or may belong to a space different from the target block in time.

[0216] Here, in the case of having a vertical relationship, the related block can be located in the space to which the target block belongs. That is, the related block does not belong to a space different in time from the target block, but according to the encoding setting, the related block can have a vertical relationship based on an area corresponding to the target block in a space different in time.

[0217] Figure 6 Examples of various arrangements of related blocks having a horizontal relationship with the target block are shown. Figure 6 , blocks placed in a horizontal relationship with the target block can be classified into blocks belonging to the same space (Curr) in time and blocks belonging to a different space (Diff) in time.

[0218] Here, even if the related block belongs to a color component different from that of the target block (X), it is considered to belong to the same space in time, but some limitations of the horizontal relationship are changed (there is a related block having the same size and position as the target block). Here, blocks belonging to the same space can be classified into blocks adjacent to (or closest to) the target block (UL, U, UR, L, DL) and blocks not adjacent to (or far away) the target block (F0, F1, F2).

[0219] Among the blocks belonging to the same space, the block adjacent to the target block may be the block closest to the left, top, upper left, upper right, lower left, etc. This is a block that has been encoded by considering the raster scan order (or Z scan. In the case of 2x2, it is upper left->upper right->lower left->lower right). That is, the positions of the adjacent blocks can be determined according to a predetermined scanning order, and changes such as removing blocks at the above positions or adding blocks at new positions (right, bottom, lower right, etc.) may occur depending on the type of scanning order (reverse Z scan <lower right->lower left->upper right->upper left>, clockwise scan <upper left->upper right->lower right->lower left>, counterclockwise scan <upper left->lower left->lower right->upper right>, etc.).

[0220] In addition, the block that is not adjacent to the target block may be a block that has already been encoded. In this case, it may belong to the same block unit (e.g., the largest coding block) as the target block, or may belong to the same segmentation unit (slice, tile, etc.). That is, limited settings may be supported, such as placing a range for an area that is not adjacent but may be included as a related block (based on the target block being within the range of x_offset and y_offset in the horizontal and vertical directions). In the present invention, it is assumed that a block having a horizontal relationship with the target block has already been encoded, but is not limited thereto.

[0221] For encoding of a target block, encoding information / reference settings of related blocks having a horizontal relationship may be used (referenced).

[0222] For example, the pixel values ​​of the relevant blocks can be used to generate the prediction values ​​of the target blocks. In detail, in intra prediction, the prediction value of the target block can be obtained by applying methods such as extrapolation, interpolation, averaging, or methods such as block matching or template matching to the pixel values ​​of the relevant blocks. In addition, in inter prediction, the prediction value of the target block can be obtained by using methods such as block matching or template matching using the pixel values ​​of the relevant blocks. In this case, in terms of finding the prediction value in the same space, block matching or template matching can be defined as intra prediction (Mode_Intra), or can be defined as inter prediction (Mode_Inter) according to the prediction method, or it can be classified as other coding modes that are defined separately.

[0223] Here, only the pixel values ​​in the spatial domain are targeted, but all or some coefficient values ​​in the frequency domain of the relevant block may be used as prediction values ​​of the target block (ie, for prediction of frequency components).

[0224] For example, the mode information of the related block can be used to encode the mode information of the target block. Specifically, in intra-frame prediction, the prediction information of the related block (directional mode, non-directional mode, motion vector, etc.) can be used to encode the prediction information of the target block (MPM, non-MPM, etc.). In addition, in inter-frame prediction, the prediction information of the related block (motion vector, reference picture, etc.) can be used to encode the prediction information of the target block.

[0225] Here, according to the prediction method of intra-frame prediction, not only can the related blocks belonging to the same space and the same color component in time as the target block be targeted (using a prediction mode of extrapolation, interpolation, averaging, etc.), but also the related blocks belonging to the same space and different color components in time as the target block can be targeted (a prediction mode in which data of different color components is copied).

[0226] Here, in the case of inter prediction, a motion vector and a reference picture are targeted as examples of prediction information, but various information such as a motion information encoding mode, a motion prediction direction, and a motion model may be included.

[0227] For example, for the reference setting of the target block, the reference setting of the related block can be used. In detail, in intra prediction, the MPM candidate group of the related block can be used as the MPM candidate group of the target block. In addition, in inter prediction, the motion prediction candidate group of the related block can be used as the motion prediction candidate group of the target block. That is, even if the candidate group is constructed based on the related block, this means that the candidate group of the related block can be used as it is without performing a separate candidate group construction in the target block.

[0228] In the above examples, description has been made assuming that the related block is a block having a horizontal relationship with the target block. However, there may be many related blocks in the image, and at least one related block must be specified for encoding the target block. The following case classification is only an example, and it is necessary to understand that various case configurations and definitions are possible without limitation thereto.

[0229] Here, a block belonging to the same space and adjacent to the target block may be designated as a related block (case 1). Alternatively, a block belonging to the same space and not adjacent to the target block may be designated as a related block (case 2). Alternatively, a block belonging to a different space may be designated as a related block (case 3). Alternatively, all or some of the blocks belonging to (case 1) to (case 3) may be designated as related blocks.

[0230] Here, in the case of (case 1), all or some of the left block, upper block, upper left block, upper right block, and lower left block (L, U, UL, UR, DL) adjacent to the target block may be designated as related blocks. In the case of (case 2), one or more of the blocks not adjacent to the target block may be designated as related blocks. In the case of (case 3), all or some of the center block, left block, right block, upper block, lower block, upper left block, upper right block, lower left block, and lower right block (C, L, R, U, D, UL, UR, DL, DR) adjacent to the target block and one or more of the blocks not adjacent to the target block may be designated as related blocks.

[0231] In order to specify the relevant block, there are various verification methods. First, a block including coordinates based on a predetermined position of the target block can be specified as a relevant block. First, based on the upper left coordinates of (a, b), it is assumed that the range of the target block (m×n) is (a+m-1, b+n-1).

[0232] (Case 1 or Case 3) The C block is described assuming that the position of the target block and each picture is the same. Therefore, the description of the block with the same letter in the same image Curr and the different image Diff may be common. However, (in the case of Case 3), the position of the C block may be different from the position of the target block in the picture, and in the example described later (ie, the block belonging to Diff), the pixel position may be changed according to the position of the C block.

[0233] A C block refers to a block including pixels at predetermined positions among the internal pixels of the target block, such as (a, b), (a, b+n-1), (a+m-1, b), (a+m-1, b+n-1), (a+m / 2-1, b+n / 2-1), (a+m / 2+1, b+n / 2-1), (a+m / 2-1, b+n / 2+1), (a+m / 2+1, b+n / 2+1). Also, an L block refers to a block including pixels at predetermined positions among the pixels outside the left boundary of the target block, such as (a-1, b), (a-1, b+n-1), and a U block refers to a block including pixels at predetermined positions among the pixels outside the upper boundary of the target block, such as (a, b-1), (a+m-1, b-1). In addition, a UL block refers to a block including pixels at predetermined positions among pixels outside the upper left boundary of the target block, such as (a-1, b-1), a UR block refers to a block including pixels at predetermined positions among pixels outside the upper right boundary of the target block, such as (a+m, b-1), (a-1, b+n), and a DL block refers to a block including pixels at predetermined positions among pixels outside the lower left boundary of the target block, such as (a-1, b+n). In the case of the right direction, the lower direction, and the lower right direction, they are omitted because they can be derived from the above description.

[0234] In the above description, an example is given of designating a block including pixels at one position among pixels existing in a block in each direction as a related block, but two or more related blocks may be designated in all or some directions, and two or more pixel positions may be limited therefor.

[0235] (Case 2) F k A block (k is 0 to 2) may mean a block of pixels separated by a predetermined length (off_x, off_y, etc.) in a predetermined direction of horizontal / vertical / diagonal lines, such as (a-off_x, b), (a, b-off_y), (a-off_x, b-off_y). Here, the predetermined length may be an integer of 1 or more, such as 4, 8, 16, etc., and may be set based on the horizontal and vertical lengths of the target block. Alternatively, it may be set based on the horizontal and vertical lengths of the maximum coding block, and various examples of its modification may be possible. The predetermined length may be set implicitly as in the above example, or the relevant syntax element may be generated as an explicit value.

[0236] As another method of designating the related blocks, a block having the same / similar pattern information as the encoding information of the target block may be designated as the related block. In this case, the pattern information does not refer to information to be encoded (or used / predicted, candidate group construction, etc.) at the current stage, but information that has been encoded or has a different attribute / meaning that has been determined. The determined information may be information determined in a previous step in the pattern determination process, or information restored in a previous step in the decoding process.

[0237] For example, when the target block performs inter-frame prediction using a non-translational motion model, a block encoded using a non-translational motion model among previously encoded blocks may be designated as a related block. In this case, motion vector information according to the non-translational motion model of the designated related block may be used to construct a candidate group for motion vector prediction according to the non-translational motion model of the target block.

[0238] In the above example, the motion model information refers to information with different attributes / meanings for checking the sameness / similarity with the target block, and the motion vector according to the motion model (non-translation motion model) of the relevant block can be information for constructing a motion vector prediction candidate group according to the motion model of the target block. In this case, the relevant block may be adjacent to the target block or may not be adjacent to the target block. If there is a pattern with less sameness / similarity with the target block, it may be useful when a block in an area not adjacent to the target block is also designated as a relevant block and used / referenced.

[0239] The following may be considered to determine a related block used / referenced to encode a target block.

[0240] The related block can be determined based on the information to be used / referenced for encoding the target block. Here, the information to be used / referenced for encoding the target block is pixel value information for prediction, mode information related to prediction / transformation / quantization / loop filter / entropy coding, etc., and reference candidate group information related to prediction / transformation / quantization / loop filter / entropy coding, etc.

[0241] In addition, the relevant block can be determined based on the state information of the target block and the image information to which the target block belongs. Here, the state information of the target block can be defined based on the block size, block shape, horizontal / vertical length ratio of the block, and the position in the unit such as picture / segmentation unit (slice, tile, etc.) / maximum coding block. Here, the image information to which the target block belongs can be defined based on the image type (I / P / B), color component (Y / Cb / Cr), etc.

[0242] In addition, the related block can be determined based on the encoding information of the target block. In detail, the related block can be determined based on whether the related block has information that there is sameness / similarity with the target block. Here, the information referenced for the sameness / similarity can be mode information related to prediction / transformation / quantization / loop filter / entropy coding, etc.

[0243] Taking into account all or some of the factors mentioned in the above examples, the category (the above case), number and position of the relevant blocks can be determined. In detail, it can be determined which category is selected, and the number and position of the relevant blocks supported in the selected category can be determined. In this case, the number of blocks supported in each category can be m, n, o, and these can be integers of 0 or greater, such as 0, 1, 2, 3, 5, etc.

[0244] The relevant block (block position) can be determined in a direction such as left, right, top, bottom, upper left, upper right, lower left, lower right, center, etc. centered on the target block (or a block corresponding to the target block in an image that is temporally different from the image to which the target block belongs). For example, the relevant block can be determined based on the block closest to the direction. Alternatively, the relevant block can be determined in a block that otherwise satisfies the direction and a specific range / condition. It may belong to a maximum coding block different from the maximum coding block to which the target block belongs, or it may be a block at a position (e.g., left, top, upper left direction) with a difference based on the horizontal or vertical length of the target block.

[0245] In addition, the related blocks may be determined based on the coding order, and in this case, the coding order may be defined by various scanning methods such as raster scanning and z scanning. As an example, a predetermined number of blocks may be included as related blocks (based on the close coding order), and the predetermined number may be an integer greater than or equal to 0, such as 0, 1, 2, 3, etc. That is, the related blocks may be managed in a memory management method such as FIFO (First In, First Out) according to the coding order, and it may be an example of determining the related blocks (determination to be included in the candidate group) that may occur in (Case 2).

[0246] When one related block is supported, this may mean that only the information of the corresponding block can be used / referenced. In addition, even if multiple related blocks are supported, one piece of information can be derived based on multiple related blocks according to the encoding setting. For example, in inter-frame prediction, three related blocks such as a left block, an upper block, and an upper right block are specified for motion vector prediction to support three motion vectors, but one motion vector can be derived by a method such as the median (or average) of the motion vectors of the three blocks according to the encoding setting to be used as a motion vector prediction value of the target block.

[0247] In the case of the above example, it can be a coding setting that can reduce the occurrence of the best candidate selection information generated by supporting two or more candidates. However, it may be difficult to expect to derive a candidate with a high correlation with the coding information of the target block. Therefore, a method of constructing a candidate group with multiple candidates can be more effective. Of course, as the number of candidates included in the candidate group increases, the amount of information used to represent this may increase, so it is important to construct an effective candidate group.

[0248] Therefore, relevant blocks such as the above-mentioned various examples can be supported, but it may be necessary to specify the best relevant block in consideration of general image characteristics, etc., and construct a candidate group based on this. In the present invention, it is assumed that candidates are constructed by two or more pieces of information from one or more relevant blocks.

[0249] The following shows a candidate group construction based on blocks having a horizontal relationship with a target block and encoding / decoding processing.

[0250] A block to which encoding information for a target block refers is specified (1). A candidate group is constructed in a predetermined order based on the encoding information of the specified block (2). One candidate of the candidate group is selected based on the encoding information of the target block (3). An image encoding / decoding process is performed based on the selected candidate (4).

[0251] In (1), a related block for constructing a candidate group of encoding information for a target block is specified. In this case, the related block may be a block having a horizontal relationship with the target block. It has been described that various categories of related blocks may be included as described above, and in addition to the encoding information of the target block, various information such as state information of the target block may be considered to specify the related block.

[0252] In (2), based on the encoding information of the relevant block specified by the above process, the candidate group is constructed according to a predetermined order. Here, information obtained based on the encoding information of one relevant block may be included in the candidate group, or information obtained based on the encoding information of a plurality of relevant blocks may be included in the candidate group. In this case, the order of constructing the candidate group may support a fixed order, or may support an adaptive order based on various encoding elements (elements to be considered when specifying relevant blocks, etc.).

[0253] In (3), one candidate of the candidate group is selected based on the encoding information of the target block, and in (4), image encoding / decoding processing can be performed based thereon.

[0254] The flowchart may be a process that is checked and executed in units of blocks. Here, in the case of some steps (1, 2), it may be a process that is checked and executed in the initial stage of encoding. Even if the content is not mentioned in (1) to (4) above, since it can be derived through the various embodiments described above, its detailed description will be omitted. Generally, it is difficult to confirm in advance which block has a high correlation with the target block among blocks with a horizontal relationship. A method for pre-confirming what correlation it has with the target block among blocks with a horizontal relationship will be described later. In addition, although (3) describes the case of selecting one candidate in the candidate group, two or more pieces of information may also be selected according to the type of encoding information, encoding settings, etc., and this may be a description that is generally applicable to the present invention.

[0255] The target block and the related block can be one of the units of encoding / prediction / transformation / quantization / loop filter, etc., and the target block and the related block can be set in the same unit. For example, when the target block is a coding block, the related block can also be a coding block, and modification in different unit settings is also possible according to the coding setting.

[0256] Figure 7 Examples of various arrangements of related blocks in a perpendicular relationship to the target block are shown.

[0257] Figure 7 This is a case where recursive tree-based partitioning (QT) is performed, and a description will be given centering on X and A to C blocks. Starting from a basic coding block (CTU.C block) having a partition depth of 0, as the partition depth increases, blocks B (1), X (2), and A (3) can be obtained. Here, blocks placed in a vertical relationship with the target block can be classified into higher blocks (or ancestor blocks) and lower blocks (or successor blocks). In this case, the higher block of the target block X can be a B block or a C block, and the lower block can be an A block. Here, the target block and the related block can be set as a higher block and a lower block, respectively, or can be set as a lower block and a higher block, respectively.

[0258] For example, in this example, if it has a larger value compared to the segmentation depth (k) of the target block, the related block may be a child block (k+1) or a grandchild block (k+2), and if it has a smaller value, the related block may be a parent block (k-1) or a grandparent block (k-2). That is, in addition to defining the vertical relationship between existing blocks, the detailed relationship between blocks can also be confirmed by the segmentation depth.

[0259] Here, as in the above example, one tree method is supported and comparison can be made by a common segmentation depth. When multiple tree methods are supported and a segmentation depth greater than one according to each tree method is supported, detailed relationships can be examined by considering the number of segmentations and each segmentation depth, rather than simple classification as in the above example.

[0260] For example, if QT is performed once in a 4M×4N block, a 2M×2N block can be obtained at a (QT) partition depth of 1, but when BT is performed twice, a 2M×2N block can be obtained. However, the (BT) partition depth is the same as that obtained at 2. In this case, the 4M×4N block can be a parent block (QT) or a grandparent block (BT) for the 2M×2N block, and conversely, can be a child block (QT) or a grandchild block (BT), and the detailed relationship can be determined based on the block partition result.

[0261] In the above examples, the starting unit of the partition is the maximum coding unit (the highest ancestor block, the maximum size that a block can have. Here, it is assumed that it is a coding unit or a block. When a block is a unit of prediction or transformation, it can be understood as a maximum prediction block or a maximum transformation block). It is not possible to have a vertical relationship exceeding the maximum coding unit, but the block areas having a vertical relationship can be freely designed according to a coding setting separate from a block partition setting such as a maximum coding unit. In the present invention, it is assumed that the vertical relationship does not deviate from the maximum coding unit. In addition, the relationship between blocks will be described later with the tree-based partitioning method as the center, but it will be mentioned in advance that the same or similar applications can be made for the index-based partitioning method.

[0262] For the encoding of the target block, the encoding information / reference setting of the related block having a vertical relationship can be used. For ease of explanation, it is assumed that the target block is a lower block of the related block. In this case, the higher block is not an independent unit for performing encoding, prediction, transformation, etc., and can be a temporary unit composed of multiple lower blocks. That is, it is necessary to understand that it is a starting unit or intermediate unit of the block partitioning process for obtaining an independent unit for performing encoding (i.e., a coding / prediction / transformation block, etc. in which the partition is no longer performed).

[0263] For example, the reference pixels of the related block may be used to generate the prediction value of the target block. In detail, in intra prediction, the prediction value of the target block may be obtained by applying a method such as extrapolation, interpolation, averaging, or template matching to the reference pixels of the related block. In addition, in inter prediction, the prediction value of the target block may be obtained by using a method such as template matching with the reference pixels of the related block.

[0264] Here, the reference pixel of the relevant block is not a pixel located in the relevant block, but refers to a pixel obtained by assuming that the relevant block is a unit for performing intra prediction / inter prediction. That is, it means that the pixel of the block having a horizontal relationship with the relevant block (higher block) (for example, the closest to the left, top, upper left, upper right, lower left direction) is used for intra prediction / inter prediction of the target block (lower block).

[0265] For example, for the reference setting of the target block, the reference setting of the related block can be used. In detail, in intra prediction, the MPM candidate group of the related block can be used as the MPM candidate group of the target block. In addition, in inter prediction, the motion prediction candidate group of the related block can be used as the motion prediction candidate group of the target block. That is, even if the candidate group is constructed based on the related block, it means that the candidate group of the related block can also be used as it is without going through a separate candidate group construction for the target block.

[0266] In the above examples, the prediction value and reference setting are determined based on the relevant block rather than the target block, and therefore the problem of using information with poor correlation to the target block to perform encoding may occur. However, since the relevant information is not obtained from completely spaced-apart regions, there may be a certain degree of correlation. Since the processing to be performed in each lower block unit is integrated into a common processing in the higher block, complexity can be reduced. In addition, parallel processing of lower blocks belonging to higher blocks is possible.

[0267] In the above examples, description has been made based on the assumption that the related block is a block having a vertical relationship with the target block, but there may be many related blocks in the image, and at least one related block must be specified for encoding the target block.

[0268] A description of possible support conditions / ranges that blocks having a vertical relationship may have is shown below, and may be determined based on all or some of the factors mentioned in examples to be described later.

[0269] (Case 1) The higher block may be less than or equal to a predetermined first threshold size. Here, the first threshold size may mean the maximum size that the higher block may have. Here, the first threshold size may be expressed in width (W), height (H), W×H, W*H, etc., and W and H may be integers of 8, 16, 32, 64 or more. Here, a block having a first threshold size may be set based on the size of a maximum coding block, a maximum prediction block, and a maximum transform block.

[0270] (Case 2) The lower block may be greater than or equal to a predetermined second threshold size. Here, the second threshold size may mean the minimum size that the lower block can have. Here, the second threshold size may be expressed as width (W), height (H), W×H, W*H, etc., and W and H may be integers of 4, 8, 16, 32 or greater. However, the second threshold size may be set to be less than or equal to the first threshold size. Here, the block having the second threshold size may be set based on the sizes of the smallest coding block, the smallest prediction block, and the smallest transform block.

[0271] (Case 3) The minimum size of the lower block may be determined based on the size of the higher block. Here, the minimum size of the lower block may be determined by applying a predetermined division value (p) or a shift operation value (q. right shift operation) to at least one of the width (W) or height (H) of the higher block (e.g., W%p or H>>q, etc.). Here, the division value may be an integer of 2, 4, 8 or greater, and the shift operation value q may be an integer of 1, 2, 3 or greater.

[0272] (Case 4) The maximum size of the higher block may be determined based on the size of the lower block. Here, the maximum size of the higher block may be determined by applying a predetermined multiplication value (r) or a shift operation value (s. left shift operation) to at least one of the width (W) or height (H) of the lower block (e.g., W*r or H<<s, etc.). Here, the multiplication value may be an integer of 2, 4, 8 or greater, and the shift operation value s may be an integer of 1, 2, 3 or greater.

[0273] (Case 5) The minimum size of the lower block may be determined considering the size of the higher block and the partitioning setting. Here, the partitioning setting may be determined by the partitioning type (tree type), the partitioning depth (common depth, individual depth for each tree), etc. For example, if QT is supported in the higher block, the size of the block in which m divisions are performed may be determined as the minimum size of the lower block, and if BT (or TT) is supported, the size of the block in which n divisions are performed may be determined as the minimum size of the lower block, and if QT and BT (or TT) are supported, the size of the block in which l divisions are performed may be determined as the minimum size of the lower block. Here, m to l may be integers of 1, 2, 3 or greater. The partitioning depth (m) of the tree that is divided into smaller blocks (or divided into a larger number) (due to one partitioning operation) may be set to be less than or equal to the partitioning depth (n) of the tree that is not divided into smaller blocks (or not divided into a larger number). Additionally, the partitioning depth (l) when tree partitioning is mixed may be set to be greater than or equal to the partitioning depth (m) of the tree that is divided into smaller blocks, and may be set to be less than or equal to the partitioning depth (n) of the tree that is not divided into smaller blocks.

[0274] Alternatively, the maximum size of the upper block may be determined in consideration of the size of the lower block and the partition setting. In this description, it may be inversely derived from the above example, and a detailed description will be omitted.

[0275] The following may be considered to determine a related block used / referenced for encoding a target block.

[0276] The relevant block can be determined based on the information to be used / referenced for encoding the target block. Here, the information to be used / referenced for encoding the target block can be pixel value information for prediction and reference candidate group information related to prediction / transformation / quantization / loop filter / entropy coding, etc.

[0277] In addition, the relevant block can be determined based on the state information of the target block and the image information to which the target block belongs. Here, the state information of the target block can be defined based on the block size, block shape, horizontal / vertical length ratio of the block, and the position in a unit such as a picture / segmentation unit (slice, tile, etc.) / maximum coding block. Here, the image information to which the target block belongs can be defined based on the image type (I / P / B), color component (Y / Cb / Cr), etc.

[0278] The number, size, position, etc. of the related blocks may be determined by considering all or some of the factors mentioned in the above examples. In detail, it may be determined whether information of a block having a vertical relationship is used / referenced for encoding the target block, and (when used / referenced) the position and size of the related block may be determined. Here, the position of the related block may be represented by a predetermined coordinate (e.g., upper left coordinate) within the block, and the size of the related block may be represented by a width (W) and a height (H). The related block may be specified by combining these.

[0279] For example, if there is no special range restriction for the lower block, all lower blocks (target blocks) belonging to the relevant block can use / reference the encoding information of the relevant block. Alternatively, if the range for the lower block is limited, if it belongs to the relevant block and is larger than the size for the lower block, the encoding information of the relevant block can be used / referenced. In addition, when two or more relevant blocks are supported, selection information of the relevant block can be additionally generated.

[0280] The following shows the candidate group construction and encoding / decoding processing based on blocks having a vertical relationship with the target block.

[0281] A basic block for specifying a block to be referenced is determined (1). A block to be referenced for encoding information of a target block is specified based on the determined basic block (2). A candidate group is constructed in a predetermined order using the encoding information of the specified block (3). One candidate in the candidate group is selected based on the encoding information of the target block (4). An image encoding / decoding process is performed based on the selected candidate (5).

[0282] In (1), a block (basic block) used as a standard for constructing a candidate group related to encoding information of the target block is determined from the target block or the first related block. Here, the first related block may be a block having a vertical relationship with the target block (here, a higher block).

[0283] In (2), when the basic block is determined by the above process, a second related block for constructing a candidate group related to the encoding information of the target block is specified. Here, the second related block may be a block having a horizontal relationship with the basic block. In (3), the candidate group is constructed in a predetermined order using the encoding information of the second related block specified by the above process. The description of the block having a horizontal relationship will be omitted because it can be derived not only by the candidate group construction based on the block having a horizontal relationship and the encoding / decoding process, but also by the various embodiments described above.

[0284] In (4), one candidate in the candidate group is selected based on the encoding information of the target block, and in (5), image encoding / decoding processing can be performed based thereon.

[0285] In this case, when determining the higher block and the lower block based on the vertical relationship setting between the blocks, since the process of constructing the candidate group based on the higher block is performed only once, the lower block can use / borrow the process of constructing the candidate group based on the higher block. That is, the flowchart can be a configuration that can be generated in the block in which encoding / decoding is first performed. If the construction of the candidate group based on the higher block has been completed and the basic block is determined as a related block in a certain order (2, 3), the candidate group that has been constructed can be simply used / borrowed.

[0286] In the above example, if the candidate group is constructed based on the higher block, the lower block is described as a simple use / borrow configuration, but is not limited thereto.

[0287] For example, even if a candidate group is constructed based on a higher block, some candidates may be fixed regardless of the lower block, while other candidates may be adaptive based on the lower block. That is, this means that some candidates may be deleted / added / changed based on the lower block. In this case, the deletion / addition / change may be performed based on the position and size of the lower block within the higher block.

[0288] That is, even if a basic block is determined as a related block in some steps (2, 3), a candidate group in which all or some modifications to the already constructed candidate group are reflected can be constructed.

[0289] The target block and the related block may be one of the units of coding / prediction / transformation / quantization / loop filter, etc., and the target block may be the same unit as the related block or a higher unit. For example, when the target block is a coding block, the related block may be a coding block, and when the target block is a coding block, the related block may be a prediction block or a transformation block.

[0290] Figure 8 Various arrangement examples of related blocks having a vertical relationship and a horizontal relationship with the target block are shown.

[0291] Figure 8 A case where a recursive tree-based partitioning (quadtree) is performed is shown, and the description will be centered around X, A to G, and p to t blocks. Starting from a basic coding block (CTU) with a partition depth of 0, as the partition depth increases, q / r / t (1) blocks, p / D / E / F / s (2) blocks, A / B / C / G / X (3) blocks can be obtained. Here, they can be classified into blocks having a vertical relationship with the target block and blocks having a horizontal relationship with the target block. In this case, the related blocks (higher blocks) having a vertical relationship with the target block X can be p blocks and q blocks (excluding CTU), and the related blocks having a horizontal relationship can be A blocks to G blocks.

[0292] Here, in the case of some related blocks, there are not only blocks having a smaller value than the segmentation depth (k) of the target block, but also blocks having a larger value than the segmentation depth (k) of the target block. It is assumed that the present embodiment targets the above blocks and the target block has the maximum segmentation depth (i.e., no more segmentation).

[0293] In block segmentation, the segmentation result is determined according to the characteristics of the image. Block segmentation can be minimized in flat parts such as background or areas with small time changes. Block segmentation can be performed extensively in parts with complex patterns or areas with fast time changes.

[0294] Through the above examples of the present invention, it has been mentioned that blocks with many horizontal relationships or vertical relationships can be used / referenced for encoding a target block. Through the examples to be described later, various examples of methods for more effectively using blocks with horizontal relationships and blocks with vertical relationships around the target block will be presented. Therefore, it is a premise that the description of the above horizontal relationships and vertical relationships can be applied to the content to be described later in the same or similar manner.

[0295] Next, various cases of correlation between blocks placed in a horizontal relationship according to the type (method) of block partitioning will be described. In this case, it is assumed that QT, BT, and TT are supported as partitioning schemes, BT is a symmetric partition (SBT), and TT is partitioned at a ratio of 1:2:1. In addition, it is assumed that blocks placed in a general horizontal relationship can have high or low correlation (general relationship). In each case, it is assumed that only one of the described tree methods is supported.

[0296] Fig. 9 is an example diagram of block partitioning obtained according to the tree type. Here, p to r represent examples of block partitioning of QT, BT and TT. It is assumed that block partitioning is performed when the encoding of the block itself is invalid due to different characteristics of some areas in the block.

[0297] In the case of QT(p), it is split into two horizontally and vertically, respectively, and it can be seen that at least one of the four sub-blocks has different characteristics. However, since 4 sub-blocks are acquired, it is impossible to know which specific sub-block has different characteristics.

[0298] For example, blocks A to D may all have different characteristics, only one of blocks A to D may have different characteristics, and the rest may have the same characteristics. Block A and block B may have the same characteristics, and block C and block D may have the same characteristics. Block A and block C may have the same characteristics, and block B and block D may have the same characteristics.

[0299] If block A and block B, block C and block D each have the same characteristics and support BT, horizontal segmentation can be performed in BT. However, if it is segmented by QT like p, it can be seen that block A and block B, block C and block D have different characteristics from each other. However, in this example, since it is assumed that only QT is supported, the correlation between blocks cannot be accurately found.

[0300] If only QT is supported and QT is performed on higher blocks with a partition depth of 1 difference around the target block (one of the sub-blocks), the correlation between the target block and related blocks (blocks in the sub-blocks except the target block) may be high or low.

[0301] BT(q) is split into two in either the horizontal or vertical direction, and it can be seen that the two sub-blocks (E block and F block) have different characteristics. If the characteristics between the sub-blocks are the same or similar, it can be derived as defined above assuming that they are not split. If only BT is supported and BT is performed on a higher block with a split depth of 1 difference around the target block, the correlation between the target block and the related block may be low.

[0302] In the case of TT(r), it is divided into three in either the horizontal or vertical direction, and it can be seen that at least one of the three sub-blocks has different characteristics. However, since 3 sub-blocks are obtained, it is impossible to know which specific sub-block has different characteristics.

[0303] For example, blocks G through I may all have different characteristics. Blocks G and H may have the same characteristics, while block I may have different characteristics. Blocks H and I may have the same characteristics, while block G may have different characteristics.

[0304] If blocks G and H have the same characteristics, block I has different characteristics, and BT is also supported (asymmetric), vertical partitioning (3:1) can be performed in BT, but if it is partitioned by TT like r, it can be seen that block G has different characteristics from block H and block I. However, since it is assumed that only TT is supported in this example, the correlation between blocks cannot be accurately found. If only TT is supported and TT is performed on higher blocks with a partition depth of 1 difference around the target block, the correlation between the target block and the related blocks may be high or low.

[0305] It has been described above that in the case of QT and TT in the partitioning scheme, the correlation between blocks may be high or low. Assume that only a partitioning scheme (e.g., QT) is supported, and the coding information of the remaining sub-blocks except one sub-block (e.g., D block) is known. If the coding information of the remaining sub-blocks A to C except the D block is the same or similar, the D block may have different characteristics and can therefore be segmented by QT. In this way, relevant information can also be identified by checking the coding information of the lower block belonging to the higher block. However, since the possibility of occurrence is low and it may be a complicated situation, the description of this will only mention the possibility, and its detailed description will be omitted. For reference, in the above case, since the D block has a low correlation with the block having a horizontal relationship, the coding information of the block having a vertical relationship can be used / referenced, and the coding information refers to the candidate group construction information of the higher block (including the blocks of A and D) of the D block.

[0306] In the above example, the correlation between sub-blocks when one tree split is supported for a higher block has been described. That is, when the encoding information of a block having a horizontal relationship is used to encode a target block, the encoding information of the related block can be used / referenced more efficiently by checking the split state (path) between the target block and the related block. For example, when constructing a candidate group, information about a block determined to have a low correlation can be excluded, or a low priority can be assigned to it. Alternatively, information and reference settings of blocks having a vertical relationship can be used.

[0307] In this case, the case where one tree split is supported may include the case where only one tree method is supported for block splitting, and even if multiple tree splits are supported, it may include the case where only one tree split is supported by the maximum value, the minimum value, the maximum split depth of the block according to each tree method, and wherein the block split setting of the tree not allowed at the previous split depth is not supported at the later split depth. That is, only QT is supported and QT can be used for splitting, and QT / BT / TT is supported, but only BT is possible at this stage, so BT can be used for splitting.

[0308] The following description will focus on the case where the correlation between each block is checked when multiple tree partitions are supported.

[0309] Fig.10 is an example diagram of the partitioning obtained due to QT, BT, TT. In this example, it is assumed that the maximum coding block is 64×64 and the minimum coding block is 8×8. In addition, it is assumed that the maximum value of the block supporting QT is 64×64 and the minimum value is 16×16, the maximum value of the block supporting BT is 32×32 and the minimum value is 4, which is one of the horizontal / vertical lengths of the block, and the maximum partition depth is 3. In this case, it is assumed that the partitioning setting for TT is determined together with BT (they are used in a bundle). It is assumed that QT and BT (QT, BT) are supported for higher blocks (A to M), asymmetric BT (ABT) is additionally supported for the lower left block (N to P) (QT, BT<or SBT>, ABT), and TT (QT, SBT, ABT, TT) is additionally supported for the lower right block (Q to S).

[0310] (Basic blocks: blocks including B, C, D, and E)

[0311] Blocks B to E may be sub-blocks that can be obtained through QT (division once), and may be sub-blocks that can be obtained through BT (one vertical division + two horizontal divisions, or one horizontal division + two vertical divisions. The number of divisions is 3 times).

[0312] Since the maximum block size supporting QT is 16×16, blocks B to E cannot be obtained through QT, and blocks B to E may be sub-blocks partitioned by BT. In this example, horizontal partitioning (B+C / D+E) is performed in BT, and vertical partitioning (B / C / D / E) is performed in each region. Therefore, as described above, blocks B and C, blocks D and E obtained as BT from the same higher block (parent block, partition depth difference is 1) may have low correlation.

[0313] In addition, since blocks B and D or blocks C and E are partitioned, the correlation between blocks B and D or blocks C and E may be low. This is because if the correlation is high, vertical partitioning is performed only in BT, and partitioning may not be performed in each region.

[0314] In the above example, it is mentioned that the correlation between sub-blocks obtained through BT is low, but this is limited to sub-blocks belonging to the same higher block (parent block, segmentation depth difference is 1), but in this example, the correlation between blocks can be checked by extending to the same higher block (grandparent block, segmentation depth difference is 2).

[0315] (Basic blocks: blocks including J, K, L, and M)

[0316] Blocks J to M may be sub-blocks that may be obtained through QT or sub-blocks that may be obtained through BT.

[0317] Since both QT and BT can be supported, the tree method can be selected as the best segmentation type. This is an example in which QT is performed. Through the above example, it is mentioned that the correlation between sub-blocks obtained by QT may be high or low. However, in this example, since multiple tree segmentations are supported, the correlation between sub-blocks can be determined differently.

[0318] Each of block J and block K or block L and block M may have a low correlation, and each of block J and block L or block K and block M may have a low correlation. This is because if the correlation between blocks adjacent in the horizontal direction and the vertical direction among blocks J to block M is high, even if QT is not performed and BT is performed, a region with a high correlation may not be segmented.

[0319] In the above example, it is mentioned that the correlation between sub-blocks obtained by QT may be high or low, but this is when a single tree method is supported, and in this example, when multiple tree methods are supported, the correlation between blocks can be checked based on the number of various cases of block segmentation.

[0320] (Basic block: including N, O, P blocks)

[0321] Blocks N to P may be sub-blocks (in a ratio of 2:1:1) obtained through BT (one horizontal partition+one horizontal partition).

[0322] (When only symmetric BT is supported <sbt>(When N blocks and O blocks are obtained from higher blocks by BT, the correlation with the area where O blocks and P blocks are bundled may be low. However, it cannot be said that N blocks have low correlation with O blocks and P blocks. Of course, N blocks can have low correlation with O blocks and P blocks. Alternatively, N blocks can have high correlation with O blocks and can have low correlation with P blocks, and vice versa, N blocks can have low correlation with O blocks and can have high correlation with P blocks.

[0323] In this example, asymmetric BT can be supported <abt>If the N block and the O block have high correlation, the regions of the N block and the O block are grouped, and horizontal partitioning of the asymmetric BT may be performed at a ratio of 3:1. However, since BT (SBT) is performed twice, the correlation between the N block and the O block may be low.

[0324] (Basic block: including Q, R, S blocks)

[0325] The Q block to the S block may be sub-blocks obtained through TT (one horizontal partition).

[0326] (When only TT is supported) Q blocks and S blocks may have high or low correlation. In this example, asymmetric BT can be supported. If the correlation between Q blocks and R blocks is high, the regions of Q blocks and R blocks are grouped, and horizontal segmentation of asymmetric BT can be performed at a ratio of 3:1. However, since TT is performed, the correlation between Q blocks and R blocks may be low.

[0327] As in the above example, the correlation between the target block and the related block having a horizontal relationship with the target block can be estimated based on the supported segmentation method and segmentation setting. Please see various cases about the correlation between blocks through the following.

[0328] Fig.11 is an example graph for examining the correlation between blocks based on the segmentation method and segmentation settings.

[0329] (a) to (c) are cases where QT, BT, and TT are performed respectively, and only QT, BT, and TT may be supported for higher blocks. As described above, in the case of QT and TT, the correlation between adjacent blocks (A and B or A and C) in the horizontal direction or the vertical direction may be high or low. This is called a general relationship. Meanwhile, in the case of BT, the correlation between blocks A and B adjacent in the horizontal direction or the vertical direction may be low. This is called a specific relationship.

[0330] (d) is a case where QT is performed, and may be a case where QT and BT may be supported. In this example, the correlation between blocks (A and B or A and C) adjacent in the horizontal direction or vertical direction may be a low-specific relationship. If the correlation is high, BT may have been applied instead of QT.

[0331] (e) is a case where BT is performed (one vertical split + one horizontal split), and it may be a case where QT and BT can be supported. In the case of A and B, since they are split from the same higher block by BT, this may be a specific relationship with low correlation. In the case of A and C, if the areas under A and C are highly correlated, they can be grouped together and splitting can be performed. However, this may be a case where C is split due to coding cost reasons. Of course, since there may be other cases, A and C may be a general relationship with high or low correlation.

[0332] (f) is a case where BT (one vertical split + one vertical split) is performed, and it may be a case where BT and TT can be supported. In the case of A and C, since they are split from the same higher block by BT, this may be a specific relationship with low correlation. In the case of A and B, there is a case where A and B are grouped together and split (the part corresponding to 2 in the 1:2:1 area of ​​TT), but other splitting of the left area occurs due to TT. In this case, since it is difficult to accurately determine the correlation, A and B may have a general relationship in which the correlation is high or low.

[0333] (g) is a case where TT (one vertical segmentation) is performed, and may be a case where BT (or SBT), ABT, and TT may be supported. In this example, the correlation between blocks (A and B or A and C) adjacent in the horizontal direction or the vertical direction may be a low specific relationship. If the correlation is high, ABT may have been applied instead of TT.

[0334] (h) is a case where QT and BT are performed (two vertical splits based on BT), and QT and BT can be supported <1> and may be cases where other TTs may be supported <2> In the case of A and B, <1> In the case of , there is no case where the basic block is split in a state where A and B are bound, so the correlation may be high or low in general. However, in <2> In the case of , there is a case where the basic block is split in a state where A and B are bound (after BT horizontal split, BT vertical split on the top side, TT vertical split on the bottom side), but it is still split using QT and BT. Therefore, this may be a specific relationship with low correlation. In this example, in order to check the relationship between blocks, the number of cases about block splitting that can be obtained from the same higher block with a difference of 1 or more (the difference is 2 in this example) can be checked.

[0335] Through the above various examples, it has been confirmed that the correlation between blocks is measured to use / reference the related block having a horizontal relationship with the target block. In this case, the related block belonging to the same space as the target block and adjacent to the target block can be used as the target. In particular, the target block and the related block can be blocks adjacent to each other in the horizontal direction or the vertical direction.

[0336] The correlation between blocks can be grasped / estimated based on various information. For example, the correlation between blocks can be checked based on the state information (size, shape, position, etc.) of the target block and the related blocks.

[0337] Here, as an example of determining the correlation based on the size of the block, if the predetermined length (horizontal length or vertical length) of the related block adjacent to the boundary (horizontal or vertical) in contact with the target block is greater than or equal to the predetermined length of the target block, the correlation between the blocks may be very high or slightly low, and this is referred to as a general relationship A. If the predetermined length of the related block is less than the predetermined length of the target block, the correlation between the blocks may be slightly high or very low, and this is referred to as a general relationship B. In this case, the horizontal length of each block may be compared when the horizontal boundary (blocks on the top side) is touched, and the vertical length of each block may be compared when the vertical boundary (blocks on the left side) is touched.

[0338] Here, as an example of determining correlation based on the shape of a block, when the target block has a rectangular shape, the correlation with the related blocks adjacent to the longer boundary in the horizontal length and the vertical length is a general relationship A, and the correlation with the related blocks adjacent to the shorter boundary can be a general relationship B.

[0339] The above description may be some examples of determining the correlation between blocks based on the state information of the blocks, and various modifications are possible. The correlation between blocks may be grasped based not only on the state information of the blocks but also on various information.

[0340] The following describes a process of checking the correlation of a related block having a horizontal relationship with a target block.

[0341] (Check the block segmentation settings in the image)

[0342] <1> Various setting information about block segmentation in an image can be checked. Support range information such as maximum block size, minimum block size, etc. of a unit such as coding / prediction / transform (assuming the target block is a coding unit in this example) can be checked. As an example, it can be checked that the maximum coding block is 128×128 and the minimum coding block is 8×8.

[0343] <2> Check the supported partitioning schemes, and check the conditions supported by each partitioning scheme such as the maximum block size, the minimum block size, and the maximum partition depth. For example, the maximum size of a block supporting QT can be 128×128, the minimum size can be 16×16, the maximum size of a block supporting BT and TT can be 128×128 and 64×64 respectively, the minimum size can usually be 4×4, and the maximum partition depth can be 4.

[0344] <3> Check the settings such as priority to be assigned to the partitioning scheme. For example, if partitioned by QT, QT may be supported for its lower blocks (sub-blocks), and if not partitioned by QT and partitioned by a different scheme, QT may not be supported for the lower blocks.

[0345] <4> When multiple segmentation schemes are supported, conditions for prohibiting certain segmentation can be checked to avoid overlapping results according to the segmentation schemes. For example, after performing TT, vertical segmentation of BT for the center area can be prohibited. That is, in order to prevent the possible overlap of segmentation results according to each segmentation scheme, setting information for prohibiting segmentation is checked in advance.

[0346] By inspection <1> to <4> All or some of the above and other setting information attached can be used to check candidate blocks that can be obtained in the image. This can be referenced to identify candidate blocks that can be obtained based on the target block and related blocks described later.

[0347] (Check block information)

[0348] State information such as the size, shape, position, etc. of the target block and the related blocks may be checked. Here, it is checked whether the position of the block is located at a unit boundary such as a picture, a slice, a tile group, a tile, a block, or a block, or is located inside.

[0349] The block in the unit can be set as the maximum coding block, and the maximum coding block can be the higher block (highest ancestor block) of the target block, but the maximum coding block is a unit that is collectively divided in the picture unit, not a form obtained according to the characteristics of the image. Therefore, if it belongs to a maximum coding block different from the maximum coding block to which the target block belongs, the correlation between the blocks cannot be checked, so it is necessary to check whether it belongs to the boundary.

[0350] In addition, since other units such as pictures and slices are composed of integer multiples of the maximum coding block or have settings that cannot be referenced, the process of checking the correlation can be performed only when it is not a boundary. In other words, the correlation can be checked only when it is not a boundary.

[0351] The size, shape, and position of the target block and the related blocks can be used to check the segmentation state of each block or the segmentation path according to which each block is obtained. This will be described in detail later.

[0352] (Check partition status and check common upper blocks)

[0353] The segmentation states of the target block and the related blocks may be checked. Here, the segmentation state may mean a segmentation path according to which each block is obtained. A process of checking a higher block of each block may be performed by checking the segmentation state, wherein the higher block may mean a block having a vertical relationship with each block. A process of checking a higher block obtained based on the segmentation path and state information of each block such as size, shape, and position is performed.

[0354] For example, state information may be obtained, such as a target block position (at the top left) of (32, 32), a width and height of 8×8, a segmentation depth of p, and a segmentation path of (QT / 1-BT / h / 0-BT / v / 1). State information may be checked, such as a related block position of (24, 32), a width and height of 8×8, a segmentation depth of q, and a segmentation path of (QT / 1-BT / h / 0-BT / v / 0). Here, the segmentation path may be expressed as a segmentation scheme / segmentation direction (h is horizontal, v is vertical, and omitted if not present) / segmentation position (0 to 3 for QT, 0 to 1 for BT, etc.).

[0355] In this example, state information may be obtained, for example, a position of an upper block (parent block, segmentation depth difference of 1) of a target block is (24, 32), a width and height are 16×8, a segmentation depth is p-1, and a segmentation path is (QT / 1-BT / h / 0). In this example, an upper block of a related block (segmentation depth of q-1) may be the same as an upper block of a target block.

[0356] For example, state information such as the target block position is (128, 64), the width and height are 16×32, the segmentation depth is p, and the segmentation path is (QT / 3-QT / 2-BT / v / 1) can be obtained. State information such as the relevant block position is (120, 64), the width and height are 8×32, the segmentation depth is q, and the segmentation path is (QT / 3-QT / 2-BT / v / 0-BT / v / 1) can be obtained.

[0357] In this example, state information may be obtained, for example, the position of the upper block (parent block, the segmentation depth difference is 1) of the target block is (112, 64), the width and height are 32×32, the segmentation depth is p-1, and the segmentation path is (QT / 3-QT / 2).

[0358] On the other hand, state information such as the position (112, 64), width and height of the upper block (parent block, segmentation depth difference of 1) of the relevant block, the segmentation depth of q-1, and the segmentation path of (QT / 3-QT / 2-BT / v / 0) can be obtained. State information such as the position (112, 64), width and height of the upper block (grandparent block, segmentation depth difference of 2) of the relevant block, the segmentation depth of q-2, and the segmentation path of (QT / 3-QT / 2) can be obtained, and it can be seen that this is the same upper block as the upper block (parent block) of the target block.

[0359] As in the above example, a process of checking an upper block of each block may be performed based on the division state, and a process of checking a common upper block may be performed.

[0360] In summary, a higher block having a segmentation depth difference of 1 or more with respect to the target block and the related block may be checked. As an example, a higher block having a segmentation depth difference of c with respect to the target block may be the same as a higher block having a segmentation depth difference of d with respect to the related block. In this case, c and d may be integers of 1, 2 or more, and c and d may be the same or different.

[0361] Here, since it is difficult to grasp the complexity or correlation, it may not be necessary to check the higher blocks with a large partition depth difference. For example, when the higher blocks are common in the largest coding block, it may be difficult to check the correlation between the blocks.

[0362] To this end, there may be a predetermined first threshold (maximum value) for c and d, and the first threshold may be an integer of 1, 2 or more. Alternatively, there may be a predetermined second threshold related to the sum of c and d, and the second threshold may be an integer of 2, 3 or more. That is, when the threshold condition is not met, the correlation between the blocks is not checked.

[0363] There may be various methods for checking whether the target block and the related block have the same higher block. For example, it may be checked by information about a predetermined position of the higher block or information based on the width and height of the block. In detail, it may be checked whether the higher blocks are the same based on the upper left coordinate of the higher block and information based on the width and height of the block.

[0364] (Check available candidates)

[0365] When a common higher block for a target block and a related block is obtained, the number of cases of various block segmentation that can be obtained from the corresponding higher block can be checked. This can be checked based on the block segmentation setting and segmentation state of the higher block. Since this has been mentioned through various examples above, detailed description will be omitted.

[0366] (Check for dependencies)

[0367] In order to check the correlation between blocks, the following can be checked in this example: In this example, it is assumed that the maximum value of the segmentation depth difference for each block is 2.

[0368] <1> If the higher block has a partition depth difference of 1 with respect to the target block and the related block, the partitioning schemes that can be supported are checked.

[0369] When only one partitioning scheme is available, the correlation can be determined according to the partitioning scheme. If it is QT or TT, it can be set to a general relationship (the correlation may be high or low), and if it is BT, it can be set to a specific relationship.

[0370] If multiple partitioning schemes are available, it is checked whether the partitioning is performed in a form in which the target block and the related block are combined. If so, it is set to a specific relationship, and if not, it is set to a general relationship.

[0371] <2> In case the higher block has a partition depth difference of 2 with respect to at least one of the target block and the related block, supportable partitioning schemes are checked.

[0372] If only one partitioning scheme is available, it can be set as a general relationship regardless of the partitioning scheme.

[0373] If multiple partitioning schemes are available, it is checked whether the partitioning is performed in a form in which the target block and the related block are combined. If so, it is set to a specific relationship, and if not, it is set to a general relationship.

[0374] The above examples are some cases for checking the correlation between blocks, and are not limited thereto, and various modifications and additions are possible. The correlation between blocks checked through the above process can be used / referenced for encoding a target block by referring to it.

[0375] In summary, in order to determine the correlation between the target block and the related blocks, all or some of the processes, such as (checking the block segmentation in the image), (checking the block information), (checking the segmentation state and checking the common higher block), (checking the candidates that can be obtained), (checking the correlation), etc., can be used, and the process of determining the correlation can be performed in various orders instead of the order listed above. In addition, it is not limited to the above-mentioned cases, and the correlation can be determined by changing some components or combining other components. Alternatively, the process of determining the correlation of other configurations can be performed, and the information of the related blocks can be used / referenced for encoding the target block based on the correlation between the blocks determined by the above process.

[0376] The correlation determined by the above process may not be an absolute fact about the characteristics between blocks, and may be prediction information that estimates the correlation between blocks in consideration of block segmentation, etc. Therefore, since it may be information that refers to a candidate group for constructing encoding information for a target block, a correlated block with a low correlation may not be included in the candidate group. Considering the possibility that the determined correlation is incorrect, the priority on the candidate group construction may be set to a lower priority, or the candidate group information of a higher block with a vertical relationship may be borrowed. In addition, in the above example, although it is assumed that the correlation between blocks is classified into two types, two, three or more classification categories may be supported.

[0377] Whether to use / reference the correlation between blocks for encoding the target block (construction of the candidate group, etc.) may be explicitly determined in units such as order, picture, slice, tile group, tile, tile, block, etc., or may be implicitly determined based on the encoding setting. Next, examples of various information constituting the encoding setting will be described.

[0378] Here, whether to refer to the correlation between blocks can be determined based on the information to be used / referenced for encoding the target block. For example, the correlation between blocks can be considered to construct an intra-frame prediction mode candidate group, and the correlation between blocks can be ignored to construct a candidate group for motion vector prediction representing a non-translational motion model in inter-frame prediction.

[0379] Here, it can be determined whether to refer to the correlation between blocks based on the state information of the target block and the image information to which the target block belongs. Here, the state information of the target block can be defined based on the block size, block shape, horizontal length / vertical length ratio of the block, and the position in the unit such as picture / segmentation unit (slice, tile, etc.) / maximum coding block. Here, the image information to which the target block belongs can be defined based on the image type (I / P / B), color component (Y / Cb / Cr), etc. For example, the correlation between blocks can be referenced only when the size of the block belongs to a predetermined range, and the correlation between blocks can be not referenced when the size of the block is not within the predetermined range. In this case, the predetermined range can be defined by a first threshold size (minimum value) and a second threshold size (maximum value), and each threshold size can be represented as W, H, W×H, W*H based on the width (W) and height (H). W and H can be integers of 1 or greater, such as 4, 8, 16.

[0380] Here, whether to refer to the correlation between blocks can be determined according to the category of the related blocks (which can be derived from the description of the positions of the related blocks having a horizontal relationship). For example, the related block may belong to the same space as the target block, and the related block and the target block may be adjacent blocks. Even if the related block and the target block belong to the same space, in the case of non-adjacent related blocks, the correlation may not be referred to.

[0381] The encoding settings may be defined in consideration of all or some of the factors mentioned in the above examples, and thus, whether to refer to the correlation between blocks may be implicitly determined.

[0382] The following shows the candidate group construction and encoding / decoding processing based on blocks having a horizontal relationship with the target block.

[0383] A correlation between a target block and a block in which a reference possibility exists is checked (1). A block to which encoding information for the target block refers is specified based on the correlation (2). A candidate group is constructed in a predetermined order using the specified encoding information (3). One candidate in the candidate group is selected based on the encoding information of the target block (4). An image encoding / decoding process is performed based on the selected candidate (5).

[0384] In (1), the correlation between the target block and the block that can be considered as the related block is checked. In (2), a block for constructing a candidate group of encoding information for the target block is specified based on the correlation checked by (1). That is, it may mean that whether to include as a related block is determined based on the checked correlation result. Of course, the contents of the specification describing the related blocks having a horizontal relationship as described above can be considered together in this example.

[0385] In (3), a candidate group is constructed according to a predetermined order using the encoding information of the related blocks specified by the above process. In this case, an adaptive order considering the related blocks included or not included by (2) can be supported. In (4), one candidate in the candidate group is selected based on the encoding information of the target block, and in (5), an image encoding / decoding process can be performed based on this.

[0386] In the flowchart, blocks determined to have low correlation based on the correlation may not be included as relevant blocks.

[0387] Another example of candidate group construction and encoding / decoding processing based on blocks having a horizontal relationship with a target block is shown below.

[0388] A block to which encoding information for a target block is to be referred is designated (1). A correlation between the target block and the designated block is checked (2). A predetermined order is determined based on the encoding information of the target block and the correlation checked by (2), and a candidate group is constructed accordingly (3). One candidate in the candidate group is selected based on the encoding information of the target block (4). An image encoding / decoding process is performed based on the selected candidate (5).

[0389] In (1), a related block to be used for constructing a candidate group of encoding information for a target block is specified. In (2), a correlation between the target block and the related block is checked. In (3), an order of included candidates may be determined based on the correlation checked by (2).

[0390] For example, if the correlation is high or low, a predetermined order may be applied. If the correlation is high, an order in which the related blocks have a high priority may be applied, and if the correlation is low, an order in which the related blocks have a low priority may be applied.

[0391] Subsequently, in (3), after the order of constructing the candidate group is determined through the above process, the candidate group can be constructed according to the order. In (4), one candidate in the candidate group is selected based on the encoding information of the target block, and in (5), the image encoding / decoding process can be performed based on this.

[0392] In the flowchart, the order of including the candidates may be adaptively set based on relevance.

[0393] An example of candidate group construction and encoding / decoding processing based on a block having a horizontal relationship or a vertical relationship with a target block is shown below. Here, it is assumed that a basic block for specifying a reference block is set as a target block.

[0394] The correlation between the target block and the block in which the reference possibility exists is checked (1).

[0395] (The number of blocks judged to have low correlation with the target block is less than / predetermined number or less)

[0396] A block referred to by encoding information for a target block is specified based on the correlation (2A). A candidate group is constructed in a predetermined order using the specified encoding information (3A).

[0397] (The number of blocks judged to have low correlation with the target block exceeds / predetermined number or more)

[0398] A basic block for specifying a reference block is changed to a predetermined higher block (2B). A block referenced by encoding information for a target block is specified based on the changed basic block (3B). A candidate group is constructed in a predetermined order using the encoding information of the specified block (4B).

[0399] One of the candidate groups is selected based on the encoding information of the target block (5). Image encoding / decoding processing is performed based on the selected candidate group (6).

[0400] In the flowchart, one of the order (P) of 1-2A-3A-5-6 or the order (Q) of 1-2B-3B-4-5-6 may be determined according to the correlation determination result. Specifically, when there are several blocks determined to have low correlation with the target block, the remaining blocks except the corresponding block are designated as the relevant blocks, and when there are many blocks determined to have low correlation with the target block, by changing the basic block constructed by the candidate group to a higher block compared with the target block, the block having a horizontal relationship with the higher block is designated as the relevant block.

[0401] In the case of the P order, since some of the flowcharts in the above flowcharts in which the blocks determined to have low correlation are not included in the related blocks are the same, the detailed description is omitted. The Q order can be a configuration based on a combination of blocks with a vertical relationship and a candidate group construction. When the block adjacent to the target block is composed of blocks with low correlation, an example of a candidate group can be constructed by changing the block unit that is the basis of the candidate group construction. In the following description, the redundant description in the previous description is omitted, and the difference is focused on.

[0402] In (2B), the block used as the candidate group standard is changed to the first relevant block. Here, the first relevant block may be a block having a vertical relationship with the target block (here, a higher block).

[0403] In (3B), a second related block used for constructing a candidate group of encoding information for a target block is specified. Here, the second related block may be a block having a horizontal relationship with a basic block, and the basic block is a higher block. In (4B), the candidate group is constructed in a predetermined order using the encoding information of the second related block specified by the above process.

[0404] Here, the criterion for determining that the correlation with the target block is low is the case of division by the number of blocks in the flowchart, but various criteria to be determined may be set.

[0405] Various relationships between blocks are described through the above various examples, and the case where encoding / decoding is performed using the example is described. When an algorithm is described based on the relationship between the above blocks in various encoding / decoding processes to be described later, it should be understood that even without adding detailed description, the settings suggested by the above various embodiments can also be applied in the same or similar manner.

[0406] (Intra-frame prediction)

[0407] In the image encoding method according to an embodiment of the present invention, intra-frame prediction may be configured as follows. The intra-frame prediction of the prediction unit may include: a reference pixel construction step, a prediction block generation step, a prediction mode determination step, and a prediction mode encoding step. In addition, the image encoding device may be configured to include: a reference pixel construction unit, a prediction block generation unit, and a prediction mode encoding unit that implement the reference pixel construction step, the prediction block generation step, the prediction mode determination step, and the prediction mode encoding step. Some of the above-mentioned processes may be omitted or other processes may be added, and may be changed in an order other than the above-mentioned order.

[0408] In a video decoding method according to an embodiment of the present invention, intra prediction may be configured as follows. The intra prediction of a prediction unit may include: a prediction mode decoding step, a reference pixel construction step, and a prediction block generation step. In addition, an image decoding device may be configured to include: a prediction mode decoding unit, a reference pixel construction unit, and a prediction block generation unit that implement the prediction mode decoding step, the reference pixel construction step, and the prediction block generation step. Some of the above processes may be omitted or other processes may be added, and may be changed in an order other than the above order.

[0409] Since the reference pixel construction unit and the prediction block generation unit of the image decoding device perform the same roles as the corresponding configurations of the image encoding device, detailed description is omitted, and the prediction mode decoding unit is performed by reversely using the method used by the prediction mode encoding unit.

[0410] (Sub-block based intra prediction)

[0411] Intra-frame prediction can be performed by setting a coding block as a prediction block, or can be performed by dividing the coding block into one or more prediction blocks. Here, the coding block refers to a unit obtained by a separate division process based on a tree or an index, and refers to a process separate from the division into prediction blocks. Here, performing intra-frame prediction by division into one or more prediction blocks is called intra-frame prediction in sub-block units. Here, it is assumed that one prediction mode occurring in a coding block is shared with all prediction blocks in the coding block, but it is not limited to this.

[0412] In intra prediction of a sub-block unit, information indicating whether to segment may be generated. For example, if the information is 1, it means that segmentation is performed, and if the information is 0, it means that segmentation is not performed. When segmentation is performed, information indicating a segmentation direction such as horizontal or vertical may be generated. For example, if the information is 0, it means horizontal segmentation, and if the information is 1, it means vertical segmentation, and the opposite configuration may be possible. Alternatively, it may be determined based on state information such as the size, shape, and position of the target block.

[0413] For example, when the width of the target block is greater than a predetermined threshold size, it can be split horizontally, and when the height of the target block is greater than a predetermined threshold size, it can be split vertically. Here, the threshold size can be an integer of 2 or greater, such as 2, 4, 8, 16, 32.

[0414] Information about the number of blocks to be partitioned may be generated additionally and may be an integer of 2, 4, 8 or more. Alternatively, it may be implicitly determined according to the encoding settings. Here, the encoding settings may be defined according to one or more factors, such as state information of the target block, e.g., size, shape, position, intra prediction mode, etc.; image type (I / P / B); color component (Y / Cb / Cr).

[0415] For example, when the size of the target block is larger than a predetermined threshold size, the number of divisions may be 4, otherwise the number of divisions may be 2. Here, the predetermined threshold size may be represented by width (W) and height (H) as W, H, W×H, W*H, and W and H may be integers of 4, 8, 16 or more.

[0416] By way of an example to be described later, a case where intra prediction is performed in units of coding blocks will be described.

[0417] (Construction of intra prediction mode candidate group)

[0418] Fig.12 is a diagram showing examples of intra prediction modes supported by an image encoding / decoding apparatus.

[0419] Reference Fig.12 , the prediction mode candidate group for intra prediction includes 67 prediction modes, which include 65 directional modes (2 to 66) and 2 non-directional modes (DC mode, planar mode). In this case, the directional mode can be identified based on slope information (e.g., dy / dx) or angle information (degrees). All or some of the prediction modes described in the above examples may be included in the prediction mode candidate group of the luminance component or the chrominance component, and other additional modes may be included in the prediction mode candidate group.

[0420] In addition, a reconstructed block of another color space that has been encoded / decoded may be used to predict a target block using the correlation between color spaces, and a prediction mode that supports this may be included. For example, in the case of a chrominance component, a prediction block of a target block may be generated by using a reconstructed block of a luminance component corresponding to the target block. That is, a prediction block may be generated based on a reconstructed block in consideration of the correlation between color spaces, and may be included as an intra prediction mode of a chrominance component.

[0421] The chrominance component may have the same prediction mode candidate group as that of the luma component, or may have some modes of the prediction mode candidate group of the luma component, and an additional prediction mode (color copy mode, color mode) related to the chrominance component may be included in the prediction mode candidate group. Here, the color copy mode may be a prediction mode related to a method of obtaining data for generating a prediction block from an area located in another color space, and the color mode may be a prediction mode related to a method of obtaining a prediction mode from an area located in another color space.

[0422] The prediction mode candidate group may be adaptively determined according to encoding / decoding settings. The number of candidate groups may be increased for the purpose of improving the accuracy of prediction, and may be reduced for the purpose of reducing the number of bits according to the prediction mode.

[0423] For example, one of candidate A (67: 65 directional modes and 2 non-directional modes), candidate B (35: 33 directional modes and 2 non-directional modes), or candidate C (18: 17 directional modes and one non-directional mode) can be selected, and the candidate group can be adaptively selected or determined according to the size and shape of the block.

[0424] In addition, depending on the encoding / decoding settings, there may be various configurations of the prediction mode candidate group. Fig.12 As shown, it is possible to switch between modes or Fig.12 The prediction mode candidate groups are evenly configured in Fig.12 , the number of modes between mode 18 and mode 34 may be greater than the number of modes between mode 2 and mode 18. Alternatively, the opposite is possible, and the candidate group may be adaptively configured according to the shape of the block (i.e., square, non-square with a width greater than the height, non-square with a height greater than the width, etc.). For example, when the width of the target block is greater than the height, the intra-prediction modes belonging to 2 to 15 are not used, and may be replaced by intra-prediction modes belonging to 67 to 80. On the other hand, in the case where the width of the target block is less than the height, the intra-prediction modes belonging to 53 to 66 are not used, and may be replaced by intra-prediction modes belonging to -14 to -1.

[0425] In the present invention, unless otherwise specified, it is assumed that intra prediction is performed using one preset prediction mode candidate group (Candidate A) having an equal mode interval, but the main elements of the present invention may be changed and applied to the adaptive intra prediction setting as described above.

[0426] Fig.12 It may be a prediction mode supported when the shape of the block is square or non-square. In addition, when the block shape is non-square, the supported prediction mode may be a prediction mode different from the above example. For example, the number of prediction mode candidates may be different, or the number of prediction mode candidates may be the same, but the prediction mode on the longer side of the block may be dense, while the prediction mode on the opposite side may be dispersed, or the prediction mode on the longer side of the block may be dispersed, while the prediction mode on the opposite side may be dense. In the present invention, as Fig.12 As shown, it is described under the following setting: prediction mode is supported regardless of block shape (equal spacing between directional modes), but other cases may also be applicable.

[0427] The indexes assigned to the prediction modes may be assigned in a specific direction order based on the preset prediction modes. Fig.12 As shown, the index can be allocated in a clockwise direction according to a certain diagonal pattern (mode 2). In addition, in the non-directional prediction mode, the index information can be allocated in priority to the directional mode, the index information can be allocated between the directional modes, or the index information can be allocated at the end, which can be determined according to the encoding / decoding setting. It is not limited to the above example, and the index can be allocated under different settings, or various modified examples are possible.

[0428] In the above description, it has been mentioned that the prediction mode candidate group can be adaptively set, but the block (basic block) as a standard for constructing the prediction mode candidate group may be a coding block or a prediction block (or a transform block). It has been described that when a coding block is determined by a block partitioning unit, the coding block may be set as a prediction block as it is, and when a coding block is determined, the coding block may be partitioned into two or more prediction blocks.

[0429] That is, the prediction mode candidate group can be constructed by setting the coding block or prediction block as a basic block. In addition, in addition to the above-mentioned units, a basic block for setting the prediction mode candidate group can also be supported. For example, it can be a block (higher block) placed in a vertical relationship with the basic block.

[0430] For the following description, it may be assumed that the coding block is the target block and the higher block having a vertical relationship with the coding block is the related block. Alternatively, it may be assumed that the prediction block is the target block and the coding block is the related block.

[0431] A basic block may be determined, and a prediction mode candidate group may be constructed based on state information such as size, shape, and position of the determined basic block.

[0432] In a method of constructing a prediction mode candidate group based on one of a plurality of basic block candidates, support may be explicitly determined in units such as sequences, pictures, slices, tile groups, tiles, blocks, and blocks. Alternatively, support may be implicitly determined based on coding settings. In this case, the coding settings may be defined based on one or more of the following factors: state information of a block (target block), such as size, shape, and position; image type (I / P / B); color component; whether intra prediction is applied in units of sub-blocks.

[0433] For example, when the size of the target block is greater than or equal to a predetermined first threshold size, the above method (setting the higher block as the basic block) can be supported. Alternatively, when the size of the target block is less than or equal to a predetermined second threshold size, the method can be supported. Here, the threshold size can be represented by width (W) and height (H) as W, H, W×H, W*H, and the threshold size is a predetermined value in the encoding / decoding device, where W and H can be integers of 1 or greater, such as 4, 8, and 16. When expressed as the sum of width and height, W*H can be an integer of 16, 32, 64, or greater. The first threshold size is less than or equal to the second threshold size.

[0434] If multiple basic blocks are supported, various definitions of related blocks are possible. (When the coding block is the target block) The related block (higher block) may be a block having a segmentation depth difference of 1 or more relative to the segmentation depth of the coding block. Alternatively, it may be a block with a predetermined width (C) and height (D) at a predetermined upper left coordinate. Here, C and D may be integers of 8, 16, 32 or more, and C and D may be greater than or equal to the width and height of the target block. In addition, C and D may be determined based on information about block size (e.g., the size of the maximum transform block, the size of the maximum coding block, etc.).

[0435] If a method of constructing a prediction mode candidate group by supporting a plurality of basic blocks is not supported, a prediction mode candidate group may be constructed based on a predefined basic block (one of a coding block or a prediction block).

[0436] (Refer to pixel structure 1)

[0437] Fig.13 : is an exemplary diagram showing a construction of reference pixels used for intra prediction according to an embodiment of the present invention. In detail, it is assumed that a basic block for reference pixel construction is M×N.

[0438] The reference pixel construction unit may construct reference pixels used for intra prediction. In this case, the reference pixels may be managed by a temporary memory (e.g., an array <array>, a primary array, a secondary array, etc.), the reference pixels may be generated and removed for each intra prediction process, and the size of the temporary memory may be determined according to the construction of the reference pixels.

[0439] In this example, it is assumed that the left block, the upper block, the upper left block, the upper right block, and the lower left block adjacent to the basic block are used for intra prediction, but the present invention is not limited thereto and a block candidate group having another configuration may be used for intra prediction.

[0440] For example, the candidate group for the neighboring blocks of the reference pixel may be an example of following a raster or Z scan, and some of the candidates may be removed according to the scanning order, or additional block candidates may be included (e.g., a right block, a bottom block, a bottom-right block, etc. are additionally configured).

[0441] like Fig.13 As shown, left block, top block, upper left block, upper right block, lower left block ( Fig.13 The neighboring pixels in the Ref_L, Ref_T, Ref_TL, Ref_TR, and Ref_BL in the basic block can be configured as reference pixels for intra prediction. The reference pixels are usually configured as pixels in the neighboring blocks closest to the basic block ( Fig.13 a in the line, which is called the reference pixel line), and other pixels ( Fig.13 b in and pixels in other external lines) can also be configured as reference pixels.

[0442] Pixels adjacent to the basic block may be classified into at least one reference pixel line. The pixel closest to the basic block may be represented by ref_0 (e.g., pixels spaced 1 from the boundary pixel of the basic block, p(-1, -1) to p(2m-1, -1) and p(-1, 0) to p(-1, 2n-1)), the second closest pixel may be represented by ref_1 (e.g., pixels spaced 2 from the boundary pixel of the basic block, p(-2, -2) to p(2m, -2) and p(-2, -1) to p(-2, 2n)), and the third closest pixel may be represented by ref_2 (e.g., pixels spaced 3 from the boundary pixel of the basic block, p(-3, -3) to p(2m+1, -3) and p(-3, -2) to p(-3, 2n+1)). That is, the reference pixel line may be defined according to the distance between the boundary pixel of the basic block and the adjacent pixel.

[0443] N or more reference pixel lines may be supported, and N may be an integer of 1, 2, 3, 4, or more. Typically, reference pixel lines are sequentially included in the reference pixel line candidate group in ascending order of distance from the basic block. However, the present disclosure is not limited thereto. For example, when N is 3, the candidate group may sequentially include reference pixel lines, such as<ref_0、ref_1、ref_2> It is also possible to configure candidate groups non-sequentially, for example<ref_0、ref_1、ref_3> or<ref_0、ref_2、ref_3> , or configure a candidate group that does not have the closest reference pixel line.

[0444] Prediction may be performed using all or some (one or more) of the reference pixel lines in the candidate group.

[0445] For example, one of a plurality of reference pixel lines may be selected according to the encoding / decoding configuration, and the reference pixel line may be used to perform intra-frame prediction. Alternatively, two or more of a plurality of reference pixel lines may be selected, and the selected reference pixel lines may be used to perform intra-frame prediction (e.g., by weighted averaging the data of the reference pixel lines).

[0446] The reference pixel line can be selected implicitly or explicitly. For example, implicit selection means that the reference pixel line is selected according to the encoding / decoding configuration defined by a combination of one or more factors (such as image type, color component, and block size / shape / position). Explicit selection means that the reference pixel line selection information can be generated at the block level.

[0447] Although the present disclosure is described in the context of performing intra prediction using the closest reference pixel line, it should be understood that various embodiments described later may be implemented in the same manner or in a similar manner when multiple reference pixel lines are used.

[0448] Assuming that the width and height of the basic block are W and H respectively, and the selected reference pixel line is refidx (if it is 0, it is the closest pixel line), the subsequent reference pixels (reference pixel range) can be obtained.

[0449] <Left side of basic block>

[0450] p(-1-refidx,-1-refidx) to p(-1-refidx,2H-1)

[0451] <Top side of basic block>

[0452] p(-refidx,-1-refidx) to p(2W-1,-1-refidx)

[0453] The reference pixel may have a position of relative coordinates calculated based on predetermined coordinates of the basic block (the upper left is set to <0, 0>), and the position of the reference pixel in the image may be checked by adding the upper left coordinates (a, b) of the basic block to the x and y components, respectively.

[0454] In the above description, the block (basic block) used as the standard for constructing the reference pixel may be a coding block or a prediction block (or a transform block). In addition, a block (higher block) having a vertical relationship with the block may be used as the basic block for constructing the reference pixel.

[0455] For the following description, it may be assumed that the coding block is the target block and the higher block having a vertical relationship with the coding block is the related block. Alternatively, it may be assumed that the prediction block is the target block and the coding block is the related block.

[0456] A basic block may be determined, and reference pixels (a reference pixel candidate group) may be constructed based on state information such as size, shape, and position of the determined basic block.

[0457] Whether a method of constructing reference pixels based on one of a plurality of basic block candidates is supported may be explicitly determined in units such as sequences, pictures, slices, tile groups, tiles, blocks, blocks, etc. Alternatively, support may be implicitly determined according to coding settings. In this case, the coding settings may be defined based on one or more of the following factors: state information of a block (target block), such as size, shape, and position; image type (I / P / B); color component; whether intra prediction is applied in units of sub-blocks.

[0458] For example, when the size of the target block is greater than or equal to a predetermined first threshold size, the method can be supported. Alternatively, when the size of the target block is less than or equal to a predetermined second threshold size, the method can be supported. Here, the threshold size can be represented by width (W) and height (H) as W, H, W×H, W*H, and the threshold size is a predetermined value in the encoding / decoding device, where W and H are 4, and W and H can be integers of 1 or greater, such as 4, 8, and 16. When expressed as the sum of width and height, W*H can be an integer of 16, 32, 64, or greater. The first threshold size is less than or equal to the second threshold size.

[0459] In addition, when it is divided into two or more blocks when applying sub-block-based intra prediction, the above method can be supported. In this case, when the prediction block (sub-block) is the target block and the coding block is the related block, the basic block can be determined as the target block, or the basic block can be determined as the related block. Here, when the method of constructing reference pixels based on one of a plurality of basic candidate blocks is not used, the basic block can be set to the prediction block, or it can be set to the opposite case.

[0460] If multiple basic blocks are supported, various definitions of related blocks are possible. (When the coding block is the target block) The related block (higher block) may be a block with a segmentation depth less than the segmentation depth of the coding block. The difference in segmentation depth between the related block and the coding block is 1 or greater. Alternatively, it may be a block with a predetermined width (C) and height (D) at a predetermined upper left coordinate. Here, C and D may be integers of 4, 8, 16 or more, and C and D may be greater than or equal to the width and height of the target block. In addition, C and D may be determined based on information about block size (e.g., the size of the maximum transform block, the size of the maximum coding block, etc.).

[0461] If a method of constructing reference pixels by supporting a plurality of basic blocks is not supported, reference pixels may be constructed based on a predefined basic block (one of a coding block or a prediction block).

[0462] Fig.14 is an example diagram of the construction of reference pixels available from basic blocks.

[0463] Reference Fig.14 , (a) represents a case where reference pixels are arranged to be adjacent to a block (shaded area) in which intra prediction is performed. (b) and (c) represent cases where reference pixels are adjacent to some boundaries but not adjacent to some boundaries in a block on which intra prediction is performed. (d) represents a case where reference pixels are arranged not to be adjacent to a block on which intra prediction is performed. This may be various cases that may occur depending on the setting of basic blocks.

[0464] Fig.15 is a conceptual diagram showing blocks adjacent to a target block for intra prediction. In detail, Fig.15 The left side of shows blocks adjacent to the target block in the current color space, and the right side shows corresponding blocks in another color space.

[0465] In particular, if certain prediction modes are supported (color copy mode), since certain areas of different color spaces can be used for intra prediction, these areas can also be considered as reference pixels.

[0466] Although the above drawings show blocks adjacent to the target block, in some examples described later, the target block may be set as a basic block.

[0467] (Intra-frame prediction preprocessing / reference pixel configuration 2)

[0468] According to the present disclosure, a reference pixel construction unit for intra prediction may include a reference pixel generator, a reference pixel interpolator, and a reference pixel filter unit. The reference pixel construction unit may include all or part of the above components.

[0469] The reference pixel construction unit may distinguish available pixels from unavailable pixels by checking the availability of the reference pixels. When a reference pixel satisfies at least one of the following conditions, the reference pixel is determined to be unavailable.

[0470] For example, if at least one of the following conditions is met, the reference pixel can be determined to be unavailable: the reference pixel is outside the picture boundary; the reference pixel does not belong to the same partition unit as the current block (e.g., a unit that does not allow mutual reference, such as a slice or a tile. However, although it is a slice or a tile, a unit that allows mutual reference is an exception); and the reference pixel has not been completely encoded / decoded. That is, if none of the above conditions are met, the reference pixel can be determined to be available.

[0471] The use of reference pixels may be restricted by the encoding / decoding configuration. For example, even if the reference pixels are determined to be available according to the above conditions, the use of the reference pixels may be restricted depending on whether constrained intra prediction is performed (for example, indicated by constrained_intra_pred_flag). When the use of a block reconstructed by referring to another image is prohibited for the purpose of robust encoding / decoding errors due to external factors such as the communication environment, constrained intra prediction can be performed.

[0472] When constrained intra prediction is disabled (eg, constrained_intra_pred_flag=0 for I picture type or P or B picture type), all reference pixel candidate blocks may be available.

[0473] In contrast, when constrained intra prediction is activated (e.g., constrained_intra_pred_flag=1 for P or B picture type), it may be determined whether to use a reference pixel candidate block according to the coding mode (Mode_Intra, Mode_InterD, Mode_InterC). However, this condition may be determined according to various other encoding / decoding factors.

[0474] Since the reference pixels are in one or more blocks, the reference pixels can be classified into three types according to their reference pixel availability: fully available, partially available, and completely unavailable. In other cases except fully available, reference pixels at the position of the unavailable candidate block can be filled or generated.

[0475] When a reference pixel candidate block is available, the pixel at the corresponding position may be included in the reference pixel memory for the target block. The pixel data of the pixel may be copied as is, or may be included in the reference pixel memory after processing such as reference pixel filtering and reference pixel interpolation. Conversely, when a reference pixel candidate block is not available, the pixel obtained by the reference pixel generation process may be included in the reference pixel memory of the target block.

[0476] Examples of generating reference pixels at the positions of unavailable blocks in various methods will be described below.

[0477] For example, an arbitrary pixel value may be used to generate a reference pixel. The arbitrary pixel value may be one of the pixel values ​​within a pixel value range (e.g., a pixel value range based on a bit depth or a pixel value range based on pixel distribution of a corresponding image) (e.g., a minimum value, a maximum value, or a median value of the pixel value range). Specifically, this example may be applicable when the entire reference pixel candidate block is unavailable.

[0478] Alternatively, reference pixels may be generated from an area in which image encoding / decoding has been completed. Specifically, reference pixels may be generated from at least one available block adjacent to an unavailable block. Reference pixels may be generated using at least one of extrapolation, interpolation, or replication.

[0479] After completing the reference pixel construction, reference pixel filtering or reference pixel interpolation may be performed. For example, only reference pixel filtering may be performed, only reference pixel interpolation may be performed, or both reference pixel filtering and reference pixel interpolation may be performed. Reference pixel filtering may be performed before or after reference pixel interpolation, or both reference pixel filtering and reference pixel interpolation may be combined and performed simultaneously.

[0480] Reference pixel interpolation and reference pixel filtering will be described below.

[0481] For the purpose of reducing the degradation remaining in the reference pixel, reference pixel filtering may be performed. In the reference pixel filtering, one of a plurality of filters that can be distinguished by setting the lengths and coefficients of various filter taps, such as [1, 2, 1] / 4, [2, 3, 6, 3, 2] / 16, may be used. In addition, a plurality of filtering may be performed by filters of different types.

[0482] Whether to perform reference pixel filtering may be explicitly determined or may be implicitly determined according to encoding settings. Here, encoding settings may be defined based on state information (size, shape, position, etc.) of a basic block (target block or related block <higher block>), image type (I / P / B), color component (Y / Cb / Cr), reference pixel line selection information, whether intra prediction is applied in units of sub-blocks, and intra prediction mode.

[0483] As an example, when the size of the basic block is greater than or equal to a predetermined first threshold size, filtering may be performed; when the size of the basic block is less than the predetermined first threshold size, filtering may not be performed. Alternatively, when the size of the basic block is less than or equal to a predetermined second threshold size, filtering may be performed; when the size of the basic block is greater than the predetermined second threshold size, filtering may not be performed. Here, the threshold size may be expressed as W, H, W×H, W*H, etc. based on the width (W) and the height (H), and W and H may be integers of 4, 8, 16, 32 or more. In particular, when expressed as the product of the width and height, such as W×H, W×H may be an integer of 16, 32, 64 or more. The first threshold size is less than or equal to the second threshold size.

[0484] For example, if the shape of the basic block is a square shape, filtering may be performed; if the shape of the basic block is not a square shape, filtering may not be performed. Alternatively, if the basic block has a non-square shape and has a width longer than the height (or a width shorter than the height), filtering may be performed; if the basic block has a square shape and does not have a width longer than the height (or a width shorter than the height), filtering may not be performed. When the ratio of the width to the height of the basic block is greater than or equal to a predetermined threshold, filtering may be performed; when the ratio of the width to the height of the basic block is less than a predetermined threshold, filtering may not be performed. Here, the threshold ratio may be set to 1:2, 1:3, 1:4, 2:3, etc. Here, it may be desirable to combine the block shape with the prediction mode to determine the setting.

[0485] For example, filtering may be performed regardless of the color component of the basic block, or may be performed only when it belongs to a predetermined color component (luminance or chrominance).

[0486] For example, when the prediction mode of the basic block is a non-directional mode, filtering may be performed; when the prediction mode of the basic block is not a non-directional mode, filtering may not be performed. Here, the non-directional mode may include both the DC mode and the plane mode, or may include only one of the DC mode and the plane mode. That is, filtering may be performed in the plane mode, and filtering may not be performed in the DC mode; or filtering may not be performed in the plane mode, and filtering may be performed in the DC mode. Alternatively, filtering may be performed in a predetermined mode in the directional mode, and filtering may not be performed in a mode other than the predetermined mode. Here, when the predetermined mode is a diagonal mode (a directional mode other than horizontal and vertical) in the directional mode, filtering may be performed; and when the predetermined mode is not a diagonal mode in the directional mode, filtering may not be performed. Here, in the diagonal mode, a mode of referring only to pixels of integer units and a mode of referring to pixels of integer units and fractional units may be included, or only a mode of referring only to pixels of integer units and a mode of referring to pixels of integer units and fractional units may be included. That is, filtering may be performed in a mode of referring to only pixels in integer units, and filtering may not be performed in a mode of referring to pixels in both integer units and fractional units; or filtering may not be performed in a mode of referring to only pixels in integer units, and filtering may be performed in a mode of referring to pixels in both integer units and fractional units. Here, the mode of referring to only pixels in integer units may include the mode of referring to only pixels in integer units. Fig.12 At least one of the modes corresponding to -14, -12, -10, -6, 2, 34, 66, 72, 78, and 80 in . That is, it may also be a mode in which reference pixel interpolation is not performed, and the prediction mode number may also be changed according to the configuration change of the prediction mode candidate group. Here, a special case of performing filtering on a horizontal mode or a vertical mode may be applied.

[0487] Whether to perform filtering may be determined by considering (combining) all or some of the factors mentioned in the above examples. The above examples are only some of the examples, and various modified examples are possible.

[0488] When filtering is performed, filter selection information may be determined explicitly or implicitly, and this may be based on encoding settings. In this case, the encoding settings may be defined by the various elements described above, and a detailed description thereof will be omitted because the description thereof may be derived by the above examples.

[0489] In the reference pixel interpolation, as described above, the reference pixel interpolation may not be performed in a prediction mode that refers to only pixels of integer units, and the reference pixel interpolation may be performed in a prediction mode that refers to pixels of fractional units.

[0490] According to the prediction mode (e.g., the directionality of the prediction mode, dy / dx, etc.) and the positions of the reference pixel and the predicted pixel, the position of the pixel to be interpolated (i.e., the fractional unit to be interpolated, ranging from 1 / 2 to 1 / 64) can be determined. In this case, one filter can be applied (e.g., the same filter is assumed in the equation for determining the filter coefficient or the length of the filter tap, but the filter is assumed to adjust the coefficient only according to the fractional accuracy such as 1 / 32, 7 / 32 or 19 / 32), or one of a plurality of filters (e.g., filters assuming different equations are used to determine the filter coefficient or the length of the filter tap) can be selected and applied according to the fractional unit.

[0491] In the former case, integer pixels can be used as input for fractional pixel interpolation, and in the latter case, the input pixels are different in each step (for example, integer pixels are used for 1 / 2 units, and integer pixels and 1 / 2 unit pixels are used for 1 / 4 units). However, the present disclosure is not limited thereto, and will be described in the context of the former case.

[0492] Fixed filtering or adaptive filtering may be performed on reference pixel interpolation, and the fixed filtering or adaptive filtering may be determined according to the encoding setting. Here, the encoding setting may be defined based on the state information of the basic block, the image type, the color component, the reference pixel line selection information, whether intra-frame prediction is applied in units of sub-blocks, the intra-frame prediction mode, etc.

[0493] In fixed filtering, reference pixel interpolation may be performed using one filter, whereas in adaptive filtering, reference pixel interpolation may be performed using one of a plurality of filters.

[0494] In adaptive filtering, one of a plurality of filters may be implicitly or explicitly determined according to the encoding / decoding configuration. The filter may be configured by selecting one or more from a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, and an 8-tap Kalman filter. The supported filter candidate groups may be defined differently according to the color component (e.g., the types of filters are partially the same or different, and the length of the filter taps is short or long).

[0495] Due to the overlap of filtering performed in the reference pixel interpolation process and filtering performed in the reference pixel filtering process, some pixels may be distorted. For this reason, the reference pixel interpolation filter setting may be determined in consideration of the reference pixel filtering setting. For reference, in the case of a prediction mode in which reference pixel interpolation is performed, it may be an example in which reference pixel filtering is not performed.

[0496] In the above description, a basic block may be determined as one of a coding block, a prediction block, or a higher block, and a description thereof may be derived through the above examples.

[0497] (Refer to pixel structure 3)

[0498] Reference pixel construction when performing intra prediction in sub-block units will be described. Some settings such as the above-mentioned multiple reference pixel lines and reference pixel filtering can be configured or set in coding block units. Intra prediction can be performed by selecting one of the multiple reference pixel lines, and reference pixel filtering can be performed to reduce degradation of reference pixels.

[0499] However, since it is impossible to know exactly what correlation the characteristics of the target block and the neighboring blocks have, some explicit settings may be required for reference pixel construction (reference pixel line selection information) and some restricted settings may be required (disabling reference pixel filtering for certain prediction modes).

[0500] For example, if a block adjacent to a target block has a high correlation with the target block, using two or more reference pixel lines for prediction can reduce quantization error and further improve prediction accuracy compared to a method of selecting one of multiple reference pixel lines.

[0501] Alternatively, reference pixel filtering may be performed even in a predetermined prediction mode, and thus, degradation of reference pixels may be effectively reduced.

[0502] Intra prediction in sub-block units can be a case where the correlation between the known target block (prediction block) and the adjacent blocks is high. In particular, since it has a characteristic in which the edge of the higher block (coding block) is shared as one information (prediction mode), it can be known in advance that all adjacent prediction blocks have the same prediction direction.

[0503] When intra prediction is performed in sub-block units, the following construction and setting of reference pixels are possible. First, it is assumed that one of multiple reference pixel lines around the target block (coding block) can be selected, and reference pixel filtering and reference pixel interpolation can be performed according to the coding settings defined by various coding elements.

[0504] For example, one reference pixel line (nearest line) defined for a target block (prediction block) is supported, and reference pixel filtering and reference pixel interpolation may be performed by fixed settings (eg, no filtering or using only one interpolation filter, etc.).

[0505] Alternatively, multiple reference pixel lines predefined for the target block are supported (implicitly). In detail, the first prediction block in the coding block (adjacent to the upper boundary or left boundary of the coding block) performs intra prediction using one reference pixel line. The second prediction block can perform intra prediction by setting multiple reference pixel lines based on the reference pixel lines adjacent to the coding block and the encoded first prediction block. The third block and other blocks can perform intra prediction by setting multiple reference pixel lines based on previous prediction blocks (in this example, the first target block and the second target block) that have already completed encoding.

[0506] In addition, reference pixel filtering can be performed by considering not only reference pixels placed on one reference pixel line of the prediction block but also reference pixels placed on another reference pixel line. In detail, when the partition direction of the subblock is the horizontal direction and the prediction mode is the vertical mode, reference pixel filtering can be applied to reference pixels placed in the vertical direction on multiple reference pixel lines. Reference pixel interpolation can also be performed by applying an adaptive setting.

[0507] The above examples are some of the cases where the setting for intra prediction is performed in sub-block units, and are not limited thereto, and various modifications may be made. That is, in consideration of a case where the setting for intra prediction in sub-block units has characteristics different from those of general intra prediction (sharing the prediction direction within the block), reference pixel construction settings may be supported.

[0508] (Perform prediction and determine patterns)

[0509] The prediction block generator may generate a prediction block in at least one prediction mode and may use reference pixels based on the prediction mode. The reference pixels may be used in a method such as extrapolation (directional mode) or a method such as interpolation, averaging (DC) or replication (non-directional mode).

[0510] The prediction mode decision unit performs a process of selecting an optimal mode from a plurality of prediction mode candidate groups. Generally, the optimal mode can be determined according to the coding cost using a rate-distortion scheme in which block distortion (e.g., distortion, sum of absolute difference (SAD) and sum of square difference (SSD) of a target block and a reconstructed block) and the number of bits generated according to the mode are considered. The prediction block generated based on the prediction mode determined in the above process can be sent to the subtraction unit and the addition unit.

[0511] All prediction modes of the prediction mode candidate group may be searched to determine the best prediction mode, or the best prediction mode may be selected in a different decision process to reduce the amount of calculation / complexity. For example, in the first step, some modes with good performance in terms of image quality degradation are selected from all intra-frame prediction mode candidates, and in the second step, the best prediction mode may be selected in consideration of the number of bits generated and the image quality degradation of the mode selected in the first step. In addition to this method, various other methods for determining the best prediction mode with reduced calculation amount / complexity may be applied.

[0512] In addition, although the prediction mode decision unit can generally be included only in the encoder, for example, when template matching is included as a prediction method or an intra-frame prediction mode is derived from an area adjacent to a target block, the prediction mode decision unit may also be included in the decoder according to the encoding / decoding configuration. In the latter case, it can be understood that a method of implicitly obtaining a prediction mode in the decoder is used.

[0513] Assume that the above processing is performed in units of target blocks rather than basic blocks. That is, the prediction mode candidate group construction and reference pixel construction may not be set based on the target block, but the process of obtaining and determining the prediction block may be based on the target block.

[0514] (Prediction mode candidate group configuration and encoding)

[0515] The prediction mode encoder may encode the prediction mode selected by the prediction mode decision unit. Information about the index of the prediction mode in the prediction mode candidate group may be encoded, or the prediction mode may be predicted and information about the prediction mode may be encoded. The former method may be applied to a luminance component but is not limited to the luminance component, and the latter method may be applied to a chrominance component but is not limited to the chrominance component.

[0516] As an example, a priority may be assigned to a predetermined prediction mode of the prediction mode candidate group to be set as a prediction value of the prediction mode. Priority may be assigned to a prediction mode that usually and frequently occurs. In this document, the directional mode may include at least one of a vertical, horizontal, or predetermined diagonal mode, and the non-directional mode may include at least one of a planar mode or a DC mode. Here, the predetermined diagonal mode may be a prediction mode having predetermined slope information, for example, Fig.12 At least one of the modes 2, 34, and 66 has an angle of 45 degrees or -45 degrees relative to the horizontal mode or the vertical mode. Priorities may be assigned to other additional diagonal modes.

[0517] At least one of the prediction modes assigned with priorities may be set as the prediction value of the priority allocation mode. In this case, the prediction modes selected as the prediction value of the priority allocation mode may be prioritized according to a predetermined order, and may be allocated, for example, in the order of a planar mode, a DC mode, a vertical mode, and a horizontal mode.

[0518] If one priority allocation mode is supported, the flat mode can be set as the predicted value, and if two priority levels are supported, the flat mode and the DC mode can be set as the predicted values.

[0519] The priority allocation mode may be determined according to one or more elements among status information of a basic block (assumed to be a target block in this example), an image type, and a color component.

[0520] As an example, the candidate group may be constructed by a method for selecting a predetermined prediction mode from the prediction mode candidate group to be set as a prediction value of the prediction mode. Here, the method of selecting the predetermined prediction mode may include: obtaining a prediction mode of a block adjacent to the basic block; obtaining a preset prediction mode; obtaining based on a prediction mode already included in the candidate group, etc.

[0521] In the present invention, the prediction mode included in the candidate group is referred to as the most probable mode (MPM). The MPM may include one prediction mode or multiple prediction modes. The number of MPMs may be determined according to the number of prediction mode candidate groups (k. k is an integer greater than or equal to 1, such as 1, 2, 3, 6, etc.). When the MPM includes multiple prediction modes, it may be referred to as an MPM candidate group.

[0522] The MPM candidate group may be supported under a fixed setting or under an adaptive setting according to various encoding / decoding elements. As an example of an adaptive setting, the candidate group construction may be determined according to which reference pixel layer is used among multiple reference pixel layers, and the candidate group construction may be determined according to whether intra prediction is performed in units of sub-blocks. A fixed setting may mean that a predefined candidate group construction rule is followed regardless of the information.

[0523] For ease of explanation, it is assumed that the MPM candidate group is constructed under one setting, and it should be understood that not only the MPM candidate group but also the candidate group construction for other intra prediction modes can also be adaptive.

[0524] The MPM is a concept supported to efficiently encode a prediction mode, and a candidate group may be constructed with a prediction mode having a high probability of occurrence as a prediction mode of a target block.

[0525] For example, the MPM candidate group can be constructed using a preset prediction mode (or at least one of a statistically frequently occurring prediction mode, a DC mode, a plane mode, a vertical mode, a horizontal mode, or some diagonal modes) or a prediction mode of an adjacent block (at least one position in a left block, an upper block, an upper left block, an upper right block, or a lower left block. For example, a left block, an upper block). Fig.15 The prediction modes of the neighboring blocks are obtained from L0 to L3 (left block), T0 to T3 (upper block), TL (upper left block), R0 to R3 (upper right block), and B0 to B3 (lower left block).

[0526] If an MPM candidate group can be constructed based on two or more sub-block positions (e.g., L0 and L2) in an adjacent block (e.g., a left block), a prediction mode for the corresponding block can be constructed in the candidate group according to a predefined priority (e.g., L0—L1—L2). Alternatively, when the MPM candidate group cannot be configured based on two or more sub-block positions, a prediction mode for a sub-block at a predefined position (e.g., L0) can be configured in the candidate group. Specifically, prediction modes at positions L3, T3, TL, R0, and B0 in the adjacent block can be selected as prediction modes for the adjacent block and included in the MPM candidate group. The above description is for a case in which a prediction mode for an adjacent block is configured in the candidate group, and it should not be understood as limiting the present disclosure. In the following examples, it is assumed that a prediction mode at a predetermined position is configured in the candidate group.

[0527] When one or more prediction modes are included in the MPM candidate group, a mode derived from the one or more previously included prediction modes may be additionally configured in the MPM candidate group. Specifically, when the k-th mode (directional mode) is included in the MPM candidate group, a mode derivable from the mode (a mode spaced at a distance of +a or -b from the k-th mode, where each of a and b is an integer equal to or greater than 1, for example, 1, 2, or 3) may be additionally included in the MPM candidate group.

[0528] Modes may be prioritized for use in constructing an MPM candidate group. An MPM candidate group may be constructed to include prediction modes in the order of prediction modes of adjacent blocks, preset prediction modes, and derived prediction modes. The process of constructing an MPM candidate group may be accomplished by filling a maximum number of MPM candidates according to priority. In the above process, if a prediction mode is the same as a previously included prediction mode, the prediction mode may not be included in the MPM candidate group, and a priority candidate may be taken and subjected to a redundancy check.

[0529] The prediction mode can be derived by selectively using the priority allocation mode or the MPM candidate. A flag related to the selection information can be generated, and the flag can indicate whether the prediction mode of the target block is set to the priority allocation mode. When the flag is 1, the prediction mode of the target block is set to the priority allocation mode. If the flag is not 1, information about whether the prediction mode of the target block is derived from the MPM candidate, the MPM index, etc. can be generated. Here, when the prediction mode of the target block is not derived in the priority allocation mode or the MPM candidate, non-MPM index information, etc. can be generated to derive the prediction mode of the target block in other prediction modes other than the priority allocation mode and the MPM candidate.

[0530] For ease of explanation, the following will focus on the description of the MPM candidate construction (MPM candidate / non-MPM candidate), and the priority allocation mode is assumed to be some candidates of the MPM candidate group construction. However, in the case of a configuration in which the priority allocation mode is explicitly supported, it is necessary to understand that the consideration of the priority allocation mode is removed in the implementation described later.

[0531] The following description is given under the assumption that the MPM candidate group includes 6 prediction modes.

[0532] For example, the MPM candidate group may be formed in the order of LT-TL-TR-BL-plane-DC-vertical-horizontal-diagonal. The prediction modes of the neighboring blocks may be included in the MPM candidate group with priority, and then in this case a preset prediction mode may be additionally configured.

[0533] Alternatively, the MPM candidate group can be in the form of LT-plane-DC-<L+1> - <l-1> - <T+1> - <t-1>-Vertical-Horizontal-Diagonal order. In this case, some prediction modes of neighboring blocks and some preset prediction modes may be included with priority, and modes derived based on the assumption that a prediction mode in a direction similar to the prediction mode of the neighboring blocks will be generated and some preset prediction modes may be additionally included.

[0534] The above example is only a part of the MPM candidate group configuration. The present disclosure is not limited thereto, and various modifications may be available.

[0535] The MPM candidate group can be represented by binarization such as unary binarization or truncated Rice binarization based on the index within the candidate group. That is, short bits can be assigned to candidates with small indices, and long bits can be assigned to candidates with large indices to represent the mode bit.

[0536] A mode not included in the MPM candidate group may be classified as a non-MPM candidate group. Two or more non-MPM candidate groups may be defined according to encoding / decoding configuration.

[0537] The following description is given under the assumption that 67 modes including directional modes and non-directional modes are included in the prediction mode candidate group, 6 MPM candidates are supported, and thus the non-MPM candidate group includes 61 prediction modes.

[0538] If a non-MPM candidate group is constructed, this means that there are remaining prediction modes that are not included in the MPM candidate group, and therefore no additional candidate group construction process is required. Therefore, binarization such as fixed-length binarization and truncated unary binarization can be used based on the index within the non-MPM candidate group.

[0539] Based on the assumption that two or more non-MPM candidate groups are constructed, the non-MPM candidate groups are classified into non-MPM_A (candidate group A) and non-MPM_B (candidate group B). It is assumed that candidate group A (p candidates, where p is equal to or greater than the number of MPM candidates) includes a prediction mode having a higher probability of occurrence as a prediction mode of a target block than candidate group B (q candidates, where q is equal to or greater than the number of candidates in candidate group A). ​​In this article, a process of constructing candidate group A can be added.

[0540] For example, some equidistant prediction modes (e.g., modes 2, 4, and 6) in the directional mode may be included in candidate group A, or a preset prediction mode (e.g., a mode derived from the prediction modes included in the MPM candidate group) may be included in candidate group A. The remaining prediction modes after the MPM candidate group construction and the candidate group A construction may form candidate group B, and no additional candidate group construction process is required. Binarization such as fixed length binarization and truncated unary binarization may be used based on the indexes in candidate group A and candidate group B.

[0541] The above example is a part of the case where two or more non-MPM candidate groups are constructed. The present disclosure is not limited thereto, and various modifications are available.

[0542] The process of predicting and encoding the prediction mode will be described below.

[0543] Information (mpm_flag) indicating whether the prediction mode of the target block is consistent with the MPM (or a mode in the MPM candidate group) may be checked.

[0544] When the prediction mode of the target block matches the MPM, the MPM index information (mpm_idx) may be additionally checked according to the MPM configuration (one or more configurations). Then, the encoding process of the target block is completed.

[0545] When the prediction mode of the target block does not match any MPM, if there is a configured non-MPM candidate group, the non-MPM index information (remaining_idx) may be checked, and then the encoding process of the target block is completed.

[0546] If a plurality of non-MPM candidate groups (two in this example) are constructed, information (non_mpm_flag) indicating whether the prediction mode of the target block is consistent with any prediction mode in candidate group A may be checked.

[0547] If the prediction mode of the target block matches any candidate in candidate group A, the index information (non_mpm_A_idx) about candidate group A may be checked, and if the prediction mode of the target block does not match any candidate in candidate group A, the candidate B index information (remaining_idx) may be checked. Then, the encoding process of the target block is completed.

[0548] When the prediction mode candidate group construction is fixed, the prediction mode supported by the target block, the prediction mode supported by the neighboring block, and the preset prediction mode may use the same prediction number index.

[0549] When the prediction mode candidate group construction is adaptive, the prediction mode supported by the target block, the prediction mode supported by the neighboring block, and the preset prediction mode may use the same prediction number index or different prediction number indexes. Fig.12 , the following description is given.

[0550] In the prediction mode encoding process, a process of unifying (or adjusting) the prediction mode candidate group to configure the MPM candidate group, etc. may be performed. For example, the prediction mode of the target block may be one of the prediction modes in the prediction mode candidate group having mode -5 to mode 61, and the prediction mode of the adjacent block may be one of the prediction modes in the candidate group having mode 2 to mode 66. In this case, since it may not be supported to use a part of the prediction mode of the adjacent block (mode 66) as the prediction mode of the target block, a process of unifying the prediction modes in the prediction mode encoding process may be performed. That is, when the fixed intra-frame prediction mode candidate group construction is supported, this process may not be required, and when the adaptive intra-frame prediction mode candidate group construction is supported, this process may be required, and it will not be described in detail herein.

[0551] Unlike the MPM-based method, encoding may be performed by assigning an index to a prediction mode of a prediction mode candidate group.

[0552] For example, the prediction modes are indexed according to their predefined priorities. When a prediction mode is selected as the prediction mode of the target block, the index of the selected prediction mode is encoded. This means a case where a fixed prediction mode candidate group is constructed and a fixed index is assigned to the prediction mode.

[0553] Alternatively, when the prediction mode candidate group is adaptively constructed, the fixed index allocation method may not be suitable. Therefore, the prediction mode can be indexed according to the adaptive priority. When the prediction mode is selected as the prediction mode of the target block, the selected prediction mode can be encoded. This method can achieve efficient encoding of the prediction mode because the index of the prediction mode is changed due to the adaptive construction of the prediction mode candidate group. That is, the adaptive priority can be intended to assign a candidate with a high probability of being selected as the prediction mode of the target block to an index in which a short mode bit is generated.

[0554] The following description is based on the assumption that, in the prediction mode candidate group (the case of a chroma component), 8 prediction modes including a preset prediction mode (a directional mode and a non-directional mode), a color copy mode, and a color mode are supported.

[0555] For example, assume that four preset prediction modes of planar mode, DC mode, horizontal mode, vertical mode, and diagonal mode (lower left diagonal in this example) are supported; one color mode C; and three color copy modes CP1, CP2, and CP3. The prediction modes may be indexed substantially in the order of preset prediction modes, color copy modes, and color modes.

[0556] In this case, the preset prediction mode as the directional mode and the non-directional mode and the color copy mode are prediction modes distinguished by the prediction method and can therefore be easily identified. However, the color mode may be the directional mode or the non-directional mode, and the color mode may overlap with the preset prediction mode. For example, when the color mode is the vertical mode, the color mode may overlap with the vertical mode as one of the preset prediction modes.

[0557] In the case where the number of prediction mode candidates is adaptively adjusted according to the encoding / decoding configuration, when there is a redundant case, the number of candidates can be adjusted (8 —> 7). Alternatively, in the case where the number of prediction mode candidates remains fixed, when there is a redundant prediction mode, an index can be allocated by adding and considering an additional candidate. In addition, even when a variable mode such as a color mode is included, an adaptive prediction mode candidate group can be supported. Therefore, the adaptive index allocation case can be regarded as an example of configuring an adaptive prediction mode candidate group.

[0558] Now, a description will be given of adaptive index allocation according to the color mode. It is assumed that the indexes are basically allocated in the order of Plane (0) - Vertical (1) - Horizontal (2) - DC (3) - CP1 (4) - CP2 (5) - CP3 (6) - C (7). In addition, if the color mode does not match any preset prediction mode, it is assumed that the index allocation is performed in the above order.

[0559] For example, if the color mode matches one of the preset prediction modes (planar mode, vertical mode, horizontal mode, and DC mode), the prediction mode matching index 7 of the color mode is filled. The preset prediction mode (lower left diagonal) is filled at the index (one of 0 to 3) of the matching prediction mode. Specifically, when the color mode is the horizontal mode, the indexes may be allocated in the order of planar (0) - vertical (1) - lower left diagonal (2) - DC (3) - CP1 (4) - CP2 (5) - CP3 (6) - horizontal (7).

[0560] Alternatively, when the color mode matches one of the preset prediction modes, the matching prediction mode is filled at index 0, and the preset prediction mode is filled at index 7 of the color mode (lower left diagonal). In this case, if the filled prediction mode is not the existing index 0 (i.e., it is not the planar mode), the existing index configuration may be adjusted. Specifically, when the color mode is the DC mode, the indexes may be allocated in the order of DC (0) - planar (1) - vertical (2) - horizontal (3) - CP1 (4) - CP2 (5) - CP3 (6) - lower left diagonal (7).

[0561] The above example is only a part of adaptive index allocation. The present disclosure is not limited to this, and various modifications may be available. In addition, binarization such as fixed length binarization, unary binarization, truncated unary binarization, and truncated Rice binarization may be used based on the index in the candidate group.

[0562] Another example of performing encoding by assigning an index to a prediction mode belonging to a prediction mode candidate group will be described.

[0563] For example, prediction modes and prediction methods are classified into a plurality of prediction mode candidate groups, and indexes are assigned to prediction modes belonging to corresponding candidate groups, and then encoded. In this case, encoding of candidate group selection information may precede index encoding. For example, a directional mode, a non-directional mode, and a color mode, which are prediction modes that perform prediction in the same color space, may belong to one candidate group (referred to as candidate group S), and a color copy mode, which is a prediction mode that performs prediction in a different color space, may belong to another candidate group (referred to as candidate group D).

[0564] The following description is based on the assumption that, in the prediction mode candidate group (the case of the chrominance component), 9 prediction modes including a preset prediction mode, a color copy mode, and a color mode are supported.

[0565] For example, it is assumed that: four preset prediction modes from among planar mode, DC mode, horizontal mode, vertical mode, and diagonal mode; one color mode C; and four color copy modes CP1, CP2, CP3, and CP4 are supported. Candidate group S may include 5 candidates as preset prediction modes and color modes, and candidate group D may include 4 candidates as color copy modes.

[0566] Candidate group S is an example of adaptively constructing a prediction mode candidate group. An example of adaptive index allocation has been described above, so it will not be described in detail herein. Since candidate group D is an example of a fixed prediction mode candidate group, a fixed index allocation method can be used. For example, the index can be allocated in the order of CP1 (0) - CP2 (1) - CP3 (2) - CP4 (3).

[0567] Binarization such as fixed-length binarization, unary binarization, truncated unary binarization, and truncated Rice binarization may be used based on the index within the candidate group. The present disclosure is not limited to the above examples, and various modifications may also be available.

[0568] A candidate group for prediction mode encoding, such as an MPM candidate group, may be constructed at the block level. Alternatively, the process of constructing a candidate group may be omitted, and a predetermined candidate group or a candidate group obtained in various methods may be used. This may be a configuration supported for the purpose of reducing complexity.

[0569] In Example 1, one predefined candidate group may be used, or one of a plurality of predefined candidate groups may be used according to the encoding / decoding setting. For example, in the case of an MPM candidate group, a predefined candidate group may be used, such as {plane-DC-vertical-horizontal-lower-left-diagonal ( Fig.12 66) - bottom right diagonal ( Fig.12 Alternatively, the candidate group constructed when no adjacent blocks are available in the MPM candidate group construction may be applied.

[0570] In Example 2, a candidate group for blocks that have been completely encoded can be used. The encoding block can be selected based on the encoding order (predetermined scanning scheme, such as z-scan, vertical scan, horizontal scan, etc.), or from blocks adjacent to the basic block (e.g., left block, upper block, upper left block, upper right block, and lower left block). However, the adjacent blocks may be limited to blocks at the position of a partition unit that can be referenced with the basic block (e.g., a partition unit having properties that can be referenced even if the blocks belong to different slices or tiles, such as different tiles belonging to the same tile group). If the adjacent block belongs to a partition unit that does not allow reference (e.g., when each block belongs to a different slice or tile and has properties that may not be referenced to each other, such as when each block belongs to a different block group), the block at that position may be excluded from the candidate.

[0571] In this case, the adjacent blocks can be determined according to the state of the basic block. For example, when the basic block is a square, the candidate group of available blocks can be borrowed (or shared) from the block positioned according to the predetermined first priority. Alternatively, when the basic block is a rectangle, the candidate group of available blocks can be borrowed from the block positioned according to the predetermined second priority. The second priority or the third priority can be supported according to the aspect ratio of the block. In order to select the candidate block to be borrowed, the priority can be set in various configurations, such as left-top-top right-bottom left-top left, or top-left-top left-top right-bottom left. In this case, all priorities from the first priority to the third priority may have the same configuration or different configurations, or a part of the priorities may have the same configuration.

[0572] A candidate group of basic blocks can be borrowed from a neighboring block only when the candidate group of basic blocks is at or greater than / greater than a predetermined boundary value, or only when it is at or less than / less than a predetermined boundary value. The boundary value can be defined as the minimum block size or maximum block size that allows borrowing of the candidate group. The boundary value can be expressed as the width (W) of the block, the height (H) of the block, W×H, W*H, etc., where each of W and H can be an integer equal to or greater than 4, 8, 16, or 32.

[0573] In Example 3, a common candidate group may be formed of higher blocks as a predetermined block group. A common candidate group may be used for lower blocks belonging to higher blocks. The number of lower blocks may be an integer equal to or greater than 1, for example, 1, 2, 3, or 4.

[0574] In this case, the higher block may be an ancestor block (including a parent block) of the lower block, or may be any group of blocks. The ancestor block may refer to a pre-divided block (divided depth difference is 1 or more) in a previous step during the partitioning for obtaining the lower block. For example, Figure 4 The parent blocks of the 4N×2N sub-blocks 0 and 1 in candidate b can be Figure 4 The 4N×4N of candidate a in .

[0575] The candidate group of the higher block can be borrowed (or shared) from the lower block only when the candidate group of the higher block is at or greater than / larger than a predetermined first boundary value, or only when it is at or less than / less than a predetermined second boundary value.

[0576] The boundary value may be defined as the minimum or maximum size of a block that a candidate group is allowed to borrow. Only one or two of the boundary values ​​may be supported, and the boundary value may be expressed as a width W, height H, W×H, W*H, etc. of a block, where each of W and H is an integer of 8, 16, 32, 64 or higher.

[0577] On the other hand, the candidate group of the lower block can be borrowed from the higher block only when the candidate group of the lower block is at or greater than / greater than a predetermined third boundary value. Alternatively, the candidate group of the lower block can be borrowed from the higher block only when the candidate group of the lower block is at or less than / less than a predetermined fourth boundary value.

[0578] In this case, the boundary value may be defined as the minimum or maximum size of the block that the candidate group is allowed to borrow. Only one or two of the boundary values ​​may be supported, and the boundary value may be expressed as the width W, height H, W×H, W*H, etc. of the block, where each of W and H is an integer of 4, 8, 16, 32 or higher.

[0579] In this case, the first boundary value (or the second boundary value) may be equal to or greater than the third boundary value (or the fourth boundary value).

[0580] Candidate borrowing (or sharing) may be selectively used based on any one of the above-described embodiments, and candidate group borrowing may be selectively used based on a combination of at least two of the first to third embodiments. In addition, candidate group borrowing may be selectively used based on any detailed configuration of the embodiments, and candidate group borrowing may be selectively used according to a combination of one or more detailed configurations.

[0581] In addition, information indicating whether to borrow a candidate group, information about block attributes (size / type / position / aspect ratio) involved in candidate group borrowing, and information about partitioning status (partitioning scheme, partitioning type, partitioning depth, etc.) can be explicitly processed. In addition, encoding factors such as image type and color component can serve as input variables in candidate group borrowing settings. Candidate group borrowing can be performed based on this information and encoding / decoding settings.

[0582] The prediction related information generated by the prediction mode encoder may be transmitted to the coding unit and included in the bitstream.

[0583] In the above description, the basic block may be determined as one of a target block (coding block, prediction block) or a related block (higher block), and the description thereof may be derived through the above-mentioned examples.

[0584] In addition, the explanation is based on the following assumption: the referenced adjacent block used to construct the prediction mode candidate group may have a high correlation or a low correlation with the basic block, but the prediction mode of the corresponding block is not included in the candidate group when the candidate group is constructed based on the correlation between the blocks. Alternatively, it can be derived from the above example that a setting for changing the candidate group construction level to a lower level can be applied.

[0585] The method of the present disclosure can be implemented as a program instruction that can be executed by various computer devices and recorded in a computer-readable medium. The computer-readable medium can include program instructions, data files and data structures individually or in combination. The program instructions recorded on the computer-readable medium can be specially designed for the present disclosure, or are known to those skilled in the art of computer software, and are therefore available.

[0586] The computer readable medium may include a hardware device that is particularly suitable for storing and executing program instructions, such as a read-only memory (ROM), a random access memory (RAM), a flash memory, etc. The program instructions may include a machine language code generated by a compiler or a high-level language code that can be executed by an interpreter in a computer. The above-mentioned hardware device may be configured to operate as one or more software modules to perform operations according to the present disclosure, and vice versa.

[0587] Furthermore, the above-described methods or apparatuses may be implemented in combination or separation of all or part of their configurations or functions.

[0588] Although the present disclosure has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and variations may be made to the present disclosure without departing from the scope and spirit of the present disclosure.

[0589] [Industrial Application]

[0590] The present disclosure may be used to encode / decode images. < / l-1> < / abt> < / sbt> < / qt>

Claims

1. An image decoding method performed by an image decoding device, comprising: Determine a reference pixel line of the current block from the reference pixel line candidate set; Determine the intra prediction mode of the current block from a candidate set of intra prediction modes; as well as The current block is predicted based on the intra prediction mode of the current block and the reference pixel line.

2. The method according to claim 1, wherein: The reference pixel line candidate set includes a first reference pixel line, a second reference pixel line, and a third reference pixel line, the first reference pixel line is adjacent to the left side or above the current block, the second reference pixel line is adjacent to the left side or above the first reference pixel line, and the third reference pixel line is adjacent to the left side or above the second reference pixel line.

3. The method according to claim 1, wherein: The reference pixel line of the current block is determined based on reference pixel line information indicating the reference pixel line from the reference pixel line candidate set. 4 . The method according to claim 1 , when the current block is a luma block, wherein the intra prediction mode of the current block is determined from an intra prediction mode candidate set based on intra prediction information.

5. The method according to claim 4, wherein: The intra prediction mode candidate set includes at least one priority allocation mode and a most probable mode MPM, and The priority allocation mode is one of a planar mode, a DC mode, a vertical mode and a horizontal mode.

6. The method according to claim 5, wherein: The MPM includes a first MPM which is an intra prediction mode of a neighboring block of the current block and a second MPM which is an intra prediction mode derived from the first MPM, and The index of the second MPM is 1 or 2 greater or less than the index of the first MPM.

7. The method according to claim 4, wherein: The intra-frame prediction information includes priority allocation mode flag information, MPM flag information, MPM index information and non-MPM index information. The MPM index information is binarized by truncated Rice binarization, and The non-MPM index information is binarized by truncated unary binarization.

8. The method according to claim 1, wherein adaptively determining the intra prediction mode candidate set based on the shape of the current block, When the width of the current block is greater than the height of the current block, the intra prediction mode candidate set does not include modes with indexes between 2 and 15 having a lower left direction, but includes modes with indexes between 67 and 80 having an upper right direction, and When the height of the current block is greater than the width of the current block, the intra prediction mode candidate set does not include modes with indexes between 53 and 66 having an upper right direction, but includes modes with indexes between -14 and -1 having a lower left direction.

9. An image encoding method performed by an image encoding device, comprising: Determine a reference pixel line of the current block from the reference pixel line candidate set; Determine the intra prediction mode of the current block from a candidate set of intra prediction modes; as well as The current block is predicted based on the intra prediction mode of the current block and the reference pixel line.

10. A method for sending a bit stream, comprising: generating a bitstream to be decoded by an image decoding method; and sending the bit stream, and Wherein, the image decoding method comprises: Determine a reference pixel line of the current block from the reference pixel line candidate set; Determine an intra prediction mode of the current block from an intra prediction mode candidate set; and The current block is predicted based on the intra prediction mode of the current block and the reference pixel line.

Images