Image encoding / decoding method and apparatus

Abstract

An image encoding / decoding method and apparatus determine an intra prediction mode of a current block in a video signal; constructing a reference sample of the current block; generating a prediction block of the current block based on the intra prediction mode and the constructed reference sample; reconstructing the current block on the basis of the prediction block and the residual block of the current block, the constructed reference samples including left and right reference samples, the left and right reference samples being reconstructed blocks adjacent to the left and right boundaries of the current block, respectively, generating a prediction block of a current block includes generating prediction samples of a current sample in the current block with a specific number of reference samples among constructed reference samples, the specific number being 5, the specific number of reference samples including a top reference sample, two left-side and one right-side reference samples, and the specific number of reference samples including two left-side and one right-side reference samples, when an intra prediction mode is a non-directional mode. The two left side reference samples comprise the bottommost reference sample in the left side reference samples, the position of the bottommost reference sample relative to the left upper side sample of the current block is (-1, H-1), and H refers to the height of the current block.

Description

[0001] This application is a divisional application of the invention patent application with the application date of September 9, 2019, application number 201980058599.5, and invention name “Image encoding / decoding method and device”. Technical Field

[0002] The present disclosure relates to image encoding / decoding methods and devices. Background Art

[0003] With the provision of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data has increased rapidly. Therefore, in order to perform various services or tasks through image prediction in various systems, the demand for improving the performance and efficiency of image processing systems has increased significantly, but the research and development results that can respond to this atmosphere are not enough.

[0004] Thus, in the conventional image encoding and decoding methods and apparatuses, it is necessary to improve the performance of image processing, especially the performance of image encoding or image decoding. Summary of the invention

[0005] Technical issues

[0006] An object of the present disclosure is to provide a method and apparatus for deriving an intra prediction mode according to a color component.

[0007] An object of the present disclosure is to provide a method and apparatus for configuring reference pixels for intra prediction.

[0008] The object of the present disclosure is to provide an image encoding / decoding method and apparatus for modifying intra-frame prediction using arbitrary pixels.

[0009] Technical Solutions

[0010] The image encoding / decoding method and apparatus according to the present disclosure may determine an intra prediction mode of a target block, generate a prediction block of the target block based on the intra prediction mode, and modify the prediction block.

[0011] In the image encoding / decoding method and apparatus according to the present disclosure, the intra prediction mode of the target block may be determined as a mode in the prediction mode candidate group according to the state information of the target block.

[0012] In the image encoding / decoding method and device according to the contents of the present disclosure, when the color component of the target block is a luminance component, a prediction mode candidate group including a directional mode and a non-directional mode can be referenced, and when the color component of the target block is a chrominance component, a prediction mode candidate group supporting at least one of a directional mode, a non-directional mode, a color mode, or a color copy mode can be referenced.

[0013] In the image encoding / decoding method and apparatus according to the present disclosure, prediction mode candidate groups may be classified into a plurality of categories by considering the maximum number or priority of prediction modes that can be included in each category.

[0014] In the image encoding / decoding method and apparatus according to the present disclosure, the prediction mode candidate group may be classified into a first category including a non-directional mode and a directional mode and a second category including a color copy mode.

[0015] In the image encoding / decoding method and apparatus according to the present disclosure, first information and second information may be obtained, the first information specifying any one of a plurality of categories, and the second information specifying an intra prediction mode of a target block in the category according to the first information.

[0016] In the image encoding / decoding method and apparatus according to the present disclosure, the intra prediction mode of the target block may be determined from a designated category based on the first information and the second information.

[0017] In the image encoding / decoding method and apparatus according to the present disclosure, when only one prediction mode is included in the category according to the first information, the second information may not be acquired.

[0018] In the image encoding / decoding method and apparatus according to the present disclosure, reference pixels for intra-frame prediction may also be constructed, but the reference pixels may belong to all or part of a plurality of reference pixel rows supported in the decoding device.

[0019] In the image encoding / decoding method and apparatus according to the present disclosure, it may further include: performing at least one of a weighting filter or an interpolation filter on the constructed reference pixels.

[0020] In the image encoding / decoding method and device according to the contents of the present disclosure, modification of the prediction block can be selectively performed based on predetermined coding information, and the coding information may include at least one of the following: image type, color component, status information, coding mode, intra-frame prediction mode, whether to apply intra-frame prediction based on sub-block units, or reference pixel rows.

[0021] According to the content of the present disclosure, the method for decoding a video signal using a decoding device includes: determining an intra-frame prediction mode of a current block in the video signal; constructing a reference sample of the current block; and generating a prediction block of the current block based on the intra-frame prediction mode and the constructed reference sample; reconstructing the current block based on the prediction block and a residual block of the current block, wherein the constructed reference sample includes a left reference sample and a right reference sample, wherein the left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block and adjacent to the right boundary of the current block. A reconstructed block adjacent to the current block, wherein generating the prediction block of the current block includes, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, wherein the specific number is determined to be 5, wherein the specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample, wherein the two left reference samples include a bottommost reference sample in the left reference samples, and wherein a position of the bottommost reference sample relative to an upper left sample of the current block is (-1, H-1), and H refers to a height of the current block.

[0022] According to the content of the present disclosure, a method for encoding a video signal using an encoding device includes: determining an intra-frame prediction mode of a current block in the video signal; constructing a reference sample of the current block; using a prediction block to generate a residual block of the current block, the prediction block is generated based on the intra-frame prediction mode and the reference sample; and encoding the residual block into a bit stream, wherein the constructed reference sample includes a left reference sample and a right reference sample, wherein the left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block and adjacent to the right boundary of the current block. An adjacent reconstructed block, wherein generating a prediction block of the current block includes, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, wherein the specific number is determined to be 5, wherein the specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample, wherein the two left reference samples include a bottommost reference sample in the left reference samples, and wherein a position of the bottommost reference sample relative to an upper left sample of the current block is (-1, H-1), and H refers to a height of the current block.

[0023] A non-transitory computer-readable recording medium according to the present disclosure stores a bit stream generated by a method for encoding a video signal using an encoding device, the method comprising: determining an intra-frame prediction mode of a current block in the video signal; constructing a reference sample of the current block; generating a residual block of the current block using a prediction block, the prediction block being generated based on the intra-frame prediction mode and the reference sample; and encoding the residual block into a bit stream, wherein the constructed reference sample comprises a left reference sample and a right reference sample, wherein the left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block. a reconstructed block adjacent to a right boundary of the current block, wherein generating the prediction block of the current block includes, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, wherein the specific number is determined to be 5, wherein the specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample, wherein the two left reference samples include a bottommost reference sample among the left reference samples, and wherein a position of the bottommost reference sample relative to an upper left sample of the current block is (-1, H-1), and H refers to a height of the current block.

[0024] According to the content of the present disclosure, a method for decoding a video signal using a decoding device includes: determining an intra-frame prediction mode applied to intra-frame prediction of a current block; generating a prediction block of the current block based on the intra-frame prediction mode; and reconstructing the current block based on the prediction block and a residual block of the current block, wherein generating the prediction block of the current block includes: configuring reference samples for the intra-frame prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and deriving a value of a second prediction sample within the prediction block based on at least one of the configured reference samples and the first prediction sample within the prediction block.

[0025] A method for encoding a video signal using an encoding device according to the present disclosure includes: determining an intra-frame prediction mode applied to intra-frame prediction of a current block; generating a prediction block of the current block based on the intra-frame prediction mode; and generating a residual block of the current block based on the prediction block; and encoding the residual block into a bitstream, wherein generating the prediction block of the current block includes: configuring reference samples for the intra-frame prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and deriving a value of a second prediction sample within the prediction block based on at least one of the configured reference samples and the first prediction sample within the prediction block.

[0026] A non-transitory computer-readable recording medium according to the present disclosure stores a bitstream generated by a method for encoding a video signal, the method comprising: determining an intra-frame prediction mode applied to intra-frame prediction of a current block; generating a prediction block of the current block based on the intra-frame prediction mode; and generating a residual block of the current block based on the prediction block; and encoding the residual block into a bitstream, wherein generating the prediction block of the current block comprises: configuring reference samples for the intra-frame prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and deriving a value of a second prediction sample within the prediction block based on at least one of the configured reference samples and the first prediction sample within the prediction block.

[0027] Beneficial Effects

[0028] According to the present disclosure, the intra prediction mode can be efficiently derived.

[0029] According to the present disclosure, the efficiency of intra prediction can be improved by selectively using reference pixel rows and predetermined filtering.

[0030] According to the present disclosure, intra prediction accuracy and encoding performance may be improved by modifying a prediction block. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present disclosure.

[0032] Figure 2 It is a component block diagram of an image encoding device according to an embodiment of the present disclosure.

[0033] Figure 3 It is a component block diagram of an image decoding device according to an embodiment of the present disclosure.

[0034] Figure 4 is an exemplary diagram showing various division shapes that can be obtained in the block division unit of the present disclosure.

[0035] Figure 5 is an example diagram showing a prediction mode in intra prediction according to an embodiment of the present disclosure.

[0036] Figure 6 is an arrangement diagram of a target block and blocks adjacent thereto according to an embodiment of the present disclosure.

[0037] Figure 7 is a flowchart illustrating a modification method of intra prediction according to an embodiment of the present disclosure.

[0038] Figure 8 is an arrangement diagram of pixels of a target block and a neighboring block according to an embodiment of the present disclosure.

[0039] Fig. 9 is an exemplary diagram of a modification method based on multiple reference pixel rows according to an embodiment of the present disclosure.

[0040] Fig.10 is a flowchart illustrating a modification method of intra prediction according to an embodiment of the present disclosure.

[0041] Fig.11 is an example diagram of an arbitrary pixel used to modify a predicted pixel according to an embodiment of the present disclosure.

[0042] Fig.12 is an example diagram of performing modification based on an arbitrary pixel according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0043] Best Practices

[0044] The image encoding / decoding method and apparatus according to the present disclosure may determine an intra prediction mode of a target block, generate a prediction block of the target block based on the intra prediction mode, and modify the prediction block.

[0045] In the image encoding / decoding method and apparatus according to the present disclosure, the intra prediction mode of the target block may be determined as a mode in the prediction mode candidate group according to the state information of the target block.

[0046] In the image encoding / decoding method and device according to the contents of the present disclosure, when the color component of the target block is a luminance component, a prediction mode candidate group including a directional mode and a non-directional mode can be referenced, and when the color component of the target block is a chrominance component, a prediction mode candidate group supporting at least one of a directional mode, a non-directional mode, a color mode, or a color copy mode can be referenced.

[0047] In the image encoding / decoding method and apparatus according to the present disclosure, prediction mode candidate groups may be classified into a plurality of categories by considering the maximum number or priority of prediction modes that can be included in each category.

[0048] In the image encoding / decoding method and apparatus according to the present disclosure, the prediction mode candidate group may be classified into a first category including a non-directional mode and a directional mode and a second category including a color copy mode.

[0049] In the image encoding / decoding method and apparatus according to the present disclosure, first information specifying any one of a plurality of categories and second information specifying an intra prediction mode of a target block in the category according to the first information may be obtained.

[0050] In the image encoding / decoding method and apparatus according to the present disclosure, the intra prediction mode of the target block may be determined from a designated category based on the first information and the second information.

[0051] In the image encoding / decoding method and apparatus according to the present disclosure, when only one prediction mode is included in the category according to the first information, the second information may not be obtained.

[0052] In the image encoding / decoding method and apparatus according to the present disclosure, reference pixels for intra-frame prediction may also be constructed, but the reference pixels may belong to all or part of a plurality of reference pixel rows supported in the decoding device.

[0053] In the image encoding / decoding method and apparatus according to the present disclosure, it may further include: executing at least one of a weighting filter or an interpolation filter on the constructed reference pixels.

[0054] In the image encoding / decoding method and device according to the contents of the present disclosure, modification of the prediction block can be selectively performed based on predetermined encoding information, and the encoding information may include at least one of the following: image type, color component, status information, encoding mode, intra-frame prediction mode, whether to apply intra-frame prediction based on sub-block units, or reference pixel rows.

[0055] Embodiments of the present invention

[0056] The present invention may be varied and modified in various ways and will be described with reference to different exemplary embodiments, some of which will be described and illustrated in the accompanying drawings. However, these embodiments are not intended to limit the present invention, but are to be understood as including all modifications, equivalents and alternatives within the spirit and technical scope of the present invention. The same reference numerals in the drawings always represent the same elements.

[0057] Although the terms first, second, etc. can be used to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teachings of the present invention, a first element can be referred to as a second element, and similarly a second element can be referred to as a first element. The term "and / or" includes any and all combinations of multiple associated listed items.

[0058] It will be understood that when an element is referred to as being "connected to" or "coupled to" another element, the element may be directly connected or directly coupled to the other element or there may be intervening elements. Conversely, when an element is referred to as being "directly connected to" or "directly coupled to" another element, there are no intervening elements.

[0059] The technical terms used in this article are only used to describe specific embodiments and are not intended to limit the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. It will also be understood that when used in this specification, the terms "including" and / or "having" specify the presence of stated features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or their groups.

[0060] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meanings as those generally understood by those of ordinary skill in the art to which the present disclosure belongs. Terms commonly used and defined in dictionaries should be interpreted as having the same contextual meanings as those in the relevant art, and unless explicitly defined in the present disclosure, they should not be interpreted as ideal or overly formal meanings.

[0061] Typically, one or more color spaces can be configured according to the color format of the image. One or more pictures of a specific size or one or more pictures of different sizes can be configured according to the color format. In the example, in the YCbCr color configuration, color formats such as 4:4:4, 4:2:2, 4:2:0, monochrome (configured using only Y), etc. can be supported. In the example, for YCbCr4:2:0, 1 luminance component (Y in this example) and 2 chrominance components (Cb / Cr in this example) can be configured, and in this case, the configuration ratio of the chrominance component to the luminance component can be an aspect ratio of 1:2. In the example, for 4:4:4, the same aspect ratio can be used. As in the above example, when an image is configured with one or more color spaces, the image can be divided into each color space.

[0062] Since images can be classified into I, P, B, etc. according to image types (e.g., picture type, slice type, tile group type, tile type, brick type, etc.), image type I may represent an image that is self-encoded without using a reference image, image type P may represent an image that is encoded by using a reference image but only allows forward prediction, and image type B may represent an image that is encoded by using a reference image and allows forward / backward prediction, but some of the above types may be combined (combine P and B) according to the encoding settings, or image types in other configurations may be supported.

[0063] Various encoding / decoding information generated in the present disclosure may be processed explicitly or implicitly. In this regard, it is understood that explicit processing generates encoding / decoding information in sequences, slices, tile groups, tiles, bricks, blocks, sub-blocks, etc. to store it in a bitstream, and parses the relevant information in the decoder in the same unit as the encoder to reconstruct the relevant information into decoded information. In this case, it is understood that implicit processing processes the encoding / decoding information in the encoder and decoder with the same processing, rules, etc.

[0064] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present disclosure.

[0065] Reference Figure 1 The image encoding device 105 and the image decoding device 100 can be user terminals such as personal computers (PCs), notebooks, personal digital assistants (PDAs), portable multimedia players (PMPs), portable play stations (PSPs), wireless communication terminals, smart phones or TVs, or server terminals such as application servers, business servers, etc., and the image encoding device 105 and the image decoding device 100 may include various devices equipped with communication devices, for example, a communication modem for communicating with various instruments or wired and wireless communication networks, a memory (120, 125) for storing various programs and data for inter-frame prediction or intra-frame prediction for encoding or decoding images, or a processor (110, 115) for performing program operations and control by running programs, etc.

[0066] In addition, the image encoded as a bit stream by the image encoding device 105 can be sent to the image decoding device 100 in real time or non-real time through a wired and wireless communication network such as the Internet, a wireless local area network, a wireless LAN network, a wireless broadband network, or a mobile radio communication network, or through various communication interfaces such as a cable or a universal serial bus, and decoded in the image decoding device 100. And the bit stream can be reconstructed into an image and played. In addition, the image encoded as a bit stream by the image encoding device 105 can be sent from the image encoding device 105 to the image decoding device 100 through a computer-readable recording medium.

[0067] The above-mentioned image encoding device and image decoding device may be separate devices, but they may be made into one image encoding / decoding device according to an embodiment. In this case, some configurations of the image encoding device may be implemented to include at least the same structure as some configurations of the image decoding device as substantially the same technical element or to perform at least the same function.

[0068] Therefore, in the detailed description of the following technical elements and their operating principles, etc., repeated description of the corresponding technical elements will be omitted. In addition, since the image decoding device corresponds to a computing device that applies the image encoding method performed in the image encoding device to decoding, the image encoding device will be mainly described below.

[0069] The computing device may include: a memory storing a program or software module embodying the image encoding method and / or the image decoding method; and a processor connected to the memory to execute the program. In this case, the image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder, respectively.

[0070] Figure 2 It is a component block diagram of an image encoding device according to an embodiment of the present disclosure.

[0071] Reference Figure 2 , the image encoding device 20 may include a prediction unit 200, a subtraction unit 205, a transformation unit 210, a quantization unit 215, an inverse quantization unit 220, an inverse transformation unit 225, an addition unit 230, a filter unit 235, a coded picture buffer 240 and an entropy encoding unit 245.

[0072] The prediction unit 200 can be implemented by using a prediction module, a software module, and can generate a prediction block for a block to be encoded in an intra-frame prediction method or an inter-frame prediction method. The prediction unit 200 can generate a prediction block by predicting the target block to be encoded in the current image. In other words, the prediction unit 200 can generate a predicted pixel value for each pixel by predicting the pixel value of each pixel in the target block to be encoded in the image according to intra-frame prediction or inter-frame prediction to generate a prediction block. In addition, the prediction unit 200 can cause the coding unit to encode information about the prediction mode by sending information necessary for generating the prediction block (for example, information about a prediction mode such as an intra-frame prediction mode or an inter-frame prediction mode) to the coding unit. In this case, the processing unit that performs the prediction and the processing unit that determines the prediction method and specific content can be determined according to the encoding setting. For example, the prediction method, prediction mode, etc. can be determined in the prediction unit, and the prediction can be performed in the transform unit.

[0073] In the inter prediction unit, the inter prediction unit can be divided into time prediction and space prediction based on the reference picture. For time prediction, it can be a prediction method that finds motion in a picture that is different from the current picture in time, and for space prediction, it can be a prediction method that finds motion in the current picture (encoding completed area, a preset area adjacent to the target block that is the same as the current picture in time). This can be integrated and managed through a reference picture list, or it can be managed by dividing the coding mode.

[0074] In addition, in the inter-frame prediction unit, the inter-frame prediction unit can be divided into a translation motion model and a non-translation motion model according to the motion prediction method. For the translation motion model, prediction can be performed by considering only parallel translation, while for the non-translation motion model, prediction can be performed by considering motions such as rotation, distance, zooming in / out, and parallel translation. When assuming unidirectional prediction, the translation motion model may require one motion vector, but the non-translation motion model may require one or more motion information (for example, one motion vector + rotation angle / scaling factor, more than 2 motion vectors, etc. It is assumed below that more than 2 motion vectors are used). For the non-translation motion model, each motion vector can be information applied to a preset position of a target block (such as the upper left vertex, upper right vertex, lower left vertex, etc. of the target block), and the position of the target block area to be predicted by the corresponding motion vector can be obtained in units of pixels or sub-blocks (integers greater than or equal to 2, such as 4×4, 8×8). For the inter-frame prediction unit, according to the motion model, some of the processes described later can be applied together, and some of the processes can be applied separately.

[0075] The inter prediction unit may include a reference picture construction unit, a motion prediction unit, a motion compensation unit, a motion information determination unit, and a motion information encoding unit. The reference picture construction unit may include a picture encoded before or after the current picture in the reference picture list (L0, L1). When the current image is configured with the reference picture according to the encoding setting, the prediction block may be obtained from the reference picture included in the reference picture list and the reference picture included in at least one of the reference picture lists.

[0076] In the inter-frame prediction unit, the reference picture construction unit may include a reference picture interpolation unit, and perform interpolation processing on the fractional pixels according to the interpolation accuracy. For example, an 8-tap interpolation filter based on DCT may be applied to the luminance component, and a 4-tap interpolation filter based on DCT may be applied to the chrominance component.

[0077] In the inter-frame prediction unit, the motion prediction unit is a process of searching for a block having a high correlation with a target block through a reference picture, and various methods may be used, such as FBMA (block matching algorithm based on full search), TSS (three-step search), etc., and the motion compensation unit means a process of obtaining a prediction block through motion prediction processing.

[0078] In the inter prediction mode, the motion information determination unit may perform processing for selecting optimal motion information of the target block, and may encode the motion information by a motion information encoding mode such as a skip mode, a merge mode, a competitive mode, etc. The mode may be configured by combining supported modes according to the motion model, and skip mode (shift), skip mode (non-shift), merge mode (shift), merge mode (non-shift), competitive mode (shift), and competitive mode (non-shift) may be examples thereof. Depending on the encoding setting, some of the modes may be included in the candidate group.

[0079] The motion information coding mode can obtain the predicted value of the motion information (motion vector, reference picture, prediction direction, etc.) of the target block in at least one candidate block, and when two or more candidate blocks are supported, the best candidate selection information can be generated. In the skip mode (without residual signal) and the merge mode (with residual signal), the predicted value can be used as the motion information of the target block, and in the competitive mode, the motion information of the target block and the difference information with the predicted value can be generated.

[0080] The candidate group of motion information prediction values ​​for the target block can be adaptive and have various configurations according to the motion information encoding mode. The motion information of the blocks spatially adjacent to the target block (e.g., left block, upper block, upper left block, upper right block, lower left block, etc.) can be included in the candidate group, the motion information of the blocks temporally adjacent to the target block (e.g., left block, right block, upper block, lower block, upper left block, upper right block, lower left block, lower right block, etc., including the block <center> in other images that responds to or corresponds to the target block) can be included in the candidate group, and the mixed motion information of the spatial candidate and the temporal candidate (e.g., the motion information of the spatially adjacent blocks and the motion information of the temporally adjacent blocks, the information obtained by the average value, center value, etc. of more than 2 candidates. The motion information can be obtained in units of the target block or the sub-block of the target block, and can be included in the candidate group.

[0081] There may be a priority for configuring the motion information predictor candidate group. The order included in the predictor candidate group configuration may be determined according to the priority, and the candidate group may be configured when the number of candidate groups (determined according to the motion information encoding mode) is reached according to the priority. In this case, the priority may be determined in the order of motion information of spatial neighboring blocks, motion information of temporal neighboring blocks, and mixed motion information of spatial candidates and temporal candidates, but the priority may also be modified.

[0082] For example, among spatially adjacent blocks, the blocks may be included in the candidate group in an order of left-top-top right-bottom left-top left, etc., and for temporally adjacent blocks, the blocks may be included in the candidate group in an order of bottom right-middle-right-bottom, etc.

[0083] The subtraction unit 205 can generate a residual block by subtracting the prediction block from the target block. In other words, the subtraction unit 205 can generate a residual block (block-shaped residual signal) by calculating the difference between the pixel value of each pixel of the target block to be encoded and the predicted pixel value of each pixel of the prediction block generated in the prediction unit. In addition, the subtraction unit 205 can generate a residual block according to a unit other than a block unit obtained in a block division unit described later.

[0084] The transform unit 210 may transform a signal belonging to the spatial domain into a signal belonging to the frequency domain, and the signal obtained in the transform process is referred to as a transform coefficient. For example, a transform block having a transform coefficient may be obtained by transforming a residual block having a residual signal sent from a subtraction unit, and an input signal is determined according to an encoding setting, and the input signal is not limited to the residual signal.

[0085] The transformation unit can transform the residual block by using a transformation method such as Hadamard Transform, DST-based transformation (discrete sine transform), DCT-based transformation (discrete cosine transform), etc., and various transformation methods improved and modified thereto can be used, but the present invention is not limited thereto.

[0086] At least one of the transform methods may be supported, and at least one detailed transform method in each transform method may be supported. In this case, the detailed transform method may be a transform method in which a portion of a basic vector is configured differently in each transform method.

[0087] For example, for DCT, more than one detailed transform method between DCT-1 to DCT-8 may be supported, and for DST, more than one detailed transform method between DST-1 to DST-8 may be supported. The transform method candidate group may be configured by configuring a part of the detailed transform method. In the example, DCT-2, DCT-8, and DST-7 may be configured as a transform method candidate group to perform transform.

[0088] Transformation can be performed in the horizontal / vertical direction. For example, by performing a total two-dimensional transformation, performing a one-dimensional transformation in the horizontal direction using a transformation method of DCT-2 and performing a one-dimensional transformation in the vertical direction using a transformation method of DST-7, the pixel value in the spatial domain can be transformed into the frequency domain.

[0089] The transformation can be performed by using a fixed transformation method or by adaptively selecting a transformation method according to the encoding setting. In this case, for the adaptive case, the transformation method can be selected by using an explicit method or an implicit method. For the explicit case, each transformation method selection information or transformation method set selection information applied to the horizontal and vertical directions can be generated in units of blocks, etc. For the implicit case, the encoding setting can be defined according to the image type (I / P / B), color component, block size / shape / position, intra-frame prediction mode, etc., and accordingly, a predetermined transformation method can be selected.

[0090] In addition, some transforms may be omitted depending on the encoding settings. In other words, this means that one or more of the various horizontal / vertical units may be omitted explicitly or implicitly.

[0091] In addition, the transform unit may send information required to generate the transform block to the encoding unit to encode the information and store the encoded information in a bitstream to send it to the decoder, and the decoding unit of the decoder may parse the information for use in the inverse transform process.

[0092] The quantization unit 215 may quantize the input signal, and in this case, the signal obtained in the quantization process is referred to as a quantization coefficient. For example, a quantization block having a quantization coefficient may be obtained by quantizing a residual block having a residual transform coefficient sent from a transform unit, and the input signal is determined according to an encoding setting, and the input signal is not limited to the residual transform coefficient.

[0093] The quantization unit may quantize the transformed residual block by using a quantization method such as dead zone uniform threshold quantization, a quantization weighting matrix, etc., and may use various improved and modified quantization methods thereof, but the present invention is not limited thereto.

[0094] The quantization process can be omitted according to the encoding setting. For example, the quantization process (including the inverse process) can be omitted according to the encoding setting (for example, the quantization parameter is 0, that is, the lossless compression environment). In another example, when the compression performance through quantization is not exhibited according to the characteristics of the image, the quantization process can be omitted. In this case, the area where the quantization process is omitted in the quantization block (M×N) can be all or part of the area (M / 2×N / 2, M×N / 2, M / 2×N, etc.), and the quantization omission selection information can be determined implicitly or explicitly.

[0095] The quantization unit may send information required to generate a quantization block to the encoding unit to encode the information, and store the encoded information in a bit stream to send it to the decoder, and the decoding unit of the decoder may parse the information to use the parsed information for an inverse quantization process.

[0096] In the above example, it is described that the residual block is assumed to be transformed and quantized by the transform unit and the quantization unit, but the residual block having the transform coefficient may be generated by transforming the residual signal of the residual block, and the quantization process may not be performed, only the quantization process may be performed without transforming the residual signal of the residual block into the transform coefficient, and both the transform process and the quantization process may not be performed. This may be determined according to the encoding setting.

[0097] The inverse quantization unit 220 inversely quantizes the residual block quantized by the quantization unit 215. In other words, the inverse quantization unit 220 generates a residual block having frequency coefficients by inversely quantizing the quantized frequency coefficient column.

[0098] The inverse transform unit 225 inversely transforms the residual block inversely quantized by the inverse quantization unit 220. In other words, the inverse transform unit 225 generates a residual block having a pixel value, that is, a reconstructed residual block, by inversely transforming the frequency coefficients of the inversely quantized residual block. In this case, the inverse transform unit 225 can perform inverse transformation by inversely using the transformation method used in the transform unit 210.

[0099] The adding unit 230 reconstructs the target block by adding the prediction block predicted in the prediction unit 200 and the residual block reconstructed by the inverse transform unit 225. The reconstructed target block may be stored in the coded picture buffer 240 as a reference picture (or reference block), and may be used as a reference picture when encoding the next block of the target block, other blocks, or other pictures in the future.

[0100] The filter unit 235 may include one or more post-processing filter processes, such as a deblocking filter, SAO (sample adaptive offset), ALF (adaptive loop filter), etc. The deblocking filter can remove block distortion generated on the boundary between blocks in the reconstructed picture. The ALF can perform filtering based on a value obtained by comparing the reconstructed image after the block filter is filtered by the block filter with the original image. The SAO can reconstruct the offset difference with the original image in pixels for the residual block to which the deblocking filter is applied. Such a post-processing filter can be applied to the reconstructed picture or block.

[0101] The coded picture buffer 240 may store the block or picture reconstructed in the filter unit 235. The reconstructed block or picture stored in the coded picture buffer 240 may be provided to the prediction unit 200 which performs intra prediction or inter prediction.

[0102] The entropy encoding unit 245 scans the generated quantized frequency coefficient column according to various scanning methods to generate a quantized coefficient column, and encodes and outputs the quantized coefficient column by using an entropy encoding method, etc. The scanning pattern may be set to one of various patterns such as a zigzag, a diagonal line, a grid shape, etc. In addition, encoded data including encoding information transmitted from each construction unit may be generated and output in a bit stream.

[0103] Figure 3 It is a component block diagram of an image decoding device according to an embodiment of the present disclosure.

[0104] Reference Figure 3 , the image decoding device 30 can be configured by including: an entropy decoding unit 305, a prediction unit 310, an inverse quantization unit 315, an inverse transform unit 320, an adder-subtractor 325, a filter 330 and a decoded picture buffer 335.

[0105] In addition, the prediction unit 310 may be configured by including an intra prediction module and an inter prediction module.

[0106] First, when an image bit stream transmitted from the image encoding device 20 is received, it may be transmitted to the entropy decoding unit 305 .

[0107] The entropy decoding unit 305 may decode the decoded data including the quantized coefficients and the decoded information transmitted to each construction unit by decoding the bit stream.

[0108] The prediction unit 310 may generate a prediction block based on the data transmitted from the entropy decoding unit 305. In this case, based on the reference image stored in the decoded picture buffer 335, a reference picture list using a default configuration method may be configured.

[0109] The inter prediction unit may include a reference picture construction unit, a motion compensation unit, and a motion information decoding unit, and some may perform the same process as the encoder and some may perform an inverse derivation process.

[0110] The inverse quantization unit 315 may inverse quantize the quantized transform coefficient transmitted in the bitstream and decoded in the entropy decoding unit 305 .

[0111] The inverse transform unit 320 may generate a residual block by applying inverse DCT, inverse integer transform, or an inverse transform method similar thereto to the transform coefficient.

[0112] In this case, the inverse quantization unit 315 and the inverse transform unit 320 may be implemented in various ways while inversely performing the processes performed in the transform unit 210 and the quantization unit 215 of the above-described image encoding device 20. For example, inverse transform and the same processes shared with the transform unit 210 and the quantization unit 215 may be used, and the transform and inverse quantization processes may be performed inversely using information on the transform and quantization processes in the image encoding device 20 (e.g., transform size, transform shape, quantization type, etc.).

[0113] The residual block after the inverse quantization and inverse transform process may be added to the prediction block obtained by the prediction unit 310 to generate a reconstructed image block. Such addition may be performed by the adder-subtractor 325.

[0114] If necessary, the filter 330 may apply a deblocking filter to the reconstructed image block to remove blocking phenomena, and additionally use other loop filters before and after the decoding process to improve image quality.

[0115] The constructed and filtered image blocks may be stored in the decoded picture buffer 335 .

[0116] Although not shown in the drawings, a block dividing unit may be additionally included in the image encoding / decoding apparatus.

[0117] The block division unit can be divided into blocks of various units and sizes. The basic coding unit (or maximum coding unit. Coding tree unit CTU) may refer to the basic (or starting) unit for prediction, transformation, quantization, etc. in the image coding process. In this case, the basic coding unit can be configured to have a luminance basic coding block (or, maximum coding block. Coding tree block CTB) and two basic chrominance coding blocks according to the color format (YCbCr in this example), and the size of each block can be determined according to the color format. In addition, a coding block (CB) can be obtained according to the division process. A coding block can be understood as a unit that is not divided into more coding blocks according to a fixed limit, and can be set as a starting unit for division into lower units. In the present disclosure, a block can be understood as a broad concept including various shapes such as triangles, circles, etc. without being limited to squares.

[0118] It should be understood that what is mentioned below is for one color component, but can be changed proportionally and applied to other color components according to the ratio of the color format (for example, for YCbCr 4:2:0, the aspect ratio of the luminance component and the chrominance component is 2:1). In addition, it should be understood that block division depending on other color components (for example, in the case of depending on the result of block division of Y from Cb / Cr) is possible, but block division independent of each color component is also possible. In addition, a common block division setting can be used (considering proportional to the length ratio), but it is also necessary to consider and understand the use of separate block division settings according to the color component.

[0119] In the block division unit, the block can be described as M×N, and the maximum value and minimum value of each block can be obtained within a certain range. For example, when the maximum value of the block is set to 256×256 and the minimum value of the block is set to 4×4, a block of size 2m×2n (in this example, m and n are integers from 2 to 8), a block of size 2m×2m (in this example, m and n are integers from 2 to 128), or a block of size m×n (in this example, m and n are integers from 4 to 256) can be obtained. In this case, m and n may be the same or different, and one or more ranges of blocks supporting the maximum value, minimum value, etc. may be generated.

[0120] For example, information about the maximum size, minimum size, etc. of a block may be generated, and information about the maximum size, minimum size, etc. of a block may be generated in some block partition settings. In this case, the former may be range information about the maximum size and minimum size that can be generated in an image, and the latter may be information about the maximum size and minimum size that can be generated according to some partition settings. In this case, the partition settings may be defined by: image type (I / P / B), color component (YCbCr, etc.), block type (encoding / prediction / transformation / quantization, etc.), partition type (index or type), partition method (QT, BT, TT, etc. in the tree method, SI2, SI3, and SI4, etc. in the index method), and the like.

[0121] In addition, there may be restrictions on the aspect ratio (shape of the block) that a block can have, and a boundary value condition may be set for this. In this case, only blocks less than / equal to an arbitrary boundary value (k) may be supported, and k may be limited according to an aspect ratio such as A / B (A is a value longer than or equal to the width or height, and B is the remaining value), and k may be a real number greater than or equal to 1 (e.g., 1.5, 2, 3, 4, etc.). As in the above example, a restriction condition on the shape of a block in an image may be supported, or one or more restriction conditions may be supported according to a partition setting.

[0122] In summary, whether block partitioning is supported can be determined by the above-mentioned range and conditions, the partitioning settings mentioned later, etc. For example, when the block conditions of the candidate (child block) supporting the partitioning according to the block (parent block) are met, the corresponding partitioning can be supported, and otherwise, the corresponding partitioning may not be supported.

[0123] The block partitioning unit may be set to be associated with each construction unit for an image encoding device and an image decoding device, and the size and shape of the block may be determined by the process. In this case, the setting block may be defined differently according to the construction unit, and the prediction block of the prediction unit, the transform block of the transform unit, the quantization block of the quantization unit, etc. may correspond to the setting block. However, the block unit according to other construction units may be defined additionally without limitation thereto. The present disclosure mainly describes the case where the input and output are rectangular in each construction unit, but input / output of other shapes (e.g., right triangles, etc.) are possible in some construction units.

[0124] The size and shape of the initial (or starting) block in the block division unit may be determined by a higher unit. The initial block may be divided into smaller blocks, and when the optimal size and shape according to the division of the block is determined, the block may be determined as the initial block in the lower unit. In this case, the higher unit may be a coding block, and the lower unit may be a prediction block or a transform block, but they are not limited thereto and various modifications may be made. As in the above example, when the initial block in the lower unit is determined, a division process for finding the optimal size and shape may be performed as in the higher unit.

[0125] In summary, the block partitioning unit can divide the basic coding block (or the maximum coding block) into at least one coding block, and can divide the coding block into at least one prediction block / transform block / quantization block. In addition, the prediction block can be divided into at least one transform block / quantization block, and the transform block can be divided into at least one quantization block. In this case, some blocks may have a dependency relationship (i.e., defined by a higher unit and a lower unit) or an independent relationship with other blocks. In the example, the prediction block can be a higher unit for the transform block or an independent unit relative to the transform block, and various relationship settings can be made according to the block type.

[0126] Depending on the encoding setting, it can be determined whether the higher unit and the lower unit are combined. In this case, the combination between the units means that the encoding process in the lower unit (e.g., prediction unit, transform unit, inverse transform unit, etc.) is performed by the block in the higher unit (size and shape) without division from the higher unit to the lower unit. In other words, this may mean that the division process in multiple units is shared and the division information is generated in one of them (e.g., the higher unit).

[0127] In an example, (when a coding block is combined with a prediction block and a transform block), prediction, transform and inverse transform processes may be performed in the coding block.

[0128] In an example, (when a coding block is combined with a prediction block), a prediction process may be performed in the coding block, and transform and inverse transform processes may be performed in a transform block that is the same as or smaller than the coding block.

[0129] In an example, (when a coding block is combined with a transform block), a prediction process may be performed in a prediction block that is the same as or smaller than the coding block, and transform and inverse transform processes may be performed in the coding block.

[0130] In an example, (when a prediction block is combined with a transform block), a prediction process may be performed in a prediction block that is the same as or smaller than a coding block, and transform and inverse transform processes may be performed in the prediction block.

[0131] In the example, (when combining is not performed in any block), prediction processing can be performed in a prediction block that is the same as or smaller than the coding block, and transform and inverse transform processing can be performed in a transform block that is the same as or smaller than the coding block.

[0132] The above examples describe various cases regarding encoding, prediction, and transform blocks, but are not limited thereto.

[0133] For the combination between units, fixed settings can be supported in the image, or adaptive settings can be supported by considering various coding elements. In this case, the coding elements may include image type, color component, coding mode (intra / inter), partition setting, block size / shape / position, aspect ratio, prediction-related information (e.g., intra prediction mode, inter prediction mode, etc.), transform-related information (e.g., transform method selection information), quantization-related information (e.g., quantization area selection information, quantization transform coefficient encoding information, etc.), and the like.

[0134] As described above, when a block having an optimal size and shape is found, mode information (e.g., partition information, etc.) may be generated for the block. The mode information may be stored in a bitstream together with information generated in a structural unit to which the block belongs (e.g., prediction-related information, transform-related information, etc.), and transmitted to a decoder, and may be parsed in the same unit in the decoder and used for image decoding processing.

[0135] Hereinafter, the division method will be described, and for convenience of explanation, it is assumed that the initial block has a square shape, but it may be equally or similarly applied even when the initial block has a rectangular shape, and the present invention is not limited thereto.

[0136] The block partitioning unit may support various partitions. For example, it may support tree-based partitioning or index-based partitioning, and may support other methods. Tree-based partitioning may use various information (e.g., whether to partition, tree type, partition direction, etc.) to determine the partition shape, and index-based partitioning may use predetermined index information to determine the partition shape.

[0137] Figure 4 is an exemplary diagram showing various division shapes that can be obtained in the block division unit of the present disclosure.

[0138] In this example, it is assumed that a partition (or process) is performed to obtain Figure 4 The division shape can be obtained by multiple division actions, but it is not limited to this. Figure 4 Other division shapes not shown in the figure are also possible.

[0139] (Tree-based partitioning)

[0140] In the tree-based partitioning of the present disclosure, quadtree (QT), binary tree (BT), ternary tree (TT), etc. can be supported. When one tree method is supported, it can be called single-tree partitioning, and when more than two tree methods are supported, it can be called multi-tree partitioning.

[0141] QT refers to a method (n) in which a block is divided into two (i.e., quartered) in the horizontal and vertical directions, respectively, BT refers to a method (b to g) in which a block is divided into two in the horizontal or vertical direction, and TT refers to a method (h to m) in which a block is divided into three in the horizontal or vertical direction.

[0142] In this case, QT can support 4 division methods (o, p) by limiting the division direction to one of the horizontal and vertical directions. In addition, BT can support only methods with uniform sizes (b, c) or only methods with uneven sizes (d to g), or can mix and support both methods. In addition, TT can support only methods with divisions tilted in a specific direction (1:1:2, 2:1:1, etc. in the direction from left to right or from top to bottom) (h, j, k, m), or can support only methods (i, l) arranged along the center (1:2:1, etc.) or can mix and support both methods. In addition, a method (q) in which the division direction is divided into 4 (i.e., 16 divisions) in the horizontal and vertical directions, respectively, can also be supported.

[0143] In addition, in the tree method, the z-division method (b, d, e, h, i, j, o) may be supported only in the horizontal division direction or only in the vertical direction (c, f, g, k, l, m, p) or both methods may be mixed and supported. In this case, z may be an integer greater than or equal to 2, such as 2, 3, and 4.

[0144] In the present disclosure, description is made under the assumption that QT supports n, BT supports b and c, and TT supports i and l.

[0145] According to the encoding setting, one or more methods of tree partitioning may be supported. For example, QT may be supported, QT / BT may be supported, or QT / BT / TT may be supported.

[0146] The above example is the case where the basic tree partitioning is QT, and BT and TT are included in the additional partitioning method depending on whether other trees are supported, but various modifications are possible. In this case, information about whether other trees are supported (bt_enabled_flag, tt_enabled_flag, bt_tt_enabled_flag, etc. It can have a value of 0 or 1, and if it is 0, it is not supported, and if it is 1, it is supported) can be implicitly determined according to the encoding setting, or can be explicitly determined in units of sequences, pictures, slices, slice groups, tiles, bricks, etc.

[0147] Information about whether to split (tree_part_flag or qt_part_flag, bt_part_flag, tt_part_flag, bt_tt_part_flag, which may have a value of 0 or 1, and in the case of 0, do not split, and in the case of 1, split) may be included in the split information. In addition, according to the splitting method (BT and TT), information about the splitting direction (dir_part_flag or bt_dir_part_flag, tt_dir_part_flag, bt_tt_dir_part_flag. It may have a value of 0 or 1, and for 0, it is <width / horizontal>, and for 1, it is <height / vertical>) may be added, which may be generated when the splitting is performed.

[0148] When multiple tree partitions are supported, various partition information can be configured. Next, a description will be given under the following assumption: how to configure the partition information in one depth level (ie, the supported partition depth is set to be greater than or equal to 1, and recursive partitioning is possible, but this is for ease of explanation).

[0149] In example (1), information on whether to divide is confirmed. In this case, when the division is not performed, the division ends.

[0150] When performing the division, the selection information about the division type (e.g., tree_idx. QT for 0, BT for 1, and TT for 2) is confirmed. In this case, the division direction information is additionally confirmed according to the selected division type and proceeds to the next step. (If additional division is possible because the division depth has not reached the maximum value, start again from the beginning, and when division is not possible, the division ends).

[0151] In example (2), information about whether a certain tree method (QT) performs division is confirmed, and the process proceeds to the next step. In this case, when division is not performed, information about whether a certain tree method (BT) performs division is confirmed. In this case, division is not performed, and information about whether a certain tree method (TT) performs division is confirmed. In this case, when division is not performed, division ends.

[0152] When a certain tree method (QT) performs the partition, proceed to the next step. In addition, when a certain tree method (BT) performs the partition, confirm the partition direction information and proceed to the next step. In addition, when a certain tree partition method (TT) performs the partition, confirm the partition direction information and proceed to the next step.

[0153] In example (3), information about whether a certain tree method (QT) performs division is confirmed. In this case, when division is not performed, information about whether certain tree methods (BT and TT) perform division is confirmed. In this case, when division is not performed, division ends.

[0154] When a certain tree method (QT) performs the division, proceed to the next step. In addition, when some tree methods (BT and TT) perform the division, confirm the division direction information and proceed to the next step.

[0155] The above examples may be cases where the priority of tree division exists (Example 2 and Example 3) or does not exist (Example 1), but various modifications may be made thereto. In addition, in the above examples, a case is described where the division in the current step is independent of the result of the division in the previous step, but the division in the current step may also be set to depend on the result of the division in the previous step.

[0156] For example, in the case of Examples 1 to 3, when partitioning of a certain tree method (QT) is performed in the previous step to proceed to the current step, partitioning of the same tree method (QT) can also be supported in the current step.

[0157] On the other hand, when the partitioning of a certain tree method (QT) is not performed in the previous step and the partitioning of certain tree methods (BT or TT) is performed to proceed to the current step, the partitioning of certain tree methods (BT and TT) other than the partitioning of the certain tree method (QT) can also be set to be supported in the next step including the current step.

[0158] The above situation means that the tree configuration supported by the block partitioning can be adaptive, which means that the above partitioning information configuration can also be configured differently. (Assume that the example described later is Example 3). In other words, in the above example, when the partitioning of a certain tree method (QT) is not performed in the previous step, the partitioning process can be performed in the current step without considering the certain tree method (QT). In addition, the partitioning information about the relevant tree method (for example, information about whether to partition, partitioning direction information, etc.) can be removed and configured. In this example <qt>The information about whether to divide is in the middle).

[0159] The above example is about the case of adaptive splitting information configuration for the case where block splitting is allowed (for example, the block size is within the range between the maximum value and the minimum value, the splitting depth of each tree method does not reach the maximum depth <allowable depth>, etc.), and adaptive splitting information configuration is possible even when block splitting is restricted (for example, the block size is not within the range between the maximum value and the minimum value, the splitting depth of each tree method reaches the maximum depth, etc.).

[0160] As mentioned above, a recursive method can be used to perform the tree-based partitioning in the present disclosure. For example, when the partition flag of the coding block with a partition depth of k is 0, the coding block is encoded in the coding block with a partition depth of k, and when the partition flag of the coding block with a partition depth of k is 1, the coding block is encoded in N sub-coding blocks with a partition depth of k+1 according to the partitioning method (in this case, N is an integer greater than or equal to 2, such as 2, 3, 4).

[0161] The sub coding block may be set as coding block (k+1) and divided into sub coding block (k+2) in the above process, and such a hierarchical division method may be determined according to division settings such as a division range, an allowable division depth, and the like.

[0162] In this case, a bitstream structure for representing the split information may be selected in one or more scanning methods. For example, a bitstream for split information may be configured based on the order of split depths, or may be configured based on whether to split.

[0163] For example, if it is based on the order of division depth, it is a method of obtaining division information in the next level of depth after the first block obtains the division information in the current level of depth, and if the method is based on whether to divide, it means a method of preferentially obtaining additional division information in the block divided based on the first block, and other additional scanning methods can be considered.

[0164] The size of the maximum block or minimum block may have a common setting regardless of the tree type (or all trees), or may have a separate setting for each tree, or may have a common setting for more than two trees. In this case, the size of the maximum block may be set equal to or less than the maximum coding block. If the size of the maximum block according to the predetermined first tree is not the same as the size of the maximum coding block, the division is implicitly performed using the predetermined second tree method until the maximum block size of the first tree is reached.

[0165] In addition, a common partition depth may be supported regardless of the tree type, or a separate partition depth may be supported according to each tree, or a common partition depth of more than two trees may be supported. Alternatively, a partition depth may be supported for some trees, while a partition depth may not be supported for some trees.

[0166] Explicit syntax elements for setting information may be supported, and some setting information may be determined implicitly.

[0167] (Partitioning based on index)

[0168] In the index-based partitioning of the present disclosure, a CSI (Constant Partition Index) method, a VSI (Variable Partition Index) method, and the like may be supported.

[0169] The CSI method may be a method of obtaining k sub-blocks by dividing in a predetermined direction, and k may be an integer greater than or equal to 2, such as 2, 3, 4. In detail, the CSI method may be a dividing method for a configuration in which the size and shape of the sub-block are determined based on the k value regardless of the block size and shape. In this case, for the predetermined direction, one or two or more directions of the horizontal direction, the vertical direction, and the diagonal direction (upper left->lower right direction or lower left->upper right direction, etc.) may be combined.

[0170] The index-based CSI partitioning method of the present disclosure may include being partitioned into z candidates in one direction (horizontally or vertically). In this case, z may be an integer greater than or equal to 2, such as 2, 3, 4, and one of the width or height of each sub-block may be the same and the other of the width or height may be the same or different. The aspect ratio of the sub-block may be A1:A2:...:AZ, and A1 to AZ may be an integer greater than or equal to 1, such as 1, 2, 3.

[0171] In addition, candidates that are divided into x and y in the horizontal direction and vertical direction, respectively, may be included. In this case, x and y may be integers greater than or equal to 1, such as 1, 2, 3, 4, but when x and y are both 1 (because they already exist), they may be restricted. Figure 4 A case where the ratio of the width or height of each sub-block is the same is shown, but candidates including different cases may be included.

[0172] In addition, candidates divided into w in a certain diagonal direction (upper left->lower right direction) or in one of the diagonal directions (lower left->upper right direction) may be included, and w may be an integer greater than or equal to 2, such as 2, 3.

[0173] Reference Figure 4 , blocks can be classified into symmetrical division shapes (b) and asymmetrical division shapes (d, e) according to the length ratio of each sub-block, and blocks can be classified into division shapes (k, m) inclined in a specific direction and division shapes (k) arranged along the center. The division shape can be defined by various coding elements including the shape of the sub-block and the length ratio of the sub-block, and the supported division shapes can be implicitly or explicitly determined according to the coding settings. Therefore, the candidate group in the index-based division method can be determined based on the supported division shapes.

[0174] On the other hand, the VSI method may be a method of obtaining more than one sub-block by dividing in a predetermined direction when the width (w) or height (h) of the sub-block is fixed, and w and h may be integers greater than or equal to 1, such as 1, 2, 4, 8, etc. In detail, the VSI method may be a dividing method for a configuration in which the number of sub-blocks is determined based on the size and shape of the block and the w or n value.

[0175] The index-based VSI partitioning method of the present disclosure may include partitioning the candidates by fixing one of the width or height of the sub-block. Alternatively, the candidates may be partitioned by fixing the width and height of the sub-block. Since the width or height of the sub-block is fixed, uniform partitioning in the horizontal direction or the vertical direction may be allowed, but is not limited thereto.

[0176] When the block before division is M×N, and the width of the sub-block is fixed (w) or the height is fixed (h) or the width and height are fixed (w, h), the number of sub-blocks obtained can be (M*N / w), (M*N / h), (M*N / w / h) respectively.

[0177] According to encoding settings, only the CSI method may be supported, only the VSI method may be supported, or both methods may be supported, and information about the supported methods may be determined implicitly or explicitly.

[0178] In the present disclosure, description is made under the assumption that the CSI method is supported.

[0179] According to encoding settings, a candidate group may be configured by including two or more candidates in index partitioning.

[0180] For example, candidate groups such as {a, b, c}, {a, b, c, n}, {a to g, n} can be configured, which may be the case where block shapes predicted to be significantly generated based on general statistical features are configured as candidate groups, such as block shapes divided into two (2) in the horizontal direction or the vertical direction, or divided into two (2) in the horizontal direction or the vertical direction, respectively.

[0181] Alternatively, a candidate group such as {a, b}, {a, o}, {a, b, o} or {a, c}, {a, p}, {a, c, p} may be configured, the candidate group including candidates that are divided into two (2) and four (4) in the horizontal direction and vertical direction, respectively. This may be the case where a block shape predicted to be significantly divided in a specific direction is configured as a candidate group.

[0182] Alternatively, a candidate group such as {a, o, p} or {a, n, q} may be configured, which may be the case where a block shape predicted to be significantly divided in a size smaller than that of the block before division is configured as a candidate group.

[0183] Alternatively, a candidate group such as {a, r, s} may be configured, which may be a case where a non-square division shape is configured as a candidate group based on the following determination: the best division result that can be obtained in a rectangular shape by other methods (tree method) is obtained in the block before division.

[0184] As in the above examples, various candidate group configurations are possible, and more than one candidate group configuration may be supported by considering various coding elements.

[0185] When the candidate group configuration is completed, various partition information configurations are possible.

[0186] For example, the index selection information may be generated in a candidate group configured by including undivided candidates (a) and divided candidates (b to s).

[0187] Alternatively, information indicating whether to divide (whether the division shape is a) may be generated, and when division is performed (if not a), index selection information may be generated in a candidate group configured with candidates for division (b to s).

[0188] Various partition information configurations other than those described above are possible, and in addition to the information indicating whether to partition, binary bits may be allocated to the index of each candidate in the candidate group by various methods such as fixed-length binarization, variable-length binarization, etc. When the number of candidate groups is 2, 1 bit may be allocated to the index selection information, and when the number of candidate groups is greater than or equal to 3, 1 bit or more may be allocated to the index selection information.

[0189] Compared to the tree-based partitioning method, the index-based partitioning method may be a method of selectively configuring a partition shape predicted to be significantly generated in a candidate group.

[0190] In addition, since the amount of bits used to show index information can be increased according to the number of supported candidate groups, the index-based partitioning method may be a method suitable for single-layer partitioning (for example, the partitioning depth is limited to 0) rather than tree-based layered partitioning (recursive partitioning). In other words, the method may be a method that supports a single partitioning operation and the sub-blocks obtained by the index-based partitioning may not be further partitioned.

[0191] In this case, it may mean that additional division into smaller blocks of the same type is impossible (for example, a coding block obtained by the index division method may not be additionally divided into coding blocks), but additional division into blocks of different types may also be set to be impossible (for example, not only division from a coding block into coding blocks is impossible, but division into prediction blocks is also impossible). Of course, it is not limited to the above example, and other modified examples are also possible.

[0192] Next, a case where a block division setting is determined based on a block type in a coding element will be described.

[0193] First, the coding block can be obtained in the partitioning process. In this case, for the partitioning process, a tree-based partitioning method can be used, and according to the type of tree, such as Figure 4 The results of the partitioning shapes are a (no partition), n (QT), b, c (BT), i, l (TT), etc. Depending on the encoding settings, each tree type can be combined in various ways, such as QT / QT+BT / QT+BT+TT, etc.

[0194] The following example shows a process of finally dividing a prediction block and a transform block based on a coding block obtained in the process, and assumes a case where prediction, transform, and inverse transform processes are performed based on each division size.

[0195] In example (1), the prediction process can be performed when the prediction block is set to the same size as the coding block, and the transform and inverse transform processes can be performed when the transform block is set to the same size as the coding block (or prediction block). Since the prediction block and the transform block are set based on the coding block, there is no separately generated partition information.

[0196] In example (2), the prediction process can be performed when the prediction block is set to the same size as the coding block. For the transform block, the transform block can be obtained in the division process based on the coding block (or prediction block), and the transform and inverse transform processes can be performed based on the obtained size.

[0197] In this case, a tree-based partitioning method can be used for the partitioning process, and according to the type of tree, it is possible to derive Figure 4 The result of the partitioning shape is a (no partition), b, c (BT), i, l (TT), n (QT), etc. Depending on the encoding settings, each tree type can be combined in various ways, such as QT / BT / QT+BT / QT+BT+TT, etc.

[0198] In this case, the index-based partitioning method can be used for the partitioning process, and the index type can be used to derive the following information: Figure 4 The result of dividing the shapes into a (no segmentation), b, c, d, etc. Depending on the encoding settings, various candidate group configurations such as {a, b, c}, {a, b, c, d}, etc. are possible.

[0199] In example (3), for the prediction block, the prediction block can be obtained in the division process based on the coding block, and the prediction process can be performed based on the obtained size. The transform block can be set to the same size as the coding block to perform the transform and inverse transform processes. This example can correspond to the case where the prediction block and the transform block have an independent relationship with each other.

[0200] In this case, an index-based partitioning method can be used in the partitioning process, and the index type can be derived. Figure 4 The result of partitioning shapes such as a (no partitioning), b to g, n, r, s, etc. Depending on the encoding settings, a variety of candidate group configurations such as {a, b, c, n}, {a to g, n}, {a, r, s}, etc. are possible.

[0201] In example (4), for the prediction block, the prediction block can be obtained in the division process based on the coding block, and the prediction process can be performed based on the obtained size. The transform block can be set to the same size as the prediction block to perform the transform and inverse transform processes. This example can be a case where the transform block is set to the same size as the obtained prediction block or vice versa (for example, the prediction block is set to the same size as the transform block).

[0202] In this case, a tree-based partitioning method can be used in the partitioning process, and a tree type such as Figure 4 The tree types can be divided into a (no partition), b, c (BT), n (QT), etc. Depending on the encoding settings, each tree type can be combined in various ways, such as QT / BT / QT+BT, etc.

[0203] In this case, an index-based partitioning method can be used in the partitioning process, and a partitioning method such as Figure 4 In the example, the candidate group is a (no segmentation), b, c, n, o, p, etc. The candidate group is a partition shape such as a (no segmentation), b, c, n, o, p, etc. According to the encoding setting, various candidate group configurations such as {a, b}, {a, c}, {a, n}, {a, o}, {a, p}, {a, b, c}, {a, o, p}, {a, b, c, n}, {a, b, c, n, p}, etc. are possible. In addition, the candidate group can be configured by a separate VSI method or by mixing the VSI method with the CSI method in the index-based partitioning method.

[0204] In example (5), for the prediction block, the prediction block can be obtained in the division process based on the coding block, and the prediction process can be performed based on the obtained size. In addition, for the transform block, the prediction block can be obtained based on the coding block in the division process, and the transform and inverse transform processes can be performed based on the obtained size. This example may be a case where the prediction block and the transform block are respectively divided based on the coding block.

[0205] In this case, a tree-based partitioning method and an index-based partitioning method may be used in the partitioning process, and the candidate group may be configured the same as or similar to Example 4.

[0206] Although the above examples describe some cases that can be generated according to whether the division process of each type of block can be shared, etc., it is not limited thereto, and various modified examples are possible. In addition, the block division setting can be determined by considering various encoding elements and block types.

[0207] In this case, the coding elements may include image type (I / P / B), color component (YCbCr), block size / shape / position, aspect ratio of the block, block type (coding block, prediction block, transform block, quantization block, etc.), division status, coding mode (intra / inter), prediction related information (intra prediction mode, inter prediction mode, etc.), transform related information (transform method selection information, etc.), quantization related information (quantization area selection information, quantization transform coefficient encoding information, etc.), and the like.

[0208] Figure 5 is an example diagram showing a prediction mode in intra prediction according to an embodiment of the present disclosure.

[0209] Reference Figure 5 , 95 prediction modes can be supported for intra prediction, and among them, 93 prediction modes are directional modes and 2 prediction modes are non-directional modes (DC, planar mode). In this case, the directional modes can be classified by slope (e.g., dy / dx) or angle information (degrees).

[0210] The non-directional mode may be a method of performing prediction by methods such as averaging, interpolation, etc. of reference pixels adjacent to a block, while the directional mode may be a method of performing prediction by methods such as extrapolation, interpolation, etc. of reference pixels adjacent to a block.

[0211] The directional mode can have vertical (up->down / down->up), horizontal (left->right / right->left), diagonal A (left-up->right-down / right-down->left-up), diagonal B (right-up->left-down / left-down->right-up), etc.

[0212] Generally, when encoding is performed by raster scanning or the like, there may be blocks adjacent to the left, top, upper left, upper right, and lower left directions, and a non-directional mode using them as reference pixels or starting points may be supported, or prediction or prediction modes in the vertical (up -> down), horizontal (left -> right), diagonal A (upper left -> lower right), diagonal B-1 (upper right -> lower left), and diagonal B-2 (lower left -> upper right) directions. When other scans besides raster scanning are supported, the directional mode may be defined differently.

[0213] For example, when the left block is available and the right block is not available, the horizontal mode may perform prediction by extrapolation in the (right-left) direction. Alternatively, when both the left block and the right block are available, prediction may be performed by extrapolation in a predetermined direction (left->right or right->left), or prediction may be performed by interpolation of the two blocks. In the present disclosure, encoding is performed according to a raster or z-scan order, and is described based on the assumption that reference pixels are located in the left, top, left-top, right-top, left-bottom directions.

[0214] In addition, a reconstructed block of another color space encoded by using the correlation between color spaces can be used for prediction of the target block, and a prediction mode supporting it can be included. For example, for a chrominance component, a prediction block of the target block can be generated by using a reconstructed block of a luminance component corresponding to the target block. In other words, a prediction block can be generated based on a reconstructed block by considering the correlation between color spaces, and can be included as an intra prediction mode of a chrominance component.

[0215] The chrominance component may have the same candidate group as the prediction mode candidate group of the luma component, or some modes in the prediction mode candidate group of the luma component and an additional prediction mode (color copy mode, color mode) for the chrominance component may be included in the prediction mode candidate group. In this case, for the color copy mode, it may be a prediction mode related to a method of obtaining data for generating a prediction block from an area located in another color space, and for the color mode, it may be a prediction mode related to a method of obtaining a prediction mode from an area located in another color space. For the color copy mode and the color mode, m mode and n mode (m, n are integers such as 0, 1, 2, 3 or more) may be supported, respectively.

[0216] When supporting 1 color copy mode, the method of obtaining predetermined data for generating a prediction block can be pre-defined, and when supporting two or more color copy modes, the method of obtaining data for generating a prediction block can be classified (for example, the position referenced in order to obtain correlation, etc.). <1> Left area, <2> Upper area, <3> left + top area, etc.) and support.

[0217] When one color mode is supported, a predetermined position for obtaining a prediction mode may be predefined, and when two or more color modes are supported, multiple positions for obtaining a prediction mode may be supported (for a corresponding block, for example, <1> middle, <2> Upper left, <3> Top right, <4> Lower left <5> lower right, etc.).

[0218] All or part of the prediction modes described in the above examples may be included in the prediction mode candidate group for the luma component or the chroma component, and other additional modes may be included in the prediction mode candidate group.

[0219] The prediction mode may mean all candidates for supported intra prediction modes, and the prediction mode candidate group may be configured to have all or part of them. In this case, the prediction mode candidate group may be configured according to the size, shape (aspect ratio), etc. of the block.

[0220] For example, the number of prediction mode candidate groups may be determined according to the block size. In this case, the block size may be classified into one of more than two ranges divided based on one or more predetermined threshold sizes (A×B, C×D, etc.), and may be determined as one of 11, 35, 67, etc., and a plurality of candidate groups according to the classified range or whether a color copy mode, a color mode, etc. are supported, and the number may be determined. In this case, the threshold size may be expressed as width (W), height (H), W×H, etc., and W and H may be integers greater than 2, such as 4, 8, 16, 32, 64, etc.

[0221] Alternatively, according to the shape of the block (or the aspect ratio of the block), the configuration of the prediction mode candidate group can be determined. In this case, the aspect ratio of the block can be classified into one of two or more ranges divided based on one or more predetermined thresholds, and according to the classified range, the configuration of the candidate group can be determined. In this case, the threshold value can be shown as W / H (or H / W) or the like, and can have an integer value greater than 1 (e.g., 1, 2, 4, 8, 16, etc.), or a fractional value between 0 and 1 (e.g., 1, 1 / 2, 1 / 4, 1 / 8, 1 / 16, etc.).

[0222] The intra prediction of the present disclosure assumes the following: for directional mode and non-directional mode, the following can be supported: Figure 5 95 prediction modes in and can support color modes and color copy modes. In addition, it is assumed that for the prediction mode candidate group of the luminance component, 67 directional modes and non-directional modes are supported, and for the prediction mode candidate group of the chrominance component, a total of 8 directional modes and 4 non-directional modes, 1 color mode and 3 color copy modes are supported, but not limited thereto. There may be a predetermined configuration standard for selecting a mode included in the prediction mode candidate group other than the assumption.

[0223] An example of configuring a prediction mode candidate group according to the aspect ratio of a block is shown below.

[0224]

Table 1

[0225] Aspect Ratio Prediction Model (Including) Forecast model (not included) 16 67～80 2～15 8 67～78 2～13 4 67～76 2～11 2 67～72 2～7 1 - - 1 / 2 -6～-1 61～66 1 / 4 -10～-1 57～66 1 / 8 -12～-1 55～66 1 / 16 -14～-1 53～66

[0226] When the block has a square shape, the table is set based on the case where the 0th and 1st prediction modes and the 2nd to 66th prediction modes are included in the prediction mode candidate group. When the block has a rectangular block shape, the candidate configuration shown in the table is possible by adding a prediction mode to the prediction mode candidate group of the square block (excluding the non-directional mode) or excluding the prediction mode from the prediction mode candidate group of the square block, which may be an example set based on the assumption that it is advantageous in terms of prediction accuracy to configure the prediction mode more widely in a longer block than in a shorter block, but the opposite configuration is also possible.

[0227] Reference pixels for intra prediction may be configured in a reference pixel construction unit. In this case, reference pixels may be managed in a temporary memory (e.g., an array, a first array, a second array, etc.), and reference pixels may be generated and removed during each intra prediction process, and the size of the temporary memory may be determined according to the configuration of the reference pixels.

[0228] Description is made based on the assumption that left, above, upper left, upper right, and lower left blocks are used for intra prediction based on a target block, but is not limited thereto and block candidate groups of other configurations may be used for intra prediction.

[0229] For example, the candidate groups of neighboring blocks for a reference pixel may be examples of the following cases: following a raster scan or a Z scan and depending on the supported scan order, some candidate groups may be configured or removed by including other block candidate groups (e.g., with additional configurations of right, bottom, bottom-right blocks, etc.).

[0230] On the other hand, the pixels adjacent to the target block can be classified into at least one reference pixel layer, and can be classified into: ref_0, that is, the nearest pixel of the target block {the pixel whose pixel value difference with the boundary pixel of the target block is 1. p(-1,-1)~p(2m-1,-1), p(-1,0)~p(-1,2n-1)}; ref_1, that is, the next adjacent pixel {the pixel value difference with the boundary pixel of the target block is 2. p(-2,-2)~p(2m,-2), p(-2,-1)~p(-2,2n)}; ref_2, that is, the next adjacent pixel {the pixel value difference with the boundary pixel of the target block is 3. p(-3,-3)~p(2m+1,-3), p(-3,-2)~p(-3,2n+1)}. In other words, the reference pixels can be classified into multiple reference pixel layers according to the distance of the pixels adjacent to the boundary pixels of the target block.

[0231] The reference pixel rows supported herein may be above N, and N may be, for example, an integer of 1, 2, 3, 4, or more. In this case, they are generally included in the reference pixel row candidate group sequentially from the reference pixel row of the nearest neighboring target block, but are not limited thereto. For example, if N is 3, then the reference pixel row candidate group may be as follows:<ref_0,ref_1,ref_2> This can be done by sequentially configuring the candidate groups, or by excluding<ref_0,ref_1,ref_3> ,<ref_0,ref_2,ref_3> ,<ref_1,ref_2,ref_3> Such non-contiguous or nearest neighboring reference pixel rows are used to configure the candidate group.

[0232] Prediction can be performed by using all reference pixel rows in the candidate set or by using some reference pixel rows (more than one).

[0233] For example, depending on the encoding setting, one of the multiple reference pixel rows may be selected to perform intra-frame prediction by using the corresponding reference pixel row. Alternatively, two or more of the multiple reference pixel rows may be selected to perform intra-frame prediction by using the corresponding reference pixel row (e.g., applying a weighted average to each reference pixel row data, etc.).

[0234] In this case, the selection of the reference pixel row may be determined implicitly or explicitly. For example, for the implicit case, this means that it is determined according to a coding setting defined by a combination of more than one or two of elements such as image type, color component, block size / shape / position, etc. In addition, for the explicit case, this means that information on the selection of the reference pixel row may be generated in units of blocks, etc.

[0235] The present disclosure mainly describes the case where intra prediction is performed by using the nearest reference pixel row. However, the reference pixel row referenced for prediction can be considered as one of the main coding elements in the present disclosure. In other words, the intra prediction settings can be determined independently of the selected reference pixel row, but the intra prediction settings can be determined accordingly.

[0236] The reference pixel construction unit for intra prediction in the present disclosure may include a reference pixel generation unit, a reference pixel interpolation unit, a reference pixel filter unit, etc., and may be configured by including all or part of each configuration.

[0237] In the reference pixel construction unit, available reference pixels and unavailable reference pixels may be classified by confirming the availability of the reference pixels. In this case, when at least one of the following conditions is satisfied, it is determined that the reference pixel is unavailable.

[0238] For example, it can be determined as unavailable when at least one of the following conditions is met: the reference pixel is outside the picture boundary; the reference pixel and the target block do not belong to the same division unit (for example, units such as slices, tiles, etc. that are not referenced to each other. However, even when units such as slices or tiles can be referenced to each other despite not being the same division unit, they are still excluded); and the encoding is not completed. In other words, it can be determined as available when any condition is not met.

[0239] In addition, the use of reference pixels can be restricted by coding settings. For example, although it is judged to be available according to conditions, the use of reference pixels is restricted according to whether restricted intra prediction (e.g., constrained_intra_pred_flag) is performed. Restricted intra prediction can be performed to prohibit the use of reconstructed blocks referenced from other images as reference pixels to prevent the propagation of errors caused by external factors including the communication environment, etc.

[0240] When constrained intra prediction is disabled (eg, constrained_intra_pred_flag=0 for I picture type or P picture type or B picture type), all reference pixel candidate blocks may be available.

[0241] Alternatively, when constrained intra prediction is activated (e.g., constrained_intra_pred_flag=1 for P picture type or B picture type), the condition for determining whether to use a reference pixel candidate block is assumed according to the coding mode (Mode_Intra, Mode_InterD, Mode_InterC), but the condition may be determined according to various other coding elements.

[0242] In this case, Mode_Intra, Mode_Inter_D, Mode_Inter_C can respectively represent intra-frame prediction, inter-frame prediction (block matching, etc.) referenced in other pictures, and inter-frame prediction referenced in the current picture, and in the case of Mode_Intra, reference is possible, while in the case of Mode_Inter, reference is impossible. The setting may be general, but not limited to this.

[0243] Since the reference pixels include more than one block, after checking the possibility of the reference pixels, the reference pixels can be classified into three cases, such as <all available>, <some available>, and <all unavailable>. In other cases except the "all available" case, the reference pixels at the positions of the unavailable candidate blocks can be filled or generated.

[0244] When the reference pixel candidate block is available, the pixels at the corresponding position may be included in the reference pixel memory of the target block. In this case, the pixel data may be copied as is, or the pixel data may be included in the reference pixel memory in processes such as reference pixel filtering, reference pixel interpolation, etc. In addition, when the reference pixel candidate block is not available, the pixels obtained in the process of generating the reference pixels may be included in the reference pixel memory of the target block.

[0245] Examples in which reference pixels at the position of an unavailable block are generated by using various methods are shown below.

[0246] For example, a reference pixel may be generated by using an arbitrary pixel value. In this case, the arbitrary pixel value may be a pixel value (e.g., a minimum value, a maximum value, a center value, etc. in the range of pixel values) belonging to a range of pixel values ​​(e.g., a range of pixel values ​​based on a bit depth or according to pixel distribution in a corresponding image). In detail, this may be an example applied when all reference pixel candidate blocks are unavailable.

[0247] Alternatively, the reference pixel may be generated from the region in which the image is encoded. In detail, the reference pixel may be generated from at least one available block adjacent to the unavailable block. In this case, at least one method such as extrapolation, interpolation, replication, etc. may be used.

[0248] For the above example, the reference pixel can be directed to an adjacent area centered on the target block, and the reference pixel can be considered by including an area corresponding to the target block (an area where prediction data is obtained or an area adjacent to it) in other color spaces referenced in some prediction modes (e.g., color copying mode, etc.).

[0249] After completing the reference pixel configuration, reference pixel filtering or reference pixel interpolation may be performed. For example, only reference pixel filtering may be performed, only reference pixel interpolation may be performed, or both reference pixel filtering and reference pixel interpolation may be performed. Reference pixel filtering may be performed before or after reference pixel interpolation, and may also be performed simultaneously by combining reference pixel interpolation.

[0250] Reference pixel filtering may be a process performed to reduce the degradation remaining in the reference pixel. Reference pixel filtering may be one of a plurality of filters that may be classified by setting various filter tap lengths and coefficients (e.g., [1, 2, 1] / 4, [2, 3, 6, 3, 2] / 16, etc.). In addition, a plurality of filters may be performed by changing the filter type.

[0251] Whether to perform reference pixel filtering may be explicitly determined according to encoding settings, or may be implicitly determined. In this case, the encoding settings may be defined based on the state information (block size, shape, position, etc.) of the target block, the image type (I / P / B), the color component (Y / Cb / Cr), the reference pixel row selection information, whether to apply intra-frame prediction in sub-block units, the intra-frame prediction mode, etc.

[0252] Reference pixel interpolation may not be performed in a prediction mode that refers to only pixels in integer units, and may be performed in a prediction mode that refers to pixels in fractional units.

[0253] The position of the pixel to be interpolated (i.e., the pixel to be interpolated in fractional units) may be determined according to the prediction mode (e.g., the direction used for the prediction mode, dy / dx, etc.) and the positions of the reference pixel and the predicted pixel. In this case, one filter may be applied regardless of the accuracy of the fractional unit, or one of a plurality of filters may be applied by selection in fractional units (e.g., a filter assuming that an equation for determining a filter coefficient or a filter tap length is divided).

[0254] The former may be an example in which pixels in integer units are used as input for interpolation of pixels in decimal units, and the latter may be an example in which the input pixels are changed in each step (for example, for 1 / 2 unit, integer pixels are used, for 1 / 4 unit, integer and 1 / 2 unit pixels are used, etc.), but are not limited to this, and the description in the present disclosure is based on the former.

[0255] Fixed filtering or adaptive filtering can be performed for reference pixel interpolation, which can be determined according to the encoding settings. In this case, the encoding settings can be defined based on the state information of the target block, the image type, the color component, the reference pixel row selection information, whether to apply intra-frame prediction in sub-block units, the intra-frame prediction mode, etc.

[0256] Fixed filtering may perform reference pixel interpolation by using one filter, and adaptive filtering may perform reference pixel interpolation by using one of a plurality of filters.

[0257] In this case, for adaptive filtering, one of a plurality of filters may be explicitly or implicitly determined according to encoding settings. In this case, the filter type may be configured after selecting more than one from a 4-tap DCT-IF filter, a 4-tap cubic filter, a 4-tap Gaussian filter, a 6-tap Wiener filter, an 8-tap Kalman filter, etc. The supported filter candidate groups may also be differently defined according to color components (e.g., some filter types are the same or different, and the length of the filter taps is short or long, etc.).

[0258] A prediction block may be generated according to at least one prediction mode in a prediction block generation unit, and reference pixels may be used based on the prediction mode. In this case, the reference pixels may be used for methods such as extrapolation according to the prediction mode, and the reference pixels may be used for methods such as interpolation, DC, or replication.

[0259] A process for selecting an optimal mode from among a plurality of prediction mode candidate groups is performed in a prediction mode determination unit. In general, the optimal mode in terms of coding overhead can be determined by using block distortion {e.g., distortion of a target block and a reconstructed block. Distortion. SAD (sum of absolute difference), SSD (sum of square difference), etc.}, and rate distortion (where the amount of generated bits according to the corresponding mode is taken into account). A prediction block generated based on the prediction mode determined in the process can be sent to a subtraction unit and an addition unit.

[0260] The prediction mode selected in the prediction mode determination unit may be encoded in the prediction mode encoding unit. In the prediction mode candidate group, index information corresponding to the prediction mode may be encoded, or the prediction mode may be predicted to encode information about it. The former may be a method applied to the luminance component, and the latter may be a method applied to the chrominance component, but is not limited thereto.

[0261] When the prediction mode is predicted and encoded, the prediction mode may be classified into a plurality of categories (K, K is an integer, such as 2, 3, 4 or more) and the prediction mode may be managed. For example, when the prediction mode is classified into 2 categories, one category may include a prediction mode predicted to be the same as the prediction mode of the target block, and the other category may include a prediction mode predicted to be unlikely to be the same as the prediction mode. In other words, the other category may include a prediction mode that is not selected as a mode predicted to be the same as the prediction mode of the target block.

[0262] The above example is a case where the prediction mode is classified into 2 categories, and the prediction mode may be classified into more categories. When the prediction mode is classified into 3 categories, the first category may include a prediction mode predicted to be likely to be the same as the prediction mode of the target block, the second category may include a prediction mode predicted to be likely to be the same as the prediction mode of the target block, and the third category may include a prediction mode predicted to be unlikely to be the same as the prediction mode of the target block. In other words, the second category may include a prediction mode predicted to be the same as the prediction mode of the target block among the prediction modes that do not belong to the first category, and the third category may include a prediction mode that does not belong to the first category and the second category.

[0263] For other examples of classification categories, they can be classified according to the prediction method. For example, they can be classified by a prediction method according to extrapolation, a prediction method according to interpolation / average, a prediction method according to replication, and the like. In this case, the prediction method according to extrapolation may mean a directional mode, the prediction method according to interpolation / average may mean a non-directional mode, the prediction method according to replication may be classified as a color replication mode, and other various classifications are possible. When a color mode derives a prediction mode in other color spaces, it can be distinguished from a directional mode or a non-directional mode, but for ease of explanation, it is assumed that it is classified as a directional mode / non-directional mode.

[0264] When classified into two categories according to the prediction method, the first category may include a directional mode / non-directional mode, and the second category may include a color copy mode. Alternatively, the first category may include a directional mode, and the second category may include a non-directional mode. The former may be applied to intra-frame prediction of a chrominance component, and the latter may be applied to intra-frame prediction of a luminance component. In the following description, it is assumed that when a color copy mode is included, it may be applied to the intra-frame prediction mode of the chrominance component, but it should be understood that even when a color copy mode is not included, it is not applied only to the intra-frame prediction mode of the luminance component.

[0265] When classified into 3 categories, the first category may include a directional mode, the second category may include a non-directional mode, and the third category may include a color copy mode. Alternatively, the first category may include a predetermined mode in the directional mode (e.g., vertical, horizontal, diagonal A mode, etc.), the second category may include a predetermined mode in the directional mode (e.g., diagonal B mode, etc.), and the third category may include a color copy mode. In this case, the diagonal A mode may include a diagonal mode that uses only integer pixels in prediction, and the diagonal B mode may include a diagonal mode that uses integer pixels and fractional pixels in prediction, but is not limited thereto.

[0266] The above description is an example of configuration for a category, and it can be classified according to various predetermined standards without being limited thereto. In addition, a plurality of predetermined standards can be combined and applied to the configuration of a category.

[0267] When the prediction mode is classified into multiple categories, selection information related to the category of the prediction mode for generating the target block and candidate selection information in each category can be generated. When the candidate in the category is 1, the candidate selection information can be omitted. Various syntax elements for category selection and candidate selection in the category can be configured. Next, it is assumed that 3 categories are supported.

[0268] *Supports a syntax element for category selection.

[0269] In this example, the value of the syntax element supported by the category selection may have a value of 0 to 2 (a value higher than 1). In the example, for the first to third categories, they may be assigned <0> , <10> , <11> In other words, one syntax element for category selection may be supported and may be configured to apply the type of index allocation and binarization for each category.

[0270] *Support for multiple syntax elements regarding category selection.

[0271] In this example, the value of the syntax element supported by the category selection may have a value of 0 or 1 (may also be a value higher than 1).

[0272] In one example, for a syntax element (eg, the first syntax element), <0> Assigned to the first and second categories, and can be <1> Assigned to the third category. In this case, the syntax element has <0> When the value is , another syntax element (second syntax element) regarding subcategory selection that supports selecting one of the first category and the second category can be supported, and for the corresponding syntax element, <0> , <1> Assigned to the first and second categories.

[0273] In an example, for a syntax element (eg, the first syntax element), it may be <0> Assigned to the first category and can be <1> Assigned to the second and third categories. In this case, the syntax element has <1> When the value is , another syntax element (second syntax element) regarding subcategory selection that supports selecting one of the second category and the third category can be supported, and for the corresponding syntax element, <0> , <1> Assigned to the second and third categories.

[0274] The above examples are examples regarding syntax element configurations for category selection, and various modifications and configurations are possible.

[0275] Each of the above categories may include at least one prediction mode. The number of prediction modes in the first to third categories may be a, b, c, and a may be less than or equal to b and c, and b may be less than or equal to c. In this example, it is assumed that a is an integer between 1 and 5, b is an integer between 3 and 8, and c is the number obtained by subtracting the number of a and b from the number of prediction mode candidate groups. Variable length binarization or fixed length binarization (e.g., Truncated Rice binarization, TruncatedBinary binarization, k-th Exp-Golomb binarization, etc.) may be applied to prediction modes belonging to this category.

[0276] In the examples mentioned later, description is made based on the assumption that the configuration and setting of the categories are fixed, but adaptive setting is possible according to various coding elements (e.g., the number of categories, the configuration of related flags, etc.). In this case, the coding elements can be defined based on color components (luminance / chrominance), state information of blocks (e.g., block size, shape, aspect ratio, position, etc.), image types (I / P / B), whether intra prediction in sub-block units is applied, reference pixel row selection information, etc.). In the example, the number of categories can be determined to be one of 2 or 3 based on the coding information.

[0277] Next, an example will be described in which the prediction mode is classified into a plurality of categories based on the probability of matching the prediction mode (3) and the prediction mode is encoded based on the category selection information and the candidate selection information in the category. However, an example in which the criteria for category classification according to the prediction method are combined for category configuration is possible. In addition, when the second category of the first category and the second category is excluded or the description about the first category and the second category is integrated, an explanation using two category configurations can be obtained, so a detailed description thereof is omitted.

[0278] Figure 6 is an arrangement diagram of a target block and blocks adjacent thereto according to an embodiment of the present disclosure.

[0279] The target block may have a high correlation with a block adjacent thereto, and prediction may be performed not only by using adjacent reference pixels, but also by using the prediction mode of the adjacent block to predict the prediction mode of the target block. Therefore, the prediction mode of the block adjacent to the target block may be selected as a candidate included in the first category or the second category. In this case, all or part of the prediction modes of the left, top, upper left, upper right, and lower left blocks of the target block may be considered as candidates included in the (previous) category. In the example, (left / top), (left / top / upper right), (left / top / lower left), (left / top / upper left / lower right / upper left), etc. may be considered.

[0280] When the prediction modes of the neighboring blocks are considered as candidates for inclusion in the category, at least one prediction mode can be selected from the blocks in each direction. For example, for the left block, one can be selected from L0 to L3, and for the top block, one can be selected from U0 to U3.

[0281] By considering the case where adjacent blocks in a predetermined direction (in this example, the left block, the top block) are divided into a plurality of sub-blocks, the sub-blocks used to select the prediction mode at a specific position can be limited (for example, the left block is L3, the top block is U3). When the prediction mode of the block at the corresponding position is not available (for example, when the corresponding block is not encoded, when it is in a position that cannot be referenced, when the encoding mode <intra-frame / inter-frame> is different, etc.), the prediction mode may not be selected in the block in the corresponding direction.

[0282] Alternatively, a priority (e.g., L3->L0->L1->L2, etc.) for selecting a prediction mode (sub-block) may be supported, so that the prediction mode of an available sub-block with a previous priority may be selected as the prediction mode in the corresponding direction. When the prediction mode of the sub-block at all positions is unavailable, the prediction mode may not be selected in the block in the corresponding direction.

[0283] Furthermore, it is possible to support priorities for prediction modes obtained from blocks in each direction in the category. For example, settings such as left->upper->upper right->lower left->upper left, left->upper->lower left->upper right->upper left, upper->left->lower left->upper right->upper left, etc. are possible, but not limited to this.

[0284] Many prediction modes are included in the prediction mode candidate group, but there may be many prediction modes that are usually generated among them. For example, edges in the vertical direction and the horizontal direction may be common image features, and many flat areas such as backgrounds may be found. Therefore, a predetermined prediction mode predicted to be significantly generated may be selected as a candidate included in the first category or the second category.

[0285] Non-directional prediction modes plane (number 0), DC mode (number 1) and horizontal (number 18), vertical (number 50) and diagonal mode (number 2, number 34, number 66, etc.) may be targets predicted as candidates for preset prediction modes that are significantly generated. Priorities for including prediction modes in categories may be supported. For example, settings such as plane->DC->horizontal->vertical, etc. are possible, but are not limited thereto.

[0286] When all or part of the prediction modes of neighboring blocks are unavailable, more than one preset prediction mode (eg, planar, DC, etc.) may be used to replace and fill the prediction mode.

[0287] In the above description, the prediction mode of the neighboring blocks of the target block and the preset prediction mode are referred to as the prediction mode considered for the category configuration. However, when there are many prediction mode candidate groups, it may be difficult to effectively predict the prediction mode of the target block having the above configuration. Figure 5 When a difference of 1 and 2 is generated under the prediction mode in the above configuration, the prediction may fail.

[0288] To this end, a prediction mode derived based on a prediction mode of an adjacent block and a preset prediction mode may be considered for configuring a category candidate group. In an example, a k-interval prediction mode (a directional mode in this example) based on a prediction mode may be considered as a derived prediction mode and may be selected as a candidate included in the first category or the second category. In this case, k may be, for example, an integer of 1, 2, 3, 4 or more.

[0289] In this case, when the prediction mode based on the neighboring block or the preset prediction mode is assumed to be Z, there is a priority of the derived prediction mode, such as Z-1, Z+1, Z-2. In this example, it is assumed that the priority is determined in the order of absolute values, such as 1, 2, and (an absolute value) <-> code is followed by a <+> code, but is not limited to this. In other words, this means that the distance information used to fill the derived prediction mode may not start from 1 (for example, an integer such as 4, 8, 12, etc.). And, the k distances of the prediction modes derived from the neighboring block prediction mode and the preset prediction mode may be the same or different.

[0290] In addition, when the prediction mode based on the prediction mode of the neighboring block or the preset prediction mode is assumed to be X, Y, there is a priority of the derived prediction mode, such as X-1, X+1, Y-1, Y+1 or X-1, Y-1, X+1, Y+1. In this example, it is assumed that X has a higher priority than Y, and the <-> code and the <+> code are alternated with each other, or one code (a negative number in this example) is derived first (based on an absolute value) and then another code (a positive number in this example) is derived, but it is not limited thereto.

[0291] In addition, multiple modes (X, Y in this example) can be used as basic modes for deriving modes, or a predetermined mode of multiple modes (i.e., one of X, Y) can be used as a basic mode. Multiple modes can be classified according to predetermined criteria, and they can be determined based on: the position of the obtained block (for example, the mode of any specific block between the left block and the upper block is set as the basic mode); the shape of the target block (for example, the mode of the upper block between the left block and the upper block in the case of a horizontally long rectangular shape and the mode of the left block in the case of a vertically long rectangular shape are set as the basic mode. Or, vice versa, etc.); and slope information of the prediction mode (for example, a mode that is more inclined in a specific direction <left or right, top or bottom> is set as the basic mode based on a vertical mode or a horizontal mode).

[0292] In summary, a prediction mode derived based on a preset prediction mode, a prediction mode of a neighboring block, and a previous prediction mode may be included in the (previous) category.

[0293] Each category may include at least one prediction mode among a preset prediction mode, a prediction mode of a neighboring block, and a derived prediction mode, and for convenience of explanation, each category is respectively referred to as Group_A, Group_B, and Group_C.

[0294] For example, Group_A may be configured in the first category, Group_A, Group_B, and Group_C may be configured in the second category, and other modes (which may be Group_C in this case) may be configured in the third category.

[0295] Alternatively, Group_B may be configured in the first category, Group_A, Group_B, and Group_C may be configured in the second category, and other modes may be configured in the third category.

[0296] Alternatively, Group_A and Group_B may be configured in the first category, Group_A, Group_B, and Group_C may be configured in the second category, and other modes may be configured in the third category.

[0297] The above example is an example of mode configuration according to category classification, and for Group_A and Group_B included in multiple categories, it can be understood that other modes not configured in the previous category are configured in the subsequent category. In addition, mode configuration according to various classification categories is possible without being limited to the above example.

[0298] Next, assume that priority of prediction modes for category configuration is supported, and when modes are configured as many as the previously ranked category (e.g., the first category) according to the category priority (e.g., 1-2-3 category order), the remaining modes are configured in subordinate categories (e.g., the second category, the third category, etc.).

[0299] In the examples mentioned below, it is assumed that priority is supported, such as plane (assuming number 0 index is assigned) -> L -> A -> DC -> <l-1> -><L+1>-> <a-1> -><A+1>-> <l-2> -><L+2>-> <a-2>-><A+2> ->See->Hor-> <ver-4>-><Ver+4>-> <hor-4>-><Hor+4> wait.

[0300] For example, when the number of prediction modes in the first category is 1, the prediction mode belonging to Group_A may be planar (0th index), and as the number of prediction modes increases, Group_A may be configured with subordinate prediction modes.

[0301] In addition, when the number of prediction modes in the second category is 1, the prediction mode belonging to Group_B can be from the next prediction mode (index number m+1) of the prediction mode (index number m) finally configured in the first category. And as the number of prediction modes increases, Group_B can be configured with subordinate prediction modes.

[0302] In addition, for the third category, Group_C may be configured with the remaining prediction modes of the next prediction mode (number n+1 index) from the prediction mode (number n index) finally configured in the second category.

[0303] The number of prediction modes in the first to third categories (in this example, when the prediction mode candidate group is 67) can be set differently, for example, (1,5,61), (2,4,61), (4,8,55), etc., and can be determined based on the number of prediction mode candidate groups.

[0304] In the above example, the preset prediction mode may be a mode that is fixedly supported regardless of the encoding environment, and the derived prediction mode may be obtained by taking a prediction mode that already exists in the candidate group as a basic mode.

[0305] On the other hand, there may be a case where it is difficult to obtain the prediction mode of the neighboring block according to the state of the neighboring block (e.g., the boundary of the division unit, whether it is encoded, the encoding mode, etc.). In this case, the state of the neighboring block can be classified into (1) a case where all neighboring blocks are unavailable, (2) a case where some neighboring blocks are unavailable, and (3) a case where all neighboring blocks are available, and an example of priority can be described by assuming a case where all neighboring blocks are available.

[0306] In addition, the prediction modes of adjacent blocks may be obtained from two or more blocks, and an example of priority may be described by assuming a case where the prediction mode of each block does not overlap.

[0307] In addition, an example of priority may be described by assuming a case where the prediction mode of the neighboring block does not overlap with the preset prediction mode.

[0308] The priority for class configuration can be adaptively set by considering the above elements. For example, the index corresponding to the prediction mode of the adjacent block can be removed from the priority, and the index of the mode obtained by taking the prediction mode of the adjacent block as the basic mode can also be removed. Both can be generally applied to the case where the prediction mode of the adjacent block is a directional mode and the case where the prediction mode of the adjacent block is a non-directional mode.

[0309] In addition, when the prediction modes of adjacent blocks overlap, the index corresponding to the other overlapping prediction modes and the index of the derived mode may be removed. Alternatively, multiple priorities of consideration elements may be supported and the categories may be configured accordingly.

[0310] In the intra prediction of the present disclosure, it is described that certain supported prediction modes can be configured as prediction mode candidate groups. This may mean that each individual prediction mode (eg, directional mode) can be supported according to the state information of the target block (eg, block size, shape, etc.).

[0311] For prediction mode coding, it can be unified into a common prediction mode candidate group configuration before coding. For example, it can be unified into a prediction mode candidate group configuration within a predetermined range for prediction mode coding, and prediction mode coding can be performed based on the prediction mode candidate group configuration. In this case, the predetermined range can be configured with Figure 5 The prediction modes corresponding to numbers 0 to 66 in the figure are, but not limited to, those.

[0312] In the unified, mapping in a predetermined range can be performed, and mapping can be performed based on the direction of the prediction mode. In one example, for a prediction mode outside the predetermined range, mapping can be performed by a prediction mode having similar characteristics to the corresponding prediction mode (for example, in the case where the edge direction is the same, etc., in other words, including the case where the starting point of the prediction and the prediction direction are different but the edge direction is the same), and various other methods can be supported.

[0313] The prediction-related information generated in the process may be sent to the coding unit and stored in the bitstream.

[0314] In the above description, the unit for performing intra prediction is referred to as a target block. In this case, the target block can be set to various block units according to block setting.

[0315] In the example, when the coding block is the same unit as the prediction block and the transform block (i.e., when the coding block is directly set as a prediction block or a transform block), a target block for intra-frame prediction for reference pixel area setting, reference pixel configuration (reference pixel filtering / reference pixel interpolation), prediction mode candidate group configuration, prediction performance, prediction mode encoding, etc. may be selected as the coding block.

[0316] In the example, when a coding block is a unit that may be the same as or different from a prediction block and a transform block (i.e., a coding block may be divided into two or more sub-blocks <prediction block or transform block>), a target block for intra prediction may be set to one of the coding block, the prediction block, and the transform block.

[0317] For example, a coding block may be set as a target block for (prediction mode encoding, prediction mode candidate group configuration / reference pixel configuration), etc., and a prediction block or a transform block may be set as a target block for (reference pixel area setting / prediction performance), etc.

[0318] Alternatively, the coding block may be set as a target block for (prediction mode encoding / prediction mode candidate group configuration) or the like, and the prediction block or transform block may be set as a target block for (reference pixel area setting / reference pixel configuration / prediction performance) or the like.

[0319] In summary, the target block may be set to one of a coding block, a prediction block, and a transform block, and a target block unit of a detailed configuration may be determined according to coding settings.

[0320] In an image decoding method according to an embodiment of the present disclosure, intra-frame prediction may be configured as follows. Intra-frame prediction in a prediction unit may include decoding a prediction mode, configuring reference pixels, and generating a prediction block. In addition, an image decoding device may be configured to include a prediction mode decoding unit, a reference pixel construction unit, and a prediction block generation unit that embody prediction mode decoding, reference pixel configuration, and prediction block generation. A portion of the above-mentioned processes may be omitted or other processes may be added, and may be changed in other orders rather than the order input above.

[0321] Since the reference pixel construction unit and the prediction block generation unit of the image decoding device play the same role as the configuration corresponding to the image encoding device, the detailed description is omitted, and the prediction mode decoding unit can be performed by reversely using the method used in the prediction mode encoding unit.

[0322] Figure 7 is a flowchart illustrating a modification method of intra prediction according to an embodiment of the present disclosure.

[0323] Reference Figure 7 , an intra prediction mode of a target block may be determined (S700), and a prediction block may be generated (S710) based on the intra prediction mode. A modification setting of the prediction block may be determined (S720) and modification of the prediction block may be performed (S730).

[0324] Intra prediction of the target block may be selected among a prediction mode candidate group obtained based on state information of the target block, and one prediction mode among them may be determined.

[0325] The luma component may refer to a prediction mode candidate group configured with a directional mode and a non-directional mode, and the chroma component may refer to a prediction mode candidate group supporting a color mode or a color copy mode for the directional mode and the non-directional mode.

[0326] In this case, it can be classified into directional mode / non-directional mode / color copy mode according to the prediction method, and each method can use extrapolation, interpolation or averaging, copying, but is not limited to this, and other modifications are possible. In the example, according to the reference pixel area setting, interpolation (bidirectional prediction) as well as extrapolation can be supported in the directional mode.

[0327] In summary, according to each prediction method, the reference area for intra prediction can be set differently, and for extrapolation, at least one of the blocks in the left, top, upper left, upper right, and lower left directions of the target block can be set as the reference area, and for interpolation, in addition to extrapolation, at least one of the blocks in the right, bottom, and lower right directions of the target block can be set as the reference area. In this case, when the adjacent area is not encoded, it can be derived and filled from one or more pixels in the encoded area. In addition, for replication, a block corresponding to the target block in other color spaces can be set as a reference area.

[0328] Based on the data in the reference region and the intra prediction mode, a prediction mode may be generated.

[0329] Modifying the prediction block can be performed to reduce discontinuous boundary characteristics with adjacent blocks in the prediction block generated according to the predetermined prediction mode. However, modification according to image characteristics may cause adverse effects, so it is necessary to perform modification by considering various coding elements.

[0330] As to whether modification of the prediction block is supported, relevant information can be explicitly generated or implicitly determined. Moreover, even if it is determined that modification in higher units such as sequences, pictures, etc. is supported, the modification settings of the target block can be limited based on the image type, color component, state information of the target block, coding mode, intra-frame prediction mode, whether to apply intra-frame prediction in sub-block units, reference pixel row selection information, etc. In addition, setting information related to the modification (e.g., a mark on whether to perform the modification in the block unit, weight information applied to the modification, etc.) can be explicitly generated in a lower unit (e.g., for the previous information, block, sub-block, etc.) or in a higher unit (e.g., for the next information, sequence, picture, slice, tile, etc.).

[0331] The modification setting may be configured by including whether to perform modification, the number, position, weight, etc. of reference pixels used for modification, etc. Although the modification is performed, the modification may be performed on all or part of the pixels in the target block. For example, when the target block is mxn, the modification may be performed on 1 to (mxn) pixels.

[0332] In this case, whether to apply modification (in pixels), the number (1 to 5), position, weight, etc. of reference pixels used for modification (when performing modification), etc. can be set based on the position of the pixel in the block and the modification setting.

[0333] In this case, the intra prediction mode supporting the modification may be determined based on whether it belongs to a predetermined prediction mode group. For example, it may be determined in plane, DC, horizontal, vertical, and diagonal modes (e.g., upper right direction mode, lower right direction mode, lower left direction mode, etc.), Figure 5 The prediction mode group is configured in the numbered 2, 34, 66 modes in the prediction mode group) and the color copy mode. Alternatively, the prediction mode group can be configured by additionally including a prediction mode derived from the prediction mode group (for example, a k-interval mode based on a directional mode. In this case, k is an integer with an absolute value of 1, 2, 3, 4 or more).

[0334] The target block supporting the modification may be smaller than or equal to a predetermined first threshold size. In this case, the first threshold size may mean the maximum size that the target block may have.

[0335] In addition, the block supporting the modification may be greater than or equal to a predetermined second threshold size. In this case, the second threshold size may mean the minimum size that the target block may have.

[0336] In this case, the first threshold size and the second threshold size may be shown as width (W), height (H), Wx H, W*H, etc., and W and H may be integers such as 4, 8, 16, 32 or more, in which case the first threshold size may be greater than or equal to the second threshold size.

[0337] Figure 8 is an arrangement diagram of pixels of a target block and a neighboring block according to an embodiment of the present disclosure.

[0338] Reference Figure 8 , which includes pixels (a to p) in the target block, pixels (A to Q) in the coded adjacent blocks, and pixels (aa to ii) in the uncoded adjacent blocks. In this case, the modification can be performed on the pixels in the target block, and the modification can be performed with reference to the pixels in the adjacent blocks.

[0339] In this case, the reference pixel in the adjacent block may be one of the values ​​obtained without or after performing a reference pixel preprocessing process (eg, reference pixel filtering). The reference pixel in the adjacent block may be determined in units of blocks or in units of pixels.

[0340] The following represents an equation for obtaining a modified pixel after applying the modification to the predicted pixel.

[0341] Z=(z*w0)+(a*w1)+(b*w2)+(c*w3)

[0342] In this case, z and Z represent predicted pixels and modified pixels, respectively, a to c represent reference pixels used for modification, and w0 to w3 represent weights applied to the modification. According to the number of reference pixels used for modification, the reference pixels and weights may be determined. In this case, a normalized value, including 0, may be considered for weighted assignment.

[0343] According to the prediction mode of the target block, the location and number of reference pixels can be determined.

[0344] For example, when the prediction mode of the target block is planar, DC, vertical, or horizontal mode, a, b, and c may be set to the upper left reference pixel outside the target block (e.g., <-1, -1>), the reference pixel at the top of the target block corresponding to or corresponding to the x component of the target pixel (e.g.,<x,-1> ), the reference pixel on the left side of the target block corresponding to or corresponding to the y component of the target pixel (e.g., <-1,y>).

[0345] In this case, for planar, DC mode, weights may be assigned to w2 and w3 to reflect the reference pixel gradients above and to the left of the target block. This may be an example of a weight setting based on how much the reference pixels corresponding to the x and y components of the target pixel have changed (e.g., W1 is 0).

[0346] In this case, for the vertical mode, a weight with a negative code may be assigned to w0, and a weight with a positive code (i.e., an inverse code) may be assigned to w3 to reflect the reference pixel gradient corresponding to the prediction direction. This may be an example of a weight setting based on how much the reference pixel corresponding to the y component of the target pixel has changed from the upper left reference pixel (e.g., w2 is 0). For the horizontal mode, the relevant description may be derived in the vertical mode, so the detailed description is omitted.

[0347] Alternatively, when the prediction mode of the target block is the diagonal mode, a, b, c may be set to the upper left reference pixel outside the target block, the reference pixel corresponding to or corresponding to the starting point in the prediction direction of the target block (e.g.,<x+y+1,-1> ) and a reference pixel corresponding to or corresponding to the opposite side of the starting point in the prediction direction of the target block (for example, <-1, x+y+1>). In this case, when the reference pixel is not acquired in integer units (that is, when interpolation is required in fractional units), the reference pixel may be set by one of the following methods: a method of obtaining the reference pixel by replacing it with an adjacent reference pixel or a method of obtaining the reference pixel by interpolating the reference pixels adjacent to both sides.

[0348] In this case, for the diagonal pattern in the upper right direction, a weight may be assigned to w3 to reflect the gradient of the reference pixel located at the starting point of the prediction direction and located on the opposite side. This may be an example of weight setting based on how much the reference pixel in the prediction direction from the target pixel has changed (e.g., W1 and W2 are 0). For the diagonal pattern in the lower left direction, a related description may be derived in this mode, so a detailed description is omitted.

[0349] Fig. 9 A and 9B are exemplary diagrams of a modification method based on multiple reference pixel rows according to an embodiment of the present disclosure.

[0350] The example mentioned below may be a configuration that is supported separately from the above-mentioned configuration in which a reference pixel row for intra prediction is selected from a plurality of reference pixel rows and will be described based thereon, but may also be applied to configurations combined with each other.

[0351] Reference Fig. 9 A represents an example about modification in the horizontal mode or the vertical mode. In detail, it may be an example of a case where modification is performed using reference pixels in two or more reference pixel rows.

[0352] When the prediction mode of the target block is the vertical mode, the accuracy of the modification can be improved by additionally setting a* as the reference pixel. Alternatively, a** can be obtained by applying a weighted average to a and a* to apply a** to the above-mentioned modification equation.

[0353] This may be the same reason as using reference pixels adjacent to a target block as prediction pixels because block partitioning may not be performed to have a boundary that accurately divides image characteristics.

[0354] When the prediction mode of the target block is the horizontal mode, the accuracy of the modification can be improved by additionally setting b* as a reference pixel of b, and b** can be obtained by applying weighted averaging to b and b* to apply b** to the above-mentioned modification equation.

[0355] Reference Fig. 9 B, represents an example of modification in a diagonal mode. In detail, it may be an example of a case where modification is performed using reference pixels in two or more reference pixel rows.

[0356] When the prediction mode of the target block is in the lower left direction, the accuracy of the modification can be improved by additionally setting a* as the reference pixel of a at the prediction starting point, or a** can be obtained by applying weighted averaging to a and a* to apply a** to the modification equation mentioned above.

[0357] In addition, the accuracy of modification can be improved by additionally setting b* as a reference pixel of b at a position relative to the starting point of prediction, or b** can be obtained by applying weighted averaging to b and b* to apply b** to the above-mentioned modification equation.

[0358] For the diagonal mode, the reference pixel is obtained based on one or more reference pixels at the starting point of the prediction or at a position opposite to the starting point. In this case, the reference pixel can be selected for the pixel in the direction of the prediction mode.

[0359] In the above example, when reference pixels on multiple reference pixel rows are used for modification, whether support is determined based on the image type, color component, status information of the target block, intra-frame prediction mode, whether the intra-frame prediction mode based on sub-block units is applied, reference pixel row selection information, etc.

[0360] Fig.10 is a flowchart illustrating a modification method of intra prediction according to an embodiment of the present disclosure.

[0361] Reference Fig.10 , an arbitrary pixel for modification of a target block may be obtained (S900), a modification setting may be determined based on the target block and an intra prediction mode (S910), and modification of the prediction block may be performed (S920).

[0362] For modification of the prediction block, blocks adjacent to the target block may be configured while being skewed in a specific direction (eg, in an upper left direction) that is generated by a coding order according to a raster scan or Z scan method.

[0363] In addition to performing prediction and modification on reference pixels that are skewed in a specific direction, the accuracy of the prediction can also be improved by using data in an area where the encoding is not completed. To this end, a process of obtaining arbitrary pixels can be performed. In this case, the number of arbitrary pixels can be, for example, 1, 2 or a larger integer.

[0364] For example, in Figure 8 The arbitrary pixel used for modification of the target block is determined from pixels aa to ii in . In detail, in a block adjacent to the target block, a pixel belonging to an area where encoding has not yet been completed can be set as an arbitrary pixel.

[0365] Alternatively, you can Figure 8 In detail, since the target block also belongs to pixels that have not been encoded, the target block can be included in the target selected as an arbitrary pixel.

[0366] Fig.11 A, 11B, 11C, 11D, and 11E are example diagrams of arbitrary pixels for modifying predicted pixels according to an embodiment of the present disclosure.

[0367] Reference Fig.11 A, 11B, 11C, 11D and 11E show example diagrams, where for Fig.11 A takes the lower right pixel in the target block, Fig.11 B takes the lower right pixel outside the target block and Fig.11 C takes the lower and left pixels outside the target block and Fig.11 D takes the lower pixels outside the target block and Fig.11 E determines the right pixel outside the target block as an arbitrary pixel (X, X1, X2).

[0368] As to whether to perform modification by using an arbitrary pixel, relevant information may be explicitly generated or implicitly determined. Furthermore, even if it is determined to use an arbitrary pixel in a higher unit such as a sequence, a picture, etc., the setting of the arbitrary pixel may be limited based on the image type, the color component, the state information of the target block, the encoding mode, whether to apply an intra prediction mode in sub-block units, the reference pixel row selection information, etc. In addition, setting information related to the arbitrary pixel (such as a flag as to whether to perform modification by using an arbitrary pixel in a block unit, etc.) may be explicitly generated.

[0369] The setting of arbitrary pixels may be configured by including whether to use arbitrary pixels (ie, whether to use arbitrary pixels for modification), the number of arbitrary pixels, positions, and the like.

[0370] In this case, the intra prediction mode in which the modification is performed by using an arbitrary pixel can be determined according to whether it belongs to a predetermined prediction mode group. For example, in a plane, DC / horizontal, vertical / diagonal mode (e.g., upper right directional mode, lower right directional mode, lower left directional mode, etc.), Figure 5 The prediction group can be configured in the numbered 2, 34, 66 modes in the prediction mode group) and the color copy mode. Alternatively, the prediction group can be configured by additionally including a prediction mode derived in the prediction mode group (for example, a k-interval mode based on a directional mode. In this case, k is an integer with an absolute value of 1, 2, 3, 4 or more).

[0371] The target block modified by using any pixel may be less than or equal to a predetermined first threshold size, and the first threshold size may mean the maximum size that the target block may have. Alternatively, the target block may be greater than or equal to a predetermined second threshold size, and the second threshold size may mean the minimum size that the target block may have. In this case, the threshold size may be shown as width (W), height (H), W x H, W*H, etc., and W and H may be integers such as 4, 8, 16, 32 or more, and the first threshold size may be greater than or equal to the second threshold size.

[0372] In the above description, it can be understood that the modification settings mentioned in the previous embodiment belong to the lower configuration or are combined with the lower configuration, so the overlapping description is omitted. In the example mentioned later, it is assumed that any pixel is set to Fig.11 A.

[0373] Since the position of any pixel is an area that has not been encoded, the pixel value at the corresponding position can be obtained by various methods.

[0374] For example, data about the position of an arbitrary pixel may be explicitly encoded. The pixel value at the corresponding position may be encoded as is (e.g., based on the bit depth), or a value obtained by a predetermined division value (e.g., quantization. The quantization value is used for modification) may be encoded.

[0375] Alternatively, two or more data candidate values ​​for the position of an arbitrary pixel can be obtained in a coding area adjacent to the target block, and can be encoded by configuring index information for it. In the example, when a total of 2 candidates are supported, the pixel value at the position of an arbitrary pixel is 67, the first candidate value obtained in the area adjacent to the target block (e.g., the upper area) is 70, and the second candidate value obtained in the area adjacent to the target block (e.g., the left area) is 85, and the candidate value selection information (e.g., selecting the first candidate value. 1-bit flag) can be encoded. In this case, obtaining the candidate value can be mentioned in the example mentioned later. Candidate values ​​can be supported up to, for example, 2, 3, 4 or more integers.

[0376] For the above example, it may be the case that information on data at an arbitrary pixel position is explicitly encoded, and data at an arbitrary pixel position may be implicitly obtained.

[0377] For example, data at an arbitrary pixel position may be obtained from a reference pixel at a predetermined position or a reference pixel corresponding to or corresponding to the position of the arbitrary pixel. As an example of a predetermined position, it may be obtained from reference pixels at upper left, upper and left positions outside the target block ( Figure 8 A, E, M in the target block) and based on the reference pixels ( Figure 8 I, Q) in can be obtained at this position.

[0378] In addition, as an example of a reference pixel corresponding to the position of an arbitrary pixel, it can be obtained from a reference pixel corresponding to the x component or y component of the arbitrary pixel (for example,<x,-1> , <-1,y>,<x+y+1,-1> , <-1,x+y+1>, etc.)

[0379] The data value of one of the reference pixels at the position may be obtained as the data value at the position of the arbitrary pixel, or a value derived based on two or more reference pixels may be obtained as the data value at the position of the arbitrary pixel.

[0380] For example, a reference pixel at a preset position among a plurality of reference pixels for the position of an arbitrary pixel may be obtained as a data value at the position of the arbitrary pixel. Alternatively, a value obtained in a predetermined process (e.g., normalization, maximum value, minimum value, center value, etc.) between a plurality of reference pixels may be obtained as a data value at the position of the arbitrary pixel. Alternatively, a value obtained in a predetermined process (e.g., weighted average, etc.) between a plurality of reference pixels may be obtained as a data value at the position of the arbitrary pixel.

[0381] The following describes a method for obtaining the data value at the position of an arbitrary pixel based on multiple reference pixels. Figure 8 A, E, M in are used as reference pixels, and in the examples mentioned later, they are referred to as x, y, z.

[0382]

Table 2

[0383]

[0384] Candidates numbered 0 to 2 in the table mean a case where a reference pixel at a predetermined position is obtained as a data value at the position of an arbitrary pixel. Candidates numbered 3 to 6 mean a case where data at an arbitrary pixel position is obtained by reflecting the gradient of the target block. In detail, this may correspond to a case where the gradient of the target block is obtained based on the reference pixel at a preset position.

[0385] It should be understood that this example can be an example of an equation derived from multiple pixels assuming that the target block has a square shape and can be changed and applied according to the aspect ratio of the block. In addition, the data value at an arbitrary pixel position can be obtained in various ways without being limited to this example.

[0386] Fig.12 12A, 12B, 12C, 12D, 12E, and 12F are example diagrams of performing modification based on arbitrary pixels according to an embodiment of the present disclosure.

[0387] Reference Fig.12 A, Fig.12 A denotes an interpolation process using reference pixels corresponding to an upper block and arbitrary pixels, and denotes a process of performing modification by using reference pixels in left, right, upper, upper-left, and upper-right directions of a target block obtained therefrom.

[0388] When the prediction mode of the target block is a vertical mode, this process can be applied when the modification is performed by reflecting the gradient of the reference pixel in the left and right directions. This can be different from the existing case when the modification is performed based on the gradient of the reference pixel in the left direction.

[0389] Reference Fig.12 C, Fig.12 C indicates an interpolation process using reference pixels corresponding to a left block and arbitrary pixels, and indicates a process of performing modification by using reference pixels in the upper, lower, left, upper left, and lower left directions of a target block obtained therefrom.

[0390] When the prediction mode of the target block is the horizontal mode, the process can be applied when the modification is performed by reflecting the gradient of the reference pixel in the upper and lower directions. It can be different from the existing case when the modification is performed based on the gradient of the reference pixel in the upper direction.

[0391] Reference Fig.12 E, Fig.12 E represents interpolation processing using reference pixels corresponding to the left block and the upper block and arbitrary pixels, and represents a modified process performed by using reference pixels in the upper, lower, left, right, upper left, upper right, lower left, and lower right directions of the target block obtained therefrom.

[0392] The following represents an equation for obtaining a modified pixel by applying the modification to the predicted pixel. In this case, the reference pixel may be configured differently from the previous equation because an arbitrary pixel or a reference pixel obtained based on an arbitrary pixel is included.

[0393] Z=(z*w0)+(a*w1)+(b*w2)+(c*w3)+(d*w4)+(e*w5)

[0394] In this case, z and Z represent predicted pixels and modified pixels, respectively, a to e represent reference pixels for modification, and w0 to w5 represent weights applied to the modification. According to the number of reference pixels for modification, the reference pixels and weights may be determined. In this case, a value considering row normalization, including 0, may be assigned to the weights.

[0395] According to the prediction mode of the target block, the location and number of reference pixels can be determined, and Fig.12 A. Fig.12 B. Fig.12 C. Fig.12 D. Fig.12 E and Fig.12 Various cases of F are possible. Since other explanations on modifying the settings can be derived in the section on the previous equations, the detailed explanation is omitted.

[0396] The case where modification is performed in intra prediction is described through various embodiments. In this case, only the case where modification is performed after obtaining the prediction block is described, but modification can be performed in the intra prediction process (ie, the process of generating the prediction block) after being reflected.

[0397] In addition, the flag information generation in the modification process can be encoded / decoded in a configuration separate from the intra-frame prediction mode, but a configuration combined with the intra-frame prediction mode or dependent on the intra-frame prediction mode is possible. In other words, it should be understood that it is not limited to an additional or subsequent part after the intra-frame prediction process.

[0398] The method according to the present disclosure can be recorded in a computer-readable medium after being implemented in the form of program instructions that can be executed by various computer devices. The computer-readable medium can include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded in the computer-readable medium can be specially designed and configured for the present disclosure, or can be available after being notified to a technician in the field of computer software.

[0399] Examples of computer-readable media may include hardware organizations, such as ROM, RAM, flash memory, etc., which are specifically configured to store and execute program instructions. In addition to machine language codes generated by a compiler, examples of program instructions may include high-level language codes that can be run by a computer with an interpreter, etc. The above-mentioned hardware devices may be configured to operate as at least one software module that performs the actions of the present disclosure, and vice versa.

[0400] In addition, the above-mentioned methods or apparatuses may be implemented after all or part of their configurations or functions are combined or implemented separately.

[0401] Although described above with reference to the desired embodiments of the present disclosure, those skilled in the relevant art will appreciate that various modifications and changes may be made to the present disclosure without departing from the spirit and scope of the present disclosure as described in the appended claims.

[0402] Industrial Applicability

[0403] The present disclosure may be used to encode / decode video signals. < / l-2> < / a-1> < / l-1> < / qt>

Claims

1. A method for decoding a video signal using a decoding device, comprising: determining an intra prediction mode for a current block in the video signal; Constructing a reference sample of the current block; as well as Generating a prediction block of the current block based on the intra prediction mode and the constructed reference sample; reconstructing the current block based on the prediction block and the residual block of the current block, The constructed reference samples include left reference samples and right reference samples. The left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block and adjacent to the right boundary of the current block, wherein generating the prediction block of the current block comprises, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, Wherein, the specific number is determined to be 5, The specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample. wherein the two left reference samples include the bottommost reference sample among the left reference samples, and The position of the bottommost reference sample relative to the upper left sample of the current block is (-1, H-1), and H refers to the height of the current block.

2. The method according to claim 1, wherein: Constructing reference samples of the current block includes determining availability of the reference samples.

3. The method according to claim 2, wherein: When at least one of the reference samples is located outside a picture boundary, the at least one of the reference samples is not included in the same tile as the current block, or decoding of the at least one of the reference samples is not completed, the at least one of the reference samples is determined to be unavailable.

4. The method according to claim 3, wherein: When restricted intra prediction is used for the current block, the availability of the left reference sample is determined based on the encoding mode of the reference sample candidate block adjacent to the left side of the current block, and the availability of the right reference sample is determined based on the encoding mode of the reference sample candidate block adjacent to the right side of the current block.

5. The method according to claim 1, wherein: The specific number of reference samples includes a first reference sample determined based on a position of a current sample within the current block and a second reference sample determined based on a width or a height of the current block.

6. The method according to claim 5, wherein: Generate a prediction sample of the current sample using any sample and the first reference sample determined based on the position of the current sample in the specific number of reference samples, and The arbitrary sample is obtained by weighted summing the second reference samples in the specific number of reference samples.

7. The method according to claim 6, wherein: A prediction sample of the current sample is generated by applying weights to the arbitrary sample and the first reference sample respectively. The method of claim 7 , wherein the weight is determined based on a position of the current sample in the current block.

9. A method for encoding a video signal using an encoding device, comprising: determining an intra prediction mode for a current block in the video signal; Constructing a reference sample of the current block; Generate a residual block of the current block using a prediction block, wherein the prediction block is generated based on the intra prediction mode and the reference sample; as well as encoding the residual block into a bitstream, The constructed reference samples include left reference samples and right reference samples. The left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block and adjacent to the right boundary of the current block, wherein generating the prediction block of the current block comprises, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, Wherein, the specific number is determined to be 5, The specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample. wherein the two left reference samples include the bottommost reference sample among the left reference samples, and The position of the bottommost reference sample relative to the upper left sample of the current block is (-1, H-1), and H refers to the height of the current block.

10. A non-transitory computer-readable storage medium storing a bitstream generated by a method of encoding a video signal using an encoding device, the method comprising: determining an intra prediction mode for a current block in the video signal; Constructing a reference sample of the current block; Generate a residual block of the current block using a prediction block, wherein the prediction block is generated based on the intra prediction mode and the reference sample; as well as encoding the residual block into a bitstream, The constructed reference samples include left reference samples and right reference samples. The left reference sample belongs to a reconstructed block adjacent to the left side of the current block and adjacent to the left boundary of the current block, and the right reference sample belongs to a reconstructed block adjacent to the right side of the current block and adjacent to the right boundary of the current block, wherein generating the prediction block of the current block comprises, when the intra prediction mode is a non-directional mode, using a specific number of reference samples in the constructed reference samples to generate prediction samples of the current sample in the current block, Wherein, the specific number is determined to be 5, The specific number of reference samples includes a top reference sample, two left reference samples and one right reference sample. wherein the two left reference samples include the bottommost reference sample among the left reference samples, and The position of the bottommost reference sample relative to the upper left sample of the current block is (-1, H-1), and H refers to the height of the current block.

11. A method for decoding a video signal using a decoding device, comprising: determining an intra prediction mode to be applied to intra prediction of a current block; generating a prediction block of the current block based on the intra prediction mode; as well as reconstructing the current block based on the prediction block and a residual block of the current block, Wherein, generating the prediction block of the current block includes: configuring reference samples for the intra prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and A value of a second prediction sample within the prediction block is derived based on at least one of the configured reference samples and the first prediction sample within the prediction block.

12. The method according to claim 11, wherein: Deriving a value of the second prediction sample is performed based on interpolation using at least one of the configured reference samples and the first prediction sample.

13. The method according to claim 11, wherein: Deriving the value of the first prediction sample includes averaging two reference samples adjacent to each other.

14. A method for encoding a video signal using an encoding device, comprising: determining an intra prediction mode to be applied to intra prediction of a current block; generating a prediction block of the current block based on the intra prediction mode; as well as generating a residual block of the current block based on the prediction block; as well as encoding the residual block into a bitstream, Wherein, generating the prediction block of the current block includes: configuring reference samples for the intra prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and A value of a second prediction sample within the prediction block is derived based on at least one of the configured reference samples and the first prediction sample within the prediction block.

15. A non-transitory computer-readable recording medium storing a bit stream generated by a method for encoding a video signal, the method comprising: determining an intra prediction mode to be applied to intra prediction of a current block; generating a prediction block of the current block based on the intra prediction mode; as well as generating a residual block of the current block based on the prediction block; as well as encoding the residual block into a bitstream, Wherein, generating the prediction block of the current block includes: configuring reference samples for the intra prediction of the current block; deriving a value of a first prediction sample within the prediction block of the current block based on at least one of the configured reference samples; and A value of a second prediction sample within the prediction block is derived based on at least one of the configured reference samples and the first prediction sample within the prediction block.

Images