Image processing method and computer readable recording medium

Abstract

The present application provides an image processing method and a computer readable recording medium, wherein the image processing method comprises: a step of dividing an image of an image into a plurality of coding units which are basic units for performing inter-frame prediction or intra-frame prediction; and a step of selectively constructing, for a coding unit which is intra-frame predicted among the divided coding units, a prediction mode list for deriving a prediction direction of a decoding target block from intra-frame prediction directions of surrounding blocks adjacent to the decoding target block, the coding unit dividing step comprising a step of dividing the image or the divided coding unit into a binary tree structure.

Description

[0001] This application is a divisional application of patent application filed on October 12, 2017, with application number 201780063502.0, entitled "Image processing method, image decoding and encoding method thereon". Technical Field

[0002] This invention relates to an image encoding and decoding method, and more specifically, to a method for segmenting a video image into multiple blocks for intra-frame prediction and transformation. Background Technology

[0003] Image compression methods divide an image into multiple blocks of predetermined size for encoding. Furthermore, to improve compression efficiency, inter-prediction and intra-prediction techniques are used to remove redundancy between multiple images.

[0004] At this point, residual signals are generated using intra-frame prediction and inter-frame prediction. The reason for calculating residual signals is that when encoding using residual signals, the data compression rate becomes higher due to the small data volume. The better the prediction, the smaller the value of the residual signal.

[0005] Intra-frame prediction methods use the surrounding pixels of the current block to predict the data for that block. The difference between the actual value and the predicted value is called the residual signal. In the case of HEVC, the intra-frame prediction method increases the number of prediction modes from the 9 used in the existing H.264 / AVC to 35, thus providing more refined predictions.

[0006] In inter-frame prediction methods, the current block is compared with blocks within multiple surrounding pixels to search for the most similar block. The positional information (Vx, Vy) of the searched block is then called a motion vector. The difference between the intra-block pixel values ​​of the predicted block predicted using the current block and the motion vector is called the motion-compensated residual block.

[0007] Thus, although further refining intra-frame prediction and inter-frame prediction reduces the amount of residual signal data, the computational load for video processing increases significantly.

[0008] In particular, the process of determining the intra-image segmentation structure used for image encoding and decoding becomes more complex, which makes it difficult to implement pipelines, etc. Existing block segmentation methods and the size of the resulting blocks may not be suitable for encoding high-resolution images. Summary of the Invention

[0009] The present invention is proposed to solve the problems described above, and its purpose is to provide an image processing method suitable for encoding and decoding high-resolution images, as well as an image decoding and encoding method using the same.

[0010] To address the aforementioned problems, an image processing method according to an embodiment of the present invention includes the following steps: segmenting an image into a plurality of coding tree units; obtaining at least one coding unit from the coding tree units among the plurality of coding tree units; and determining a prediction mode list for deriving intra-prediction modes of the coding unit from the at least one coding unit among the coding units based on a plurality of flags including a first flag and a second flag, using intra-prediction modes of at least one peripheral block adjacent to the coding unit, wherein the first flag corresponds to the most probable mode, the second flag corresponds to the selected mode, and the first flag indicates whether to derive intra-prediction modes from the plurality of intra-prediction modes in the prediction mode list. In the formula, a first intra-frame prediction mode is determined as the prediction mode of the coding unit, a second flag indicates whether a second prediction mode different from the plurality of intra-frame prediction modes in the prediction mode list is determined as the prediction mode of the coding unit, and when the first flag indicates that the first intra-frame prediction mode is not determined from the plurality of intra-frame prediction modes in the prediction mode list as the prediction mode of the coding unit, based on the information obtained from the bitstream for deriving a third intra-frame prediction mode, the third prediction mode different from the plurality of intra-frame prediction modes in the prediction mode list and the second prediction mode are determined as the prediction mode of the coding unit.

[0011] An image processing method according to an embodiment of the present invention includes the following steps: segmenting an image into a plurality of coding tree units; determining at least one coding unit obtained from the coding tree units among the plurality of coding tree units; and determining a plurality of flags including a first flag and a second flag based on whether a prediction mode list for deriving an intra prediction mode of the coding unit is determined from at least one of the coding units by using intra prediction modes of at least one peripheral block adjacent to the coding unit, wherein the first flag corresponds to the most probable mode, the second flag corresponds to the selection mode, the first flag indicates whether a first intra prediction mode is determined as the prediction mode of the coding unit from the plurality of intra prediction modes in the prediction mode list, the second flag indicates whether a second prediction mode different from the plurality of intra prediction modes in the prediction mode list is determined as the prediction mode of the coding unit, and when the prediction mode of the coding unit is a third prediction mode and the second prediction mode different from the plurality of intra prediction modes in the prediction mode list, based on information obtained from the bitstream for deriving the third intra prediction mode, the first flag indicates not to determine the first intra prediction mode as the prediction mode of the coding unit from the plurality of intra prediction modes in the prediction mode list.

[0012] Furthermore, according to an embodiment of the present invention, a computer-readable recording medium stores a bitstream encoded by an image processing method executed by an image processing apparatus, the image processing method comprising the steps of: segmenting an image into a plurality of coding tree units; determining at least one coding unit obtained from the coding tree units among the plurality of coding tree units; and determining a plurality of flags including a first flag and a second flag based on whether a prediction mode list for deriving an intra prediction mode for the coding unit is determined from at least one of the coding units by using an intra prediction mode of at least one peripheral block adjacent to the coding unit, wherein the first flag corresponds to the most probable mode, the second flag corresponds to a selected mode, and the first flag indicates that... Whether to determine a first intra-prediction mode from among the multiple intra-prediction modes in the prediction mode list as the prediction mode of the coding unit, and whether to determine a second prediction mode different from the multiple intra-prediction modes in the prediction mode list as the prediction mode of the coding unit. When the prediction mode of the coding unit is a third prediction mode and the second prediction mode different from the multiple intra-prediction modes in the prediction mode list, based on the information obtained from the bitstream for deriving the third intra-prediction mode, the first flag indicates not to determine the first intra-prediction mode from among the multiple intra-prediction modes in the prediction mode list as the prediction mode of the coding unit.

[0013] Invention Effects

[0014] According to embodiments of the present invention, by segmenting the coding units that perform inter-frame prediction or intra-frame prediction into a binary tree structure, and constructing a prediction mode list for intra-frame prediction from the surrounding blocks for the segmented coding units, the coding efficiency for high-resolution images can be improved. Attached Figure Description

[0015] Figure 1 This is a block diagram illustrating the structure of an image encoding apparatus according to an embodiment of the present invention.

[0016] Figures 2 to 5 This is a diagram illustrating a first embodiment of a method for processing images in block units.

[0017] Figure 6 This is a block diagram illustrating an embodiment of a method for performing inter-frame prediction in an image coding apparatus.

[0018] Figure 7 This is a block diagram illustrating the structure of an image decoding apparatus according to an embodiment of the present invention.

[0019] Figure 8 This is a block diagram illustrating an embodiment of a method for performing inter-frame prediction in an image decoding apparatus.

[0020] Figure 9 This is a diagram illustrating a second embodiment of a method for processing images in block units.

[0021] Figure 10 This is a diagram illustrating one embodiment of the syntax structure used when processing images in block units.

[0022] Figure 11 This is a diagram illustrating a third embodiment of a method for processing images in block units.

[0023] Figure 12 This is a diagram illustrating an embodiment of a method for constructing a transform unit by dividing a coding unit into a binary tree structure.

[0024] Figure 13 This is a diagram illustrating a fourth embodiment of a method for processing images in block units.

[0025] Figures 14 to 16 This is a diagram illustrating another embodiment of a method for processing images in block units.

[0026] Figure 17 and Figure 18 This is a diagram illustrating an embodiment of a method for determining the segmentation structure of a transform unit by performing rate distortion optimization.

[0027] Figure 19 This is a diagram illustrating a first embodiment of the block-based adaptive loop filter method according to the present invention.

[0028] Figure 20 This is a diagram illustrating a second embodiment of the block-based adaptive loop filter method according to the present invention.

[0029] Figure 21 This is a flowchart illustrating the operation of a decoding apparatus that performs intra-frame prediction according to an embodiment of the present invention.

[0030] Figures 22 to 24 This is a reference diagram used to illustrate how an MPM mode is constructed from a peripheral block according to an embodiment of the present invention.

[0031] Figure 25 This is a flowchart illustrating an MPM directory construction method according to an embodiment of the present invention.

[0032] Figure 26 and Figure 27 This is a flowchart illustrating an intra-frame prediction mode decoding method in a non-MPM mode according to an embodiment of the present invention.

[0033] Figure 28 This is a flowchart illustrating the update-based decoding method for MPM mode and non-MPM mode according to embodiments of the present invention. Detailed Implementation

[0034] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings. When describing the embodiments of the present invention, detailed descriptions of relevant known structures or functions may be omitted if it is determined that such detailed descriptions would obscure the main points of this specification.

[0035] When a structural element is referred to as being "connected" or "linked" to another structural element, it should be understood that it can be directly connected or linked to other structural elements, but other constituent elements may also exist in between. Furthermore, the description in this invention as "comprising" a specific structure does not exclude structures other than the corresponding structure, but may mean that additional structures may be included within the scope of the implementation of this invention or the technical concept of this invention.

[0036] The terms "first" and / or "second" can be used to describe various structural elements, but the structural elements are not limited to these terms. The terms are used for the purpose of distinguishing a structural element from other structural elements. For example, without departing from the scope of the invention, a first structural element may be referred to as a second structural element, and similarly, a second structural element may be referred to as a first structural element.

[0037] The constituent components shown in embodiments of the invention are illustrated independently to represent their distinct functional characteristics. Therefore, this does not imply that each component is composed of separate hardware or software units. In other words, for convenience, each constituent part includes each of the listed components. Thus, at least two constituent parts of each constituent part can be combined to form a single constituent part, or a constituent part can be divided into multiple constituent parts to perform each function. Embodiments combining each constituent part and embodiments separating one of the constituent parts are also included within the scope of the invention without departing from its spirit.

[0038] Furthermore, some components may not be essential for performing the basic functions of the invention, but rather selective components that merely improve its performance. In addition to components for improving performance, the invention can be implemented by including only the essential components necessary to achieve the essence of the invention. Structures containing only essential components, besides selective components for improving performance, are also included within the scope of the invention.

[0039] Figure 1 This is a block diagram illustrating the structure of an image encoding apparatus according to an embodiment of the present invention. The image encoding apparatus 10 includes: an image segmentation unit 110, a transformation unit 120, a quantization unit 130, a scanning unit 131, an entropy encoding unit 140, an intra-frame prediction unit 150, an inter-frame prediction unit 160, an inverse quantization unit 135, an inverse transformation unit 125, a post-processing unit 170, an image storage unit 180, a subtraction unit 190, and an addition unit 195.

[0040] Reference Figure 1 The image segmentation unit 110 analyzes the input video signal, segments the image into coding units, determines the prediction mode, and determines the size of the prediction unit according to the coding unit.

[0041] The image segmentation unit 110 transmits the prediction unit to be encoded to the intra-frame prediction unit 150 or the inter-frame prediction unit 160 according to the prediction mode (or prediction method). Furthermore, the image segmentation unit 110 transmits the prediction unit to be encoded to the subtraction unit 190.

[0042] Here, the image is composed of multiple slices, which can be divided into multiple coding tree units (CTUs) used to segment the image as the basic unit.

[0043] The coding tree unit can be divided into one or more coding units (CUs) that are basic units for performing inter-prediction or intra-prediction.

[0044] The coding unit (CU) can be divided into one or more prediction units (PUs) that are the basic units that perform prediction.

[0045] At this time, the encoding device 10 determines one of the inter-frame prediction and intra-frame prediction for the segmented encoding units CU using a prediction method. However, different prediction blocks can be generated for each prediction unit PU.

[0046] On the one hand, the coding unit (CU) can be divided into one or more transformation units (TUs) that are the basic units for transforming the residual block.

[0047] At this time, the image segmentation unit 110 transmits the image data to the subtraction unit 190 in block units (e.g., prediction unit PU or transformation unit TU) as described above.

[0048] Reference Figure 2 The coding tree unit (CTU) with a maximum size of 256×256 pixels is divided into a quadtree structure, which can then be divided into four coding units (CU) with a square shape.

[0049] The four coding units (CUs) with a square shape can be further divided into quadtree structures. As mentioned above, the depth of the coding unit (CU) divided into a quadtree structure can have an integer value from 0 to 3.

[0050] The coding unit (CU) can be divided into one or more prediction units (PU) according to the prediction mode.

[0051] When the size of the coding unit (CU) is 2N×2N in intra-frame prediction mode, the size of the prediction unit (PU) can be as follows: Figure 3 (a) shows a 2N×2N or Figure 3 (b) shows the N×N.

[0052] On the one hand, when the size of the coding unit (CU) is 2N×2N in the inter-frame prediction mode, the size of the prediction unit (PU) can be as follows: Figure 4 (a) shows a 2N×2N, Figure 4 (b) shows 2N×N, Figure 4 (c) shows N×2N, Figure 4 (d) shows the N×N, Figure 4 (e) shows 2N×nU, Figure 4 (f) shows 2N×nD, Figure 4 (g) shows nL×2N and Figure 4 (h) represents one of the sizes of nR×2N.

[0053] Reference Figure 5The encoding unit CU is divided into a quad tree structure, which can then be divided into four transformation units TU with a square shape.

[0054] The four coding units CU, which are square in shape, can be further divided into quadtree structures. As mentioned above, the depth of the transformation unit TU, which is divided into a quadtree structure, can have an integer value from 0 to 3.

[0055] Here, when the coding unit CU is in inter-frame prediction mode, the prediction unit PU and the transform unit TU segmented from the coding unit CU can have independent segmentation structures.

[0056] When the coding unit CU is in intra-prediction mode, the transform unit TU segmented from the coding unit CU cannot be larger than the size of the prediction unit PU.

[0057] Moreover, the transformation unit segmented as described above has a maximum size of 64×64 pixels.

[0058] The transformation unit 120 transforms the original block of the input prediction unit PU and the residual block of the residual signal between the prediction block generated by the intra-frame prediction unit 150 or the inter-frame prediction unit 160, and the transformation is performed with the transformation unit TU as the basic unit.

[0059] During the transformation process, different transformation matrices can be determined according to the prediction mode (intra or inter). The residual signal of intra-frame prediction has directionality according to the intra-frame prediction mode. Therefore, the transformation matrix can be adaptively determined according to the intra-frame prediction mode.

[0060] The transformation unit can be transformed by two (horizontal and vertical) one-dimensional transformation matrices. For example, in the case of inter-frame prediction, a predetermined transformation matrix can be determined.

[0061] On the one hand, in the case of intra-frame prediction, when the intra-frame prediction mode is horizontal, the probability of the residual block having vertical directionality increases. Therefore, an integer matrix based on DCT is suitable for the vertical direction, and an integer matrix based on DST or KLT is suitable for the horizontal direction. When the intra-frame prediction mode is vertical, an integer matrix based on DST or KLT is suitable for the vertical direction, and an integer matrix based on DCT is suitable for the horizontal direction.

[0062] Furthermore, in DC mode, integer matrices based on DCT can be applied in both directions.

[0063] Moreover, in the case of intra-frame prediction, the transformation matrix can be adaptively determined based on the size of the transform unit TU.

[0064] The quantization unit 130 determines the quantization step size for quantizing the coefficients of the residual block transformed by the transformation matrix, and the quantization step size can be determined according to a quantization unit of a preset size or larger.

[0065] The size of the quantization unit can be 8×8 or 16×16. The quantization unit 130 quantizes the coefficients of multiple transform blocks using a quantization matrix determined according to the quantization step size and prediction mode.

[0066] Furthermore, the quantization unit 130, as a predictor of the quantization step size of the current quantization unit, can utilize the quantization step size of the quantization unit adjacent to the current quantization unit.

[0067] The quantization unit 130 searches in the order of the left quantization unit, the top quantization unit, and the top left quantization unit of the current quantization unit, and can generate a quantization step size predictor for the current quantization unit using one or two valid quantization step sizes.

[0068] For example, the quantization unit 130 may determine the first valid quantization step size obtained from the sequential search as the quantization step size predictor, or determine the average of the two valid quantization steps obtained from the sequential search as the quantization step size predictor, or determine the quantization step size predictor when only one quantization step size is valid.

[0069] If the quantization step size predictor is determined, the quantization unit 130 transmits the difference between the quantization step size of the current quantization unit and the quantization step size predictor to the entropy coding unit 140.

[0070] On the one hand, the left-side coding unit, the top-side coding unit, and the top-left-side coding unit of the current coding unit may all be absent, or there may be previously existing coding units in the coding order within the largest coding unit.

[0071] Therefore, according to the encoding order, the quantization step size of the quantization unit adjacent to the current encoding unit and the quantization unit immediately preceding the largest encoding unit becomes a candidate.

[0072] At this point, the priority order can be set as follows: 1) the left quantization unit of the current coding unit, 2) the top quantization unit of the current coding unit, 3) the top-left quantization unit of the current coding unit, and 4) the quantization unit immediately preceding the current coding unit. This order can be changed, and the top-left quantization unit can be omitted.

[0073] On the one hand, as described above, the quantized transform block is transmitted to the inverse quantization unit 135 and the scanning unit 131.

[0074] The scanning unit 131 scans the coefficients of the quantized transform block and transforms them into one-dimensional quantized coefficients. Since the coefficient distribution of the quantized transform block depends on the intra-prediction mode and is non-zero, the scanning method can be determined according to the intra-prediction mode.

[0075] Furthermore, the coefficient scanning method can be determined according to the size of the transformation unit, and the scanning method can vary according to the directional intra-frame prediction mode. In this case, the scanning order of the quantization coefficients can be scanned in opposite directions.

[0076] When the quantized coefficients are divided into multiple subsets, the same scanning method is applied to the quantized coefficients within each subset. The scanning method between subsets can be a zigzag scan or a diagonal scan.

[0077] On the one hand, preferably, the scanning method scans from the principal subset including DC to the remaining subset in the forward direction; however, a reverse scan is also possible.

[0078] In addition, the scanning method between subsets can be set in the same way as the scanning method of the quantized coefficients in the subset, and the scanning method between subsets can be determined according to the intra-frame prediction mode.

[0079] On the other hand, the encoding device 10 enables information indicating the position of the last non-zero quantization coefficient in the transformation unit PU and the position of the last non-zero quantization coefficient in each subset to be included in the bit stream and transmitted to the decoding device 20.

[0080] The inverse quantization unit 135 performs inverse quantization on the quantized coefficients as described above, and the inverse transformation unit 125 performs inverse transformation with transformation unit TU, which can reconstruct the inverse quantized transformation coefficients into residual blocks of the spatial region.

[0081] The adder 195 adds the residual block reconstructed by the inverse transform 125 and the prediction block received from the intra-frame prediction 150 or the inter-frame prediction 160 to generate a reconstructed block.

[0082] In addition, the post-processing unit 170 can perform a deblocking filtering process to eliminate the block effect generated in the reconstructed image; a sample adaptive offset (SAO) application process to compensate for the difference between the original image and the original image on a pixel-by-pixel basis; and adaptive loop filtering (ALF) to compensate for the difference between the original image and the original image on a coding unit basis.

[0083] The unblocking filtering process can be applied to the boundaries of prediction units PU or transformation units TU with a size greater than a predetermined size.

[0084] For example, the unblocking filtering process may include: determining the bounary to be filtered; determining the boundary filter strength applicable to the bounary; determining whether a unblocking filter is applicable; and, if a unblocking filter is applicable, selecting a filter applicable to the bounary.

[0085] On the other hand, whether the deblocking filter is applicable may depend on i) whether the boundary filtering intensity is greater than 0; and ii) whether the pixel values ​​of the boundary portions of the two blocks (P block, Q block) adjacent to the boundary to be filtered are less than a first reference value determined by a quantization parameter representing the degree of change.

[0086] Preferably, there are at least two filters. If the absolute value of the difference between two pixels located at the block boundary is greater than or equal to the second reference value, then a filter that performs relatively weak filtering is selected.

[0087] The second reference value is determined by the quantization parameter and the boundary filter strength.

[0088] Furthermore, the Sample Adaptive Offset (SAO) application process is used to reduce the distortion between pixels in the image to which the unblocking filter is applied and the original pixels. Whether to apply SAO can be determined on an image or slice unit.

[0089] An image or slice can be divided into multiple offset regions, and an offset type can be determined for each offset region. The offset types may include a predetermined number (e.g., 4) of edge offset types and 2 band offset types.

[0090] For example, when the offset type is an edge offset type, the edge type of each pixel is determined to apply the corresponding offset. The edge type can be determined based on the distribution of the values ​​of the two pixels adjacent to the current pixel.

[0091] The Adaptive Loop Filtering (ALF) process can filter based on comparing the values ​​of the reconstructed image and the original image after the unblocking filtering process or the adaptive offset application process.

[0092] The image storage unit 180 receives post-processed image data from the post-processing unit 170 and reconstructs the image in image units for storage. The image can be an image in frame units or an image in field units.

[0093] The inter-frame prediction unit 160 can perform motion estimation using at least one reference image stored in the image storage unit 180, and can determine the reference image index and motion vector representing the reference image.

[0094] At this time, based on the determined reference image index and motion vector, a prediction block corresponding to the prediction unit to be encoded can be extracted from the reference image used for motion estimation among multiple reference images stored in the image storage unit 180.

[0095] The intra-prediction unit 150 can perform intra-prediction coding using reconstructed pixel values ​​from the image including the current prediction unit.

[0096] The intra-prediction unit 150 receives the current prediction unit to be predicted and selects one of a predetermined number of intra-prediction modes to perform intra-prediction based on the size of the current block.

[0097] The intra-prediction unit 150 can adaptively filter reference pixels to generate intra-prediction blocks, and when reference pixels cannot be used, it can use available reference pixels to generate reference pixels.

[0098] The entropy coding unit 140 can entropy code the quantization coefficients quantized by the quantization unit 130, the intra-prediction information received from the intra-prediction unit 150, and the motion information received from the inter-prediction unit 160.

[0099] Figure 6 This is a block diagram illustrating an embodiment of the structure in which inter-frame prediction is performed in the encoding apparatus 10. The illustrated inter-frame prediction encoder may include a motion information determination unit 161, a motion information encoding mode determination unit 162, a motion information encoding unit 163, a prediction block generation unit 164, a residual block generation unit 165, a residual block encoding unit 166, and a multiplexer 167.

[0100] Reference Figure 6 The motion information determination unit 161 determines the motion information of the current block. The motion information includes a reference image index and a motion vector. The reference image index can represent one of the images that have been encoded and reconstructed.

[0101] It may include a reference image index representing a reference image belonging to list 0 (L0) when performing one-way inter-frame prediction coding on the current block, and a reference image index representing a reference image belonging to list 0 (L0) and a reference image index representing a reference image belonging to list 1 (L1) when performing two-way prediction coding on the current block.

[0102] Furthermore, it may also include an index of one or two reference images representing the composite list (L / C) generated by combining list 0 and list 1 when bidirectional predictive coding is performed on the current block.

[0103] The motion vector represents the position of the predicted block in the image indicated by each reference image index, and the motion vector can be in pixel units (integer units) or subpixel units.

[0104] For example, the motion vector may have a resolution of 1 / 2, 1 / 4, 1 / 8, or 1 / 16 pixels, and when the motion vector is not an integer unit, the prediction block may be generated from integer units of pixels.

[0105] The motion information encoding mode determination unit 162 can determine the encoding mode of the motion information of the current block by one of the skip mode, merge mode and AMVP mode.

[0106] The skip mode is applicable when there is a skip candidate with the same motion information as the current block, and when the residual signal is 0. The skip mode can be applied when the size of the prediction unit PU, i.e., the current block, is the same as that of the encoding unit CU.

[0107] The merging mode is applicable when there is a merging candidate with the same motion information as the current block, and when there is a residual signal when the size of the current block is different from or the same as the coding unit (CU). On one hand, the merging candidate can be the same as the skipping candidate.

[0108] The AMVP mode is applicable when the skip mode and merge mode are not used, and it selects the AMVP candidate with the motion vector that is most similar to the motion vector of the current block as the AMVP predictor.

[0109] The motion information encoding unit 163 can encode motion information according to the method determined by the motion information encoding mode determination unit 162.

[0110] For example, the motion information encoding unit 163 performs a merged motion vector encoding process when the motion information encoding mode is skip mode or merge mode, and performs an AMVP encoding process when the AMVP mode is used.

[0111] The prediction block generation unit 164 generates a prediction block using the motion information of the current block. When the motion vector is an integer unit, it copies the block corresponding to the position represented by the motion vector in the image indicated by the reference image index and generates the prediction block of the current block.

[0112] On the one hand, when the motion vector is not an integer unit, the prediction block generation unit 164 can generate pixels of the prediction block from integer unit pixels in the image indicated by the reference image index.

[0113] At this point, an 8-tap interpolation filter can be used to generate predicted pixels for luminance pixels, and a 4-tap interpolation filter can be used to generate predicted pixels for chrominance pixels.

[0114] The residual block generation unit 165 generates a residual block using the current block and the prediction block of the current block. When the size of the current block is 2N×2N, the residual block can be generated using the current block and the prediction block of the current block with a size of 2N×2N.

[0115] On the one hand, when the size of the current block used for prediction is 2N×N or N×2N, after calculating the prediction block for each of the two 2N×N blocks that are constructed, the two 2N×N prediction blocks can be used to generate the final prediction block of size 2N×2N.

[0116] Furthermore, a residual block of size 2N×2N can be generated using a prediction block of size 2N×2N. To address the discontinuity at the boundary of the two prediction blocks of size 2N×N, overlapping smoothing can be applied to the pixels at the boundary.

[0117] The residual block encoding unit 166 divides the residual block into one or more transform units TU, and can perform transform encoding, quantization and entropy encoding on each transform unit TU.

[0118] The residual block coding unit 166 can use integer-based transform moments to transform the residual blocks generated by the inter-frame prediction method, and the transform moments can be integer-based DCT matrices.

[0119] On one hand, the residual block encoding unit 166 uses a quantization matrix to quantize the coefficients of the residual block transformed by the transformation matrix, and the quantization matrix can be determined by quantization parameters.

[0120] The quantization parameter can be determined according to the coding unit CU of a predetermined size or larger. When the current coding unit CU is smaller than the predetermined size, the quantization parameter of the first coding unit CU is encoded in the coding order within the predetermined size, while the quantization parameter of the remaining coding units CU is the same as the parameter, so no encoding is required.

[0121] Alternatively, the coefficients of the transform block can be quantized using a quantization matrix determined based on the quantization parameters and the prediction mode.

[0122] The quantization parameters of the coding unit CU adjacent to the current coding unit CU can be used to predictively encode the quantization parameters determined according to the coding unit CU of the predetermined size or above.

[0123] The search is performed in the order of the left coding unit CU and the upper coding unit CU of the current coding unit CU, and one or two valid quantization parameters are used to generate the quantization parameter predictor of the current coding unit CU.

[0124] For example, the first valid quantization parameter searched in the above order can be identified as the quantization parameter predictor, and the search is performed in the order of the left coding unit CU and the coding unit CU immediately preceding the coding unit in the coding order to identify the first valid quantization parameter as the quantization parameter predictor.

[0125] The coefficients of the quantized transform block are scanned and transformed into one-dimensional quantized coefficients. Furthermore, the scanning method can be set according to different entropy coding modes.

[0126] For example, when encoding with CABAC, the quantization coefficients of inter-frame predictive coding can be scanned in a predetermined manner (zigzag or diagonal raster scanning). When encoding with CAVLC, the scanning can be performed in a different manner.

[0127] For example, zigzag scanning is used during inter-frame scanning, and the scanning method can be determined according to the intra-frame prediction mode during intra-frame scanning. Furthermore, the coefficient scanning method can be determined according to the size of the transform unit.

[0128] On the one hand, the scanning method can vary depending on the directional intra-frame prediction mode, and the scanning order of the quantization coefficients can be scanned in opposite directions.

[0129] The multiplexer 167 performs multiplexing on the motion information encoded by the motion information encoding unit 163 and the residual signal encoded by the residual block encoding unit 166.

[0130] The motion information may vary depending on the encoding mode. For example, in skip mode or merge mode, the motion information only includes an index indicating the predictor, while in AMVP mode, the motion information may include the reference image index of the current block, the differential motion vector, and the AMVP index.

[0131] The following will be explained in detail. Figure 1 An embodiment of the operation of the intra-frame prediction unit 150 shown.

[0132] First, the intra-prediction unit 150 receives prediction mode information and the size of the prediction unit PU from the image segmentation unit 110, and reads reference pixels from the image storage unit 180 to determine the intra-prediction mode of the prediction unit PU.

[0133] The intra-prediction unit 150 determines whether to generate a reference pixel by checking whether there is an unusable reference pixel, and the reference pixel can be used to determine the intra-prediction mode of the current block.

[0134] When the current block is located at the top boundary of the current image, the pixels adjacent to the top of the current block are not defined. When the current block is located at the left boundary of the current image, the pixels adjacent to the left of the current block are not defined. It can be determined that the pixels are not available pixels.

[0135] Furthermore, if the current block is located at the slice boundary and the pixel adjacent to the slice above or to the left is not a pixel that has been reconstructed from an encoded pixel, it can also be determined that the pixel is not usable.

[0136] As mentioned above, when there are no pixels adjacent to the left or top of the current block, or when there are no pre-encoded and reconstructed pixels, the intra-prediction mode of the current block can be determined using only the available pixels.

[0137] On the one hand, the available reference pixels of the current block can be used to generate reference pixels for unavailable locations. For example, when the pixels of the upper block are unavailable, the upper pixels can be generated by using some or all of the pixels on the left, and vice versa.

[0138] That is, a reference pixel can be generated by copying the nearest available reference pixel in a predetermined direction from a reference pixel at an unavailable position, or by copying the nearest available reference pixel in the opposite direction when there is no available reference pixel in the predetermined direction.

[0139] On the one hand, if there are pixels above or to the left of the current block, they can also be determined as unusable pixels based on the encoding mode of the block to which the pixel belongs.

[0140] For example, if the reference pixel adjacent to the upper side of the current block belongs to a block that has been reconstructed by inter-frame coding, the pixel can be determined as an unusable pixel.

[0141] At this time, usable reference pixels can be generated using pixels belonging to blocks that are reconstructed by intra-frame coding adjacent to the current block. The encoding device 10 transmits the information of the usable reference pixels to the decoding device 20 according to the encoding mode.

[0142] The intra-prediction unit 150 uses the reference pixels to determine the intra-prediction mode of the current block. The number of intra-prediction modes that can be allowed in the current block can vary depending on the size of the block.

[0143] For example, when the current block size is 8×8, 16×16, or 32×32, there can be 34 intra-frame prediction modes; when the current block size is 4×4, there can be 17 intra-frame prediction modes.

[0144] The 34 or 17 intra-frame prediction modes may consist of at least one non-directional mode and multiple directional modes.

[0145] At least one non-directional mode can be a DC mode and / or a planar mode. When the DC mode and the planar mode are non-directional modes, there may be 35 intra-frame prediction modes regardless of the size of the current block.

[0146] In this case, it can include 2 non-directional modes (DC mode and planar mode) and 33 directional modes.

[0147] In planar mode, a predicted block for the current block is generated using at least one pixel value (or a predicted value of the pixel value, hereinafter referred to as the first reference value) located on the lower right side of the current block and a reference pixel.

[0148] The composition of an image decoding apparatus according to an embodiment of the present invention can be seen from reference to Figures 1 to 6 The composition of the image encoding device 10 described herein is derived, for example, by performing a reference in reverse. Figures 1 to 6 The process of the image encoding method described can decode the image.

[0149] Figure 7 This is a block diagram illustrating the structure of a video decoding apparatus according to an embodiment of the present invention. The decoding apparatus 20 includes an entropy decoding unit 210, an inverse quantization / inverse transform unit 220, an adder 270, a deblocking filter 250, an image storage unit 260, an intra-frame prediction unit 230, a motion compensation prediction unit 240, and an intra-frame / inter-frame switching switch 280.

[0150] The entropy decoding unit 210 receives the bit stream encoded on the image encoding device 10, decodes it, divides it into intra-frame prediction mode index, motion information, quantization coefficient sequence, etc., and transmits the decoded motion information to the motion compensation prediction unit 240.

[0151] The entropy decoding unit 210 transmits the intra-prediction mode index to the intra-prediction unit 230 and the inverse quantization / inverse transform unit 220, and transmits the inverse quantization coefficient sequence to the inverse quantization / inverse transform unit 220.

[0152] The inverse quantization / inverse transform unit 220 transforms the quantization coefficient sequence into a two-dimensional arrangement of inverse quantization coefficients. To perform the transformation, one of multiple scanning modes can be selected. For example, the scanning mode can be selected based on the prediction mode of the current block (i.e., intra-frame prediction or inter-frame prediction) and the intra-frame prediction mode.

[0153] The inverse quantization / inverse transform unit 220 applies a quantization matrix selected from multiple quantization matrices to reconstruct the quantization coefficients of the two-dimensional arrangement of inverse quantization coefficients.

[0154] On the one hand, different quantization matrices can be applied depending on the size of the current block to be reconstructed. On the other hand, for blocks of the same size, the quantization matrix can be selected based on at least one of the prediction mode of the current block and the intra-frame prediction mode.

[0155] The inverse quantization / inverse transform unit 220 performs an inverse transform on the reconstructed quantization coefficients to reconstruct the residual block. The inverse transform process can be performed using the transform unit TU as the basic unit.

[0156] Adder 270 adds the residual block reconstructed by inverse quantization / inverse transform unit 220 and the prediction block generated by intra-frame prediction unit 230 or motion compensation prediction unit 40 to reconstruct the image block.

[0157] The deblocking filter 250 performs deblocking filtering on the reconstructed image generated by the adder 270, which can reduce the truncation artifacts caused by image loss during quantization.

[0158] The image storage unit 260 is a frame memory used to store decoded images that have undergone truncation filtering processing by the deblocking filter 250.

[0159] The intra-prediction unit 230 reconstructs the intra-prediction mode of the current block based on the intra-prediction mode index received from the entropy decoding unit 210, and generates a prediction block based on the reconstructed intra-prediction mode.

[0160] The motion compensation prediction unit 240 generates a prediction block for the current block from the image stored in the image storage unit 260 based on motion vector information. When fractional precision motion compensation is applied, the selected interpolation filter is used to generate the prediction block.

[0161] The intra / inter-frame switching switch 280 can provide the adder 270 with a prediction block generated in either the intra-frame prediction unit 230 or the motion compensation prediction unit 240, based on the coding mode.

[0162] Figure 8 This is a block diagram illustrating an embodiment of the components used in the image decoding apparatus 20 to perform inter-frame prediction. The inter-frame prediction decoder includes: a multiplexer 241, a motion information coding mode determination unit 242, a merged mode motion information decoding unit 243, an AMVP mode motion information decoding unit 244, a prediction block generation unit 245, a residual block decoding unit 246, and a reconstruction block generation unit 247.

[0163] Reference Figure 8 The multiplexer 241 performs multiplexing from the received bit stream to the currently encoded motion information and the encoded residual signal, transmits the multiplexed motion information to the motion information encoding mode determination unit 242, and transmits the multiplexed residual signal to the residual block decoding unit 246.

[0164] The motion information encoding mode determination unit 242 determines the motion information encoding mode of the current block. When the skip_flag of the received bit stream is 1, it can be determined that the motion information encoding mode of the current block is encoded in the skip encoding mode.

[0165] The motion information encoding mode determination unit 242 can determine that the motion information encoding mode of the current block is encoded in the merge mode when the skip_flag of the received bit stream is 0 and the motion information received by the multiplexer 241 only has a merge index.

[0166] The motion information encoding mode determination unit 242 can determine that the motion information encoding mode of the current block is encoded in AMVP mode when the skip_flag of the received bit stream is 0 and the motion information received by the multiplexer 241 has a reference image index, a differential motion vector, and an AMVP index.

[0167] The merge mode motion information decoding unit 243 is activated when the motion information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is either skip mode or merge mode. The AMVP mode motion information decoding unit 244 is activated when the motion information encoding mode determination unit 242 determines that the motion information encoding mode of the current block is AMVP mode.

[0168] The prediction block generation unit 245 generates a prediction block for the current block using the motion information reconstructed by the merged mode motion information decoding unit 243 or the AMVP mode motion information decoding unit 244.

[0169] When the motion vector is in integer units, the block corresponding to the position represented by the motion vector in the image indicated by the reference image index can be copied to generate the prediction block of the current block.

[0170] On the one hand, when the motion vector is not an integer unit, the pixels of the prediction block are generated from the integer unit pixels in the image indicated by the reference image index. When it is a luminance pixel, an 8-pole interpolation filter can be used to generate the prediction pixel. When it is a chrominance pixel, a 4-tap interpolation filter can be used to generate the prediction pixel.

[0171] The residual block decoding unit 246 performs entropy decoding on the residual signal and performs inverse scanning on the coefficients after entropy decoding to generate a two-dimensional quantization coefficient block. The inverse scanning method may vary depending on the entropy decoding method.

[0172] For example, the diagonal raster inverse scanning method is used when decoding based on CABAC, and the zigzag inverse scanning method is used when decoding based on CAVLC. Furthermore, the inverse scanning method can be determined differently depending on the size of the prediction block.

[0173] The residual block decoding unit 246 performs inverse quantization on the coefficient block generated as described above using an inverse quantization matrix. To derive the quantization matrix, the quantization parameters can be reconstructed. Here, the quantization step size can be reconstructed using coding units of a predetermined size or larger.

[0174] The residual block decoding unit 246 performs inverse transformation on the inverse quantization coefficient block to reconstruct the residual block.

[0175] The reconstruction block generation unit 247 adds the prediction block generated by the prediction block generation unit 245 and the residual block generated by the residual block decoding unit 246 to generate a reconstruction block.

[0176] Next, refer to Figure 7 This describes an embodiment of the process of reconstructing the current block through intra-frame prediction.

[0177] First, the intra-prediction mode of the current block is decoded from the received bit stream. To do this, the entropy decoding unit 210 reconstructs the first intra-prediction mode index of the current block with reference to one of a plurality of intra-prediction mode tables.

[0178] The multiple intra-prediction mode tables are tables shared by the encoding device 10 and the decoding device 20, and can be used to select any table for multiple blocks adjacent to the current block according to the distribution of intra-prediction modes.

[0179] For example, if the intra prediction mode of the block to the left of the current block is the same as the intra prediction mode of the block above the current block, the first intra prediction mode index of the current block can be reconstructed using the first intra prediction mode table. If they are different, the first intra prediction mode index of the current block can be reconstructed using the second intra prediction mode table.

[0180] As another example, if the intra-prediction modes of the upper block and the left block of the current block are both directional intra-prediction modes, and the direction of the intra-prediction mode of the upper block is within a predetermined angle of the direction of the intra-prediction mode of the left block, the first intra-prediction mode index of the current block can be reconstructed using the first intra-prediction mode table. If the direction exceeds the predetermined angle, the first intra-prediction mode index of the current block can be reconstructed using the second intra-prediction mode table.

[0181] The entropy decoding unit 210 transmits the first intra-prediction mode index of the reconstructed current block to the intra-prediction unit 230.

[0182] The intra-prediction unit 230, which receives the first intra-prediction mode index, can determine the maximum possible mode of the current block as the intra-prediction mode of the current block when the index has a minimum value (i.e., 0).

[0183] On one hand, the intra-prediction unit 230 compares the index represented by the maximum possible mode of the current block with the first intra-prediction mode index when the index has a value other than 0. If the comparison result shows that the first intra-prediction mode index is greater than or equal to the index represented by the maximum possible mode of the current block, the intra-prediction mode corresponding to the second intra-prediction mode index which is the first intra-prediction mode index plus 1 is determined as the intra-prediction mode of the current block. If not, the intra-prediction mode corresponding to the first intra-prediction mode index can be determined as the intra-prediction mode of the current block.

[0184] The intra-prediction modes that are allowed for the current block can consist of at least one non-directional mode and multiple directional modes.

[0185] At least one non-directional mode may be a DC mode and / or a planar mode. Furthermore, one of the DC mode and the planar mode may be adaptively included in the set of permissible intra-prediction modes.

[0186] Therefore, the image header or slice header may include information about non-directional modes specifically included in the set of allowed intra-frame prediction modes.

[0187] Next, the intra-prediction unit 230 reads reference pixels from the image storage unit 260 in order to generate intra-prediction blocks and determines whether there are unusable reference pixels.

[0188] The determination can be performed based on the presence or absence of reference pixels used when generating an intra-prediction block using the decoded intra-prediction mode applicable to the current block.

[0189] Next, when it is necessary to generate reference pixels, the intra-frame prediction unit 230 can generate reference pixels for unavailable locations using the pre-reconstructed available reference pixels.

[0190] The definition of unavailable reference pixels and the method for generating reference pixels can be compared with... Figure 1 The operation is the same as that of the intra-prediction unit 150, but the reference pixels used when generating the intra-prediction block can be selectively reconstructed according to the decoded intra-prediction mode of the current block.

[0191] Furthermore, the intra-prediction unit 230 determines whether a filter is applicable to the reference pixel in order to generate a prediction block, that is, based on the decoded intra-prediction mode and the size of the current prediction block, it determines whether a filter is applicable to the reference pixel for the intra-prediction block used to generate the current block.

[0192] Furthermore, the larger the block size, the greater the problem of unblocking artifacts. Therefore, a larger block size can increase the number of prediction patterns that filter the reference pixel. However, when the block size is larger than a predetermined size, it will be regarded as a flat region. Therefore, in order to reduce complexity, the reference pixel may not be filtered.

[0193] If it is determined that filtering of the reference pixel is necessary, the intra-frame prediction unit 230 uses a filter to filter the reference pixel.

[0194] Depending on the degree of difference between the reference pixels, at least two filters can be adaptively applied. The filtering coefficients of the filters are relatively optimal.

[0195] In addition, at least two filters can be adaptively applied according to the size of the current block, and when applying filters, narrowband filters are applied to blocks with small sizes, and wideband filters are applied to blocks with large sizes.

[0196] In DC mode, since the prediction block is generated from the average value of the reference pixel, no filter is required. In vertical modes where the image is correlated in different vertical directions, no filter is required for the reference pixel. Similarly, in horizontal modes where the image is correlated in the horizontal direction, no filter is required for the reference pixel.

[0197] Thus, whether filtering is applicable is also related to the intra-prediction mode of the current block. Therefore, reference pixels can be adaptively filtered based on the intra-prediction mode of the current block and the size of the prediction block.

[0198] Next, the intra-prediction unit 230 generates a prediction block using reference pixels or filtered reference pixels based on the reconstructed intra-prediction mode. The generation of the prediction block can be the same as that in the coding device 10, so a detailed description of it will be omitted.

[0199] The intra-prediction unit 230 determines whether to filter the generated prediction block, and can determine whether to filter based on the information contained in the slice header or coding unit header or the intra-prediction mode of the current block.

[0200] If it is determined that the generated prediction block is to be filtered, the intra-prediction unit 230 can generate new pixels by filtering the pixels at a specific position of the prediction block generated using available reference pixels adjacent to the current block.

[0201] For example, in DC mode, reference pixels that are in contact with the predicted pixels can be used to filter the predicted pixels that are in contact with the reference pixels.

[0202] Therefore, the predicted pixel can be filtered using one or two reference pixels based on its position, and the filtering of the predicted pixel in DC mode can be applied to prediction blocks of all sizes.

[0203] On the one hand, in vertical mode, the prediction pixels in the prediction block that are in contact with the left reference pixels can be changed by using reference pixels other than the top pixel used when generating the prediction block.

[0204] Similarly, in horizontal mode, the prediction pixels in the generated prediction block that are in contact with the upper reference pixels can be changed using reference pixels other than the left-side pixels used when generating the prediction block.

[0205] The current block can be reconstructed by using the predicted block of the current block reconstructed in this way and the residual block of the decoded current block.

[0206] Figure 9 This is a diagram illustrating a second embodiment of a method for processing images by dividing them into block units.

[0207] Reference Figure 9 The coding tree unit (CTU) with a maximum size of 256×256 pixels is first divided into a quadtree structure, which can then be divided into four coding units (CUs) with a square shape.

[0208] Here, at least one of the coding units that is divided into the quadtree structure is divided into a binary tree structure, and then can be further divided into two coding units CU with a rectangular shape.

[0209] On the one hand, at least one of the coding units that is divided into the quadtree structure is divided into a quadtree structure, and then it can be further divided into four coding units CU with a square shape.

[0210] On the one hand, at least one of the coding units that is further divided into the binary tree structure is further divided into a binary tree structure, and then it can be divided into two coding units CU with square or rectangular shapes.

[0211] On the one hand, one of the coding units that is further divided into the quadtree structure is further divided into a quadtree structure or a binary tree structure, and then it can be divided into coding units CU with square or rectangular shapes.

[0212] As mentioned above, the coded blocks (CBs) constructed by partitioning into a binary tree structure can be used for prediction and transformation without further partitioning. That is, belonging to... Figure 9 The sizes of the prediction unit PU and the transform unit TU of the coded block CB shown can be the same as the size of the corresponding coded block CB.

[0213] As mentioned above, references can be used Figure 3 and Figure 4 The method described divides the coding unit, which is divided into a quadtree structure, into more than one prediction unit (PU).

[0214] Furthermore, the coding units segmented into quadtree structures as described above can utilize references Figure 5 The method described is to divide the data into one or more transformation units TU, and the divided transformation unit TU can have a maximum pixel size of 64×64.

[0215] Figure 10 An example of a syntax structure used when processing images in block units is shown.

[0216] Reference Figure 10 The `split_cu_flag` flag is used to indicate whether to split the reference. Figure 9 The coding unit (CU) is described, and binary_depth is used to represent the depth of the coding unit (CU) using binary tree partitioning.

[0217] Furthermore, the binary_split_flag can be used to indicate whether the encoding unit (CU) is split into a binary tree structure.

[0218] By reference Figure 9 and Figure 10 The method described applies to the segmentation of blocks (e.g., coding unit CU, prediction unit PU, and transform unit TU). Figures 1 to 8 The method described is used to encode and decode images.

[0219] Below, refer to Figures 11 to 16 Other embodiments of the method for dividing the coding unit CU into more than one transform unit TU are described.

[0220] According to an embodiment of the present invention, the encoding unit CU is divided into a binary tree structure, and can be further divided into transformation units TU, which are the basic units for transforming residual blocks.

[0221] Reference Figure 11 The structure is divided into a binary tree structure, and at least one of the rectangular coding blocks (CB0, CB1) of size N×2N or 2N×N is further divided into a binary tree structure, thereby dividing it into rectangular transformation units (TU0, TU1) of size N×N.

[0222] As described above, block-based image coding methods can perform prediction, transformation, quantization, and entropy coding steps.

[0223] In the prediction step, a prediction signal is generated by referring to the block currently being encoded and existing encoded images or surrounding images. This prediction signal can be used to calculate the difference signal with the current block.

[0224] Furthermore, in the transformation step, the differential signal is input and various transformation functions are used to perform the transformation. The transformed signal is divided into DC coefficients and AC coefficients, and energy compression is performed, which can improve coding efficiency.

[0225] Furthermore, in the quantization step, quantization is performed by inputting transform coefficients, and then entropy encoding is performed on the quantized signal, thereby enabling the encoding of the image.

[0226] On the one hand, the image decoding method is executed in the reverse order of the encoding process described above, and on the other hand, image quality distortion occurs during the quantization step.

[0227] To improve coding efficiency while reducing image quality distortion, the size or shape of the transform unit TU and the type of transform function can be set in various ways, depending on the distribution of the differential signal input in the transform step and the characteristics of the image.

[0228] For example, in the prediction step, when searching for blocks similar to the current block through a block-based motion estimation process, the distribution of the differential signal can have diverse forms depending on the characteristics of the image, using cost determination methods such as SAD (sum of absolute differences) or MSE (mean squared error).

[0229] Therefore, by selectively determining the size or shape of the transform unit CU based on the distribution of various differential signals, encoding can be performed effectively.

[0230] Reference Figure 12 When a differential signal occurs in a certain coded block CBx as shown in 12(a), as Figure 12 As shown in (b), the coded block CBx is divided into a binary tree structure and then into two transformation units TU, thus enabling efficient transformation.

[0231] For example, we can say that the DC value usually represents the average value of the input signal. Therefore, when receiving an input signal during the transformation process, such as... Figure 12 When the difference signal is shown in (a), the coding block CBx is divided into two transformation units TU, so that the DC value can be effectively represented.

[0232] Reference Figure 13 The rectangular coding unit (CU0) of size 2N×2N is divided into a binary tree structure, which can be further divided into rectangular transformation units (TU0, TU1) of size N×2N or 2N×N.

[0233] According to another embodiment of the present invention, as described above, the step of dividing the coding unit CU into a binary tree structure can be repeated more than twice, thereby dividing it into multiple transformation units TU.

[0234] Reference Figure 14 The rectangular coding block (CB1) of size N×2N is divided into a binary tree structure, and the divided N×N block is further divided into a binary tree structure to construct a rectangular block of size N / 2×N or N×N / 2. Then, the N / 2×N or N×N / 2 block is further divided into a binary tree structure, thereby dividing into rectangular transformation units (TU1, TU2, TU4, TU5) of size N / 2×N / 2.

[0235] Reference Figure 15 The rectangular coding block (CB0) of size 2N×2N is divided into a binary tree structure, and the divided block of size N×2N is further divided into a binary tree structure to construct a square block of size N×N. Then, the block of size N×N is further divided into a binary tree structure, thereby dividing it into rectangular transformation units (TU1, TU2) of size N / 2×N.

[0236] Reference Figure 16 The rectangular coding block (CB0) of size 2N×N is divided into a binary tree structure, and the divided block of size N×N is further divided into a quadtree structure. Thus, the block of size N / 2×N / 2 can be divided into rectangular transformation units (TU1, TU2, TU3, TU4).

[0237] By referring to Figures 11 to 16 The method described applies to the segmentation of blocks (e.g., coding unit CU, prediction unit PU, and transform unit TU) and is applicable to reference. Figures 1 to 8 The same method applies to non-zero values, thus enabling the encoding and decoding of images.

[0238] The following describes an embodiment of the method for determining the block segmentation structure according to the encoding device 10 of the present invention.

[0239] The image segmentation unit 110 provided in the image coding apparatus 10 performs rate distortion optimization (RDO) in a predetermined order, and can determine the segmentation structure of the coding unit CU, prediction unit PU and transform unit TU that can be segmented as described above.

[0240] For example, in order to determine the block segmentation structure, the image segmentation unit 110 can determine the optimal block segmentation structure in terms of bitstream and distortion while performing rate distortion optimization (RDO).

[0241] Reference Figure 17 When the encoding unit (CU) has a pixel size of 2N×2N, Figure 17(a) shows the 2N×2N pixel size. Figure 17 (b) shows the N×N pixel size, Figure 17 (c) shows the N×2N pixel size, Figure 17 (d) shows the 2N×N pixel size transform unit PU segmentation structure, which sequentially performs rate distortion optimization to determine the optimal segmentation structure of the transform unit PU.

[0242] Reference Figure 18 When the encoding unit (CU) has an N×2N or 2N×N pixel size, it is... Figure 18 (a) shows the pixel size of N×2N (or 2N×N). Figure 18 (b) shows the N×N pixel size, Figure 18 (c) shows the pixel dimensions of N / 2×N (or N×N / 2) and N×N. Figure 18 (d) shows the pixel dimensions of N / 2×N / 2, N / 2×N, and N×N. Figure 18 (e) shows the N / 2×N pixel-sized transform unit PU segmentation structure, which sequentially performs rate distortion optimization to determine the optimal segmentation structure of the transform unit PU.

[0243] Although the block segmentation method of the present invention has been illustrated above by taking the case of determining the block segmentation structure by performing rate distortion optimization (RDO), the image segmentation unit 110 determines the block segmentation structure by utilizing the sum of absolute differences (SAD) and the mean square error (MSE), thereby reducing complexity and maintaining appropriate efficiency.

[0244] According to one embodiment of the present invention, whether adaptive loop filtering (ALF) is applicable can be determined based on the coding unit CU, prediction unit PU, or transform unit TU segmented as described above.

[0245] For example, the suitability of an adaptive loop filter (ALF) can be determined at the coding unit (CU) level, and the size or coefficients of the applicable loop filter can vary depending on the coding unit (CU).

[0246] At this point, information indicating whether the Adaptive Loop Filter (ALF) is applicable to each coding unit (CU) can be included in the header of each slice.

[0247] In the case of chroma signals, the suitability of an adaptive loop filter (ALF) can be determined in image units. Unlike luminance, the loop filter can also have a rectangular shape.

[0248] Furthermore, the applicability of the adaptive loop filter (ALF) can be determined based on the slice. Therefore, information indicating whether the adaptive loop filter (ALF) is applicable to the current slice can be included in the slice header or the image header.

[0249] If it indicates that adaptive loop filtering has been applied to the current slice, the slice header or image header may additionally include information indicating the horizontal and / or vertical filter lengths of the luminance components used in the adaptive loop filtering process.

[0250] Slice headers or image headers may include information indicating the number of filter banks. When there are two or more filter banks, the filter coefficients can be encoded using prediction methods.

[0251] Therefore, slice headers or image headers can include information indicating whether filter coefficients are encoded using a prediction method, which can include predicted filter coefficients when a prediction method is used.

[0252] On the one hand, in addition to luminance, chrominance components can also be adaptively filtered. In this case, the information indicating whether each chrominance component is filtered can be included in the slice header or image header. In order to reduce the number of bits, it can be jointly encoded together with the information indicating whether Cr and Cb are filtered (i.e., multiple encoding).

[0253] At this point, in the case of chrominance components, in order to reduce complexity, it is frequently possible that neither Cr nor Cb will be filtered. Therefore, when neither Cr nor Cb is filtered, the smallest index can be assigned to perform entropy coding.

[0254] When filtering is applied to both Cr and Cb, the largest index can be assigned to perform entropy coding.

[0255] Reference Figure 19 ,like Figure 19 As shown in (a), the coding tree unit (CTU) with a maximum size of 256×256 pixels is divided into a quadtree structure, which can be divided into 4 coding units (CU) with a square shape.

[0256] In addition, the four square-shaped coding units CU can be further divided into quadtree structures, and the depth of the coding unit CU divided into quadtree structures as described above can be an integer from 0 to 3.

[0257] like Figure 19 As shown in (a), it can be determined whether the adaptive loop filter (ALF) is applicable to the segmented coding unit (CU), and the segmentation structure of the coding unit (CU), including whether the adaptive loop filter (ALF) is applicable, can be as follows: Figure 19 (b) indicates.

[0258] Here, split_cu_flag indicates whether to split the node. When split_cu_flag is 0, the corresponding node's coding unit (CU) is not split into smaller blocks. When split_cu_flag is 1, the corresponding node's coding unit (CU) is split into smaller coding units (CU).

[0259] Furthermore, ALF_on / off_flag indicates whether the adaptive loop filter is applied to the node. When ALF_on / off_flag is 0, the adaptive loop filter ALF is not applied to the coding unit (CU) of the node. When ALF_on / off_flag is 1, the adaptive loop filter ALF is applied to the coding unit (CU) of the node.

[0260] Reference Figure 20 ,like Figure 20 As shown in (a), the coding tree unit (CTU) with a maximum size of 256×256 pixels is first divided into a quadtree structure, and then can be divided into 4 coding units (CU) with a square shape.

[0261] At this point, at least one of the coding units divided into the quadtree structure is divided into a binary tree structure, and then it can be further divided into two coding units CU with rectangular shapes.

[0262] On the one hand, at least one of the coding units divided into the quadtree structure is divided into a quadtree structure, thereby it can be further divided into 4 coding units CU with a square shape.

[0263] Furthermore, at least one of the coding units that is further divided into the binary tree structure is further divided into a binary tree structure, thereby dividing it into two coding units CU with square or rectangular shapes.

[0264] On the one hand, at least one of the coding units that is further divided into the quadtree structure is further divided into a quadtree structure or a binary tree structure, thereby, it can be divided into coding units CU with square or rectangular shapes.

[0265] like Figure 20 As shown in (a), it can be determined whether the adaptive loop filter (ALF) is applied to each segmented coding unit (CU). The segmentation structure of the coding unit (CU) including whether the adaptive loop filter (ALF) is applied is as follows: Figure 20 (b) indicates.

[0266] Figure 20 In (b), the solid line connecting the nodes indicates that the node on the upper side is divided into a quadtree structure to form the node on the lower side, and the dashed line connecting the nodes indicates that the node on the upper side is divided into a binary tree structure to form the node on the lower side.

[0267] Figure 21 This is a flowchart illustrating the operation of a decoding apparatus 20 that performs intra-frame prediction according to an embodiment of the present invention.

[0268] According to another embodiment of the present invention, the image encoding apparatus 10 can encode and transmit intra-prediction modes. When the image encoding apparatus 10 encodes and transmits intra-prediction modes for encoding target blocks, in order to reduce the number of bits to be transmitted and improve encoding efficiency, a method for predicting intra-prediction modes in the aforementioned intra-prediction direction by the intra-prediction unit 150 can be used. The decoding apparatus 20 receives the signal information for intra-prediction and can perform intra-prediction decoding using prediction modes selected from intra-prediction mode candidates.

[0269] In particular, the prediction mode of the target block is highly likely to be the same as the prediction mode of its neighboring blocks. Therefore, the image coding apparatus 10 can encode the prediction mode of the target block using the prediction modes of the neighboring blocks adjacent to the target block through the intra-frame prediction unit 150. Hereinafter, the prediction mode used as the prediction value of the intra-frame prediction mode of the target block is called MPM (Most Probable Mode), and the neighboring block refers to the block adjacent to the current block.

[0270] The intra-prediction unit 150 can derive MPM candidate modes from peripheral blocks adjacent to the target block, and generate an MPM list using the derived MPM candidate modes. At this time, the intra-prediction unit 150 can assign the MPM candidate modes derived from the peripheral blocks to the MPM list. Furthermore, the intra-prediction unit 150 can encode the intra-prediction modes using the MPM list.

[0271] The image coding apparatus 10 can perform intra-frame predictive coding on the prediction modes of the target block using the MPM list. Thus, when the current coding block is encoded using the method derived from the prediction modes of surrounding blocks, the image coding apparatus 10 can send other flags to the decoder.

[0272] More specifically, the prediction mode candidates for the surrounding blocks can be constructed from the MPM list. When the prediction mode of the current block is derived using at least one prediction mode from the MPM list, the image coding device 10 uses the MPM list to determine whether to perform intra-frame prediction coding and can assign a predetermined value to the MPM flag accordingly.

[0273] Furthermore, the MPM flag can be sent to the image decoding device 20, which constructs the MPM list in the same manner as the image encoding device 10, and can perform decoding on the current block. At this time, the MPM flag can be used as a flag to determine whether the current block is encoded using the MPM list. As an embodiment, the MPM flag sent to the image decoding device 20 can be expressed using statements such as `prev_intra_luma_pred_flag` or `MPM_Flag`. For example, if the current block is a block that performs encoding in a direction that induces prediction within the MPM list, the MPM flag can be assigned 1; otherwise, it can be assigned 0.

[0274] On the other hand, when the image encoding device 10 is not using MPM mode encoding (MPM Flag == False), it performs intra-predictive mode encoding according to either the selected mode or the non-selected mode.

[0275] When performing encoding in selected mode, the image encoding device 10 performs fixed-length coding on the predicted directional modes not included in the MPM mode according to the number of directional modes, and transmits it to the image decoding device 20. At this time, the image encoding device 10 can transmit a selected mode flag to the image decoding device 20 to indicate whether to perform intra-frame predictive coding in selected mode.

[0276] Here, when the image encoding device 10 performs intra-frame prediction coding in selected mode, it can assign 1 to Selected_mode_Flag; when performing intra-frame prediction coding in non-selected mode, it can assign 0 to Selected_mode_flag. When performing intra-frame prediction coding in non-selected mode, the image encoding device 10 encodes one of the prediction directions not selected in MPM mode and selected mode using truncated binary coding and transmits it to the image decoding device 20.

[0277] Furthermore, when an MPM candidate mode that is identical to the prediction mode of the encoded object block exists in the MPM list, the encoder can transmit to the decoder an index indicating which MPM candidate mode induced the prediction mode of the encoded object block from within the MPM list. This index will be referred to as the MPM index. As an example, the MPM index can be represented by mpm_idx.

[0278] In the step of constructing the MPM list, the image encoding device 10 can remove redundant prediction modes by using a trimming process or the like. Furthermore, in the order of constructing the MPM list, the image encoding device 10 can construct the MPM list using in-frame prediction modes from the surrounding blocks of the current block and arbitrary prediction modes.

[0279] As an example of constructing the MPM list, the image coding apparatus 10 can construct the MPM list in the order of Left block, Above block, Below-Left block, Above-Right block, Above-Left block, etc. of the current block. The MPM list can be constructed by removing redundant prediction modes from surrounding blocks to fill the MPM list and then constructing DC or Planar modes. When constructed in this way, the existing / initial modes of the MPM list may include Left, Above, Planar, DC, Below-Left, Above-Right, and Above-left, and the order can be adjusted according to standards and coding efficiency.

[0280] In particular, according to embodiments of the present invention, at least one of the coding units segmented into the quadtree structure is segmented into a binary tree structure, and then further segmented into two coding units CU with a rectangular shape, or further segmented into four coding units CU with a square shape; or at least one of the coding units segmented into the binary tree structure is further segmented into a binary tree structure, thereby segmenting into two coding units CU with a square or rectangular shape. Furthermore, information about the segmentation structure can be used to construct the MPM candidate list.

[0281] At this point, by combining the quadtree and binary tree structures described above, additional processing may be required during the encoding and decoding process to improve efficiency, depending on whether MPM candidate modes are used or not.

[0282] As a detailed embodiment of in-frame predictive coding, as described above Figure 9As shown, the encoding unit used to perform encoding can be divided into square blocks and horizontally or vertically oriented rectangular blocks. When performing intra-frame prediction using horizontally / vertically oriented rectangular blocks, the direction of prediction can be determined based on the characteristics of the block, the characteristics of surrounding blocks, and the conditions under which surrounding blocks are encoded. Therefore, the image encoding apparatus 10 can adjust the composition order of the MPM list. The characteristics of the block can include its shape, depth, width, height, and size. An example of adjusting the composition order of the MPM list for the current block performing intra-frame prediction encoding based on the characteristics of the encoded surrounding blocks is as follows: When one of the surrounding blocks has already been encoded in the horizontal intra-direction, and the current block performing intra-frame prediction is a horizontally oriented rectangular block, the image encoding apparatus 10 can adjust the composition order of the MPM list, prioritizing or having the highest priority on surrounding blocks encoded in prediction directions similar to the current block in terms of characteristics and shape.

[0283] Similarly, when one of the peripheral blocks has been encoded in the vertical intra-direction, and the current block performing intra-direction prediction is a vertically oriented rectangular block, the image coding apparatus 10 can adjust the composition order of the MPM list, prioritizing or giving the highest priority to peripheral blocks encoded in the prediction direction that are similar to the characteristics and shape of the current block in the MPM list.

[0284] When adjusting the MPM list as described in one embodiment, for example, when the existing or initial MPM list under normal circumstances is {Planar, DC, 2, Hor, Ver, Diagonal}, based on the prediction mode directionality of surrounding blocks and the characteristics of the current block, prediction modes with horizontal or vertical orientations can be adjusted preferentially and with the highest priority, such as {Hor, Planar, DC, 2, Ver, Diagonal} or {Ver, Planar, DC, 2, Hor, Diagonal}. In this case, the image encoding device 10 can determine whether to make an adjustment based on the decoding device 20 and predetermined rules, or by additionally transmitting a specific signal to the decoding device 20.

[0285] More specifically, the decoding device 20 derives the prediction direction of the block currently being decoded from the intra-prediction directions of the peripheral blocks of the structure to determine the intra-prediction mode, which can be as follows: Figure 21 To carry out the work.

[0286] First, the decoding device 20 determines whether it is in MPM mode based on the MPM flag (prev_intra_luma_pred_flag or MPM_Flag) (S101).

[0287] Furthermore, in MPM mode, the decoding device 20 uses the MPM index (MPM_idx) to construct a candidate list for predicting the MPM mode (S103).

[0288] Here, the decoding device 20 can construct an MPM candidate list by utilizing the prediction patterns of surrounding blocks that can be referenced from the currently decoded block. For example, the decoding device 20 can construct a list of up to 6 MPMs based on the prediction patterns of at least 5 surrounding blocks.

[0289] Furthermore, in order to predict the block currently being decoded, the decoding device 20 analyzes the MPM mode flag and MPM index to derive the prediction direction of the current block and can perform intra-frame predictive decoding. For example, in the prediction direction, 65 modes and DC and Planar modes can be used as candidate references to encode at least 5 blocks from the current block into the surrounding left / top-left / bottom-left / top / top-right intra-frame blocks, thereby constructing the MPM list.

[0290] If the MPM list cannot be filled using the prediction modes of surrounding blocks, the decoding device 20 can apply offset values ​​such as +1, +2, +3, ... or -1, -2, -3, ... to the already filled prediction modes (Angular modes) to construct the MPM list using unique prediction modes. If the MPM list cannot be constructed using the methods described above, it can be constructed from any predefined MPM List table or a predefined MPM List set.

[0291] On the one hand, when not in MPM mode, two modes can be considered. The first is the selection mode, which can be represented by the selected mode flag. The decoding device 20 can determine the selection mode based on the value of the selected mode flag (Selected_Mode_Flag) being 0 or 1 (S105).

[0292] In non-MPM mode, to derive the prediction direction, additional patterns can be included, constructed with non-repeating directions by predetermined constraints. In this case, the index can be processed by fixed-length coding or truncated binary coding. Differentiating between the two cases for constructing the pattern set allows for separate processing using selected and non-selected modes.

[0293] In the selection mode, as described above, the decoding device 20 constructs a predetermined selection mode set and performs prediction based on it (S107).

[0294] The selection mode list can be constructed based on multiples of a predetermined natural number. Furthermore, the decoding device 20, upon receiving a selection mode index or a selection mode flag (Selected mode Flag == True), can derive a prediction direction from the set of selection modes. Additionally, when predictively encoding the selection mode index, a fixed-length 4-bit encoding can be used, for example.

[0295] In the non-selected mode (If Selected Mode Flag == False), the decoding device 20 constructs a set of non-selected modes composed of modes not included in the selected mode, and performs prediction based on this (S107).

[0296] Furthermore, the decoding device 20 receives the non-selection mode index and can derive a prediction direction from the set of non-selection modes. Also, when performing predictive encoding on the index of the selection mode, truncated binary encoding may be used, for example.

[0297] As one embodiment of the decoding method based on selected and non-selected modes, the 61 prediction modes not included in the MPM list comprise a selected mode set with 16 prediction modes and a non-selected mode set with the remaining 45 prediction modes. In this case, the selected modes can consist of 16 sets, and the initial prediction modes can consist of multiples of 4, or be composed of any offset value applied to the initial multiples of 4. The non-selected modes can be composed of prediction modes that do not overlap with the prediction modes included in the MPM list and the selected mode set, determined by the encoder and a preset method.

[0298] At this point, the in-frame prediction decoding method of the selected mode decodes the selected mode flag (Selected mode flag == true) received from the encoder and the prediction mode selected in the selected mode set using 4-bit fixed-length encoding. Thus, the current block can be decoded using the decoded prediction mode.

[0299] The decoding method for predicting within a non-selection mode frame can be executed when the value of the selection mode flag is 0 or false, and the truncated binary code received from the encoder can be decoded. A prediction mode can be decoded from the set of non-selection modes to decode the current block. Figures 22 to 24 This is a reference diagram used to illustrate how an MPM mode is constructed from a peripheral block according to an embodiment of the present invention.

[0300] Reference Figure 22 According to an embodiment of the present invention, the decoding device 20 can construct and export six MPM lists from the predictors of the prediction mode (Angular mode) of at least five blocks surrounding the current block during the encoding step.

[0301] For example, such as Figure 22 As shown, when performing inter-frame coding on peripheral blocks, the image coding device 10 can refer to the top block above the current block, the top-left block above the current block, and the top-right block above the current block, and select these as part of the MPM candidates, and determine whether to add them to the MPM list based on coding efficiency.

[0302] Furthermore, when the image encoding device 10 refers to the left-hand block (Left) and the left-down block (Left-Down) located on the left side of the current block, it can select these as part of the MPM candidates and determine whether to add them to the MPM list based on the encoding efficiency.

[0303] Furthermore, as mentioned above, such as Figure 9 When the peripheral blocks are encoded into a square or rectangular structure, the blocks are binary-segmented on the left or top, and can have more than one top block candidate or more than one left block candidate. In this case, up to 7 intra-frame prediction block candidates can be guided to the MPM list.

[0304] Reference Figure 23 , Figure 23 This diagram illustrates the angular patterns predicted for 67 intra-frame scenarios, which can be categorized as 0: Planar / 1: DC mode. Then, based on the direction, there are 65 prediction patterns (angular patterns). In this specification, the direction towards the lower left is referred to as non-zero Diagonal_2, and the direction towards the upper right is referred to as Diagonal_65.

[0305] Furthermore, during the encoding and decoding process based on the MPM prediction mode, in order to fill at least 6 MPM lists from 5 peripheral blocks, after determining the mode according to the first and second methods described later, if there is no mode to be determined from the peripheral blocks, a list can be constructed according to the third method. In this case, even if the prediction mode cannot be derived from the peripheral blocks in the case of Planar and DC, it can still be determined by the first method.

[0306] First method: When the peripheral block is intra-coded, the image coding device 10 sets the peripheral block as a reference block, and the prediction mode (angle mode) and DC / Planar mode attached to the reference block that does not repeat in the list can construct the MPM list.

[0307] The second method: If the first method cannot fill all 6 MPM lists, the image encoding device 10 can append a prediction pattern calculated from the already filled patterns using variable values ​​(addition or difference @ values, @ = 1, 2, 3, etc.) to the list. If the 6 MPM lists still cannot be filled and the repeated prediction patterns (angle patterns) do not have DC or Planar patterns in the MPM lists, the image encoding device 10 adds the Planar and DC patterns to the lists.

[0308] Third method: When the image encoding device 10 cannot construct a list of 6 MPMs using the second and third methods, it constructs a list of 6 MPMs based on vertical / horizontal / diagonal line_2 (from Left-Down towards Right-Top Conner) / Diagonal_65 (from Right-Top towards Left-Down Conner). Planar and DC already exist in the list using the first method.

[0309] Figure 24 This is an embodiment of the present invention, which uses MPM mode to encode intra-frame C in a quadtree block segmentation structure.

[0310] In the existing quadtree block partitioning, a segmented prediction unit (PU) cannot exist in a coding unit (CU). When coding in intra-frame blocks, only N×N blocks or 2N×2N block partitions can exist.

[0311] However, according to embodiments of the present invention, considering the above-described quadtree block partitioning that can be divided by a binary tree, when the terminal node determines that the prediction unit and the coding unit have the same size, it can be as follows: Figure 24 The segmentation is performed as shown. Therefore, as in the past, the intra-prediction direction cannot be referenced on all five surrounding blocks. In this case, as described above, a method for constructing the MPM list can be proposed as described above.

[0312] Figure 24 As shown, the image coding apparatus 10 can construct an MPM list for the Planar and DC modes of intra-frame C using the first method. Furthermore, since there is no reference prediction mode (angle mode) for the image coding apparatus 10, the second method can be skipped. The image coding apparatus 10 can then construct six MPM lists by constructing the remaining MPM lists in a vertical / horizontal and two diagonal order using the third method. Even so, if the MPM list is not filled according to the inherent prediction direction, as described above, prediction modes calculated from the already filled modes using variable values ​​(addition or difference @ values, @ = 1, 2, 3, etc.) can be added to the list.

[0313] Figure 25 This is a flowchart illustrating a method for constructing an MPM list according to an embodiment of the present invention.

[0314] As in Figures 22 to 24 As explained, the decoding apparatus 20 according to an embodiment of the present invention can sequentially construct an MPM list for MPM mode using the first method to the third method via the intra-frame prediction unit 230. The method for constructing the list is described step by step as follows.

[0315] First, the decoding device 20 determines whether the current block is an intra-coded block (S201). If it is not intra-coded, it performs inter-frame prediction decoding (S203).

[0316] Furthermore, when an intra-frame coded block is encountered, the decoding device 20 determines whether it is in MPM mode based on the MPM flag (S205).

[0317] When not in MPM mode, the decoding device 20 performs decoding in non-MPM mode (S207), which will be described in detail later.

[0318] Furthermore, in MPM mode, the decoding device 20 determines whether the upper end or the upper surrounding block of the current block can be used for intra-frame directional mode prediction (S209).

[0319] When available, the decoding device 20 adds the prediction mode based on the upper end and the upper peripheral block to the MPM list (S211).

[0320] If unavailable, the decoding device 20 re-determines whether the left and left-side peripheral blocks can be used. If available, the prediction mode based on the left and left-side peripheral blocks is added to the MPM list (S213).

[0321] Then, the decoding device 20 determines whether the MPM list has been filled (S215).

[0322] If the list is not full, the decoding device 20 determines whether there are any prediction patterns that can be appended to the MPM list (S217) and adds the prediction patterns that can be appended to the MPM list (S219).

[0323] If the DC or Planner prediction mode is not present, the decoding device 20 may add the DC or Planner prediction mode to the MPM list respectively (S221).

[0324] Then, the decoding device 20 can determine whether the MPM list is still not full (S223). If it is not full, it can add to the MPM list using the basic mode (default mode) (S225).

[0325] Here, the basic mode can exist as multiple basic mode sets (Default Mode Sets) composed of arbitrary prediction mode sets, depending on coding efficiency and standards. In this case, when selecting a basic mode set (Default Mode Set) to fill the MPM list, one of the multiple basic mode sets is selected according to the block structure and characteristics, or selected by other signaling signals to form the MPM list.

[0326] On the one hand, when the MPM list is full, for example, when the number of MPM lists becomes 6, the decoding device 20 ends the construction of the MPM list, analyzes its MPM index information, and decodes according to the intra-frame prediction mode of the current block.

[0327] Figure 26 and Figure 27 This is a flowchart illustrating an intra-prediction mode decoding method in a non-MPM mode according to an embodiment of the present invention.

[0328] Reference Figure 26 First, the decoding device 20 can determine multiple non-MPM modes based on whether it is an MPM mode. These multiple non-MPM modes can be any of the preset modes Non_MPM_mode_1, Non_MPM_mode_2, ..., and can be determined according to their respective flags. Here, the names Non_MPM_mode_1 and Non_MPM_mode_2 can be changed according to encoding standards, etc. As an example, such as... Figure 21 The encoding and decoding methods are as follows: Non_MPM_mode_1 is executed by the selected mode, and Non_MPM_mode_2 is executed by the non-selected mode. Intra-frame predictive encoding and decoding can be performed.

[0329] like Figure 27 As shown, the decoding device 20 enters its respective prediction mode (MPM, selection mode and non-selection mode) according to whether the MPM mode flag and the selection mode flag are true (TRUE), and can derive the prediction mode from the MPM list, the selection mode set or the non-selection mode set.

[0330] First, the decoding device 20 can identify the MPM mode by analyzing the MPM mode flag mentioned above.

[0331] Furthermore, in MPM mode, the decoding device 20 can perform decoding in MPM mode. As described above, in MPM mode, decoding can be processed through 67 intra-frame prediction modes, which may include 65 angle modes and DC / Planar mode.

[0332] On the one hand, when encoding is not performed in MPM mode, the decoding device 20 can process in non-MPM mode. In this case, a selection mode and a non-selection mode can be used for decoding through one of the 61 intra-frame modes not included in MPM mode.

[0333] At this point, when performing fixed-length encoding using the existing method, a total of 6 bits are required. Therefore, the decoding device 20 classifies the remaining 61 intra-frame modes into multiple non-MPM modes, and can construct a set of non-MPM modes according to a predetermined algorithm based on the classified non-MPM modes.

[0334] When the current decoded block is encoded using a non-MPM mode, the decoding device 20 decodes the syntax of the selected non-MPM mode type and the index of the selected non-MPM mode set to derive the predicted mode (angle mode) of the decoded block. When non-MPM mode blocks are encoded and decoded in this way, bit saving can be achieved based on the number of classified non-MPM modes and the rules for constructing each non-MPM mode set, and encoding efficiency can be improved.

[0335] As an example, such as Figure 26 As shown, the non-MPM mode can be divided into two. One of the non-MPM modes is the selected mode, which uses the prediction mode set (Angular set) as the selected mode set.

[0336] On the one hand, there is another mode, which can be called a non-selected mode. The angular set of the non-selected mode can be called the non-selected mode set. In this case, the non-selected mode can be constructed only by angular modes that are not included in the selected mode set.

[0337] For example, if the selection pattern set is constructed in multiples of 4, then a total of 16 sets such as Set A = {0, 4, 8, 12, 16, ... 60} can be constructed from the angle patterns 2 to 67 that are not included in the Planar and DC modes and the MPM list. The decoding device 20 selects one of the indices for the angle patterns selected in the encoding step.

[0338] On the one hand, when selecting a decoded block in a selection mode in non-MPM mode, the decoding device 20 can decode the angle mode. Additionally, since encoding and decoding can be performed using 4-bit fixed-length encoding, compression efficiency can be improved compared to existing methods.

[0339] Next, the decoding device 20 can construct Set B = {1, 2, 3, 5, 6, 7, ..., 59} from 45 angle modes that are not included in the MPM list (6 prediction direction modes) and the selected mode (16 prediction direction modes) in the non-selection mode set (Non-MPM mode set), and decode it by truncating binary encoding, thereby improving encoding and decoding efficiency.

[0340] Figure 28 This is a flowchart illustrating the update-based decoding method for MPM mode and non-MPM mode according to embodiments of the present invention.

[0341] Reference Figure 28 In blocks decoded via non-MPM mode, the intra angular mode is derived from a set of multiple non-MPM modes set to initial values ​​until an update. Then, the angular prediction mode of the decoded block can be derived by constructing a set of non-MPM modes with updated base settings.

[0342] Therefore, the decoding device 20 first determines whether it is an MPM mode (S301). If it is an MPM mode, it derives the prediction mode from the MPM list (S315). If it is not an MPM mode but a non-MPM mode (S303), it determines whether there is update information for the non-MPM mode set (S305).

[0343] To this end, the image encoding device 10 may include and transmit non-MPM update information in the slice header or image header. The non-MPM update information may include update information of the mode selected in the non-MPM mode or update information of predetermined rules constructed from a set.

[0344] When there is no update, the decoding device 20 can derive the prediction mode (Angularmode) from the initial non-MPM mode set (S311).

[0345] At this time, when the non-MPM mode is constructed from 2, the decoding device 20 constructs one mode as the selection mode and the other mode as the non-selection mode. Then, it constructs the initial non-MPM set by taking the selection mode as a set of multiples of 4 and the non-selection mode as a set of modes that include and do not include the selection mode.

[0346] If updated information is included, the decoding device 20 can update the information of the angular mode of the selected mode set and the non-MPM mode set (S307), and the prediction mode (Angular mode) can be derived from the updated non-MPM mode set.

[0347] For example, the decoding device 20 constructs a basic mode set, obtains additional information corresponding to multiple basic mode sets (Default mode sets) of non-MPM modes from image headers, slice headers, or tile headers, or obtains change values ​​corresponding to a predetermined set, thereby updating multiple non-MPM mode sets.

[0348] Here, when the surrounding blocks are encoded with different prediction modes, there is a possibility of reduced compression efficiency when the MPM list is filled with the basic value (Default) of the MPM mode. Therefore, the image encoding apparatus 10 can effectively derive the angle of the non-MPM mode by transmitting as a differential signal value an index of the default set used to select a default set of non-MPM modes and an additional prediction mode (angle mode) of the directional prediction mode corresponding to the index.

[0349] For example, if the basic mode set is constructed as Default mode set = {2, 18, 34, 50, 60}, then the image encoding device 10 can transmit only the differential signals {1, 0, 3, 3, 2} corresponding to the indices. Therefore, after constructing the basic mode set, the decoding device 20 receives the differential signals to obtain an updated mode set {3, 18, 37, 52, 62}.

[0350] Furthermore, when using multiple mode lists for the non-MPM modes, the decoding device 20 can update the current GOP by selecting a non-MPM mode with a predetermined frequency or higher using the preceding GOP, based on the GOP unit.

[0351] Furthermore, the decoding device 20 performs decoding on the current intra-frame block based on the prediction mode selected through step S311, step S309, or step S315 (S317).

[0352] The method of the present invention as described above is made into a computer-executable program, which is stored on a computer-readable recording medium. Examples of recording media include read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also include those embodied in the form of a carrier wave (e.g., propagated via the Internet).

[0353] Computer-readable recording media are distributed across network-connected computer systems, thus computer-readable code is stored and executed in a distributed manner. The functional programs, codes, and coded segments used to implement the method can be readily deduced by programmers in the art to which this invention pertains.

[0354] The preferred embodiments of the present invention have been shown and described above, but the present invention is not limited to the specific embodiments described. Those skilled in the art to which this invention pertains may make various modifications to the present invention without departing from the spirit of the invention as required by the claims. Such modifications should not be understood separately from the technical concept or prospect of the present invention.

Claims

1. An image processing method, comprising the following steps: The image is segmented into multiple coding tree units; At least one coding unit is obtained from the coding tree units among the plurality of coding tree units; and Based on multiple flags including a first flag and a second flag, a list of prediction modes is determined by using the intra prediction modes of at least one peripheral block adjacent to the coding unit, for deriving the intra prediction modes of the coding unit from at least one coding unit in the coding unit. The first flag corresponds to the most likely pattern, and the second flag corresponds to the selection pattern. The first flag indicates whether a first intra-prediction mode is determined from a list of multiple intra-prediction modes in the prediction mode list as the prediction mode for the coding unit. The second flag indicates whether a second prediction mode, which is different from the plurality of intra-frame prediction modes in the prediction mode list, is determined as the prediction mode of the coding unit, and, When the first flag indicates that the first intra-prediction mode is not determined as the prediction mode of the coding unit from the plurality of intra-prediction modes in the prediction mode list, based on the information obtained from the bitstream for deriving the third intra-prediction mode, the third prediction mode, which is different from the plurality of intra-prediction modes in the prediction mode list, and the second prediction mode are determined as the prediction mode of the coding unit.

2. An image processing method, comprising the following steps: The image is segmented into multiple coding tree units; Determine that at least one coding unit is obtained from the coding tree units among the plurality of coding tree units; as well as Based on whether a prediction mode list for deriving the intra prediction mode of a coding unit is determined from at least one of the coding units using the intra prediction modes of at least one peripheral block adjacent to the coding unit, a plurality of flags, including a first flag and a second flag, are determined. The first flag corresponds to the most likely pattern, and the second flag corresponds to the selection pattern. The first flag indicates whether a first intra-prediction mode is determined from a list of multiple intra-prediction modes in the prediction mode list as the prediction mode for the coding unit. The second flag indicates whether a second prediction mode, which is different from the plurality of intra-frame prediction modes in the prediction mode list, is determined as the prediction mode of the coding unit, and, When the prediction mode of the coding unit is a third prediction mode and a second prediction mode that are different from the plurality of intra-prediction modes in the prediction mode list, based on the information obtained from the bitstream for deriving the third intra-prediction mode, the first flag indicates that the first intra-prediction mode is not determined from the plurality of intra-prediction modes in the prediction mode list as the prediction mode of the coding unit.

3. A computer-readable recording medium storing a bitstream encoded by an image processing method executed by an image processing apparatus, the image processing method comprising the following steps: The image is segmented into multiple coding tree units; Determine that at least one coding unit is obtained from the coding tree units among the plurality of coding tree units; as well as Based on whether a prediction mode list for deriving the intra prediction mode of a coding unit is determined from at least one of the coding units using the intra prediction modes of at least one peripheral block adjacent to the coding unit, a plurality of flags, including a first flag and a second flag, are determined. The first flag corresponds to the most likely pattern, and the second flag corresponds to the selection pattern. The first flag indicates whether a first intra-prediction mode is determined from a list of multiple intra-prediction modes in the prediction mode list as the prediction mode for the coding unit. The second flag indicates whether a second prediction mode, which is different from the plurality of intra-frame prediction modes in the prediction mode list, is determined as the prediction mode of the coding unit, and, When the prediction mode of the coding unit is a third prediction mode and a second prediction mode that are different from the plurality of intra-prediction modes in the prediction mode list, based on the information obtained from the bitstream for deriving the third intra-prediction mode, the first flag indicates that the first intra-prediction mode is not determined from the plurality of intra-prediction modes in the prediction mode list as the prediction mode of the coding unit.

Images