Image decoding method and image encoding method

Abstract

An image decoding method and an image encoding method are provided. According to an embodiment of the present invention, a method for decoding an image comprises the following steps: identifying a parallel motion information prediction unit index of a current block whose motion information is to be decoded; obtaining motion information of at least one neighboring block of the current block that is not a block belonging to a previously indexed parallel motion information prediction unit; and processing motion information prediction decoding for the current block based on the obtained motion information.

Description

[0001] This application is a divisional application of an invention patent application with an application date of February 27, 2018, application number "201880023458.5", and title "Image processing method for processing motion information for parallel processing, method for decoding and encoding using the image processing method, and device for the method". Technical Field

[0002] The present invention relates to an image processing method, a method for decoding and encoding an image using the image processing method, and a device used for the method. More specifically, the present invention relates to an image processing method for processing motion information for parallel processing, a method for decoding and encoding an image using the image processing method, and a device used for the method. Background Art

[0003] Digital video technology can be used in a holistic manner for a wide variety of digital video devices, including, for example, digital televisions, digital live broadcast systems, wireless broadcast systems, personal digital assistants (PDAs), laptop or desktop computers, digital cameras, digital recording devices, video game devices, video game consoles, mobile phones, satellite radios, etc. Digital video devices can implement video compression technology (such as MPEG-2, MPEG-4, or ITU-T H.264 / MPEG-4, Part 10, Advanced Video Coding (AVC), H.265 / High Efficiency Video Coding (HEVC)) to more efficiently send and receive digital video information. Video compression technology performs spatial prediction and temporal prediction to eliminate or reduce redundancy inherent in video sequences.

[0004] As such image compression techniques, there are various techniques, such as an inter-frame prediction technique for predicting pixel values ​​included in a current picture from a previous picture or a subsequent picture of the current picture, an intra-frame prediction technique for predicting pixel values ​​included in the current picture using pixel information in the current picture, and an entropy coding technique for assigning short codes to values ​​with a high frequency of occurrence and assigning long codes to values ​​with a low frequency of occurrence, and image data can be effectively compressed using these image compression techniques and can be transmitted or stored.

[0005] In order to economically and efficiently cope with various resolutions, frame rates, etc. according to these applications, it is necessary to have a video decoding device that can be easily controlled according to the performance and functions required in the application.

[0006] For example, in an image compression method, a picture is partitioned into a plurality of blocks each having a predetermined size to perform encoding. In addition, an inter-frame prediction technique and an intra-frame prediction technique for eliminating redundancy between pictures are used to improve compression efficiency.

[0007] In this case, a residual signal is generated by using intra prediction and inter prediction. The reason for obtaining the residual signal is that when encoding is performed using the residual signal, the amount of data is small, making the data compression rate higher, and the better the prediction, the smaller the value of the residual signal.

[0008] The intra prediction method predicts the data of the current block by using the pixels around the current block. The difference between the actual value and the predicted value is called a residual signal block. In the case of HEVC, since the number of prediction modes increases from 9 prediction modes used in the existing H.264 / AVC to 35 prediction modes, the intra prediction method is performed while being further divided.

[0009] In the case of the inter-frame prediction method, the current block is compared with the blocks in the adjacent picture to find the most similar block. Here, the position information (Vx, Vy) of the found block is called a motion vector. The difference in pixel values ​​between the current block and the prediction block predicted by the motion vector is called a residual signal block (motion compensation residual block).

[0010] In this way, although the intra prediction and inter prediction are further divided so that the data amount of the residual signal is reduced, the amount of calculation for processing the video is greatly increased.

[0011] Specifically, the increased complexity of the process of determining a partition structure in a picture for image encoding and decoding and the existing block partitioning method result in difficulties in pipeline execution, and the existing block partitioning method and the size of the blocks resulting from the partitioning operation may not be suitable for encoding high-resolution images.

[0012] For example, in the parallel processing of motion information for inter-frame prediction, since the current block can be processed only after the encoding or decoding of the left neighboring block, the upper neighboring block, and the upper-left neighboring block are completed, when implementing a parallel pipeline, it is necessary to wait until the block mode and partition size of the neighboring block processed in another pipeline for the current block are determined, thereby causing a problem of pipeline stagnation.

[0013] To solve this problem, a merging method for merging motion information of some prediction units within an arbitrary block size has been proposed in HEVC. However, when calculating the motion information of the first prediction unit in an arbitrary block, the same problem of requiring motion information of neighboring blocks arises, resulting in a significant reduction in coding efficiency. Summary of the invention

[0014] Technical issues

[0015] The present invention has been given to solve the above-mentioned problems, and an object of the present invention is to provide a method and apparatus for decoding and encoding an image, wherein the method and apparatus improve encoding efficiency by parallel processing of motion information prediction decoding and encoding of a high-resolution image.

[0016] Solution

[0017] In order to solve the above-mentioned objectives, a method for decoding motion information according to an embodiment of the present invention includes: identifying a parallel motion information prediction unit index of a current block whose motion information is to be decoded; obtaining motion information of at least one neighboring block among the remaining blocks excluding blocks belonging to a previously indexed parallel motion information prediction unit from neighboring blocks of the current block; and processing motion information prediction decoding for the current block based on the obtained motion information.

[0018] In order to solve the above-mentioned objectives, a method for processing motion information according to an embodiment of the present invention includes: identifying a parallel motion information prediction unit index of a current block whose motion information will be encoded; obtaining motion information of at least one neighboring block among the remaining blocks excluding blocks belonging to a previously indexed parallel motion information prediction unit from neighboring blocks of the current block; and processing motion information prediction encoding for the current block based on the obtained motion information.

[0019] Furthermore, in order to solve the above-mentioned object, the method according to the embodiment of the present invention can be implemented as a program for executing the method on a computer and a non-volatile recording medium storing the program and readable by a computer.

[0020] Beneficial Effects

[0021] According to an embodiment of the present invention, decoded blocks may be sequentially grouped into predetermined parallel motion information prediction units.

[0022] According to an embodiment of the present invention, in motion information decoding, motion information decoding may be performed using motion information of remaining blocks excluding blocks belonging to a previous parallel motion information prediction unit from neighboring blocks of a current block.

[0023] Therefore, pipeline processing for each parallel motion information prediction unit can be independently performed, and pipeline stalling can be prevented in advance, thereby improving encoding and decoding efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a block diagram showing the configuration of an image encoding device according to an embodiment of the present invention.

[0025] Figures 2 to 5 is a diagram illustrating a first embodiment of a method of partitioning an image into block units and processing the image.

[0026] Figure 6 is a block diagram illustrating an embodiment of a method of performing inter-frame prediction in an image encoding apparatus.

[0027] Figure 7 is a block diagram showing the configuration of an image decoding apparatus according to an embodiment of the present invention.

[0028] Figure 8 is a block diagram illustrating an embodiment of a method of performing inter-frame prediction in an image decoding apparatus.

[0029] Fig. 9 is a diagram illustrating a second embodiment of a method of partitioning an image into block units and processing the image.

[0030] Fig.10 is a diagram illustrating a third embodiment of a method of partitioning an image into block units and processing the image.

[0031] Fig.11 is a diagram illustrating an embodiment of a method of partitioning a coding unit to construct a transformation unit using a binary tree structure.

[0032] Fig.12 is a diagram illustrating a fourth embodiment of a method of partitioning an image into block units and processing the image.

[0033] Figures 13 to 14 is a diagram illustrating still another embodiment of a method of partitioning an image into block units and processing the image.

[0034] Fig.15 and Fig.16 is a diagram illustrating an embodiment of a method of determining a partition structure of a transform unit by performing rate-distortion optimization (RDO).

[0035] Fig.17 and Fig.18 is a flowchart illustrating an image processing method for processing motion information for parallel processing according to an embodiment of the present invention.

[0036] Figures 19 to 22 : are diagrams illustrating a motion information processing method according to an embodiment of the present invention for each case. DETAILED DESCRIPTION

[0037] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description of embodiments of the present invention, detailed descriptions of known functions and configurations included herein will be omitted when they may make the subject matter of the present disclosure rather unclear.

[0038] It will be understood that when an element is referred to as being "connected" or "coupled" to another element, the element may be directly connected or directly coupled to the other element, but there are other elements between the two elements. In addition, the description of "comprising" a specific configuration in the present invention does not exclude configurations other than the configuration, but means that additional configurations may be included within the practical scope of the present invention or the technical scope of the present invention.

[0039] The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are only used for the purpose of distinguishing one component from another component. For example, without departing from the scope of the present invention, a first component may be referred to as a second component, and similarly, a second component may also be referred to as a first component.

[0040] In addition, the constituent units shown in the embodiments of the present invention are shown separately to present different characteristic functions, and each component does not mean that each constituent unit is configured with a separate hardware or software constituent unit. That is, for ease of explanation, each constituent unit is included in the list, and at least two constituent units in the constituent unit can be combined to form a constituent unit, or a constituent unit can be divided into multiple constituent units to perform each function. The integrated embodiment and the separated embodiment of such constituent units are also included in the scope of the present invention, unless they depart from the essence of the present invention.

[0041] In addition, some components are not essential components for performing essential functions, but are optional components only for improving performance in the present invention. The present invention can be implemented only with components necessary for realizing the essence of the present invention except for components for improving performance, and structures including essential components except for optional components for improving performance are also included in the scope of the present invention.

[0042] Figure 1 1 is a block diagram showing the configuration of an image encoding device according to an embodiment of the present invention. The image encoding device 10 includes a picture partition module 110, a transform module 120, a quantization module 130, a scanning module 131, an entropy encoding module 140, an intra-frame prediction module 150, an inter-frame prediction module 160, an inverse quantization module 135, an inverse transform module 125, a post-processing module 170, a picture storage module 180, a subtractor 190, and an adder 195.

[0043] Reference Figure 1 , the picture partition module 110 analyzes the input video signal and partitions the picture into coding units to determine a prediction mode and determine a prediction unit size for each coding unit.

[0044] In addition, the picture partition module 110 transmits the prediction unit to be encoded to the intra prediction module 150 or the inter prediction module 160 according to the prediction mode (or prediction method). In addition, the picture partition module 110 transmits the prediction unit to be encoded to the subtractor 190.

[0045] Here, a picture of an image consists of a plurality of slices, and the slice may be partitioned into a plurality of coding tree units (CTUs) which are basic units of picture partition.

[0046] The coding tree unit may be partitioned into one or at least two coding units (CUs) which are basic units of inter prediction or intra prediction.

[0047] Here, the maximum sizes of the coding tree unit and the coding unit may be different from each other, and signaling information about the maximum size may be transmitted to the decoding device 20. This will be referred to later. Fig.17 is described in more detail.

[0048] A coding unit (CU) may be partitioned into one or at least two prediction units (PUs) which are basic units of prediction.

[0049] In this case, the encoding device 10 determines any one of inter prediction and intra prediction as a prediction method as a prediction method for each coding unit (CU) obtained from the partitioning operation, but for each prediction unit (CU), prediction blocks may be generated differently from each other.

[0050] In addition, a coding unit (CU) may be partitioned into one or two or more transform units (TUs), wherein the transform unit (TU) is a basic unit for performing transform on a residual block.

[0051] In this case, the screen partition module 110 may transmit the image data to the subtractor 190 in units of blocks (eg, prediction units (PUs) or transform units (TUs)) resulting from the partitioning operation as described above.

[0052] Reference Figure 2 , a coding tree unit (CTU) having a maximum size of 256×256 pixels is partitioned into four coding units (CUs) each having a square shape using a quadtree structure.

[0053] Each of the four coding units (CUs) having a square shape may be further partitioned using a quadtree structure. The depth of a coding unit (CU) has any integer from 0 to 3.

[0054] A coding unit (CU) may be partitioned into one or at least two prediction units (PUs) according to a prediction mode.

[0055] In the case of intra prediction mode, when the size of the coding unit (CU) is 2N×2N, the prediction unit (PU) has Figure 3 The size of 2N × 2N shown in (a) or Figure 3 The size of N×N is shown in (b).

[0056] In addition, in the case of inter prediction mode, when the size of the coding unit (CU) is 2N×2N, the prediction unit (PU) has any one of the following sizes: Figure 4 2N×2N shown in (a), Figure 4 (b) shows 2N×N, Figure 4 (c) shows N×2N, Figure 4 (d) shows N×N, Figure 4 2N×nU shown in (e), Figure 4 2N×nD shown in (f), Figure 4 nL×2N shown in (g) and Figure 4 nR×2N shown in (h).

[0057] Reference Figure 5 , a coding unit (CU) may be partitioned into four transform units (TUs) each having a square shape using a quadtree structure.

[0058] Each of the four transform units (TUs) having a square shape may be further partitioned using a quadtree structure. The depth of the transform unit (TU) resulting from the quadtree partitioning operation may have any integer value from 0 to 3.

[0059] Here, when a coding unit (CU) is in an inter prediction mode, a prediction unit (PU) and a transformation unit (TU) obtained by partitioning the corresponding coding unit (CU) may have partition structures independent of each other.

[0060] When a coding unit (CU) is in intra prediction mode, the size of a transform unit (TU) obtained by partitioning the coding unit (CU) may not be larger than the size of a prediction unit (PU).

[0061] Furthermore, a transform unit (TU) resulting from the partitioning operation as described above may have a maximum size of 64×64 pixels.

[0062] The transform module 120 transforms a residual block, wherein the residual block is a residual signal between an original block of an input prediction unit (PU) and a prediction block generated by the intra prediction module 150 or the inter prediction module 160, wherein the transform can be performed in the transform module 120 by using a transform unit (TU) as a basic unit.

[0063] In the transform process, different transform matrices may be determined according to a prediction mode (intra or inter), and an intra-predicted residual signal has directionality according to the intra-prediction mode, so that the transform matrix may be adaptively determined according to the intra-prediction mode.

[0064] The basic unit of transformation may be transformed by two (horizontal and vertical) one-dimensional transformation matrices. For example, in the case of inter-frame prediction, a predetermined transformation matrix may be determined.

[0065] In addition, in the case of intra prediction, when the intra prediction mode is horizontal, the probability that the residual block has directionality in the vertical direction is high. Therefore, a DCT-based integer matrix is ​​applied in the vertical direction, and a DST-based or KLT-based integer matrix is ​​applied in the horizontal direction. When the intra prediction mode is vertical, a DST-based or KLT-based integer matrix is ​​applied in the vertical direction, and a DCT-based integer matrix is ​​applied in the horizontal direction.

[0066] Furthermore, in case of DC mode, a DCT-based integer matrix is ​​applied in both directions.

[0067] In case of intra prediction, a transform matrix may be adaptively determined based on the size of a transform unit (TU).

[0068] The quantization module 130 determines a quantization step size for quantizing coefficients of the residual block transformed by the transformation matrix, wherein the quantization step size may be determined for each quantization unit having a predetermined size or more in the quantization module 130 .

[0069] The size of the quantization unit may be 8×8 or 16×16, and the quantization module 130 quantizes coefficients of the transform block using a quantization matrix determined according to a quantization step size and a prediction mode.

[0070] In addition, the quantization module 130 may use the quantization step size of a quantization unit adjacent to the current quantization unit as a quantization step size predictor of the current quantization unit.

[0071] The quantization module 130 searches for one or two valid quantization steps in the order of the left quantization unit, the upper quantization unit, and the upper-left quantization unit of the current quantization unit, and generates a quantization step prediction factor of the current quantization unit using the valid quantization step.

[0072] For example, the quantization module 130 determines the first valid quantization step obtained by searching in the above order as the quantization step prediction factor, or determines the average of two valid quantization step sizes obtained by searching in the above order as the quantization step prediction factor, or when only one quantization step size is valid, the quantization module 130 determines the quantization step size as the quantization step prediction factor.

[0073] When the quantization step size predictor is determined, the quantization module 130 sends a difference between the quantization step size of the current quantization unit and the quantization step size predictor to the entropy encoding module 140 .

[0074] In addition, there are no left coding units, upper coding units, and upper-left coding units of the current coding unit, or there may be coding units that previously exist in the encoding order within the maximum coding unit.

[0075] Therefore, a quantization unit adjacent to a current coding unit in a maximum coding unit and a quantization step size of an immediately preceding quantization unit in an encoding order may be candidates.

[0076] In this case, the priority may be set in the following order: 1) the left quantization unit of the current coding unit, 2) the upper quantization unit of the current coding unit, 3) the upper left quantization unit of the current coding unit, and 4) the quantization unit immediately preceding in the coding order. The order may be changed, and the upper left quantization unit may be omitted.

[0077] In addition, the quantized transform block is sent to the inverse quantization module 135 and the scanning module 131.

[0078] The scanning module 131 scans the coefficients of the quantized transform block and transforms the coefficients into one-dimensional quantized coefficients. In this case, since the coefficient distribution of the transform block after quantization may depend on the intra prediction mode, the scanning method may be determined according to the intra prediction mode.

[0079] In addition, the coefficient scanning method may be determined according to the size of the basic unit of transformation, and the scanning mode may be changed according to the directional intra prediction mode. In this case, the scanning order of the quantized coefficients may be scanned in the opposite direction.

[0080] When the quantized coefficients are divided into a plurality of subsets, the same scanning pattern may be applied to the quantized coefficients in each subset, and zigzag scanning or diagonal scanning may be applied to the scanning pattern between the subsets.

[0081] In addition, it is best to apply the scanning pattern in a forward direction starting from the main subset including DC to the remaining subsets, but the scanning pattern can also be applied in the reverse direction thereof.

[0082] Furthermore, a scan pattern between subsets may be set in the same manner as a scan pattern of quantized coefficients in a subset, and the scan pattern between subsets may be determined according to an intra prediction mode.

[0083] In addition, the encoding device 10 is configured so that information indicating the position of the last non-zero quantized coefficient in the transform unit (PU) and the position of the last non-zero quantized coefficient in each subset is included in the bitstream and transmitted to the decoding device 20 .

[0084] The inverse quantization module 135 performs inverse quantization on the quantized coefficients as described above, and the inverse transform module 125 performs inverse transform on a per transform unit (TU) basis to reconstruct the transform coefficients obtained from the inverse quantization operation into a residual block in the spatial domain.

[0085] The adder 195 may generate a reconstructed block by adding the residual block reconstructed by the inverse transform module 125 and the prediction block received from the intra prediction module 150 or the inter prediction module 160 .

[0086] In addition, the post-processing module 170 performs a deblocking filtering process for eliminating a blocking effect generated in a reconstructed picture, a sample adaptive offset (SAO) application process for compensating for a difference relative to an original image on a per-pixel basis, and an adaptive loop filtering (ALF) process for compensating for a difference relative to an original image on a per-coding unit basis.

[0087] The deblocking filtering process may be applied to a boundary of a prediction unit (PU) or a transform unit (TU) having a predetermined size or more.

[0088] For example, the deblocking filtering process may include determining a boundary to be filtered, determining a boundary filtering strength to be applied to the boundary, determining whether a deblocking filter is applied, and selecting a filter to be applied to the boundary when it is determined to apply a deblocking filter.

[0089] In addition, whether the deblocking filter is determined depends on the following factors: i) whether the boundary filtering strength is greater than 0, and ii) whether the value indicating the degree of change of the pixel value at the boundary of two blocks (P block and Q block) adjacent to the boundary to be filtered is less than the first reference value determined by the quantization parameter.

[0090] Preferably, at least two filters are used.When the absolute difference between two pixels located at the block boundary is greater than or equal to a second reference value, a filter that performs relatively weak filtering is selected.

[0091] The second reference value is determined by the quantization parameter and the boundary filter strength.

[0092] The sample adaptive offset (SAO) application process is to reduce distortion between pixels in an image to which a deblocking filter is applied and original pixels. Whether to perform the sample adaptive offset (SAO) application process may be determined on a per-picture or per-slice basis.

[0093] A picture or a slice may be partitioned into a plurality of offset regions, and an offset type may be determined for each offset region. The offset types include a predetermined number (eg, four) of edge offset types and two band offset types.

[0094] For example, when the offset type is an edge offset type, the edge type to which each pixel belongs is determined so that a corresponding offset is applied. The edge type is determined based on the distribution of values ​​of two pixels adjacent to the current pixel.

[0095] In an adaptive loop filtering (ALF) process, filtering may be performed based on a value obtained by comparing an image reconstructed through a deblocking filtering process or a subsequent adaptive offset application process with an original image.

[0096] The picture storage module 180 receives the post-processed image data from the post-processing module 170, and reconstructs and stores the image based on each picture. The picture can be an image based on each frame or an image based on each field.

[0097] The inter prediction module 160 may perform motion estimation using at least one reference picture stored in the picture storage module 180, and may determine a motion vector and a reference picture index indicating the reference picture.

[0098] In this case, a prediction block corresponding to a prediction unit to be encoded is selected from reference pictures for motion estimation among a plurality of reference pictures stored in the picture storage module 180 according to the determined reference picture index and the motion vector.

[0099] The intra prediction module 150 may perform intra prediction encoding using reconstructed pixel values ​​in a picture including a current prediction unit.

[0100] The intra prediction module 150 receives a current prediction unit to be prediction-encoded, and performs intra prediction by selecting one of a predetermined number of intra prediction modes according to the size of the current block.

[0101] The intra prediction module 150 may adaptively filter reference pixels to generate an intra prediction block, and generate reference pixels using available reference pixels when reference pixels are unavailable.

[0102] The entropy encoding module 140 may perform entropy encoding on the quantization coefficient quantized by the quantization module 130 , the intra prediction information received from the intra prediction module 150 , the motion information received from the inter prediction module 160 , and the like.

[0103] Figure 6 is a block diagram of an embodiment of a configuration for performing inter-frame prediction in the encoding device 10. Figure 6The inter-frame prediction encoder shown in includes a motion information determination module 161, a motion information encoding mode determination module 162, a motion information encoding module 163, a prediction block generation module 164, a residual block generation module 165, a residual block encoding module 166, and a multiplexer 167.

[0104] Reference Figure 6 The motion information determination module 161 determines the motion information of the current block, wherein the motion information includes a reference picture index and a motion vector, and the reference picture index represents any one of the previously encoded and reconstructed pictures.

[0105] It may include a reference picture index indicating any one of the reference pictures belonging to list 0 (L0) when the current block is unidirectionally inter-frame prediction encoded, and indicating a reference picture index of one of the reference pictures in list 0 (L0) when the current block is bidirectionally prediction encoded, and indicating a reference picture index of one of the reference pictures in list 1 (L1).

[0106] Furthermore, when the current block is bidirectionally predictively encoded, an index indicating one or two pictures of reference pictures of a synthetic list LC generated by combining list 0 and list 1 may be included.

[0107] The motion vector indicates the position of the prediction block in the picture indicated by each reference picture index, and the motion vector may be a pixel unit (integer unit) or a sub-pixel unit.

[0108] For example, the motion vector may have a resolution of 1 / 2, 1 / 4, 1 / 8, or 1 / 16 pixels. When the motion vector is not an integer unit, a prediction block may be generated from integer unit pixels.

[0109] The motion information encoding mode determination module 162 may determine the encoding mode for the motion information of the current block to be any one of the skip mode, the merge mode, and the AMVP mode.

[0110] The skip mode is applied when there is a skip candidate having the same motion information as that of the current block and the residual signal is 0. The skip mode is applied when the current block as a prediction unit (PU) is the same size as a coding unit.

[0111] The merge mode is applied when there is a merge candidate with the same motion information as the current block. The merge mode is applied when the current block is different from the coding unit (CU) size, or when there is a residual signal when the current block is the same as the coding unit (CU) size. In addition, the merge candidate and the skip candidate can be the same.

[0112] The AMVP mode is applied when the skip mode and the merge mode are not applied, and an AMVP candidate having a motion vector most similar to a motion vector of the current block may be selected as an AMVP predictor.

[0113] The motion information encoding module 163 may encode the motion information according to the method determined by the motion information encoding mode determination module 162 .

[0114] For example, when the motion information encoding mode is the skip mode or the merge mode, the motion information encoding module 163 performs a merge motion vector encoding process, and when the motion information encoding mode is the AMVP mode, the motion information encoding module 163 performs an AMVP encoding process.

[0115] The prediction block generation module 164 generates a prediction block using motion information of the current block, and generates the prediction block of the current block by copying a block corresponding to a position indicated by the motion vector in a picture indicated by a reference picture index when the motion vector is an integer unit.

[0116] In addition, when the motion vector is not an integer unit, the prediction block generation module 164 may generate pixels of the prediction block from pixels of an integer unit in a picture indicated by the reference picture index.

[0117] In this case, an 8-tap interpolation filter is used for luma pixels to generate predicted pixels, and a 4-tap interpolation filter is used for chroma pixels to generate predicted pixels.

[0118] The residual block generation module 165 generates a residual block using the current block and a prediction block of the current block. When the size of the current block is 2N×2N, the residual block generation module 165 generates a residual block using the current block and a prediction block having a size of 2N×2N corresponding to the current block.

[0119] In addition, when the size of the current block used for prediction is 2N×N or N×2N, a prediction block of each of two 2N×N blocks constituting 2N×2N is obtained, and then the two 2N×N prediction blocks can be used to generate a result prediction block of a size of 2N×2N.

[0120] Furthermore, a residual block of size 2N×2N may be generated using a prediction block of size 2N×2N. In order to resolve the discontinuity of the boundary between two prediction blocks of size 2N×N, overlap smoothing may be applied to pixels at the boundary.

[0121] The residual block encoding module 166 divides the residual block into one or at least two transform units (TUs), and each transform unit (TU) may be transform-encoded, quantized, and entropy-encoded.

[0122] The residual block encoding module 166 may transform the residual block generated by the inter prediction method using an integer-based transform matrix, and the transform matrix may be an integer-based DCT matrix.

[0123] In addition, the residual block encoding module 166 uses a quantization matrix to quantize coefficients of the residual block transformed by the transformation matrix, and the quantization matrix may be determined by a quantization parameter.

[0124] A quantization parameter is determined for each coding unit (CU) having a predetermined size or larger. When a current coding unit (CU) is smaller than the predetermined size, only the quantization parameter of the first coding unit (CU) in the coding order among the coding units having the predetermined size or smaller is encoded, and the quantization parameters of the remaining coding units (CU) are the same as the quantization parameter of the first coding unit (CU) and are therefore not encoded.

[0125] Also, coefficients of a transform block may be encoded using a quantization matrix determined according to a quantization parameter and a prediction mode.

[0126] The quantization parameter determined for each coding unit (CU) having the predetermined size or more may be prediction-encoded using quantization parameters of coding units (CU) adjacent to the current coding unit (CU).

[0127] A quantization parameter predictor of a current coding unit (CU) is generated by searching for one or two valid quantization parameters in the order of a left coding unit (CU) and an upper coding unit (CU) of the current coding unit (CU).

[0128] For example, the first effective quantization parameter obtained by searching in the above order can be determined as the quantization parameter predictor. The search is performed in the order of the left coding unit (CU) and the immediately preceding coding unit (CU) in the coding order, so that the first effective quantization parameter is determined as the quantization parameter predictor.

[0129] The quantized coefficients of the transform block are scanned and transformed into one-dimensional quantized coefficients, and the scanning method may be differently set according to the entropy encoding mode.

[0130] For example, quantized coefficients subjected to inter-frame prediction encoding may be scanned in a predetermined manner (raster scanning in a zigzag or diagonal direction) when encoded according to CABAC, and scanned in a manner different from the predetermined manner when encoded according to CAVLC.

[0131] For example, a scanning method may be determined according to a zigzag pattern in the case of inter, a scanning method may be determined according to an intra prediction mode in the case of intra, and a coefficient scanning method may be determined differently according to a size of a basic unit of transformation.

[0132] In addition, the scanning mode may be changed according to the directional intra prediction mode, and the scanning order of the quantization coefficients may be scanned in the opposite direction.

[0133] The multiplexer 167 multiplexes the motion information encoded by the motion information encoding module 163 and the residual signal encoded by the residual block encoding module 166 .

[0134] The motion information may differ depending on the encoding mode, and for example, in the case of skip or merge, the motion information may include only an index indicating a predictor, and in the case of AMVP, the motion information may include a reference picture index of the current block, a differential motion vector, and an AMVP index.

[0135] In the following, the following will be described in detail Figure 1 An embodiment of the operation of the intra prediction module 150 is shown in FIG.

[0136] First, the intra prediction module 150 receives prediction mode information and the size of a prediction unit (PU) from the picture partition module 110, and reads reference pixels from the picture storage module 180 to determine an intra prediction mode of the prediction unit (PU).

[0137] The intra prediction module 150 determines whether a reference pixel is generated by checking whether there is an unavailable reference pixel, and the reference pixel may be used to determine an intra prediction mode of a current block.

[0138] When the current block is located at the upper boundary of the current picture, the pixels adjacent to the upper side of the current block are not defined. When the current block is located at the left boundary of the current picture, the pixels adjacent to the left side of the current block are not defined, wherein these pixels can be determined as not available pixels.

[0139] Furthermore, even when the current block is located at a slice boundary so that pixels adjacent to the upper side or the left side of the slice are not previously encoded and reconstructed pixels, these pixels may be determined as not being available pixels.

[0140] When there are no pixels adjacent to the left or upper side of the current block or there are no pixels that have been previously encoded and reconstructed, the intra prediction mode of the current block may be determined using only available pixels.

[0141] In addition, the available reference pixels of the current block can be used to generate reference pixels at unavailable positions. For example, when the pixels at the upper block are unavailable, some or all of the left pixels can be used to generate the upper pixels, and vice versa.

[0142] That is, a reference pixel may be generated by copying an available reference pixel at a position closest to a reference pixel at an unavailable position in a predetermined direction, or when there is no available reference pixel in a predetermined direction, a reference pixel may be generated by copying an available reference pixel at a nearest position in a reverse direction.

[0143] In addition, even when an upper pixel or a left pixel of the current block exists, the reference pixel may be determined as an unavailable reference pixel according to the encoding mode of the block to which the upper pixel or the left pixel belongs.

[0144] For example, when a block to which a reference pixel adjacent to the upper side of the current block belongs is a block that is inter-coded and thus reconstructed, the reference pixel may be determined as an unavailable pixel.

[0145] Here, blocks adjacent to the current block are intra-coded so that available reference pixels can be generated using pixels belonging to the reconstructed block, and information about the encoding device 10 determining the available reference pixels according to the encoding mode is transmitted to the decoding device 20.

[0146] The intra prediction module 150 determines an intra prediction mode of a current block using reference pixels, and the number of intra prediction modes that can be accepted in the current block may vary according to the size of the block.

[0147] For example, when the size of the current block is 8×8, 16×16, and 32×32, there may be 34 intra prediction modes, and when the size of the current block is 4×4, there may be 17 intra prediction modes.

[0148] The 34 or 17 intra prediction modes may be configured with at least one non-directional mode and a plurality of directional modes.

[0149] The at least one non-directional mode may be a DC mode and / or a planar mode. When the DC mode and the planar mode are included in the non-directional modes, 35 intra prediction modes may exist regardless of the size of the current block.

[0150] Here, two non-directional modes (DC mode and planar mode) and 33 directional modes may be included.

[0151] In case of the planar mode, a prediction block of the current block is generated by using at least one pixel value (or a predicted value of the pixel value, hereinafter referred to as a first reference value) located at the lower right of the current block and a reference pixel.

[0152] The configuration of the image decoding device according to the embodiment of the present invention can be seen from the reference Figures 1 to 6 The configuration of the image encoding device 10 described above is derived. For example, the above-mentioned Figures 1 to 6Describes the process of image encoding method to decode the image.

[0153] Figure 7 2 is a block diagram showing the configuration of a video decoding device according to an embodiment of the present invention. The decoding device 20 includes an entropy decoding module 210, an inverse quantization / inverse transform module 220, an adder 270, a deblocking filter 250, a picture storage module 260, an intra-frame prediction module 230, a motion compensation prediction module 240, and an intra-frame / inter-frame selection switch 280.

[0154] The entropy decoding module 210 receives a bit stream encoded by the image encoding apparatus 10 and decodes the bit stream so that the bit stream is divided into an intra prediction mode index, motion information, a quantization coefficient sequence, etc., and transmits the decoded motion information to the motion compensation prediction module 240 .

[0155] In addition, the entropy decoding module 210 transmits the intra prediction mode index to the intra prediction module 230 and the dequantization / inverse transformation module 220 , and transmits the dequantization coefficient sequence to the dequantization / inverse transformation module 220 .

[0156] The inverse quantization / inverse transform module 220 transforms the quantization coefficient sequence into a two-dimensional array of inverse quantization coefficients, and may select one of a plurality of scanning modes for the transform, and for example, the scanning mode is selected based on the intra-prediction mode and the prediction mode (e.g., intra-prediction or inter-prediction) of the current block.

[0157] The inverse quantization / inverse transform module 220 applies a quantization matrix selected from a plurality of quantization matrices to the inverse quantization coefficients of the two-dimensional array to reconstruct the quantization coefficients.

[0158] In addition, quantization matrices different from each other may be selected according to the size of a current block to be reconstructed, and a quantization matrix may be selected based on at least one of an intra prediction mode and a prediction mode of the current block for blocks of the same size.

[0159] The inverse quantization / inverse transform module 220 inversely transforms the reconstructed quantization coefficient to reconstruct a residual block, and may perform the inverse transform process using a transform unit (TU) as a basic unit.

[0160] The adder 270 reconstructs an image block by adding the residual block reconstructed by the inverse quantization / inverse transform module 220 and the prediction block generated by the intra prediction module 230 or the motion compensation prediction module 240 .

[0161] The deblocking filter 250 may perform a deblocking filtering process on the reconstructed image generated by the adder 270 to reduce deblocking artifacts caused by image loss according to the quantization process.

[0162] The picture storage module 260 is a frame memory for storing the partially decoded image on which the deblocking filter 250 performs a deblocking filtering process.

[0163] The intra prediction module 230 reconstructs the intra prediction mode of the current block based on the intra prediction mode index received from the entropy decoding module 210, and generates a prediction block according to the reconstructed intra prediction mode.

[0164] The motion compensation prediction module 240 generates a prediction block of the current block from a picture stored in the picture storage module 260 based on motion vector information, and when fractional precision motion compensation is applied, the motion compensation prediction module 240 applies a selected interpolation filter to generate the prediction block.

[0165] The intra / inter selection switch 280 may provide the adder 270 with a prediction block generated in the intra prediction module 230 or the motion compensation prediction module 240 based on the encoding mode.

[0166] Figure 8 2 is a block diagram showing an embodiment of a configuration for performing inter prediction in the image decoding apparatus 20. The inter prediction decoder includes a demultiplexer 241, a motion information encoding mode determination module 242, a merge mode motion information decoding module 243, an AMVP mode motion information decoding module 244, a prediction block generation module 245, a residual block decoding module 246, and a reconstructed block generation module 247. Here, the merge mode motion information decoding module 243 and the AMVP mode motion information decoding module 244 may be included in a motion information decoding module 248.

[0167] Reference Figure 8 The demultiplexer 241 demultiplexes the currently encoded motion information and the encoded residual signal from the received bit stream, sends the demultiplexed motion information to the motion information encoding mode determination module 242, and sends the demultiplexed residual signal to the residual block decoding module 246.

[0168] The motion information encoding mode determination module 242 determines the motion information encoding mode of the current block, and when the skip flag of the received bitstream has a value of 1, the motion information encoding mode determination module 242 determines that the motion information encoding mode of the current block is encoded in the skip encoding mode.

[0169] When the skip flag of the received bitstream has a value of 0 and the motion information received from the demultiplexer 241 has only a merge index, the motion information encoding mode determination module 242 determines that the motion information encoding mode of the current block is encoded in the merge mode.

[0170] In addition, when the skip flag of the received bitstream has a value of 0 and the motion information received from the demultiplexer 241 includes a reference picture index, a differential motion vector, and an AMVP index, the motion information encoding mode determination module 242 determines that the motion information encoding mode of the current block is encoded in the AMVP mode.

[0171] When the motion information encoding mode determination module 242 determines that the motion information encoding mode of the current block is the skip mode or the merge mode, the merge mode motion information decoding module 243 is activated. When the motion information encoding mode determination module 242 determines that the motion information encoding mode of the current block is the AMVP mode, the AMVP mode motion information decoding module 244 is activated.

[0172] The prediction block generation module 245 generates a prediction block of the current block using motion information reconstructed by the merge mode motion information decoding module 243 or the AMVP mode motion information decoding module 244 .

[0173] When the motion vector is an integer unit, the prediction block of the current block may be generated by copying a block corresponding to a position indicated by the motion vector in a picture indicated by the reference picture index.

[0174] In addition, when the motion vector is not an integer unit, the pixels of the prediction block are generated from the integer unit pixels in the picture indicated by the reference picture index. Here, the predicted pixels can be generated by using an 8-tap interpolation filter in the case of luma pixels and a 4-tap interpolation filter in the case of chroma pixels.

[0175] The residual block decoding module 246 performs entropy decoding on the residual signal and inversely scans the entropy decoded coefficients to generate a two-dimensional quantized coefficient block. The inverse scanning method may vary according to the entropy decoding method.

[0176] For example, when decoding is performed based on CABAC, inverse scanning may be applied in a diagonal raster inverse scanning method, and when decoding is performed based on CAVLC, inverse scanning may be applied in a zigzag inverse scanning method. In addition, the inverse scanning method may be determined differently according to the size of the prediction block.

[0177] The residual block decoding module 246 may dequantize the generated coefficient block using the dequantization matrix and reconstruct the quantization parameter to derive the quantization matrix. Here, the quantization step size may be reconstructed for each coding unit equal to or larger than a predetermined size.

[0178] The residual block decoding module 260 reconstructs the residual block by inversely transforming the inverse quantized coefficient block.

[0179] The reconstructed block generation module 270 generates a reconstructed block by adding the prediction block generated by the prediction block generation module 250 and the residual block generated by the residual block decoding module 260 .

[0180] In the following, we will refer again to Figure 7 An embodiment of a process of reconstructing a current block through intra prediction is described.

[0181] First, the intra prediction mode of the current block is decoded from the received bitstream. To this end, the entropy decoding module 210 refers to one of a plurality of intra prediction mode tables to reconstruct a first intra prediction mode index of the current block.

[0182] The plurality of intra prediction mode tables are tables shared by the encoding apparatus 10 and the decoding apparatus 20 , and any one table selected according to distribution of intra prediction modes for a plurality of blocks adjacent to a current block may be applied.

[0183] For example, when the intra-frame prediction mode of the left block of the current block is the same as the intra-frame prediction mode of the upper block of the current block, the first intra-frame prediction mode table is applied to reconstruct the first intra-frame prediction mode index of the current block, otherwise, the second intra-frame prediction mode table can be applied to reconstruct the first intra-frame prediction mode index of the current block.

[0184] As another example, in the case where the intra-frame prediction modes of the upper block and the left block of the current block are both directional intra-frame prediction modes, when the direction of the intra-frame prediction mode of the upper block and the direction of the intra-frame prediction mode of the left block are within a predetermined angle, the first intra-frame prediction mode index of the current block is reconstructed by applying the first intra-frame prediction mode table, and when the direction of the intra-frame prediction mode of the upper block and the direction of the intra-frame prediction mode of the left block exceed the predetermined angle, the first intra-frame prediction mode index of the current block can be reconstructed by applying the second intra-frame prediction mode table.

[0185] The entropy decoding module 210 transmits the reconstructed first intra prediction mode index of the current block to the intra prediction module 230 .

[0186] When the index has a minimum value (ie, 0), the intra prediction module 230 receiving the first intra prediction mode index determines the maximum possible mode of the current block as the intra prediction mode of the current block.

[0187] In addition, when the index has a value other than 0, the intra-frame prediction module 230 compares the index indicated by the maximum possible mode of the current block with the first intra-frame prediction mode index, and as a result of the comparison, when the first intra-frame prediction mode index is not less than the index indicated by the maximum possible mode of the current block, the intra-frame prediction module 230 determines the intra-frame prediction mode corresponding to the second intra-frame prediction mode index obtained by adding 1 to the first intra-frame prediction mode index as the intra-frame prediction mode of the current block, otherwise, the intra-frame prediction module 230 determines the intra-frame prediction mode corresponding to the first intra-frame prediction mode index as the intra-frame prediction mode of the current block.

[0188] The allowable intra prediction modes for the current block may be configured with at least one non-directional mode and a plurality of directional modes.

[0189] The at least one non-directional mode may be a DC mode and / or a planar mode. In addition, the DC mode or the planar mode may be adaptively included in the allowable intra prediction mode set.

[0190] To this end, information specifying a non-directional mode included in the allowable intra prediction mode set may be included in a picture header or a slice header.

[0191] Next, the intra prediction module 230 reads reference pixels from the picture storage module 260 and determines whether there are unavailable reference pixels in order to generate an intra prediction block.

[0192] The determination may be made by applying a decoded intra prediction mode of the current block according to whether there are reference pixels used to generate the intra prediction block.

[0193] Next, when it is necessary to generate reference pixels, the intra prediction module 230 may generate reference pixels at unavailable locations by using available reference pixels that have been previously reconstructed.

[0194] The definition of unavailable reference pixels and the method for generating the reference pixels can be the same as that according to Figure 1 The operation in the intra prediction module 150 of FIG. 1 is the same. However, according to the decoded intra prediction mode of the current block, the reference pixels used to generate the intra prediction block may be selectively reconstructed.

[0195] In addition, the intra prediction module 230 determines whether to apply filtering to the reference pixels to generate a prediction block. That is, it determines whether to apply filtering to the reference pixels based on the size of the current block and the decoded intra prediction mode to generate an intra prediction block of the current block.

[0196] As the size of the block increases, the blocking effect increases. Therefore, as the size of the block increases, the number of prediction modes for filtering the reference pixels may increase. However, when the block is larger than a predetermined size, since the block is determined to be a flat area, the reference pixels may not be filtered to reduce complexity.

[0197] When it is determined that the reference pixel needs to be applied with a filter, the intra prediction module 230 filters the reference pixel using the filter.

[0198] At least two filters may be adaptively applied according to the degree of depth difference between the reference pixels. The filter coefficients of the filters are preferably symmetric.

[0199] Furthermore, the above two or more filters may be adaptively applied according to the size of the current block. When applying the filters, a filter with a narrow bandwidth is applied to a small-sized block, and a filter with a wide bandwidth is applied to a large-sized block.

[0200] In the case of DC mode, since the prediction block is generated as the average value of the reference pixels, there is no need to apply a filter. In a vertical mode where the images have correlation in the vertical direction, there is no need to apply a filter to the reference pixels, and similarly, in a horizontal mode where the images have correlation in the horizontal direction, there is no need to apply a filter to the reference pixels.

[0201] Since whether to apply filtering is related to the intra prediction mode of the current block, the reference pixels may be adaptively filtered based on the size of the prediction block of the current block and the intra prediction mode.

[0202] Next, the intra prediction module 230 generates a prediction block using reference pixels or filtered reference pixels according to the reconstructed intra prediction mode, and the generation of the prediction block is the same as the operation of the encoding apparatus 10, so that a detailed description thereof will be omitted.

[0203] The intra prediction module 230 determines whether to filter the generated prediction block, and may determine whether to filter based on information included in a slice header or a coding unit header or according to an intra prediction mode of a current block.

[0204] When it is determined that the generated prediction block is to be filtered, the intra prediction module 230 filters pixels at specific locations of the prediction block generated using available reference pixels adjacent to the current block to generate new pixels.

[0205] For example, in the DC mode, the predicted pixels adjacent to the reference pixels among the predicted pixels may be filtered using the reference pixels adjacent to the predicted pixels.

[0206] Therefore, the predicted pixel is filtered using one or two reference pixels according to the position of the predicted pixel, and the filtering of the predicted pixel in the DC mode can be applied to prediction blocks of all sizes.

[0207] In addition, in the vertical mode, a predicted pixel adjacent to a left reference pixel among predicted pixels of a prediction block may be changed using a reference pixel other than an upper pixel used to generate the prediction block.

[0208] Likewise, in the horizontal mode, a predicted pixel adjacent to an upper reference pixel among generated predicted pixels may be changed using reference pixels other than left pixels used to generate a predicted block.

[0209] The current block may be reconstructed by using the prediction block of the current block reconstructed in this manner and the residual block of the current block that has been decoded.

[0210] Fig. 9 is a diagram illustrating a second embodiment of a method of partitioning an image into block units and processing the image.

[0211] Reference Fig. 9 , a coding tree unit (CTU) having a maximum size of 256×256 pixels is partitioned into four coding units (CUs) each having a square shape using a quadtree structure.

[0212] At least one of the coding units resulting from the quadtree partitioning operation may be further partitioned into two coding units (CUs) each having a rectangular shape using a binary tree structure.

[0213] Also, at least one of the coding units resulting from the quadtree partitioning operation may be further partitioned into four coding units (CUs) each having a square shape using a quadtree structure.

[0214] At least one of the coding units resulting from the binary tree partitioning operation may be further partitioned into two coding units (CUs) each having a square shape or a rectangular shape using a binary tree structure.

[0215] Also, at least one of the coding units resulting from the quadtree partitioning operation may be further partitioned into coding units (CUs) each having a square shape or a rectangular shape using a quadtree structure or a binary tree structure.

[0216] The coding blocks (CBs) obtained from the binary tree partitioning operation and constructed as described above can be used for prediction and transformation without being further partitioned. Fig. 9 The sizes of the prediction unit (PU) and the transform unit (TU) of the coding block CB shown in may be the same as the size of the coding block (CB).

[0217] As mentioned above, you can use the reference Figure 3 and Figure 4 The described method partitions a coding unit resulting from a quadtree partitioning operation into one or at least two prediction units (PUs).

[0218] The coding units obtained from the quadtree partitioning operation as described above can be obtained by using the reference Figure 5 The described method is partitioned into one or at least two transform units (TUs), and the transform unit (TU) resulting from the partitioning operation may have a maximum size of 64×64 pixels.

[0219] The syntax structure used to partition and process an image on a per-block basis may indicate partition information using a flag. For example, whether a coding unit (CU) is partitioned is indicated by using split_cu_flag, and the depth of a coding unit (CU) partitioned by a binary tree may be indicated by using binary_depth. Whether a coding unit (CU) is partitioned using a binary tree structure may be indicated by a separate binary_split_flag.

[0220] Reference above Figures 1 to 8 The method described was applied by referring to Fig. 9 The described method partitions the resulting blocks (eg, coding units (CUs), prediction units (PUs), and transform units (TUs)) so that encoding and decoding of an image can be performed.

[0221] In the following, reference is made to Figures 10 to 15 , another embodiment of a method of partitioning a coding unit (CU) into one or at least two transform units (TUs) will be described.

[0222] According to an embodiment of the present invention, a coding unit (CU) may be partitioned into transform units (TUs) using a binary tree structure, wherein the transform unit is a basic unit for transforming a residual block.

[0223] Reference Fig.10 At least one of the rectangular coding blocks CB0 and CB1 obtained from the binary tree partitioning operation and having a size of N×2N or 2N×N is further partitioned into square transformation units TU0 and TU1 both having a size of N×N using a binary tree structure.

[0224] As described above, the block-based image encoding method may perform prediction, transformation, quantization, and entropy encoding steps.

[0225] In the prediction step, a prediction signal is generated by referring to the current coding block and an existing coding image or a surrounding image, and thus a differential signal with respect to the current block can be calculated.

[0226] In addition, in the transform step, the transform is performed using various transform functions having the differential signal as an input. The transformed signal is classified into a DC coefficient and an AC coefficient, thereby achieving energy compression and improving encoding efficiency.

[0227] Furthermore, in the quantization step, quantization is performed with the transform coefficient as an input, and then entropy encoding is performed on the quantized signal, so that the image can be encoded.

[0228] In addition, the image decoding method is performed in the reverse order of the above encoding process, and image quality distortion may occur in the quantization step.

[0229] In order to reduce image quality distortion while improving encoding efficiency, the size or shape of a transform unit (TU) and the type of transform function to be applied may vary depending on the distribution of a differential signal input in a transform step and characteristics of an image.

[0230] For example, when a block similar to the current block is found through a block-based motion estimation process in the prediction step, a cost measurement method such as the sum of absolute differences (SAD) or the mean square error (MSE) is used, and the distribution of the differential signal can be generated in various forms according to the characteristics of the image.

[0231] Therefore, efficient encoding may be performed by selectively determining a size or shape of a transform unit (CU) based on various distributions of a differential signal.

[0232] For example, when a differential signal is generated in a specific coding block CBx, the coding block CBx is partitioned into two transform units (TUs) using a binary tree structure. Since the DC value is generally referred to as the average value of the input signal, when the differential signal is received as an input of the transform process, the DC value can be effectively indicated by partitioning the coding block CBx into two transform units (TUs).

[0233] Reference Fig.11 , a square coding unit CU0 of size 2N×2N is partitioned into rectangular transformation units TU0 and TU1 of size N×2N or 2N×N using a binary tree structure.

[0234] According to another embodiment of the present invention, as described above, the step of partitioning the coding unit (CU) using the binary tree structure is repeated two or more times, thereby obtaining a plurality of transform units (TUs).

[0235] Reference Fig.12, a rectangular coding block CB1 of size N×2N is partitioned using a binary tree structure, and a block of size N×N obtained from the partitioning operation is further partitioned using a binary tree structure, thereby establishing rectangular blocks of size N / 2×N or N×N / 2. Subsequently, the blocks of size N / 2×N or N×N / 2 are further partitioned into square transform units TU1, TU2, TU4 and TU5 of size N / 2×N / 2 using a binary tree structure.

[0236] Reference Fig.13 , a square coding block CB0 of size 2N×2N is partitioned using a binary tree structure, and a block of size N×2N obtained from the partitioning operation is further partitioned using a binary tree structure, thereby constructing a square block of size N×N, and then the block of size N×N can be further partitioned into rectangular transformation units TU1 and TU2 of size N / 2×N using a binary tree structure.

[0237] Reference Fig.14 , a rectangular coding block CB0 of size 2N×N is partitioned using a binary tree structure, and a block of size N×N obtained from the partitioning operation is further partitioned using a quadtree structure, thereby obtaining square transformation units TU1, TU2, TU3 and TU4 of size N / 2×N / 2.

[0238] Reference Figures 1 to 8 The described method was applied by referring to Figures 10 to 14 The described method partitions the resulting blocks (eg, coding units (CUs), prediction units (PUs), and transform units (TUs)) so that encoding and decoding can be performed on an image.

[0239] Hereinafter, an embodiment of a method in which the encoding apparatus 10 determines a block partition structure according to the present invention will be described.

[0240] The picture partitioning module 110 provided in the image encoding apparatus 10 performs rate-distortion optimization (RDO) according to a preset sequence, and outputs a prediction unit (PU) and determines a partition structure of a coding unit (CU), a prediction unit (PU), and a transform unit (TU) that can be partitioned as described above.

[0241] For example, to determine the block partition structure, the picture partition module 110 performs rate-distortion optimization-quantization (RDO-Q) to determine the optimal block partition structure in terms of bit rate and distortion.

[0242] Reference Fig.15, when a coding unit (CU) has a pixel size of 2N×2N, RDO is performed in the order of transformation unit (PU) partitions of the pixel size of 2N×2N shown in (a), the pixel size of N×N shown in (b), the pixel size of N×2N shown in (c), and the pixel size of 2N×N shown in (d), thereby determining an optimal partition structure of the transformation unit (PU).

[0243] Reference Fig.16 , when the coding unit (CU) has a pixel size of N×2N or 2N×N, RDO is performed in order of partition structures for transform units (PUs) of the pixel size of N×2N (or 2N×N) shown in (a), the pixel size of N×N shown in (b), the pixel sizes of N / 2×N (or N×N / 2) and N×N shown in (c), the pixel sizes of N / 2×N / 2, N / 2×N and N×N shown in (d), and the pixel size of N / 2×N shown in (e), thereby determining an optimal partition structure of the transform unit (PU).

[0244] In the above description, the block partitioning method of the present invention has been described as an example of determining the block partition structure by performing rate distortion optimization (RDO). However, the picture partitioning module 110 may use the sum of absolute errors (SAD) or mean square error to determine the block partition structure, thereby maintaining efficiency while reducing complexity.

[0245] Hereinafter, an image processing method and encoding and decoding methods thereof according to an embodiment of the present invention will be described in more detail.

[0246] Fig.17 and Fig.18 is a flowchart illustrating an image processing method for processing motion information for parallel processing according to an embodiment of the present invention. Figures 19 to 22 is a diagram showing a method of processing motion information for each case.

[0247] As described above, when the motion compensation module 160 of the encoding device 10 and the motion information decoding module 248 of the decoding device 20 respectively process motion information encoding and decoding according to the merge mode and the AMVP mode, processes such as motion compensation, mode determination, and entropy coding can be implemented as a parallel pipeline system as a hardware module.

[0248] Therefore, when the mode decision of the previous block is completed later than the processing time of the current block, the motion compensation processing of the current block may not be performed, which may cause the occurrence of pipeline stall.

[0249] In order to solve this problem, according to an embodiment of the present invention, the encoding device 10 or the decoding device 20 sequentially groups a plurality of blocks into parallel motion information prediction units according to a predetermined arbitrary size (S101), obtains motion information of at least one available block among the remaining blocks excluding the block of the previous parallel motion information prediction unit belonging to the current block from the adjacent blocks (S103), and constructs a motion information prediction list based on the obtained motion information (S105), and performs motion information prediction decoding processing for each mode based on the motion information prediction list (S107). However, when there is no available block in step S103, the motion information prediction list in step S105 is constructed using a predetermined zero motion vector (zero MV) or a motion vector at the same position of the previous frame (co-located MV).

[0250] More specifically, blocks to be encoded and to be decoded may be sequentially grouped into parallel motion information prediction units (PMUs) having a predetermined size according to encoding and decoding orders.

[0251] In a case where the nth parallel motion information prediction unit to which the current block to be decoded belongs is PMU(n), when encoding and decoding processing (such as AMVP or merging) requiring motion vector information of neighboring blocks is performed in the motion compensation module 160 or the motion information decoding module 248, blocks included in PMU(n-1) that has been encoded or decoded immediately before the current block may be excluded from the neighboring blocks.

[0252] That is, in the motion compensation module 160 or the motion information decoding module 248, when constructing a motion vector prediction (MVP) candidate list of the current block, motion information of a block included in PMU (n-1) that has been encoded / decoded immediately before PMU (n) to which the current block belongs may be excluded. Therefore, the MVP candidate list corresponding to the current block may be determined from motion information of blocks not included in PMU (n-1) among neighboring blocks.

[0253] However, since the number of available motion vectors of neighboring blocks may be reduced, if Fig.19 As shown in , in order to prevent the coding efficiency from being reduced, motion information of more neighboring blocks F to I than existing neighboring blocks A to E may be used to construct a motion vector prediction (MVP) candidate list.

[0254] Therefore, at least one of motion vectors of available blocks (eg, blocks of the same picture / tile / slice) not included in PMU (n-1) among neighboring blocks may be determined as a resulting motion vector prediction (MVP) candidate list.

[0255] In addition, refer to Fig.18, when decoding motion information using a neighboring block at a predetermined position, the motion compensation unit 160 or the motion information decoding module 248 identifies a neighboring block to be used for motion estimation of a current block belonging to PUM(n) (S201), when the neighboring block is included in PMU(n-1) (S203), obtains alternative information of the motion information of the neighboring block included in PMU(n-1) (S205), and, without the motion information of the neighboring block included in PMU(n-1), uses the alternative information to process motion estimation decoding of the current block (S207).

[0256] Here, even if the actual motion vector information at the corresponding position is unknown, the substitute information may be calculated as a predetermined default value, or the substitute information may be a value derived from a neighboring block included in PMU (n). For example, substitute information of a neighboring block included in PMU (n-1) may be constructed with a zero motion vector (zero MV) or a motion vector at the same position of a previous frame (co-located MV).

[0257] For such processing, the encoding device 10 may include the size information of the PMU in the syntax form in the sequence header information, the picture header information, or the slice header information for transmission. The size information of the PMU may also be explicitly or implicitly signaled as a value related to the CTU size and the minimum size and maximum size of the CU (or PU).

[0258] In addition, the encoding device 10 may sequentially group the PMUs according to a predetermined size and assign a parallel motion information prediction unit index to each unit to index, and the assigned index information may be explicitly transmitted or the index information may be derived in the decoding device 20 .

[0259] For example, the size of the PMU may always be equal to the minimum PU size, or the size of the PMU may be equal to the size of the PU or CU currently being encoded / decoded. In this case, the PMU size information may not be explicitly sent, and the index may be identified according to the sequential grouping operation.

[0260] Reference Fig. 20 , the thick line indicates the CTU, the solid line indicates the PU, and the red dotted line indicates the PMU.

[0261] According to an embodiment of the present invention, the motion vector of the lower left end of the neighboring motion vector of the current PU may be a motion vector included in the immediately preceding PMU (n-1) according to the encoding and decoding order. Therefore, when motion vector prediction encoding and decoding are performed in the encoding device 10 and the decoding device 20, the motion vector of the neighboring block included in the immediately preceding PMU (n-1) may be excluded when constructing the motion vector list. As alternative information of the block included in the immediately preceding indexed PMU (n-1), a zero motion vector (zero MV) or a motion vector at the same position of a previous frame (co-located MV) may be used to construct the motion vector candidate list.

[0262] In addition, refer to Fig.21 , since the neighboring blocks at the upper right end are included in the immediately preceding PMU (n-1), they are excluded from the candidate motion vectors in the motion vector prediction encoding and decoding of the current block. Alternatively, a zero motion vector (zero MV) or a motion vector at the same position of a previous frame (co-located MV) may be used as a candidate motion vector.

[0263] Reference Fig. 22 , describes the case where all neighboring blocks on the left are included in the previous PMU (n-1). Fig. 22 As shown in the figure, the encoding device 10 and the decoding device 20 exclude the motion vectors of all left neighboring blocks from the candidate motion vectors in the motion vector prediction encoding and decoding process, and instead, the motion vectors of the upper (upper left and upper right) neighboring blocks can be used to construct a candidate list for motion vector prediction.

[0264] According to this construction, the dependencies between parallel motion information prediction units can be split, and motion information decoding can be processed without waiting for the processing of blocks in the previous pipeline among adjacent blocks to be completed, thereby preventing pipeline stalls in advance and improving encoding and decoding efficiency accordingly.

[0265] The above method according to the present invention can be stored in a computer-readable recording medium. The computer-readable recording medium can be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and can also be implemented in the form of a carrier wave (e.g., transmitted via the Internet).

[0266] The computer readable recording medium can be distributed on a network-connected computer system so that the computer readable code can be stored and executed in a distributed manner. In addition, the functional program, code and code segment for implementing the above method can be easily inferred by a programmer in the technical field to which the present invention belongs.

[0267] Although the exemplary embodiments of the present invention have been illustrated and described above, the present invention is not limited to the aforementioned specific embodiments, and those skilled in the art may make various modifications to the present invention without departing from the gist of the present invention defined in the claims. These modifications should not be understood as departing from the technical concept or viewpoint of the present invention.

Claims

1. An image decoding method, the method comprising: deriving a first motion information candidate from a predefined neighboring block of the current block; deriving a second motion information candidate from at least one block included in a motion information prediction unit including the current block; generating a candidate list using at least one of the first motion information candidate and the second motion information candidate; Generate a prediction block for the current block based on the candidate list; and generating a reconstructed block of the current block based on the predicted block, Wherein, the motion information prediction unit is a coding tree unit, The at least one block included in the motion information prediction unit is determined by excluding blocks of a motion information prediction unit belonging to a row different from a motion information prediction unit including the current block.

2. A method for encoding an image, the method comprising: deriving a first motion information candidate from a predefined neighboring block of the current block; deriving a second motion information candidate from at least one block included in a motion information prediction unit including the current block; generating a candidate list using at least one of the first motion information candidate and the second motion information candidate; Generate a prediction block for the current block based on the candidate list; and generating a reconstructed block of the current block based on the predicted block, Wherein, the motion information prediction unit is a coding tree unit, The at least one block is determined by excluding blocks belonging to a motion information prediction unit of a row different from a motion information prediction unit including the current block.

Images