Image encoding and decoding method and bit stream storage method for performing inter-frame prediction

Abstract

The present invention discloses an image coding and decoding method and a bit stream storage method for performing inter-frame prediction. The image decoding method for performing inter-frame prediction includes: generating a reference picture list 0 and a reference picture list 1 for a current block; specifying a reference picture of the current block based on the reference picture list 0 and the reference picture list 1; performing inter-frame prediction on the current block using the reference picture; and reconstructing the current block based on a prediction block of the current block generated by the inter-frame prediction.

Description

[0001] This application is a divisional application of the invention patent application with application date of June 7, 2016, application number 201680045604.5, and invention name “Method and device for encoding and decoding intra-frame prediction”. Technical Field

[0002] The present invention relates to image encoding and decoding technology, and more particularly to an encoding and decoding method and apparatus for performing reference pixel construction and most probable mode construction in intra-frame prediction. Background Art

[0003] With the popularization of the Internet and mobile terminals and the development of information and communication technologies, the use of multimedia data is rapidly increasing. As a result, in order to perform various services or tasks through image prediction in various systems, the demand for improving the performance and efficiency of image processing systems has greatly increased. However, research and development to meet this demand is insufficient.

[0004] With the image encoding and decoding methods and apparatuses according to the above-mentioned prior art, there is a need for performance improvement in image processing, particularly image encoding or image decoding. Summary of the Invention

[0005] Technical issues

[0006] The present invention aims to solve the above-mentioned problems, and an object of the present invention is to provide an image encoding / decoding method that improves the intra-frame prediction mode.

[0007] Another object of the present invention is to provide an image encoding / decoding device that improves the intra-frame prediction mode.

[0008] Technical Solution

[0009] In order to achieve the purpose of the present invention, an image encoding method for performing intra-frame prediction according to an aspect of the present invention includes: determining whether boundary pixels within adjacent blocks can be used as reference pixels of the current block in consideration of prediction modes of each of a current block and adjacent blocks, obtaining reference pixels of the current block based on the determined result, generating a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decoding the current block using the generated prediction block.

[0010] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may be performed by obtaining a reference pixel composed of a predetermined pixel value.

[0011] Here, determining whether a boundary pixel can be used as a reference pixel may be performed by indicating whether a predefined flag considers a prediction mode of a neighboring block.

[0012] Here, determining whether the boundary pixels are usable as reference pixels may be performed by determining that the boundary pixels are usable as reference pixels of the current picture when the prediction mode of the neighboring block is intra prediction.

[0013] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is not usable as a reference pixel of the current picture when the prediction mode of the adjacent block is inter prediction.

[0014] Here, when the prediction mode of the neighboring block is inter prediction, determining whether the boundary pixel is usable as a reference pixel may be performed by determining whether the boundary pixel is usable as a reference pixel of the current picture, considering the reference picture of the neighboring block.

[0015] Here, a reference picture may be selected from List 0 storing data of a reference picture before the current picture or List 1 storing data of a reference picture after the current picture, and the current picture may be selected from List 0 or List 1.

[0016] Here, when the reference picture of the neighboring block is the current picture, the boundary pixels may be determined to be usable as reference pixels of the current picture.

[0017] Here, if the reference picture of the neighboring block is not the current picture, it may be determined that the boundary pixel cannot be used as a reference pixel of the current picture.

[0018] Here, when the reference picture of the neighboring block is an I picture, the boundary pixels may be determined to be usable as reference pixels of the current picture.

[0019] In order to achieve other objects of the present invention, an image decoding method for performing intra-frame prediction according to another aspect of the present invention includes: receiving a bit stream including data on prediction modes of a current block and blocks adjacent to the current block, confirming the prediction modes of the adjacent blocks by extracting data from the received bit stream, determining whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block in consideration of the prediction modes of the adjacent blocks, obtaining reference pixels of the current block according to the determined result, generating a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decoding the current block using the generated prediction block.

[0020] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may be performed by obtaining a reference pixel constructed by a preset pixel value.

[0021] Here, determining whether a boundary pixel can be used as a reference pixel may indicate whether a preset flag considers a prediction mode of a neighboring block.

[0022] Here, determining whether the boundary pixels are usable as reference pixels may be performed by determining that the boundary pixels are usable as reference pixels of the current picture when the prediction mode of the neighboring block is intra prediction.

[0023] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is not usable as a reference pixel of the current picture when the prediction mode of the adjacent block is inter prediction.

[0024] Here, determining whether a boundary pixel is usable as a reference pixel may be performed by determining whether the boundary pixel is usable as a reference pixel of a current picture, considering a reference picture of a neighboring block.

[0025] Here, a reference picture may be selected from List 0 storing data of a reference picture before the current picture or List 1 storing data of a reference picture after the current picture, and the current picture may be included in List 0 or List 1 .

[0026] Here, when the reference picture of the neighboring block is the current picture, it may be determined that the boundary pixels may be used as reference pixels of the current picture.

[0027] Here, when the reference picture of the neighboring block is not the current picture, it may be determined that pixels in the neighboring block cannot be used as reference pixels of the current picture.

[0028] Here, when the reference picture of the neighboring block is an I picture, it may be determined that pixels in the neighboring block may be used as a reference picture of the current picture.

[0029] In order to achieve other objects of the present invention, in an image decoding device including one or more processors according to another aspect of the present invention, the one or more processors can receive a bit stream including data about prediction modes of a current block and blocks adjacent to the current block, extract data from the received bit stream to confirm the prediction modes of the adjacent blocks, determine whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block considering the prediction modes of the adjacent blocks, obtain reference pixels of the current block based on the determined result, generate a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decode the current block using the generated prediction block.

[0030] In order to achieve the purpose of the present invention, an image encoding method for performing intra-frame prediction according to one aspect of the present invention includes: determining whether each of the left block and the upper block of the current block can be used to construct reference pixels of the current block, selecting a most probable mode (MPM) candidate group based on the determined result, determining redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group, selecting an MPM group using the prediction mode of an alternative block adjacent to the left block or adjacent to the upper block when redundancy is determined to exist, and performing intra-frame prediction mode encoding on the current block based on the selected MPM group.

[0031] Here, determining whether each block can be used may be performed by determining as unusable when the left block and the upper block exist outside the image boundary or are encoded using inter prediction and determining as available in other cases.

[0032] Here, the MPM candidate group may be selected from the intra prediction mode of the block determined to be available among the left block and the above block, the intra prediction mode predetermined for the block determined to be unavailable, the planar prediction mode, the vertical prediction mode, and the DC prediction mode.

[0033] Here, when both the left block and the upper block are available, redundancy may be determined by determining redundancy based on variance values ​​calculated for reference pixel values ​​in two or more consecutive blocks among blocks located below and to the left, above and to the left, above and to the upper right of the current block.

[0034] Determining redundancy may include selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0035] Here, the replacement block may be selected from adjacent blocks located below and to the left, above and to the left, and above and to the right of the current block.

[0036] Here, when constructing a neighboring block with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0037] Here, the replacement block may be selected with a clockwise or counterclockwise priority among neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0038] According to another aspect of the present invention, an image decoding method for performing intra-frame prediction may include: extracting data on whether the prediction mode of the current block is included in the most probable mode (MPM) group from a received bit stream; determining whether the prediction mode of the current block is included in the MPM group based on the data; extracting MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group; confirming the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and performing intra-frame prediction on the current block based on the confirmed prediction mode.

[0039] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct the reference pixels of the current block; selecting the most likely mode MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when redundancy is determined to exist.

[0040] Here, determining whether a block can be used may be performed by determining as unusable when each of the left block and the upper block exists outside an image boundary or is encoded with inter-frame prediction and determining as available in other cases.

[0041] Here, the MPM candidate group may be selected from intra prediction modes of blocks determined to be available among left and upper blocks, predetermined intra prediction modes for blocks determined to be unavailable, planar prediction mode, vertical prediction mode, and DC prediction mode.

[0042] Here, when both the left block and the above block are available, determining the redundancy of the prediction block can be performed by determining the redundancy based on the variance value calculated for the reference pixel values ​​in two or more consecutive blocks among the blocks located to the lower left, left, upper left, above, and upper right of the current block.

[0043] Here, determining the redundancy of the prediction block may be selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0044] Here, the replacement block may be selected from adjacent blocks located below and to the left, above and to the left, and above and to the right of the current block.

[0045] When constructing a neighboring block with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0046] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority order.

[0047] To achieve the purpose, in an image decoding device including one or more processors according to other aspects of the present invention, the one or more processors may: extract data related to whether the prediction mode of the current block is included in the most probable mode MPM group from a received bit stream; determine whether the prediction mode of the current block is included in the MPM group based on the data; extract the MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group, confirm the prediction mode for the current block in the MPM group based on the extracted MPM group index data, and perform intra-frame prediction on the current block based on the confirmed prediction mode.

[0048] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct the reference pixels of the current block; selecting the MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when redundancy is determined to exist.

[0049] Here, determining whether a block can be used may be performed by determining as unusable when each of the left block and the upper block exists outside an image boundary or is encoded with inter-frame prediction and determining as available in other cases.

[0050] Determining the redundancy of the prediction block may be performed by selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0051] Beneficial effects

[0052] When the image encoding / decoding method and apparatus according to the embodiments of the present invention are used as described above, the accuracy of prediction can be improved by achieving improvements in reference pixel construction and most probable mode construction in intra prediction.

[0053] Furthermore, in intra-frame prediction, the encoding efficiency of a picture can be improved by improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0055] Figure 2 is a diagram illustrating a configuration of an image encoding apparatus according to an embodiment of the present invention.

[0056] Figure 3 is a diagram illustrating a structure of an image decoding device according to an embodiment of the present invention.

[0057] Figure 4 is an exemplary view illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.

[0058] Figure 5 is an exemplary view illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

[0059] Figure 6 is an exemplary view illustrating a case where a prediction block is generated in a single direction in an image encoding and decoding method according to an embodiment of the present invention.

[0060] Figure 7 is an exemplary view of construction of a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0061] Figure 8 is an exemplary view illustrating another example of a case where inter prediction is performed from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0062] Figure 9 is an exemplary view illustrating intra prediction in an image encoding method according to an embodiment of the present invention.

[0063] Figure 10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

[0064] Figure 11 is an exemplary view showing a process of obtaining a prediction block.

[0065] Figure 12 This is an exemplary view showing a main procedure of an image encoding method according to an embodiment of the present invention using syntax in encoding units.

[0066] Figure 13 is an exemplary view showing an example of supporting symmetric type partitioning or asymmetric type partitioning as inter prediction when a prediction block is generated by block matching in a current picture.

[0067] Figure 14 is an exemplary view showing that 2N×2N and N×N can be supported in inter prediction.

[0068] Figure 15 is an exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0069] Figure 16ais an exemplary view illustrating whether to use reference pixels in consideration of encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0070] Figure 16b is an exemplary diagram illustrating whether reference pixels are used regardless of encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0071] Figure 17 is another exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0072] Figure 18a is an exemplary view illustrating a case where an encoding mode of a neighboring block is intra prediction in intra prediction according to an embodiment of the present invention.

[0073] Figure 18b is an exemplary view illustrating a case where encoding modes and reference pictures of neighboring blocks are considered in intra prediction according to an exemplary embodiment of the present invention.

[0074] Figure 18c is another exemplary view illustrating a case where encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.

[0075] Figure 19a is an exemplary view showing a method of obtaining reference pixels when an available neighboring block is located at the lower left end.

[0076] Figure 19b is an exemplary view showing a method of obtaining a reference pixel when an available neighboring block is located at the upper right end.

[0077] Figure 19c is an exemplary view showing a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0078] Figure 19d is an exemplary view showing a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0079] Figure 20 is a flowchart of an image encoding method according to an embodiment of the present invention.

[0080] Figure 21 The diagram shows an example of an intra prediction mode of HEVC (high efficiency video coding).

[0081] Figure 22a is an exemplary view showing reference pixels used to predict a current block in a reference pixel construction step.

[0082] Figure 22b is an exemplary view showing that additional pixels of a neighboring block are included in the reference pixel candidate group.

[0083] Figure 23a 、 Figure 23b 、 Figure 24a and Figure 24b is an exemplary view showing various mode directions of a current block with a wider reference pixel that can be included in the reference pixel construction step.

[0084] Figure 25 is a flow chart illustrating a process of determining the most probable mode for a general current prediction unit (or prediction block).

[0085] Figure 26 is an exemplary view of determining a prediction mode of a current prediction unit (or block).

[0086] Figure 27 is an exemplary view of the group prediction mode.

[0087] Figures 28 to 32 is an exemplary view of a case where an MPM is constructed for a prediction mode using neighboring blocks other than an upper block and a left block.

[0088] Figure 33 is an exemplary view of a case where a most probable mode (MPM) is constructed as a substitute block when both a current block and neighboring blocks are available.

[0089] Figure 34 is a flowchart illustrating an image encoding method for performing intra-frame prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0090] Although the present invention is susceptible to various modifications and changes, specific embodiments thereof are shown by way of example in the accompanying drawings and will be described in detail herein. However, it should be understood that the present invention is not limited to specific embodiments, but includes all modifications, equivalents, and alternatives that fall within the spirit and scope of the present invention.

[0091] Terms such as first, second, A, B, etc. may be used to describe various components, but these components should not be limited by these terms. These terms are only used to distinguish one component from another component. For example, a first component may be referred to as a second component, and similarly, a second component may be referred to as a first component without departing from the scope of the present invention. The term "and / or" includes any combination of multiple related listed items or any one of multiple related listed items.

[0092] It should be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other component, but it should be understood that other components may exist between them. On the other hand, when an element is referred to as being "directly connected" or "directly coupled" to another element, it should be understood that no other elements exist between them.

[0093] The terms used herein are for the purpose of describing specific embodiments only and are not intended to limit the present invention. Unless the context clearly dictates otherwise, singular forms include plural referents. In this application, the terms "including" or "having" are used to specify the presence of the stated features, figures, steps, operations, elements, parts or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts or combinations thereof.

[0094] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the contextual meaning of the relevant art, and should not be interpreted as an ideal or excessive form of meaning unless explicitly defined in this application.

[0095] Typically, a motion picture may consist of a series of pictures, and each picture may be divided into predetermined areas such as blocks. In addition, the divided areas may be referred to as various sizes or terms such as coding tree units (CTUs), coding units (CUs), prediction units (PUs), and transform units (TUs), as well as blocks. Each unit may be constructed with one luminance block and two chrominance blocks, which may be constructed differently depending on the color format. In addition, the sizes of the luminance blocks and the chrominance blocks may be determined according to the color format. For example, in the case of 4:2:0, the size of the chrominance block may have a length of 1 / 2 the width and height of the luminance block. For these units, reference may be made to items such as existing HEVC or H.264 / AVC. In the present invention, these blocks and items are used in a mixed manner, but may be understood differently according to standard techniques, and should be understood as corresponding items or units for encoding and decoding processing according to these standard techniques.

[0096] In addition, a picture, block, or pixel referenced when encoding or decoding a current block or current pixel is referred to as a reference picture, reference block, or reference pixel. Those skilled in the art will appreciate that the term "picture" described below can be used to replace other terms with equivalent meanings such as image, frame, etc.

[0097] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

[0098] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0099] refer to Figure 1 The image encoding device 12 and the decoding device 11 can be implemented by a user terminal such as a personal computer (PC), a laptop computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a portable game console (PlayStation Portable, PSP), a wireless communication terminal, a smartphone, a television, or a server terminal such as an application server and a service server, and can include: a communication device such as a communication modem for performing communication with various machines or wired / wireless communication networks; a memory 18 for encoding or decoding images or storing various programs and data for inter-frame or intra-frame prediction used for encoding or decoding; a processor 14 for performing calculations and control through program execution, etc. In addition, the video encoded into a bit stream by the video encoding device 12 can be transmitted to the video decoding device and then decrypted by the video decoding device to be reconstructed and reproduced as an image in real time or non-real time through a wired or wireless communication network such as the Internet, a local wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, or various communication interfaces such as a cable or a universal serial bus (USB).

[0100] In addition, an image encoded by a video encoding device can be transmitted as a bit stream from the encoding device to the decoding device through a computer-readable recording medium.

[0101] Figure 2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. Figure 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

[0102] like Figure 2 As shown in , the image encoding device 20 according to this embodiment includes a prediction unit 200, a subtraction unit 205, a transform unit 210, a quantization unit 215, a dequantization unit 220, an inverse transform unit 225, an addition unit 230, a filter 235, a decoding picture buffer 240 and an entropy encoding unit 245.

[0103] like Figure 3 As shown in , the image decoding device 30 according to this embodiment includes an entropy decoding unit 305, a prediction unit 310, a dequantization unit 315, an inverse transform unit 320, an addition unit 325, a filter unit 330 and a decoded picture buffer 335.

[0104] The image encoding device 20 and the image decoding device 30 may be configured separately, but depending on the embodiment, they may be combined into a single image encoding and decoding device. In this case, the prediction unit 200, dequantization unit 220, inverse transform unit 225, addition unit 230, filter 235, and memory 240 of the image encoding device 20 and the prediction unit 310, dequantization unit 315, inverse transform unit 320, addition unit 325, filter unit 330, and memory 335 of the image decoding device 30 are substantially the same technical elements in the order disclosed, and may include the same structure or be implemented to perform at least the same functions. Furthermore, when the entropy encoding unit 245 performs its functions in reverse, it may correspond to the entropy decoding unit 305. Therefore, in the following description of the technical elements and their operating principles, repeated descriptions of the corresponding technical elements will be omitted.

[0105] Since the image decoding device 30 corresponds to a computing device that applies the image encoding method performed in the image encoding device 20 to decoding, the following description will focus on the image encoding device 20.

[0106] The computing device may include a memory storing a program or software module implementing the image encoding method and / or the image decoding method, and a processor coupled to the memory and running the program. The image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder.

[0107] Each component of the image encoding device of this embodiment will be described in more detail below.

[0108] Here, the image encoding device 20 may further include a division unit. The division unit divides the input image into blocks of a predetermined size (M×N). Here, M or N is any natural number greater than 1. In detail, the division unit can be constructed with a picture division unit and a block division unit. The size or type of the block can be determined according to the characteristics and resolution of the image, and the size or type of the block supported by the picture division unit can be an M×N square type (256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or an M×N rectangular type whose horizontal length and vertical length are expressed as a power of 2. For example, for an 8k UHD image with high resolution, the input image can be divided into 256×256, for a 1080p HD image, the input image can be divided into 128×128, and for a WVGA image, the input image can be divided into 16×16.

[0109] Information about the size or type of such a block can be set in units of sequence, picture, slice, etc., and the relevant information can be sent to the decoder. That is, it can be set in units of sequence parameter set, picture parameter set, slice header, or a combination thereof.

[0110] Here, a sequence refers to a constituent unit formed by collecting several related scenes. A picture is a series of luminance (Y) components or the sum of luminance and chrominance (Y, Cb, Cr) components in a scene or picture, and the scope of a picture can be one frame or one field.

[0111] A slice can refer to an independent slice segment and several dependent slice segments in the same access unit. An access unit refers to a collection of network abstraction layer (NAL) units associated with a single coded picture. A NAL unit is a syntactic structure of a video compression bitstream constructed in a network-friendly format in the H.264 / AVC and HEVC standards. Typically, a slice unit is constructed as a NAL unit, and the collection of NALs or NALs that constitute a frame is considered an access unit in the system standard.

[0112] Returning to the description of the picture division unit, information related to the size or type (M×N) of the block can be made by explicit flags, and specifically may include block type information, one length information when the block is square, each length information when the block is rectangular, or difference information between width and length. For example, if M and N are composed of powers of k (assuming k is 2) (M=2 m , N = 2 n), the information related to m and n can be encoded by various methods such as unary binarization and truncated unary binarization so that the relevant information can be sent to the decoding device. Alternatively, the minimum allowable partition size Minblksize supported by the picture partitioning unit can be sent I×J (for ease of explanation, if I=2 i , J = 2 j , then it is assumed that I=J), information of mI or nj. As another example, when M and N are different from each other, the difference between m and n (|mn|) can be transmitted. Alternatively, the maximum allowable partition size Maxblksize supported by the picture partitioning unit can be transmitted as I×J (for ease of explanation, if I=2 i , J = 2 j , then assume I=J), im or nj information.

[0113] In the implicit case, for example, when the syntax of the relevant information exists but cannot be verified by the encoder or decoder, the encoder or decoder can follow the preset default setting. For example, when the relevant syntax cannot be confirmed when confirming the block type information, the block type can be set to a square type, which is the default setting. Alternatively, in the step of confirming the block size information, specifically, in the step of confirming the block size information by the difference from the minimum allowable partition size Minblksize as in the above example, when the syntax related to the difference can be recognized but the syntax related to the minimum allowable partition size Minblksize cannot be recognized, the information can be obtained from the default value related to the preset minimum allowable partition size Minblksize.

[0114] As described above, the size or type of the block in the picture division unit may be explicitly transmitted as related information from the encoder or decoder, or may be implicitly determined according to the characteristics and resolution of the image, etc.

[0115] As described above, the blocks divided and determined by the picture division unit can be used as basic coding units. The blocks divided and determined by the picture division unit can be the minimum units constituting high-level units such as pictures, slices, and tiles, and can be the maximum units of blocks such as coding blocks, prediction blocks, transform blocks, quantization blocks, entropy blocks, and in-loop filtering blocks, but some blocks are not limited thereto and may be exceptions.

[0116] For example, some blocks such as an in-loop filtering block may be applied as a unit larger than the size of the above-mentioned block.

[0117] The block partitioning unit performs partitioning on blocks such as encoding, prediction, transform, quantization, entropy, and in-loop filtering. The partitioning unit is included in each component to perform its function. For example, the transform unit 210 may include a transform block partitioning unit, and the quantization unit 215 may include a quantization block partitioning unit. The size or type of the initial block in the block partitioning unit may be determined by the partitioning result of the block at the previous level or higher level. For example, in the case of a coding block, a block obtained by the picture partitioning unit as the previous level may be set as the initial block. Alternatively, in the case of a prediction block, a block obtained by the partitioning process of a higher-level coding block as the prediction block may be set as the initial block. Alternatively, in the case of a transform block, a block obtained by the partitioning process of a higher-level coding block as the transform block may be set as the initial block. The conditions for determining the size or type of the initial block are not always fixed, and part may be changed or there may be special cases. In addition, the splitting process of the current level (for example, whether it can be split, the type of divisible block, etc.) may also be affected by a combination of at least one factor such as the splitting status of blocks at the previous level or higher level (for example, the size of the coding block, the type of the coding block, etc.) and the setting conditions of the current level (for example, the size of the supported transform block, the type of transform block, etc.).

[0118] The block partitioning unit can support a quadtree-based partitioning scheme. That is, the block can be divided into four blocks, each of which has a horizontal and vertical length of 1 / 2 of the block before the partition. This means that the partitioning can be repeated until the allowed partition depth limit dep_k is reached (dep_k represents the number of allowed partitions, and the block size of the allowed partition depth limit is M>>k, N>>k).

[0119] In addition, a binary tree-based partitioning scheme can be supported. This indicates that one of the horizontal and vertical lines can be divided into two blocks with a length of 1 / 2 compared to the block before the partition. In the case of quadtree partitioning and binary tree partitioning, symmetric partitioning or asymmetric partitioning can be used, and which partitioning scheme is selected can be determined based on the settings of the encoder or decoder. The symmetric partitioning scheme will be mainly described in the image encoding method of the present invention.

[0120] The split flag (div_flag) may indicate whether each block is split. If the corresponding value is 1, the split is performed, and if the value is 0, the split is not performed. Alternatively, if the corresponding value is 1, the split is performed and additional splits are allowed, and if the value is 0, the split is not performed and additional splits are not allowed. Depending on conditions such as the minimum allowable split size and the allowable split depth limit, the flag may only consider whether the split is performed and may not consider whether additional splits are performed.

[0121] The partition flag can be used in quadtree partitioning or binary tree partitioning. In binary tree partitioning, the partition direction can be determined based on the partition depth, coding mode, prediction mode, size, type and category of the block (one of coding, prediction, transform, quantization, entropy, in-loop filtering, etc., or one of brightness and chrominance) and at least one factor such as slice type, partition allowable depth limit, minimum and maximum allowable partition size, or a combination thereof.

[0122] In this case, depending on the split direction corresponding to the split flag, the block can be split into 1 / 2 only in the horizontal direction of the block, or 1 / 2 only in the vertical direction. For example, when it is assumed that the block supports M×N (M>N) and M is greater than N, horizontal splitting is supported, and when it is assumed that the current split depth (dep_curr) is less than the allowable split depth limit that allows additional splitting to be performed, the split flag is allocated 1 bit so that horizontal splitting is performed when the corresponding value is 1, and no further splitting is performed when the value is 0. The split depth can be one split depth for quadtree splitting and binary splitting and each split depth for quadtree splitting and binary splitting. The allowable split depth limit can be one allowable split depth limit for quadtree splitting and binary splitting and each allowable split depth limit for quadtree splitting and binary splitting.

[0123] As another example, if the block is M×N (M>N) and N is equal to a predetermined minimum allowable partition size and horizontal partitioning is not supported, the partition flag is assigned 1 bit so that vertical partitioning is performed when the value is 1, and if the value is 0, no further partitioning is performed.

[0124] Furthermore, flags (div_h_flag, div_v_flag) for horizontal or vertical splitting can be supported respectively, and binary splitting can be supported according to the flag. The horizontal splitting flag (div_h_flag) or the vertical splitting flag (div_v_flag) indicates whether horizontal or vertical splitting of each block is allowed. When the horizontal splitting flag (div_h_flag) or the vertical splitting flag is 1, horizontal or vertical splitting is performed, and when the flag is 0, horizontal or vertical splitting is not performed. Or when each flag is 1, horizontal or vertical splitting is performed, and additional horizontal or vertical splitting is allowed. When the value is 0, horizontal or vertical splitting is not performed, and further horizontal or vertical splitting may not be allowed. Depending on conditions such as the minimum allowable splitting size and the allowable splitting depth limit, the flag can consider whether to perform the splitting, or whether to perform the additional splitting. Alternatively, flags (div_flag / h_v_flag) for horizontal or vertical splitting can be supported, and binary splitting can be supported according to the flag. The split flag (div_flag) can indicate whether to perform horizontal or vertical splitting, and the split direction flag (h_v_flag) can indicate the horizontal or vertical splitting direction. When the split flag (div_flag) is 1, splitting is performed and horizontal or vertical splitting is performed according to the split direction flag (h_v_flag), and horizontal or vertical splitting is not performed when the value is 0. If the value is 1, horizontal or vertical splitting is performed according to the horizontal split flag (h_v_flag), and additional horizontal or vertical splitting is allowed. When the value is 0, horizontal or vertical splitting is not performed, and further horizontal or vertical splitting is considered not allowed. Depending on conditions such as the minimum allowable split size and the allowable split depth limit, the flag can consider whether to perform splitting and whether to perform additional splitting.

[0125] These split flags can support horizontal and vertical splits respectively, and also support binary tree splits according to the flags. Moreover, when the split direction is predetermined, as in the above example, only one of the two split flags can be used, or both split flags can be used.

[0126] For example, when all flags are allowed, the possible types of blocks can be divided into any one of M×N, M / 2×N, M×N / 2, and M / 2×N / 2. In this case, the flags can be encoded as 00, 01, 10, and 11 (in the order of div_h_flag / div_v_flag). The above case is an example of a setting in which the division flags can be overlapped, rather than an example of a setting in which the division flags cannot be overlapped. For example, it is also possible to divide the division block types into M×N, M / 2×N, and M×N / 2, and in this case, the above flags are encoded as 00, 01, 10 (in the order of div_h_flag / div_v_flag), or as 00, 10, and 11 (in the order of div_flag / h_v_flag, h_v_flag is a flag indicating whether the division direction is horizontal or vertical). Here, overlapping means that both horizontal division and vertical division are performed simultaneously. The above-mentioned quadtree division and binary tree division can be used alone or in combination according to the settings of the encoder or decoder. For example, quadtree or binary tree partitioning can be determined according to block size or type. That is, when the block type is M×N and M is greater than N, horizontal partitioning is performed, when the block type is M×N and N is greater than M, binary partitioning can be supported according to vertical partitioning, and when the block type is M×N and N and M are the same, quadtree partitioning can be supported.

[0127] As another example, if the size of the block (M×M) is greater than or equal to the block partition threshold (Thrblksize), binary tree partitioning may be supported; if less than, quadtree partitioning may be supported.

[0128] In another example, quadtree partitioning is supported when M or N of the block (M×N) is greater than or equal to the first minimum allowable partition size (Minblksize1), and binary tree partitioning can be supported when M or N of the block (MxN) is greater than or equal to the second minimum allowable partition size (Minblksize2). When the first partition support range and the second partition support range that can be defined by the maximum allowable partition size and the minimum allowable partition size overlap, priority can be given to the first or second partitioning method according to the setting of the encoder or decoder. In this example, the first partitioning method is quadtree partitioning and the second partitioning method is binary tree partitioning. For example, when the first minimum allowable partition size (Minblksize1) is 16, the second maximum allowable partition size (Maxblksize2) is 64, and the block before partitioning is 64×64, it is possible to allow quadtree partitioning and binary tree partitioning to be allowed within both the first partition support range and the second partition support range. When the first partitioning method (quadtree partitioning in this example) is prioritized according to the setting, quadtree partitioning is performed, and additional quadtree partitioning is allowed when the partition flag (div_flag) is 1, and it can be considered that quadtree partitioning is not performed and no further quadtree partitioning is performed when the partition flag (div_flag) is 1. According to such conditions as the minimum allowable partition size and the allowable partition depth limit, the flag can only consider whether to perform the partition, and may not consider whether to perform further partitioning. When the partition flag (div_flag) is 1, since the block is divided into 4 blocks with a size of 32×32 larger than the first minimum allowable partition size (Minblksize 1), quadtree partitioning can continue. If the flag is 0, no further quadtree partitioning is performed, and the current block size (64×64) is within the second partition support range, it is possible to perform binary tree partitioning. If the partition flag (in the order of div_flag / h_v_flag) is 0, no further partitioning is performed. If the split flag is 10 or 11, horizontal splitting or vertical splitting can be performed. If the block before splitting is 32×32 and the split flag (div_flag) is 0, so that no further quadtree splitting is performed, and if the second maximum allowable split size (Maxblksize2) is 16, the size of the current block (32×32) is not within the second split support range, so that splitting is no longer supported. In the above description, the priority of the splitting method can be determined according to at least one or a combination of the slice type, coding mode, luminance and chrominance components, etc.

[0129] As another example, various settings can be supported based on luma and chroma components. For example, the quadtree or binary tree partitioning structure determined by the luma component can also be used in the chroma component without encoding or decoding additional information. Alternatively, when independent partitioning of luma and chroma components is supported, quadtree + binary tree partitioning can be supported for the luma component, and quadtree partitioning can be supported for the chroma component. Alternatively, when quadtree + binary tree partitioning is supported in luma and chroma components, the partitioning support range may or may not be the same as or proportional to the luma and chroma components. For example, if the color format is 4:2:0, the partitioning support range of the chroma component may be N / 2 of the partitioning support range of the luma component.

[0130] As another example, different settings can be made according to the type of slice. For example, I slices can support quadtree partitioning, P slices can support binary tree partitioning, and B slices can support quadtree + binary tree partitioning.

[0131] Quadtree partitioning and binary tree partitioning can be set and supported based on various conditions as shown in the above examples. The above examples are not limited to the above conditions and may include situations where the conditions are reversed. They may include one or more factors mentioned in the above examples, or a combination thereof, and may also be transformed into another example. The allowable partition depth limit can be determined based on at least one factor or a combination of factors such as the partitioning method (quadtree, binary tree), slice type, luminance and chrominance components, encoding mode, etc. Furthermore, the partition support range can be determined based on at least one factor or a combination of factors such as the partitioning method (quadtree), slice type, luminance and chrominance components, encoding mode, etc., as well as related information that can be represented as the maximum and minimum values ​​of the partition support range. When the related information is constructed using a display flag, it can indicate the length information of each maximum and minimum value, or the difference information between the minimum and maximum values. For example, when the maximum and minimum values ​​are composed of powers of k (assuming k is 2), the power information of the maximum and minimum values ​​can be encoded using various binarization methods and transmitted to the decoding device. Alternatively, the difference between the powers of the maximum and minimum values ​​can be transmitted. The information transmitted in this case can be the power information of the minimum value and the power difference information.

[0132] According to the above description, information related to a flag may be generated and transmitted in units such as a sequence, a picture, a slice, a tile, or a block.

[0133] The partition flag shown in the above example can represent the block partition information by a quadtree or a binary tree or a mixture of the two tree schemes, and the partition flag can be encoded by various methods such as unary binarization and truncated unary binarization to send the relevant information to the decoding device. The bit stream structure of the partition flag used to represent the partition information of the block can be selected from one or more scanning methods. For example, you can construct the bit stream of the partition flag based on the partition depth order (from dep0 to dep_k), and you can also construct the bit stream of the partition flag based on whether the partition is performed. The method based on the partition depth order allows the partition information of the current level depth to be obtained based on the initial block, and then the partition information is obtained at the next level. The method based on whether the partition is performed indicates that additional partition information is preferably obtained in the partition block based on the initial block, and other scanning methods not shown in the above examples can be included and selected. In addition, according to this embodiment, the block partitioning unit can generate and represent index information of a predefined type of block candidate group other than the above-mentioned partition flag. The types of block candidates may include M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2, etc., for example, as types of partition blocks that may be included in a block before partitioning. When a candidate group of partition blocks is determined as described above, index information of the partition block type may be encoded by various methods such as fixed-length binarization, cut-off-type binarization, and truncation-type binarization. The candidate group of partition blocks may be determined based on at least one factor or a combination of factors such as partition depth, coding mode, prediction mode, size, type, sort, and slice type, allowable partition depth limit, and maximum and minimum allowable partition sizes. For further description, it is assumed that (M×N, M / 2×N / 2) is candidate list 1, (M×N, M / 2×N, M×N / 2, and M / 2×N / 2) is candidate list 2, (M×N, M / 2×N, M×N / 2) is candidate list 3, and (M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2) is candidate list 4. For example, in the case of M×N, when (M=N), the partition block candidates of candidate list 2 can be supported, and when (M≠N), the partition block candidates of candidate list 3 can be supported.

[0134] As another example, it is possible to support the partition block candidates of candidate list 2 when M or N of M×N is greater than or equal to the boundary value blk_th, and the partition block candidates of candidate list 4 when M or N is less than the boundary value blk_th. Alternatively, it is possible to support the partition block candidates of candidate list 1 when M or N is equal to or greater than or equal to the first boundary value blk_th_1, the partition block candidates of candidate list 1 when M or N is less than the first boundary value blk_th_1 and greater than or equal to the second boundary value blk_th_2, and the partition block candidates of candidate list 4 when M or N is less than the second boundary value blk_th_2.

[0135] As another example, when the encoding mode is intra prediction, the partition block candidates of candidate list 2 may be supported, and when the encoding mode is inter prediction, the partition block candidates of candidate list 4 may be supported.

[0136] Even when partition block candidates are supported as described above, the bit construction according to the binarization of each block may be the same or different. For example, as in the case of a partition flag, when the supported partition block candidates are limited according to the size or type of the block, the bit construction according to the binarization of the block candidate may vary. For example, in the case of (M>N), it is possible to support block types according to horizontal partitioning, i.e., M×N, M×N / 2, and M / 2×N / 2, wherein the binary bits of the index of M×N / 2 according to the partition block candidate group M×N, M / 2×N, M×N / 2, M / 2×N / 2, and M×N / 2 of the current condition may be different from each other. Depending on the type of block used for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., one of the partition flag method or the partition index method may be used to represent information related to the partition and type of the block. Furthermore, the block size limit and the allowable partition depth limit for supporting the partition and block type may be different according to the type of each block.

[0137] The encoding or decoding process of the block unit can be performed according to the process of prediction block determination, transform block determination, quantization block determination, entropy block determination, and in-loop filtering determination after the coding block is determined. The order of the encoding or decoding process is not always fixed, and some orders may be changed or excluded. The size and type of each block are determined according to the encoding cost of each candidate of the block size and type, and the image data of each determined block and the partition-related information such as the size and type of each determined block can be encoded.

[0138] The prediction unit 200 can be implemented using a prediction module as a software module, and can generate a prediction block using an intra-frame prediction scheme or an inter-frame prediction scheme for the block to be encoded. Here, the prediction block is understood to be a block that closely matches the block to be encoded using the term of pixel difference, and can be determined by various methods including the sum of absolute difference (SAD) and the sum of square difference (SSD). In addition, various syntaxes used when decoding the image block can be generated at this time. The prediction block can be classified into an intra-frame block and an inter-frame block according to the coding mode.

[0139] Intra-frame prediction (Intra-frame prediction) is a prediction technique that uses spatial correlation. It refers to a method of predicting the current block using reference pixels of a block that was previously encoded and decoded in the current picture. In other words, intra-frame prediction can use the luminance values ​​reconstructed by prediction and reconstruction as reference pixels in the encoder and decoder. Intra-frame prediction is effective for flat areas with continuity and areas with constant directionality, and because it uses spatial correlation, it can be used to ensure random access and prevent error diffusion.

[0140] Inter-frame prediction uses a compression technique that removes data redundancy by using time correlation by referencing an image encoded using one or more past or future pictures. That is, inter-frame prediction can generate a prediction signal with high similarity by referencing one or more past or future pictures. The encoder can use inter-frame prediction to search for a block that has a high correlation with the block to be encoded in the reference picture, and send the position information and residual signal of the selected block to the decoder. The decoder can use the selected information of the transmitted image to generate the same prediction block as the encoder, and compensate the transmitted residual signal to construct a reconstructed image.

[0141] Figure 4 is an exemplary view illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. Figure 5 is an exemplary view illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

[0142] In the video encoding method of this embodiment, since inter-frame prediction generates a prediction block from a previously encoded picture with high temporal correlation, encoding efficiency can be improved. Current (t) may refer to a current picture to be encoded, and may include a first reference picture t-1 having a first temporal distance (t-1) earlier than a picture order count (POC) of the current picture, and a second reference picture t-2 earlier than the first temporal distance of a temporal stream based on a picture or a picture order count (POC).

[0143] like Figure 4 As shown in , in the case of inter-frame prediction that can be used in the video encoding method of this embodiment, motion prediction is performed on blocks with high correlation by block matching between the current block of the current picture current (t) and the reference blocks of the reference pictures t-1 and t-2 to find the best prediction block from the earlier encoded reference pictures t-1 and t-2. In order to perform accurate prediction, an interpolation process is performed based on a structure in which at least one sub-pixel is arranged between two adjacent pixels to find the best prediction block, and then motion compensation is performed to find the resulting prediction block.

[0144] like Figure 5 As shown in , the inter-frame prediction that can be used in the video encoding method of this embodiment allows the generation of a prediction block from reference pictures t-1 and t+1 that have been temporally encoded in two directions based on the current picture current (t). In addition, two prediction blocks can be generated from one or more reference pictures.

[0145] When encoding an image using inter-frame prediction, the motion vector information and reference picture information of the optimal prediction block are encoded. In this embodiment, when generating a prediction block in either a unidirectional or bidirectional manner, the reference block lists are constructed differently so that the prediction block can be generated from the corresponding reference picture lists. Basically, reference pictures that exist before the current picture are assigned to list 0, and reference pictures that exist after the current picture are assigned to list 1. When reference picture list 0 cannot be filled to the permissible reference picture number of reference picture list 0, a reference picture that exists after the current picture can be assigned. Similarly, when reference picture list 1 cannot be filled to the permissible reference picture number of reference picture list 1, a reference picture that exists after the current picture can be assigned. Similarly, when constructing reference picture list 1, if reference picture list 1 does not satisfy the reference picture list, a reference picture that exists before the current picture can be assigned.

[0146] Figure 6 is an exemplary view illustrating a case where a prediction block is generated in a single direction in an image encoding and decoding method according to an embodiment of the present invention.

[0147] refer to Figure 6In the image encoding and decoding method according to this embodiment, the prediction block can be found from the previously encoded reference pictures t-1, t-2, and the prediction block can also be found from the area that has been encoded in the current picture current (t).

[0148] That is, the image encoding and decoding method according to this embodiment can be implemented so that not only is a prediction block generated from previously encoded pictures t-1 and t-2 with high temporal correlation, but also a prediction block with high spatial correlation is found. Finding a prediction block with such spatial correlation can correspond to finding a prediction block using intra-frame prediction. In order to perform block matching from an area already encoded in the current picture, the image encoding method of this embodiment can construct a syntax for information related to prediction candidates mixed with intra-frame prediction mode.

[0149] For example, when n (n is an arbitrary natural number) intra-frame prediction modes are supported, one mode is added to the intra-frame prediction candidate group to support n+1 modes, and the prediction mode can be encoded using M fixed bits that satisfy 2M-1≤n+1≤2M. In addition, one of the candidate groups of prediction modes with high probability, such as the most probable mode MPM of high-efficiency video coding HEVC, can be selected. It can also be preferentially encoded in higher-level prediction mode coding.

[0150] When generating a prediction block through block matching in the current picture, the video encoding method of this embodiment can construct a syntax for related information mixed with the inter-frame prediction mode. The additional related prediction mode information can be motion or displacement related information. The motion or displacement related information can include the best candidate among various vector candidates, the difference between the best candidate vector and the actual vector, the reference direction, the reference picture information, etc.

[0151] Figure 7 is an exemplary view of reference picture list construction in an image encoding and decoding method according to an embodiment of the present invention. Figure 8 is an exemplary view showing another example of a case where inter prediction is performed from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0152] refer to Figure 7 , the image encoding method according to an embodiment of the present invention can perform inter-frame prediction on the current block of the current picture current (t) according to the reference picture lists (reference list 0, reference list 1).

[0153] refer to Figure 7 and Figure 8, reference picture list 0 may be constructed from reference pictures earlier than the current picture (t), where t-1 and t-2 indicate reference pictures having a first temporal distance (t-1) and a second temporal distance (t-2), respectively, earlier than the picture order count (POC) of the current picture (t). In addition, reference picture list 1 may be composed of reference pictures later than the current picture (t), where t+1 and t+2 indicate reference pictures having a first temporal distance (t+1) and a second temporal distance (t+2), respectively, later than the POC of the current picture (t).

[0154] The above example of reference picture list construction shows an example in which the reference picture list is composed of reference pictures having a temporal distance (based on POC in this example) difference of 1, but it can be constructed so that the temporal distance difference between reference pictures is different. This can be confirmed in the example of reference picture list construction described later.

[0155] Prediction can be performed from the reference pictures in the list according to the slice type (I, P, or B). When a prediction block is generated in the current picture (current (t)) through block matching, the current picture can be added to the reference picture list (reference list 0, reference list 1) to perform inter-frame prediction coding.

[0156] like Figure 8 As shown in , it is possible to add the current picture (t) to the reference picture list 0 or to add the current picture (t) to the reference picture list 1. That is, the reference picture list 0 can be constructed by adding the reference picture of the temporal distance (t) to the reference picture earlier than the current picture (t), and the reference picture list 1 can be constructed by adding the reference picture of the temporal distance (t) to the reference picture later than the current picture (t).

[0157] For example, when constructing reference picture list 0, a reference picture before the current picture may be assigned to reference picture list 0, and then the current picture (t) may be assigned to reference picture list 0. When constructing reference picture list 1, a reference picture may be assigned to reference picture list 1, and then the current picture (t) may be assigned to reference picture list 1. Alternatively, when constructing reference picture list 0, the current picture (t) may be assigned, and then the reference picture before the current picture may be assigned. When constructing reference picture list 1, the current picture (t) may be assigned, and then the reference picture after the current picture may be assigned. Alternatively, when constructing reference picture list 0, the reference picture before the current picture may be assigned, the reference picture after the current picture may be assigned, and then the current picture (t) may be assigned. Similarly, when constructing reference picture list 1, the reference picture after the current picture may be assigned, the reference picture before the current picture may be assigned, and then the current picture (t) may be assigned. The above examples are not limited to the above cases, but may include cases where the conditions are reversed and may be transformed into examples of other cases.

[0158] Whether to include the current picture in each reference picture list (for example, not to add to the list, to add only to list 0, to add only to list 1, or to add to both lists 0 and 1) can be set in the encoder or decoder, and information related thereto can be transmitted in units of sequence, picture, slice, etc. The information can be encoded by methods such as fixed-length binarization, cut-off-type binarization, truncated binarization, etc.

[0159] and Figure 7 Unlike the method of , the image encoding and decoding method of this embodiment allows selecting a prediction block by performing block matching on a current picture (t), configuring a reference picture list including relevant information about the prediction block, and performing encoding and decoding on an image by using the reference picture list.

[0160] In reference picture list construction, the construction order and rules for each list, as well as the permissible number of reference pictures for each list, can be set differently. These can be determined based on at least one factor, such as whether the current picture is included in the list (including the current picture as a reference picture in inter-frame prediction), the slice type, list reconstruction parameters (which can be applied to lists 0 and 1 separately and together), the position of the group of pictures (GOP), temporal layer information (temporal ID), or a combination of at least one of these factors. The relevant information can be explicitly transmitted in units such as sequences and pictures. For example, in the case of P slices, regardless of whether the current picture is included in the list, reference picture list 0 can follow list construction rule A. In the case of B slices, reference picture list 0 including the current picture in the list can follow list construction rule B, reference picture list 1 can follow list construction rule C, reference picture list 0 not including the current picture in the list can follow list construction rule D, and reference picture list 1 can follow list construction rule E. The list construction rules B and D, as well as C and E, can be constructed in the same manner as described in the reference picture list construction example, or in a modified manner. As another example, when the current picture is included in the list, a first allowable number of reference pictures may be set, and when the current picture is not included in the list, a second allowable number of reference pictures may be set. The first allowable number of reference pictures and the second allowable number of reference pictures may be the same or different, and the difference between the first allowable number of reference pictures and the second allowable number of reference pictures may be set to 1 by default. As another example, when the current picture is included in the list and list reconstruction parameters are applied, all reference pictures in slice A may be included in the list reconstruction candidate group, and only some reference pictures in slice B may be included in the list reconstruction candidate group. In this case, slices A or B may be classified based on whether the current picture is included in the list, temporal layer information, slice type, position of the group of pictures (GOP), etc. Factors determining whether to be included in the candidate group may be based on the picture order count (POC) of the reference picture or reference picture index, the reference prediction direction (earlier or later than the current picture), whether the current picture exists, etc.

[0161] According to the above construction, since a reference block encoded with inter prediction can be used in the current picture, inter prediction can be allowed or used even in motion estimation of an I slice.

[0162] Furthermore, when constructing a reference picture list, the index assignment or list construction order may differ depending on the slice type. In the case of an I slice, low indices (e.g., idx = 0, 1, 2) are used by increasing the priority level of the current picture (t), and the number of bits in image coding can be reduced by binarization (fixed-length binarization, cut-off binarization, truncation binarization, etc.) that maximizes the allowable number of reference pictures C in the reference picture list. In the case of a P slice or a B slice, when it is determined that the probability of selecting the reference picture of the current block as a prediction candidate by performing block matching in the current picture is lower than the probability of selecting the prediction candidate using other reference pictures, the number of bits in image coding can be reduced through various methods, including setting the priority of block matching for the current picture backward and setting the allowable number of reference pictures in the corresponding reference picture list using high indices (e.g., idx = C, C-1) to the maximum. In the above example, the priority setting of the current picture can be constructed in the same or modified manner as described in the reference picture list construction example. By not constructing a reference picture list based on a slice type (e.g., an I slice), information related to reference pictures can be omitted. For example, a prediction block can be generated using existing inter-frame prediction, but inter-frame prediction information can be expressed using information obtained by excluding reference picture information from motion information in inter-frame prediction mode.

[0163] The method for performing block matching in the current picture can determine whether block matching is supported based on the slice type. For example, block matching in the current block may be supported by I slices, but not P slices or B slices, or other variations are possible. In addition, the method for supporting block matching in the current picture can determine whether block matching is supported in units of pictures, slices, tiles, etc., or based on the position of the group of pictures (GOP), temporal layer information (temporal ID), etc. Such setting information can be sent from the image encoding process or encoder to the decoder in units of sequences, pictures, slices, etc. In addition, even if the above-mentioned setting information or syntax exists in a high-level unit and the setting-related operation is turned on, when the same setting information or syntax exists in a low-level unit, the setting information of the low-level unit can have priority over the setting information in the high-level unit. For example, when processing the same or similar setting information in sequences, pictures, or slices, the picture unit can have priority over the sequence unit, and the slice unit can have priority over the picture unit.

[0164] Figure 9 is an exemplary view illustrating intra prediction in an image encoding method according to an embodiment of the present invention.

[0165] refer to Figure 9 As shown, the intra-frame prediction method according to an embodiment of the present invention includes reference sample filling, reference sample filtering, intra-frame prediction and boundary filtering.

[0166] The reference pixel filling step may be an example of a reference pixel construction step, the reference pixel filtering step may be referred to as a reference pixel filter unit, intra-frame prediction may include a prediction block generation step and a prediction mode encoding step, and boundary filtering may be an embodiment of a post-processing filter step.

[0167] That is, the intra prediction performed in the image encoding apparatus of this embodiment may include a reference pixel construction step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors such as block size, block type, block position, prediction mode, prediction method, quantization parameter, and the order of the above steps, one or some of the above steps may be omitted, and other steps may be added.

[0168] The reference pixel construction step, the reference pixel filtering step, the prediction block generation step, the prediction mode encoding step, and the post-processing filtering step can be implemented so that the software module stored in the memory is executed by the processor connected to the memory. Therefore, in the following description, for the convenience of description, the reference pixel construction unit, the prediction block generation unit, the prediction mode encoding unit, and the post-processing filter unit will be referred to as the execution subject of each step, as a functional unit generated by a combination of a software module that implements each step and a processor that executes it, or a construction unit that performs the function of the functional unit.

[0169] More specifically, the reference pixel construction unit constructs reference pixels for predicting the current block by reference pixel padding. When a reference pixel does not exist or is unavailable, reference pixel padding can perform methods such as copying the value from the nearest available pixel so that the value is used as the reference pixel. The decoded picture buffer (DPB) can be used to copy the value.

[0170] That is, intra prediction uses reference pixels of previously coded blocks of the current picture to perform prediction. To this end, in the reference pixel construction step, the neighboring pixels of the current block, such as the neighboring pixels of the blocks to the left, above the left, above, and above the right, are mainly used as reference pixels. When scanning such as reverse z-scan is used as the coding order scanning method, the candidate group of neighboring blocks for reference pixels is only an example in which the coding order of blocks follows raster scanning or z-scan, and neighboring pixels of blocks other than the above blocks, such as the neighboring pixels of the blocks to the right, below the right, and below, can also be used as reference pixels.

[0171] Furthermore, depending on the implementation, additional pixels besides directly neighboring pixels may be blended with alternative or existing reference pixels according to the step-by-step construction of the intra prediction.

[0172] In addition, when prediction is performed in a directional mode among intra-frame prediction modes, reference pixels in decimal units can be generated by performing linear interpolation on the reference pixels in integer units. Modes for performing prediction using reference pixels in integer unit positions include some modes with vertical, horizontal, 45 degrees and 135 degrees, and there is no need to generate reference pixels in decimal units for the above prediction modes. Prediction modes with other directions other than the above prediction modes are interpolated with interpolation precision of 1 / 2 powers such as 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32 and 1 / 64, and can have precision of multiples of 1 / 2. This is because the interpolation precision can be determined based on the number of supported prediction modes or the prediction direction of the prediction mode. Fixed interpolation precision can always be supported in pictures, slices, tiles, blocks, etc., and adaptive interpolation precision can be supported based on the size of the block, the type of block, and the prediction direction of the supported mode. At this time, the direction of the prediction mode may be represented by tilt information or angle information based on a specific line (eg, a positive x-axis on a coordinate plane) in a direction indicated by the mode.

[0173] As an interpolation method, linear interpolation can be performed using directly adjacent integer pixels, but other interpolation methods can also be supported. For the purpose of interpolation, one or more filter types and numbers of taps can be supported, such as a 6-tap Wiener filter, an 8-tap Kalman filter, etc., and the type of interpolation to be performed can be determined based on information such as block size and prediction direction. In addition, relevant information can be transmitted in units of sequence, picture, slice, block, etc.

[0174] The reference pixel filter unit can filter the reference pixels after they are constructed to improve prediction efficiency by reducing the residual degradation in the encoding process. The reference pixel filter unit can implicitly or explicitly determine the type of filter and whether to apply filtering based on the size, type, and prediction mode of the block. That is, even in filters with the same tap, the filter coefficients can be determined differently depending on the filter type. For example, 3-tap filters such as [1, 2, 1] / 4 and [1, 6, 1] / 8 can be used.

[0175] In addition, the reference pixel filter unit can determine whether to apply filtering by determining whether to additionally transmit a bit. For example, in the implicit case, the reference pixel filter unit can determine whether to apply filtering based on the characteristics (discreteness, standard deviation, etc.) of the pixels in the peripheral reference block.

[0176] In addition, when the relevant flag satisfies the concealment condition predetermined in the residual coefficient, intra prediction mode, etc., the reference pixel filter unit can determine whether to apply filtering. For example, the number of taps of the filter is 3 taps such as [1, 2, 1] / 4 for a small block (blk) and 5 taps such as [2, 3, 6, 3, 2] / 16 for a large block (blk), and the number of applications can be determined according to whether filtering is performed, whether filtering is performed once, or whether filtering is performed twice.

[0177] Furthermore, the reference pixel filter unit can generally apply filtering to the reference pixels that are closest to the current block. Additional reference pixels other than the closest reference pixels can also be considered in the filtering process. For example, filtering can be applied to the additional reference pixels by replacing the closest reference pixels, or filtering can be applied by blending the additional reference pixels with the closest reference pixels.

[0178] Filtering can be applied fixedly or adaptively, and this can be determined based on at least one factor or a combination of factors such as the size of the current block or the size of the adjacent block, the coding mode of the current block or the adjacent block, the block boundary properties of the current block or the adjacent block (for example, it is the boundary of the coding unit or the boundary of the transform unit), the prediction mode or direction of the current block or the adjacent block, the prediction method of the current block or the adjacent block, the quantization parameter, etc. The determination can be made so that the encoder or decoder has the same settings (implicitly), or (explicitly) considering the coding cost. Typically, the filter applied is a low-pass filter. Depending on the various factors mentioned above, the number of filter taps, filter coefficients, whether the filter flag is encoded, and the number of filter applications can be determined. Its information is set in units of sequence, slice, block, etc., and the relevant information is sent to the decoder.

[0179] The prediction block generation unit may generate a prediction block by an extrapolation method using reference pixels in intra prediction, an interpolation method such as an average value DC of reference pixels or a planar pattern, and a reference pixel duplication method. When duplicating reference pixels, one reference pixel may be duplicated to generate one or more predicted pixels, or one or more reference pixels may be duplicated to generate one or more predicted pixels, and the number of duplicated reference pixels may be equal to or smaller than the number of predicted pixels.

[0180] Directional prediction methods and non-directional prediction methods can be classified according to the prediction method. In particular, directional prediction methods can be classified into linear directional methods and curved directional methods. The linear directional method uses an extrapolation method to generate pixels of a prediction block by using reference pixels located on a prediction direction line. The curved directional method uses an extrapolation method to generate pixels of a prediction block by using reference pixels located on a prediction direction line, and refers to a method that allows partial prediction directions of pixel units to be changed by considering the detailed directionality of the block (e.g., edge < edge >). In the case of the directional prediction mode in the image encoding and decoding method of the present invention, the linear directional method will be mainly described. In addition, in the case of the directional prediction method, the intervals between adjacent prediction modes may be equal or unequal, which may be determined according to the size or type of the block. For example, when the current block is obtained by a block partition unit with a size and type of M×N, if M and N are the same, the intervals between the prediction modes may be uniform, and if M and N are different, the intervals between the prediction modes may be uneven. As another example, when M is greater than N, a mode with vertical directionality can allow finer intervals to be allocated between prediction modes that are close to the vertical mode (90 degrees), and wider intervals to be allocated to modes that are not close to the vertical mode. When N is greater than M, a mode with horizontal directionality can allow finer intervals to be allocated between prediction modes that are close to the horizontal mode (180 degrees), and wider intervals to be allocated to modes that are not close to the horizontal prediction mode. The above examples are not limited to the above cases, but may include cases where the conditions are reversed from each other, and examples of other cases are also allowed. In this case, the intervals between prediction modes can be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode can be numerically expressed by tilt information or angle information of the direction.

[0181] Furthermore, the prediction block may be generated by other methods using spatial correlation in addition to the above-described methods. For example, a reference block may be generated as a prediction block using an inter-frame prediction method such as motion search and compensation using the current picture as a reference picture. The prediction block generation step may generate a prediction block using reference pixels according to the prediction method. That is, the prediction block may be generated by directional prediction or non-directional prediction such as extrapolation, interpolation, replication, and averaging of existing intra-frame prediction methods according to the prediction method, the prediction block may be generated using an inter-frame prediction method, and the prediction block may be generated by other additional methods.

[0182] Intra-frame prediction methods can be supported under the same settings of the encoder or decoder, and the intra-frame prediction method can be determined based on the slice type, block size, block type, etc. Intra-frame prediction methods can be supported based on at least one of the above prediction methods or a combination thereof. Intra-frame prediction modes can be constructed based on the supported prediction methods. The number of supported intra-frame prediction modes can be determined based on the prediction method, slice type, block size, block type, etc. Related information can be set and transmitted in units of sequences, pictures, slices, blocks, etc.

[0183] The prediction mode encoding step performed via the prediction mode encoding may determine a mode that is optimal in terms of encoding cost according to encoding cost of each prediction mode as the prediction mode of the current block.

[0184] As an example, for the purpose of reducing prediction mode bits, the prediction mode encoding unit may use one or more neighboring block modes for predicting the current block mode. The prediction mode may be included in a candidate group of most probable modes (MPMs) that may be the same as the candidate block mode, and the modes of the neighboring blocks may be included in the candidate group. For example, prediction modes of blocks such as the left, upper left, lower left, upper, and upper right blocks of the current block may be included in the above-mentioned candidate group.

[0185] The candidate group of prediction modes can be constructed based on at least one factor or a combination of factors such as the position of adjacent blocks, the priority of adjacent blocks, the priority of partitioned blocks, the size or type of adjacent blocks, a preset specific mode (in the case of chroma blocks), and the prediction mode of luminance blocks, and the relevant information can be sent in units of sequence, picture, slice, block, etc.

[0186] For example, when a block adjacent to a current block is divided into two or more blocks, it is possible to determine which block mode among the divided blocks is included as a mode prediction candidate for the current block under the same setting of the encoder or decoder. For example, the left block of the neighboring block of the current block (M×M) performs quadtree partitioning in the block partitioning unit so that the partitioned block is three blocks, and when M / 2×M / 2, M / 4×M / 4, and M / 4×M / 4 blocks are included from top to bottom, based on the block size, the prediction mode of the M / 2×M / 2 block can be included as a mode prediction candidate for the current block. As another example, the upper block among the neighboring blocks of the current block (N×N) performs binary tree partitioning in the block partitioning unit so that the partitioned block is three blocks, and when N / 4×N, N / 4×N, and N / 2×N blocks are included from left to right, according to a predetermined order (priority assigned from left to right), the prediction mode of the first N / 4×N block from the left can be included as a mode prediction candidate for the current block.

[0187] As another example, when the prediction mode of a block adjacent to the current block is a directional prediction mode, a prediction mode adjacent to the prediction direction of the corresponding mode (in terms of tilt information or angle information of the mode direction) may be included in the mode prediction candidate group of the current block. In addition, according to the prediction mode construction or the combination of adjacent blocks, a predetermined mode (plane, DC, vertical, horizontal, etc.) may be preferably included. In addition, a prediction mode with a high frequency of occurrence among the prediction modes of the adjacent blocks may be preferably included. The priority order indicates not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index in the candidate group construction (that is, a higher probability of being assigned fewer bits in the binarization process).

[0188] As another example, when the maximum number of mode prediction candidate groups for the current block is k, where the left block consists of m blocks whose lengths are smaller than the vertical length of the current block, and the upper block consists of n blocks whose lengths are smaller than the horizontal length of the current block, if the sum of the divided blocks of the adjacent blocks (m+n) is greater than k, it is possible to fill the candidate group according to a predetermined order (from left to right, from top to bottom), and if the sum of the divided blocks of the adjacent blocks (m+n) is greater than the maximum value k of the candidate group, then the prediction mode of the block other than the adjacent block (left block, upper block, etc.) in the prediction mode of the adjacent block can be included in the mode prediction candidate group of the current block. The above examples are not limited to the above cases, but may include cases where the conditions are reversed, and examples of other cases are also allowed.

[0189] In this way, the candidate blocks used to predict the current block mode are not limited to a specific block position, but can utilize prediction mode information from at least one of the blocks located to the left, above the left, below the left, above, and above the right. As in the above example, various factors can be considered to construct the prediction mode candidate group for the current block.

[0190] The prediction mode encoding unit can classify a group of most probable mode (MPM) candidates that are likely to be the same as the mode of the current block (referred to as candidate group 1 in this example) and a group of mode candidates in other situations (referred to as candidate group 2 in this example). The prediction mode encoding process can be changed depending on which candidate group the prediction mode of the current block belongs to. The total number of prediction modes can be composed of the sum of the prediction modes of candidate group 1 and the prediction modes of candidate group 2. The number of prediction modes in candidate group 1 and the number of prediction modes in candidate group 2 are determined based on at least one factor or a combination of the total number of prediction modes, the block type, etc. The same binarization or other binarization can be applied to each candidate group. For example, fixed-length binarization can be applied to candidate group 1, and cut-off binarization can be applied to candidate group 2. In the above description, although the number of candidate groups is illustrated as two, it is possible to expand to, for example, a first mode candidate group with a high probability of being the same as the mode of the current block, a second mode candidate group with a high probability of being the same as the mode of the current block, and other mode candidate groups, and variations are also allowed.

[0191] Considering the high correlation between reference pixels adjacent to the boundary between the current block and the adjacent blocks and the pixels in the current block, in the post-processing filtering step performed by the post-processing filter unit, some predicted pixels of the prediction block generated in the previous process can be replaced with values ​​generated by filtering one or more reference pixels and one or more predicted pixels adjacent to the boundary, and the predicted pixels can be replaced with values ​​obtained by applying values ​​obtained by digitizing features between reference pixels adjacent to the block boundary (for example, differences in pixel values, tilt information, etc.) to the filtering process. In addition to the above methods and with similar purposes, other methods (correcting some predicted pixels of the prediction block by reference pixels) can be added. In the post-processing filter unit, a filter and whether to apply the filter can be determined implicitly or explicitly. The position and number of reference pixels and current pixels used in the post-processing filter, the prediction mode applied, etc. can be set in the encoder or decoder, and the relevant information can be transmitted in units of sequence, picture, slice, etc.

[0192] In addition, in the post-processing filtering step, an additional post-processing step may be performed after the prediction block is generated, as in the block boundary filtering. In addition, after obtaining the residual signal, the post-processing filtering step may be performed on the current block reconstructed by adding the residual signal and the prediction signal obtained through the transformation process, the quantization process, and the dequantization process, taking into account the characteristics of the adjacent predicted pixels similar to the boundary filtering.

[0193] Finally, a prediction block is selected or obtained through the above process. The information obtained from the process may include information related to the prediction mode, and may be sent to the transform unit 210 for encoding the residual signal after the prediction block is obtained.

[0194] Figure 10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

[0195] refer to Figure 10 The image encoding method according to this embodiment may include a motion estimation step and an interpolation step. Information on a motion vector, a reference picture index, and a reference direction generated in the motion estimation step may be sent to the interpolation step. In the motion estimation step and the interpolation step, values ​​stored in a decoded picture buffer (DPB) may be used.

[0196] That is, the image coding device may perform motion estimation to find a block similar to the current block in a previously encoded picture. Furthermore, the image coding device may perform interpolation on a reference picture for prediction with finer precision than decimal units. Finally, the image coding device obtains a predicted block through a predictor, where information obtained in this process includes a motion vector, a reference picture index or reference index, a reference direction, and the like, and then performs residual signal encoding.

[0197] In this embodiment, intra prediction is performed in a P slice or a B slice, so that Figure 9 The shown method supports a combination of inter-frame prediction and intra-frame prediction.

[0198] Figure 11 is an exemplary view showing a process of obtaining a prediction block.

[0199] refer to Figure 11 , an image encoding method according to an exemplary embodiment of the present invention includes the steps of reference sample filling, reference sample filtering, intra-frame prediction, boundary filtering, motion estimation, and interpolation.

[0200] In the case where the image coding device supports block matching in the current picture, the prediction method in the I slice can be Figure 11 Instead of Figure 9 . That is, the image encoding device can use not only the prediction mode in the I slice, but also information such as the motion vector, reference picture index, and reference direction that only appears in the P slice or B slice to generate the prediction block. However, since the reference picture is a feature of the current picture, some information may be omitted. For example, when the reference picture is the current picture, the reference picture index and reference direction can be omitted.

[0201] In addition, in an image encoding device, when interpolation is applied, since blocks matching decimal units may not be necessary due to the nature of artificial images such as computer graphics, whether to perform block matching is set in the encoder, and units such as sequences, pictures, and slices can be set.

[0202] For example, the image coding device may not interpolate reference pictures used for inter-frame prediction according to the encoder's settings, and may implement various settings, such as not interpolating only when block matching is performed in the current picture. In other words, the image coding device of this embodiment can be configured to determine whether to perform interpolation of reference pictures. In this case, it can be determined whether to perform interpolation for all or some of the reference pictures that make up the reference picture list. For example, when block matching in decimal units is not required because the image of the reference block has the characteristics of an artificial image, the image coding device does not perform interpolation, and when block matching in decimal units is required for a specific current block due to a natural image, interpolation is performed.

[0203] Furthermore, the image encoding device can set whether to apply block matching to a reference picture for interpolation in block units. For example, when a natural image and an artificial image are mixed, interpolation is performed on the reference picture. When the optimal motion vector can be obtained by searching a portion of the artificial image, the motion vector can be expressed in a specific unit (here, it is assumed to be an integer unit), and when the optimal motion vector can be obtained by selectively searching a portion of the natural image, the motion vector can be expressed in another specific unit (here, it is assumed to be a quarter unit).

[0204] Figure 12 is an exemplary view illustrating a main procedure of an image encoding method according to an embodiment of the present invention using syntax in encoding units.

[0205] refer to Figure 12 , curr_pic_BM_enabled_flag indicates a flag that allows block matching in the current picture, and can be defined and transmitted in units of sequences and pictures. At this time, the process of generating a prediction block by performing block matching on the current picture can represent a case where an operation is performed through inter-frame prediction. It can be assumed that cu_skip_flag, which is an inter-frame technique for not encoding the residual signal, is a flag supported only by P slices or B slices other than I slices. In this case, when curr_pic_BM_enabled_flag is turned on, I slices can also support block matching (BM) in inter-frame prediction mode.

[0206] That is, the image encoding apparatus of this embodiment can support skipping when generating a prediction block for the current picture using block matching, and can also support skipping using on-screen techniques other than block matching. Depending on such conditions, skipping may not be supported in an I slice. Whether to perform skipping can be determined based on encoder settings.

[0207] For example, when an I slice supports skipping, a prediction block can be directly reconstructed as a reconstructed block by block matching without encoding the residual signal through prediction_unit() linked to the prediction unit with if(cu_skip_flag) as a specific flag. In addition, the image encoding device classifies the method of using a prediction block by block matching in the current picture as an inter-frame prediction technique, and can handle such classification by pred_mode_flag as a specific flag.

[0208] That is, if pred_mode_flag is 0, the image encoding device sets the prediction mode to the inter-frame prediction mode (MODE_INTER), and if pred_mode_flag is 1, the prediction mode can be set to the intra-frame prediction mode (MODE_INTRA). This is an intra-frame technique similar to the existing technology, but it can be classified as an intra-frame technique or an inter-frame technique in an I slice to distinguish it from the existing structure. That is, the image encoding device of this embodiment can use a temporal correlation structure without using temporal correlation in an I slice. part_mode indicates information about the size and type of the blocks divided in the coding unit.

[0209] Figure 13 is an exemplary view showing an example of supporting symmetric type partitioning or asymmetric type partitioning as inter prediction when a prediction block is generated by block matching in a current picture.

[0210] refer to Figure 13 In the case of generating a prediction block by block matching in the current picture, as in inter-frame prediction, the image encoding method according to the present exemplary embodiment can support symmetric partition types such as 2N×2N, 2N×N, and N×2N, and asymmetric partition types such as nL×2N, nR×2N, 2N×nU, and 2N×nD. Various block sizes and types can be determined according to the partitioning method of the block partitioning unit.

[0211] Figure 14 is an exemplary view showing that 2N×2N and N×N can be supported in inter prediction.

[0212] refer to Figure 14, the image encoding method according to this embodiment can support 2N×2N and N×N as the prediction block types used in conventional intra prediction. This is an example of a square supported by a quadtree partitioning method according to a predefined block candidate group in a block partitioning unit or a partitioning method, and other block types that can be supported by adding a rectangular shape to a binary tree partitioning method or a predefined block candidate group in intra prediction, where the setting can be set in the encoder. The encoder can also set whether to apply skipping only when performing block matching on the current picture, whether to apply skipping to existing intra predictions, or whether to apply skipping to new intra predictions. Information related thereto can be sent in units of sequence, picture, slice, etc.

[0213] The subtraction unit 205 may generate a residual block by subtracting pixel values ​​of the prediction block generated from the prediction unit 200 from pixel values ​​of the current block to be encoded to derive pixel difference values.

[0214] The transform unit 210 receives the residual block, which is the difference between the current block and the prediction block generated by intra-frame prediction or inter-frame prediction, from the subtraction unit 205 and transforms the residual block into the frequency domain. Through the transform process, each pixel of the residual block corresponds to the transform coefficient of the transform block. The size and type of the transform block can be equal to or smaller than the size of the coding unit. Furthermore, the size and type of the transform block can be the same as or smaller than the prediction unit. The image encoding device can perform the transform process by grouping several prediction units.

[0215] The size or type of the transform block can be determined by the block partitioning unit, and square or rectangular transforms can be supported according to the block partitioning. The block partitioning operation can be affected by the transform-related settings supported by the encoder or decoder (supported transform block size, type, etc.).

[0216] The size and type of each transform block are determined according to the cost of each candidate of the size and type of each transform block, and split information such as the determined image data of each transform block and the determined size and type of each transform block may be encoded.

[0217] The transformation can be performed using a one-dimensional transformation matrix (DCT, DST, etc.). Each transformation matrix can be adaptively used for horizontal and vertical units. Examples of adaptive use can be determined by various factors such as block size, block type, block type (luminance and chrominance), coding mode, prediction mode information, quantization parameter, and coding information of neighboring blocks.

[0218] For example, in the case of intra prediction, when the prediction mode is horizontal, a DCT-based transform matrix can be used in the vertical direction, and a DST-based transform matrix can be used in the horizontal direction. When the prediction mode is vertical, a DCT-based transform matrix can be used in the horizontal direction, and a DST-based transform matrix can be used in the vertical direction. The transform matrix is ​​not limited to that described above. Information related thereto can be determined using an implicit or explicit method, and can be determined based on one or more factors or a combination thereof such as the block size, block type, coding mode, prediction mode, quantization parameter, coding information, etc. of adjacent blocks, and the relevant information can be sent in units of sequence, picture, slice, block, etc.

[0219] Here, considering the case of using an explicit method, when two or more transformation matrices for horizontal and vertical directions are used as candidate groups, information about the transformation matrix used for each direction can be transmitted separately, or information about the transformation matrix used for each of the horizontal and vertical directions and information about the transformation matrix used for the horizontal and vertical directions can be transmitted using two or more pairs as candidate groups.

[0220] In addition, taking into account the characteristics of the image, partial or overall transformations can be omitted. For example, one or both of the horizontal or vertical components can be omitted. Since intra-frame prediction or inter-frame prediction cannot be performed well, when the difference between the current block and the predicted block is large (that is, when the residual component is large), the coding loss due to the transformation may increase. This can be determined based on at least one factor or a combination of coding mode, prediction mode information, block size, block type, block type (luminance and chrominance), quantization parameter, and coding information of adjacent blocks. The above conditions can be expressed using implicit or explicit methods, and information related thereto can be sent in units of sequence, picture, slice, etc.

[0221] The quantization unit 215 performs quantization of the residual component transformed by the transform unit 210. A quantization parameter is determined on a block basis, and the quantization parameter may be set in units of a sequence, a picture, a slice, a block, or the like.

[0222] For example, the quantization unit 215 may predict the current quantization parameter using one or more quantization parameters derived from neighboring blocks such as left, top left, top, top right, and bottom left of the current block.

[0223] When there is no quantization parameter predicted from a neighboring block, that is, when the block is located at a boundary of a picture, a slice, etc., the quantization unit 215 may output or transmit a difference value from basic parameters transmitted in units of a sequence, a picture, a slice, etc. When there is a quantization parameter estimated from a neighboring block, the difference value may be transmitted using the quantization parameter of the corresponding block.

[0224] The priority of the blocks from which the quantization parameters are to be derived can be preset or sent in units of sequences, pictures, slices, etc. The residual block can be quantized using dead zone uniform threshold quantization (DZUTQ), a quantization weighting matrix, or an improved scheme. One or more quantization schemes may be used as candidates, and the scheme may be determined by the coding mode, prediction mode information, etc.

[0225] For example, the quantization unit 215 can set a quantization weighting matrix to be applied to inter-frame coding, intra-frame coding units, etc., and can also set different weighting matrices according to the intra-frame prediction mode. Assuming that the size of the block is equal to the quantization block size of size M×N, the quantization weighting matrix can be constructed by changing the quantization coefficient of each position of each frequency component. The quantization unit 215 can select one of various existing quantization methods and can be used under the same settings of the encoder or decoder. Information related thereto can be sent in units of sequence, picture, slice, etc.

[0226] at the same time, Figure 2 and 3 The dequantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIG can be implemented by inversely performing the processes in the inverse transform unit 210 and the quantization unit 215. That is, the dequantization unit 220 can dequantize the quantized transform coefficients generated by the quantization unit 215, and the inverse transform unit 225 can inversely transform the dequantized transform coefficients to generate a reconstructed residual block.

[0227] Figure 2 and Figure 3 The addition units 230 and 324 shown in FIG can generate a reconstructed block by adding the pixel values ​​of the prediction block generated by the predictor to the pixel values ​​of the reconstructed residual block. The reconstructed block can be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

[0228] In-loop filters such as deblocking filters, sample adaptive offset (SAO), and adaptive loop filters (ALF) can be applied to the reconstructed block. The deblocking filter can filter the reconstructed block to eliminate distortion between block boundaries that occur during encoding and decoding. SAO is a filter process that reconstructs the difference between the original image and the reconstructed image with an offset on a pixel-by-pixel basis relative to the residual block. ALF can perform filtering to minimize the difference between the predicted block and the reconstructed block. ALF can perform filtering based on a comparison value between the reconstructed block and the current block through the deblocking filter.

[0229] The entropy encoding unit 245 may perform entropy encoding on the transform coefficient quantized by the quantization unit 215. For example, a technique such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) encoding, etc. may be performed.

[0230] The entropy coding section 245 may include the bitstream obtained by encoding the quantized coefficients and various information required for decoding the encoded bitstream as the encoded data. The encoded data may include the coding block type, the quantized coefficients, the bitstream encoding the quantized block, and information required for prediction. In the case of quantized coefficients, the two-dimensional quantized coefficients may be scanned in one dimension. The quantized coefficients may vary depending on the characteristics of the image. In particular, in the case of intra-frame prediction, since the distribution of the coefficients may have a specific distribution depending on the prediction mode, the scanning method may be set differently.

[0231] In addition, the entropy coding section 245 can be configured differently depending on the size of the block to be encoded. The scanning pattern can be preset to have at least one of various patterns such as a zigzag, a diagonal line, and a raster, or can be set to a candidate having at least one of various patterns such as a zigzag, a diagonal line, and a raster. It can be determined by the encoding mode, prediction mode information, etc., and can be used under the same settings of the encoder and decoder. Information related thereto can be transmitted in units of sequences, pictures, slices, etc.

[0232] The size of the quantized block (hereinafter referred to as the quantized block) input to the entropy encoding section 245 may be equal to or smaller than the size of the transform block. In addition, the quantized block may be divided into two or more sub-blocks. When the quantized block is divided, the scanning pattern in the divided block may be set to be the same as or different from the scanning pattern of the existing quantized block.

[0233] For example, when the scanning pattern of an existing quantization block is called a zigzag pattern, the zigzag pattern may be applied to all sub-blocks, or the zigzag pattern may be applied to the sub-block located to the upper left of the block containing the DC component, and the diagonal pattern may be applied to the other blocks. This may also be determined based on the coding mode, prediction mode information, etc.

[0234] The start position of the scanning pattern in the entropy coding section 245 is basically from the upper left corner, but may be from the upper right corner, the lower right corner, or the lower left corner depending on the characteristics of the image. Information on which one or more candidate groups have been selected may be transmitted in units of sequence, picture, slice, etc. Entropy coding technology may be used as the coding technology, but is not limited thereto.

[0235] at the same time, Figure 2 and Figure 3 The dequantization of the dequantization unit 220 and the inverse transformation of the inverse transformation unit 225 shown in are constructed inversely to the quantization of the quantization unit 215 and the transformation of the transformation unit 210 and can be implemented by combining the basic filters 235 and 330.

[0236] Hereinafter, the formation of the above-mentioned reference pixels will be described in more detail.

[0237] When performing intra prediction according to an embodiment of the present invention, the availability of reference pixels of adjacent blocks must be checked. The availability of reference pixels can be determined based on at least one factor or a combination of factors such as the position of the adjacent block, the position of the current block in the partitioned block, and the boundary characteristics of the adjacent blocks. For example, when an M×M block is divided into four M / 2×M / 2 blocks (in this example, 0, 1, 2, and 3 indexes are assigned in the raster scan direction) by a block partitioning unit, and the current block is a number of 3 M / 2×M / 2 blocks, although the adjacent blocks located in a specific direction (in this example, the upper right corner and the lower left corner in this example) belong to the current block in units of pictures, slices, or tiles, the corresponding blocks have not yet been encoded, making it determined that the blocks are unavailable, and thus the reference pixels of the blocks may not be used. As another example, whether to use the reference pixels of the adjacent blocks can be determined based on the boundary characteristics between the current block and the adjacent blocks (boundaries of slices, tiles, etc.).

[0238] Another example of using reference pixels in adjacent blocks can support settings for determining whether to use data with a high error propagation probability in the encoder / decoder in order to avoid error propagation over time in the case of P slices or B slices. Based on this setting information, it is possible to determine whether to use data with a high error propagation probability for generating prediction blocks, and flags such as constrained_intra_pred_flag (CIP) of HEVC can perform a similar role. This flag can determine whether to use adjacent blocks encoded in inter-frame prediction mode with a high error propagation probability among adjacent blocks referenced in generating intra-frame prediction blocks. If the flag is 0, the reference pixels of the block encoded with inter-frame prediction may not be used, and if the flag is 1, the reference pixels of the block encoded with inter-frame prediction may be used.

[0239] The following example shows the case where a prediction block is generated when the flag is not zero. The additional setting of the flag causes the reference pixels to be unavailable, regardless of the error propagation possibility, whereby any reference pixels may not be used for prediction. In this case, since the prediction block may not use any reference pixels, the prediction block can be generated by filling a preset value (e.g., the middle value of the pixel value range of the bit depth). Alternatively, it is possible to generate a prediction block to which edge information is applied based on a predefined value. For example, if the predefined value is A, based on an arbitrary pixel in an M×N block, the right area including the arbitrary pixel is filled with (A+B) and the other area is filled with (AC) to generate a prediction block, where B and C represent parameters for representing edge information.

[0240] Furthermore, by additionally setting a flag, the error propagation probability can be determined by the number of adjacent blocks encoded using inter-frame prediction, the number of reference pixels of the adjacent blocks, and the like. Based on this determination, it is possible to determine whether to use the reference pixels of a block encoded using intra-frame prediction. For example, if fewer than N blocks among the M adjacent blocks adjacent to the current block are blocks encoded using inter-frame prediction, the reference pixels of the block may not be used to generate the prediction block, and the reference pixels of (MN) blocks encoded using intra-frame prediction may also not be used to generate the prediction block. Since the number of blocks as a determination criterion may be affected by the size or type of the block, it may be replaced by the number of reference pixels present at the position to be used for predicting the current block, and other factors may be considered.

[0241] The generated prediction block may be affected depending on whether reference pixels are used. For example, if at least one of the neighboring blocks of the current block is available depending on whether reference pixels are used, the reference pixels of the unavailable block may be replaced by pixels derived (e.g., copied, averaged, etc.) from the reference pixels of the available block. As another example, when no available block exists depending on whether reference pixels are used, the reference pixels of the unavailable block may be replaced with a predefined value (e.g., a middle value of the pixel value range of the bit depth).

[0242] The replacement of values ​​derived from available blocks may be disabled according to the settings of the encoder or decoder. In this case, the prediction mode derived from the corresponding block may not be used.

[0243] Hereinafter, these contents will be described in more detail with reference to the accompanying drawings.

[0244] Figure 15 is an exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0245] refer to Figure 15, it is possible to determine whether the reference pixels of the corresponding block are used to generate the prediction block according to the encoding mode of the neighboring blocks of the current block. In this case, the neighboring blocks can be in the shape of an M×M square or an M×N rectangle obtained by the block division unit, and can be encoded using one of intra prediction and inter prediction according to the encoding mode. Figure 15 As shown in , when the current block is the middle block, the block located at the upper left is encoded using inter-frame prediction (inter), the two blocks located at the upper end are encoded using intra-frame prediction (intra), and the block located at the upper right is encoded using inter-frame prediction, intra-frame prediction, and intra-frame prediction from the left side thereof.

[0246] Blocks coded with inter-frame prediction can be further classified based on the reference picture. Figure 15 The expression ref=t for an inter-frame prediction coded block indicates that the reference picture is the current picture.

[0247] It will be described under the above assumptions Figure 16a and Figure 16b .

[0248] Figure 16a is an exemplary diagram for explaining whether to use reference pixels when considering encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention. Figure 16b is an exemplary view for explaining whether to use reference pixels without considering encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0249] In the following example, whether to consider the coding mode of the adjacent block will be explained using the CIP flag. Specifically, when the CIP flag indicates 1, the block encoded with a specific coding mode (inter-frame prediction in this example) may not be used to predict the current block, and when the flag indicates 0, the block encoded with the specific coding mode may be used to predict the current block.

[0250] refer to Figure 16a If the CIP flag is 1 and the neighboring block is coded with inter-frame prediction (inter), the reference pixels of the corresponding block may not be used to predict the current block. The reference blocks of the neighboring blocks of the block coded with intra-frame prediction (intra) indicated by the hatching can be used to predict the current block.

[0251] Here, pixels adjacent to the current block within a block that can be used as reference pixels may be used as reference pixels, and other additional pixels may be included.

[0252] refer to Figure 16b If the CIP flag is 0, the reference pixels of the coding block can be used to predict the current block. That is, the pixels adjacent to the current block in the upper left, upper right, upper left, left, and lower left of the current block can be used as reference pixels.

[0253] Figure 17 is another exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0254] refer to Figure 17 , the adjacent block is encoded using either intra prediction or inter prediction. When the adjacent block is encoded using inter prediction, ref=t indicates that the reference picture is the current picture, ref=t-1 indicates that the reference picture is the picture immediately earlier than the current picture, ref=t-2 indicates that the reference picture is the second earlier than the current picture, and another reference picture can be added according to the reference direction of the inter prediction. The block on the upper right represents a block located outside the picture range.

[0255] In such Figure 17 In the coding state shown in Figures 18a to 18c An example of determining availability as a reference pixel is described.

[0256] Figure 18a is an exemplary view for explaining a case where the encoding mode of a neighboring block is intra prediction according to an embodiment of the present invention. Figure 18b is an exemplary view for explaining a case where encoding modes and reference pictures of neighboring blocks are considered in intra prediction according to an exemplary embodiment of the present invention. Figure 18c is another exemplary view for explaining a case where encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.

[0257] refer to Figure 18a ,exist Figure 17 Neighboring blocks coded with intra prediction in the coding mode may be shown in the shaded area.

[0258] Since the shadow block is encoded using intra prediction, pixels in the corresponding block can be used as reference pixels.

[0259] And in this case, whether or not to consider the encoding mode of the adjacent block can be indicated according to the CIP flag, and can be an example of the case where the CIP flag is 1.

[0260] refer to Figure 18b , as mentioned above, in Figure 17 Pixels in a block coded with intra prediction in the coding mode state may be used as reference pixels.

[0261] However, in Figure 18b In

[15] , even if an adjacent block is encoded using inter-frame prediction, when the reference picture of the adjacent block is the current picture, the pixels in such a block can be used as reference pixels. Alternatively, when the reference picture of the adjacent block is the current picture, the pixels of the corresponding block may not be used as reference pixels.

[0262] Because error propagation does not occur temporally between pictures when the reference picture of the adjacent block is the current picture, pixels can be used. Conversely, because error propagation may occur when the block referenced by the current picture is a block encoded from a previous or subsequent picture, pixels may not be used.

[0263] As described above, whether to use reference pixels can be determined based on at least one factor or a combination of factors including the position of the adjacent block, the coding mode of the adjacent block, the position of the current block in the partitioned block, and the boundary characteristics of the current block and the adjacent block, and the reference picture can be added to the above factors. Specifically, considering the index of the reference picture of the adjacent block, or the picture order count (POC), it is possible to determine whether to use the reference pixels of the corresponding block for predicting the current block. In the above example, although the reference picture is specified as the current picture, it can be extended to a reference picture with a different index or picture order count (POC). Moreover, even when the reference direction of inter-frame prediction is unidirectional prediction (L0, L1) and bidirectional prediction, various settings for the reference picture indexes L0 and L1 can be constructed.

[0264] Therefore, not only can Figure 18a Pixels in the shaded block (intra-frame prediction block) and pixels in a block encoded using inter-frame prediction may also be used as reference pixels.

[0265] And in this case, since whether or not the encoding mode of the adjacent blocks is considered can be indicated according to the CIP flag, it can be an example for explaining the case where the flag is 1.

[0266] refer to Figure 18c , in principle all neighboring pixels in neighboring blocks can be used as reference pixels.

[0267] However, since the area at the upper right of the current block located in the center is an area without image data or an area outside the image, it may correspond to an area that is not available in any case.

[0268] The above reference Figures 15 to 18c The examples described are summarized below.

[0269] First, the availability of reference pixels of neighboring blocks of the current block may be determined.

[0270] When it is determined that reference pixels in all adjacent blocks are unavailable, reference pixels composed of predefined pixel values ​​can be obtained. For example, the reference pixels can be composed of values ​​in the middle of the pixel's representation range. More specifically, since a value can be assigned to a reference pixel with 1<<(bit depth-1), and the bit depth represents the number of bits used to represent a pixel, the illustrated value can represent a value in the middle of the pixel's representation range.

[0271] When it is determined that at least one neighboring block is available, the coding modes of the neighboring blocks (left, top, top left, top right, bottom left, etc.) may be considered.

[0272] Furthermore, it is possible to use a preset flag to indicate whether the coding mode is taken into account.

[0273] For example, all pixels in the adjacent blocks may be used as reference pixels regardless of the encoding mode, and at this time the preset flag may indicate that the encoding mode is not considered.

[0274] Here, if the encoding mode of the neighboring block is inter prediction, the neighboring block is determined to be unavailable, so that pixels in the neighboring block may not be used for reference pixels.

[0275] Also, even if the encoding mode of the neighboring block is inter prediction, by checking the reference picture, when the reference picture of the neighboring block is the current picture, the pixel can be used as a reference pixel.

[0276] Here, the reference picture may be selected from List 0 storing data of reference pictures earlier than the current picture or List 1 storing data of reference pictures later than the current picture, and the current picture may be included in List 0 or List 1 .

[0277] Also, even if the encoding mode of the neighboring block is inter prediction, by checking the reference picture, when the reference picture of the neighboring block is an I picture (other than P or B), the pixel may be used as a reference pixel.

[0278] However, the above constraints should be understood as examples of determining whether a pixel in a neighboring block can be used as a reference pixel, taking into account specific conditions. Specifically, the coding mode of the neighboring block is inter-frame prediction, and it is possible to determine the availability of the pixel as a reference pixel by referring to the motion vector of the neighboring block's motion information, reference picture information, etc. In this case, the above-mentioned exemplified conditions can be set for units such as sequences, pictures, and slices.

[0279] The availability determination of reference pixels according to an embodiment of the present invention may be performed according to the following procedure.

[0280] First, confirm whether the block adjacent to the current block is located outside the boundary of the picture, slice or tile. Then, it can be confirmed whether the adjacent block is encoded. Then, check the slice type of the adjacent block, and if the type is I slice, the boundary pixels within the adjacent block can be set to be used as reference pixels for the current block. If the type is P slice, the mode is unidirectional prediction. Therefore, the reference picture is confirmed by checking the reference picture list 0 (ref_idx_l0). When the reference picture is the current picture, the reference pixels of the current picture can be set to available. However, this should be understood as an example. When the current picture is not a reference picture, the reference pixels can be set to unavailable. If the type is a B slice, after confirming whether the current picture is used as a reference picture by confirming whether the prediction mode is unidirectional_L0, unidirectional_L1 or bidirectional, it can be confirmed whether the reference picture is available.

[0281] Figure 19a is an exemplary view for explaining a method of obtaining a reference pixel when an available neighboring block is located at the lower left. Figure 19b is an exemplary view for explaining a method of obtaining a reference pixel when an available neighboring block is located at the upper right end. Figure 19c is an exemplary view for explaining a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end. Figure 19d is an exemplary view for explaining a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0282] refer to Figures 19a to 19d , shaded pixels may represent adjacent pixels within a block that are available for determining adjacent blocks, and the arrow direction may indicate a direction constructed by reference pixels.

[0283] As described above, although the adjacent block may be determined to be available, it may also be determined to be unavailable. Therefore, the pixels in the block determined to be unavailable may not be used as reference pixels, and in this case, how to construct the adjacent pixels in the unavailable block may be problematic.

[0284] refer to Figure 19a and Figure 19b , when only pixels located at the upper right or lower left of the current block can be used as reference pixels, the pixels that are in contact with the unavailable area are copied and filled into the pixels adjacent to the current block in the unavailable area.

[0285] refer to Figure 19c When there are available areas both above and below the current block, the pixels can be filled by copying pixels from any available area, by obtaining the average value of pixels on both sides (such as <from bottom to top, from left to right>, <from left to right and from top to bottom>, or <from bottom to top and from right to left>), or by linear interpolation.

[0286] refer to Figure 19d When areas available as reference pixels are mixed, pixels can be filled with pixel values ​​copied in one direction (left direction, downward direction); when there are available areas on both sides (such as the upper end of the current block), pixels can be filled with values ​​generated by using pixels on both sides such as interpolation and averaging; and when there is an available area on one side of the current block such as the left side or the lower left end, pixels can be filled with values ​​obtained by copying.

[0287] More specifically, refer to Figure 19d , pixels located to the right, including P(-1, -1) and P(blk_size, -1) located to the upper right of the current block, may be available. Values ​​from P(0, -1) to P(blk_size-1, -1) may not be available. In this case, if an average value is assigned, it is possible to assign it as P(x, y) {x is 0 to blk_size-1, y is 1} = {P(-1, -1) + P(blk_size, -1)} + 1 / 2. In this case, blk_size can represent the block size.

[0288] The pixels located at the bottom including P(-1, 0) may be unavailable. In this case, P(x, y) {x is 1, y is 0 to 2*blk_size-1} may be allocated with P(-1, -1).

[0289] In addition, it is possible to determine whether to perform interpolation based on the position of the reference pixel. Figure 19c As shown in , even if there are available areas on both sides, interpolation may be limited when the positions of the available areas are not connected by a vertical line. Specifically, when comparing the coordinates of both ends of the available area, when both x and y do not match (such as P(-1, a) and P(b, -1)), interpolation may be limited. When one of x and y matches each other (such as P(a, -1), P(b, -1)), interpolation can be performed.

[0290] In addition, even in Figure 19d There are available areas on both sides shown in , and interpolation can be performed when the positions of the available areas are located on a vertical line.

[0291] Figure 20 is a flowchart of an image encoding method according to an embodiment of the present invention.

[0292] refer to Figure 20, an image encoding method for performing intra-frame prediction includes: considering a prediction mode of each of a current block and an adjacent block, determining whether boundary pixels within the adjacent block can be used as reference pixels of the current block (step S110); obtaining reference pixels of the current block according to the determined result (S120); generating a prediction block through intra-frame prediction based on the obtained reference pixels (S130); and encoding the current block using the generated prediction block (S140).

[0293] A description of each step in this case can be found in Figures 15 to 18c , and therefore a detailed description thereof will be omitted.

[0294] An image decoding method for performing intra-frame prediction according to an embodiment of the present invention includes the following steps: receiving a bit stream including data on prediction modes of a current block and blocks adjacent to the current block; extracting data from the received bit stream to confirm the prediction modes of the adjacent blocks; determining whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block in consideration of the prediction modes of the adjacent blocks; obtaining reference pixels of the current block based on the determined result; generating a prediction block for intra-frame prediction based on the obtained reference pixels; and decoding the current block using the generated prediction block.

[0295] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may obtain a reference pixel constructed by a preset pixel value.

[0296] Here, determining whether a boundary pixel can be used as a reference pixel may indicate whether a preset flag considers a prediction mode of a neighboring block.

[0297] Here, when the prediction mode of the neighboring block is intra prediction, determining whether the boundary pixel can be used as a reference pixel may determine whether the boundary pixel can be used as a reference pixel of the current picture.

[0298] Here, when the prediction mode of the adjacent block is inter prediction, determining whether the boundary pixel can be used as a reference pixel may determine that the boundary pixel cannot be used as a reference pixel of the current picture.

[0299] Here, when the prediction mode of the adjacent block is inter prediction, determining whether the boundary pixel can be used as a reference pixel may determine whether the boundary pixel can be used as a reference pixel of the current picture considering the reference picture of the adjacent block.

[0300] Here, the reference picture may be selected from List 0 storing data of reference pictures earlier than the current picture or List 1 storing data of reference pictures later than the current picture, and the current picture may be included in List 0 or List 1 .

[0301] Here, when the reference picture of the neighboring block is the current picture, it may be determined that the boundary pixels may be used as the reference picture of the current picture.

[0302] Here, when the reference picture of the neighboring block is not the current picture, it may be determined that the pixels in the neighboring block cannot be used as the reference picture of the current picture.

[0303] Here, when the reference picture of the neighboring block is an I picture, it may be determined that pixels in the neighboring block may be used as a reference picture of the current picture.

[0304] In an image decoding device including one or more processors according to an embodiment of the present invention, the one or more processors may: receive a bit stream including data on prediction modes of a current block and blocks adjacent to the current block; and confirm the prediction modes of the adjacent blocks by extracting data from the received bit stream; determine whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block in consideration of the prediction modes of the adjacent blocks; obtain reference pixels of the current block based on the determined result; generate a prediction block for intra-frame prediction based on the obtained reference pixels; and decode the current block using the generated prediction block.

[0305] Figure 21 is a diagram showing an example of the intra prediction mode of HEVC.

[0306] refer to Figure 21 , there are a total of 35 prediction modes for the luma component (Intra_fromLuma, 35). There may be prediction modes with various directions including planar (Intra_planar, 0) and average (Intra_DC, 1). For each prediction mode, the direction of each arrow may indicate the direction in which the prediction block is constructed using reference pixels.

[0307] Figure 22a is an example diagram for explaining reference pixels used for current block prediction in the reference pixel construction step.

[0308] refer to Figure 22a, pixels (A0 to P0, X0) of blocks to the left, upper left, lower left, upper, and upper right of the current block may be included in the reference pixel candidate group. For example, in the case of an M×N block, pixels p[0, -1] to p[(2×M)-1, -1], p[-1, 0] to p[(2×N)-1], and p[-1, -1] based on the upper left coordinates may be included in the candidate group. Here, p[0, -1] corresponds to A0, p[2×M-1, -1] corresponds to H0, and p[-1, -1] corresponds to X0. In the above example, 2×(M+N)+1 reference pixels may be included in the candidate group and may be determined based on at least one factor such as the size of the block, the type of the block, the prediction mode, the prediction method, the slice type, and a combination thereof. For example, some reference pixels may be added to or excluded from the candidate group depending on the prediction mode. As another example, some reference pixels may be added to or excluded from the candidate group depending on the prediction scheme.

[0309] Figure 22b is an exemplary view for explaining that additional pixels of a neighboring block are included in a reference pixel candidate group.

[0310] refer to Figure 22b For example, in the case of an M×N block, pixels of p[-1, -2] to p[(2×M)-1, -2], p[-2, -1] to p[-2, (2×N)-1], and p[-1, -1] based on the upper left coordinate of the block may be included in the new reference pixel candidate group. Pixels may be classified into a first reference pixel line (A0 to P0, X0) and additional reference pixel lines to represent differences from existing reference pixels, and other additional reference pixel lines may be included as a new reference pixel candidate group. The order of the reference pixel lines may be determined based on the distance between the reference pixels and the current block.

[0311] Reference pixel lines may be commonly applied to pixels of blocks to the left, upper left, lower left, upper, and upper right, and may be constructed from one adjacent block or two or more adjacent block units. For example, when constructing a pixel line relative to pixels of a block to the left, pixels of the corresponding block may be constructed on reference pixel line L. When constructing a pixel line relative to blocks to the upper left, upper, and upper right, pixels of the corresponding block may be constructed in reference pixel line U. This example will be described based on reference pixel lines commonly applied to adjacent blocks.

[0312] Whether to add additional reference pixel lines in addition to existing reference pixel lines can be determined based on one or more factors of the block, block type, prediction mode and prediction method, or a combination thereof. In addition, whether to use existing reference pixel lines and whether to use additional reference pixel lines can have various constructed combinations. For example, existing reference pixel lines are always used, whether to use additional pixel rows can be determined, and whether to use each existing reference pixel line and whether to use each of the additional pixel rows can be determined separately. In addition, when there are more than two additional pixel rows, various combinations are possible relative to the above settings. And, when reference pixel lines are not applied to adjacent blocks jointly but partially, it is possible to have various combinations related to the above settings.

[0313] In addition, the allowable range of the reference pixel line can be set. That is, information about how much the reference pixel line will be added (maximum value) can be set in the encoder / decoder, and this can be determined based on one factor or a combination of factors such as the type of the current or adjacent block, the size of the current or adjacent block, etc. If the order of the reference pixel line is a small number, the priority may be higher, but the assumption thereon may not result in a binarization process of the relevant information. The index information or identifier information of the reference pixel line can be generated under predetermined conditions. The reference pixel line index or identifier information can be binarized by various methods such as unary binarization, cut-off unary binarization, etc., and the binarization can be determined based on the allowable maximum value information in the reference pixel line.

[0314] When adjacent blocks are used together for reference pixel line construction or other purposes, various combinations of the above settings are possible. Information related to reference pixel lines can be sent in units of sequences, pictures, slices, etc. The encoded / decoded information can be included in the prediction mode information in the prediction mode encoding step.

[0315] After constructing a reference pixel candidate group through the above process, a process is performed to check whether reference pixels of adjacent blocks are used. Whether reference pixels are used can be determined by at least one of factors such as the position of the adjacent block, the coding mode of the adjacent block, the reference image of the adjacent block, the position of the current block in the partition block, and the boundary characteristics between the current block and the adjacent block, or a combination thereof. Prediction of the current block can be performed using pixels in the reference pixel candidate group that are determined to be available through this process, as well as replaced pixel values ​​of pixel positions that are determined to be unavailable through this process.

[0316] When prediction is performed in a directional mode among intra prediction modes, reference pixels can be generated in decimal units using various interpolation methods for integer pixel reference pixels. When adding reference pixel lines as in the above example, reference pixels from two or more reference pixel lines can be supported during the interpolation process. Not only can one-dimensional filtering, which interpolates fractional pixels between two existing pixels, but also two-dimensional filtering, which interpolates fractional pixels between four pixels, be applied.

[0317] In the reference pixel filtering step, a process of applying filtering to the reference pixels obtained from the reference pixel constructing step will be described based on a case with wider reference pixels.

[0318] Figure 23a 、 Figure 23b 、 Figure 24a and Figure 24b 1 is a diagram showing various mode directions for a current block with a wider reference pixel that can be included in the reference pixel construction step. This is for the purpose of explaining the directions of the various prediction modes described above and is not limited thereto. It can be assumed that the first reference pixel line (A0 to P0, X0) and the second reference pixel line (A1 to P1, X1, X2) are the same. 10 , X 01 , X 11 ) is constructed as a reference pixel candidate group for explanation.

[0319] When only the first reference pixel line is included in the reference pixel candidate group, filtering may be applied to pixels included in the reference pixel candidate group, that is, the most adjacent pixels of the current block. When additional reference pixel lines are included in the reference pixel candidate group, filtering may be applied that takes into account the additional reference pixels.

[0320] Figure 23a An exemplary view of a pattern (vertical) of reference pixels of a second reference pixel line, more specifically, reference pixels A1, B1, C1, and D1 located in the second row from the top of the current block are referenced so that pixels generated by applying filtering to the reference pixels are applied for prediction, and an exemplary view of a pattern of reference pixels of a first reference pixel line, specifically, reference pixels A0, B0, C0, and D0 located in the first row so that pixels generated by applying filtering to the reference pixels are applied for prediction.

[0321] Figure 23bAn exemplary view of a pattern (horizontally) of reference pixels of the second reference pixel line is shown, specifically, reference pixels I1, J1, K1, and L1 of the second row located to the left of the current block are referenced so that pixels generated by applying the generated reference pixels are applied to prediction, and an exemplary view of a pattern of reference pixels of the first reference pixel line, specifically, reference pixels I0, J0, K0, and L0 so that pixels generated by applying filtering to the reference pixels are applied to prediction.

[0322] Figure 24a An exemplary view of a pattern of reference pixels of the second reference pixel line (a directional pattern toward the lower left end) is shown, specifically, reference pixels C1, D1, E1, F1, G1, and H1 located on the second row from the upper right end of the current block are referenced so that pixels generated by applying filtering to the reference pixels are applied for prediction, and an exemplary view of a pattern of reference pixels of the first reference pixel line, more specifically, reference pixels B0, C0, D0, E0, F0, G0, and H0 located on the first row from the upper right end of the current block are also referenced so that pixels generated by applying filtering to the reference pixels are applied for prediction.

[0323] Figure 24b An exemplary view of a pattern of reference pixels of the second reference pixel line (a directional pattern toward the lower right end) is shown, specifically, reference pixels J1, I1, X01, X11, X10, A1, and B1 are referenced so that pixels generated by applying filtering to the reference pixels are applied for prediction, and an exemplary view of a pattern of reference pixels of the first reference pixel line, specifically, reference pixels K0, J0, I0, X0, A0, B0, and C0 located at the upper left of the current block so that pixels generated by applying filtering to the reference pixels are applied for prediction.

[0324] When filtering is applied to the reference pixels of the first reference pixel line, adaptive filtering may be performed according to the mode of the current block. When the reference pixels of the second reference pixel line are also applied for filtering, not only is adaptive filtering performed according to the mode of the current block, but the reference pixels applied for filtering may also be adaptive. That is, filtering on the neighboring pixels of the current block may be independent of the prediction mode direction, and when the reference pixels of the second reference pixel line are also included in the filtering process, the filtering may be performed according to the mode of the current block. Figure 22a 、 Figure 22b 、 Figure 23a and Figure 23b The pixels on the direction line of the pattern shown in are applied to the filtering.

[0325] In the prediction block generation step, a process of applying filtering to the reference pixels obtained from the reference pixel construction step will be described based on a case with wider reference pixels.

[0326] In the case of a directional prediction method using the first reference pixel line as a reference pixel candidate group, the closest reference pixel can be used to generate the prediction block. If additional reference pixel lines other than the first reference pixel line are supported, the reference pixels used to generate the prediction block can be determined. For example, depending on the prediction mode, the prediction block can be generated as the first reference pixel, the prediction block can be generated as the second reference pixel, and the prediction block can be generated as the predicted pixel of the first reference pixel line and the second reference pixel line.

[0327] In the following, the prediction mode encoding step will be described in more detail.

[0328] In this case, the following examples will be described based on the HEVC standard, and the relevant settings (number of prediction modes, prediction mode direction, etc.) are not limited to the examples described below. Since there are 35 modes in the case of HEVC, after determining the optimal mode, it is necessary to encode the effective mode information. Generally, when an image is divided into several blocks, adjacent blocks usually have similar characteristics. Therefore, when the mode of the current block is encoded, the adjacent block information is utilized. HEVC encodes the prediction mode of the current block based on the prediction modes of the left and top blocks.

[0329] Figure 25 This is a flowchart illustrating a process for determining the optimal mode for a general current prediction unit (or prediction block). Here, the most probable mode (MPM) is a prediction mode that is highly likely to be the prediction mode of the current prediction unit. Since prediction units surrounding the current prediction unit may have a mode similar to that of the current prediction unit, a mode may be selected from the modes of the surrounding blocks.

[0330] refer to Figure 25 , based on the current prediction unit (PU), the prediction mode of the prediction unit on the left is referred to as left_PU_MPM, and the prediction mode of the prediction unit on the upper side is expressed as Above_PU_MPM. Specifically, when the prediction mode of the left prediction unit is different from the prediction mode of the upper prediction unit, the first most probable mode MPM[0] is determined as the prediction mode of the left prediction unit, the second most probable mode MPM[1] is determined as the prediction mode of the upper prediction unit, and the third most probable mode MPM[2] is determined as one of non-overlapping planar, DC, and vertical modes. On the other hand, when the prediction mode of the left prediction unit is the same as the prediction mode of the upper prediction unit, it is determined whether the same prediction mode is less than mode 2. If the prediction mode is less than mode 2, the planar, DC, and vertical modes are set as the most probable modes, and when the prediction modes are larger or the same, the prediction mode of the left prediction unit, a mode smaller than it, and a mode larger than it can be set as the first, second, and third most probable modes (MPMs), respectively.

[0331] In short, if Figure 25 As shown in , the modes with high probability of being the prediction mode of the current block can be classified into most probable mode (MPM) sets or groups and encoded. When the current block has the same characteristics as the adjacent block, so that the prediction mode of the adjacent block is the same or similar to the prediction mode of the current block, the possibility of selecting one mode of the MPM group is high, so that short bits can be allocated, thereby performing efficient encoding. If this does not occur, the prediction mode of the current block can be encoded by binarizing the remaining modes except the most probable mode (MPM) among all modes and selecting one of them. In the case of HEVC, the number of modes in the best mode (MPM) group is three, so that one or two bits are used to perform encoding. If not selected in the most probable mode (MPM) group, encoding is performed using a fixed length of 5 bits. According to the embodiment of the present invention, the use of the mode as the most probable mode (MPM) for the left and above prediction units is not limited, and mode information of one or more blocks from the left, upper left and above, upper right and lower left blocks can be utilized. Furthermore, when one of the blocks is divided into two or more blocks, it is possible to prioritize information on which block among the divided blocks is referenced. For example, when the current block is 2N×2N and two N×N divided blocks are located on the left, either the upper N×N pattern or the lower N×N pattern may be used, depending on the setting rule.

[0332] The number of modes selected by the most probable mode (MPM) can also be preset in the encoding / decoding device, or sent in units of sequence, picture, slice, etc. In addition, the binarization of the number of modes can be represented by using various binarization methods such as fixed length or variable length. In addition, for the binarization of modes that are not classified as the most probable mode (MPM), various binarization methods can be used. In addition, in order to effectively binarize the modes that are not classified as the most probable mode (MPM), an MPM candidate group can be set. For example, M-2 of the M mode N Can be binarized into the most probable mode (MPM) group, and 2N patterns can be binarized into candidate groups that are not classified as the most probable mode (MPM). For example, if there are 19 patterns, N can be selected as one of 1, 2, 3, and 4, and the patterns in the most probable mode (MPM) group can be represented as variable length (if N is 4, it can be three), and the candidate group that is not classified as the most probable mode (MPM) can be represented as a fixed length (4 bits).

[0333] Hereinafter, a specific example of determining the most probable mode (MPM) for the current prediction unit (or block) according to an embodiment of the present invention will be described. For ease of explanation, it is assumed that a block having the same block type and size as the current block is located in a neighboring block. It is also assumed that the mode of the block determined to be unavailable based on the availability of reference pixels of the neighboring blocks is DC.

[0334] First, in order to obtain the mode information of the adjacent blocks, it is necessary to check the availability of the boundary pixels of the corresponding block as reference pixels. For example, it is necessary to check whether the mode is on the boundary line of the picture or slice, or whether the encoding method is the same. In this example, regardless of the reference picture, if the encoding mode is the inter-frame prediction mode, the reference pixels of the corresponding block are set to unavailable. (As in HEVC, the left and upper modes are used as MPM) When the left block generates a prediction block in the current picture by block matching and is encoded with the inter-frame prediction method, the reference pixels of the corresponding block can be determined to be unavailable. When the upper block is encoded with intra-frame prediction, the mode (vertical) of the corresponding block can be included in the MPM candidate. Since the reference pixels of the left block are unavailable, the mode of the corresponding block can be set to DC and included in the MPM candidate.

[0335] Available reference pixels and unavailable reference pixels can be classified by checking the availability of reference pixels of adjacent blocks. Reference pixels determined to be unavailable can be filled with a predefined value (e.g., the middle value of the range of pixel values ​​of the bit depth) or an alternative value derived from the reference pixels determined to be available. A prediction block can be generated by using the pixel values ​​of reference pixels that can be used for intra prediction and the alternative values ​​of unavailable reference pixels.

[0336] Figure 26 is an exemplary view for determining a prediction mode of a current prediction unit (or block).

[0337] refer to Figure 26 , available reference pixels can be displayed in black, while unavailable reference pixels can be displayed in no color. Unavailable reference pixels can be replaced by predefined values ​​or values ​​derived from available pixels. Figure 26 , the pixel values ​​of the unavailable reference pixels of blocks A, B, C, and D may be filled with values ​​obtained by copying the pixel value in the upper right corner of block E. That is, all the reference pixels of blocks A, B, C, and D may have the same value as the pixel in the upper right corner of block E. For example, all modes that generate a prediction block using blocks B and C as reference pixels may construct all the predicted pixels of the prediction block as the same pixel value (the pixel value in the upper right corner of block E).

[0338] Here, since block A or block B is determined to be unavailable, the block may have a DC mode as previously assumed. In this case, since the prediction mode of block A is set to DC and the mode of block B is also set to DC, it is possible to reduce the possibility that the mode determined as the most probable mode (MPM) (two DCs appear) is the prediction mode of the current block. Therefore, the mode can be used by replacing it with the prediction mode of an available block adjacent to the current block.

[0339] refer to Figure 26 , the mode of block A can replace the prediction mode of block E. By using DC as the prediction mode of block E and the prediction mode of block B, it is possible to increase the possibility that the prediction mode of the current block is selected as a mode of MPM. Figure 25 The flowchart of FIG. 1 may set MPM[0] = planar mode, MPM[1] = DC mode, and MPM[2] = vertical mode, which is a general method. Assuming that the pixels in the right boundary of block E are available (black cells in the figure), and the pixel values ​​on top of the boundary pixels are copied to the boundary pixels of the above-mentioned unavailable blocks to perform intra-frame prediction, all the same prediction blocks are generated from planar, DC, and vertical, thereby possibly causing mode redundancy. Therefore, as described above, using the prediction mode of the alternative block can have a more advantageous effect.

[0340] Reference again Figure 26 , if constrained intra prediction is performed, then Figure 26 The upper right pixel value of the boundary pixel of block E shown in black in FIG is copied to the boundary pixel of block A to perform intra prediction. It is also assumed that the mode of block E is not DC. Under this assumption, if the prediction block is a vertical, horizontal, group 2, group 3, and group 4 including plane and DC (refer to the following description for each prediction mode group), Figure 27 ) mode, it is possible to generate a prediction block in which the pixel values ​​of all prediction blocks generated by the prediction mode are one value (the upper right pixel value of block E). This is possible because all are copied to the same pixel value. Therefore, reference Figure 25 , when the prediction mode of block E is selected by replacing block A and encoding according to the most probable mode (MPM), the candidate group can be classified into left_PU_MPM as the prediction mode of block E, and Above_PU_MPM as the prediction mode of block B (i.e., DC). Since Left_PU_MPM and Above_PU_MPM are different from each other, MPM[0] = prediction mode of block E, MPM[1] = DC, and MPM[2] = planar or vertical (determined according to the mode of block E).

[0341] When the prediction mode of block E is planar, DC, vertical, horizontal, and one of groups 2 to 4, MPM[0] is planar, DC, vertical, horizontal, and one of groups 2 to 4, and MPM[1] is DC, then if the mode of block E is not planar, MPM[2] is planar, and if the mode of block E is planar, MPM[2] is vertical. However, when a prediction block is generated using the planar, DC, vertical, horizontal, and groups 2, 3, and 4 modes as described above, the same prediction block is generated (if the reference pixel is constructed with a pixel value of 100, all pixel values ​​are 100 regardless of which mode is used to generate the prediction block), so that the MPMs are managed in an overlapping mode.

[0342] Therefore, when the prediction mode of block E is one of planar, DC, vertical, horizontal, and groups 2 to 4 (groups 2, 3, 4), MPM[0] is determined to be one of planar, DC, vertical, horizontal, and groups 2 to 4, MPM[1] is determined to be one of the modes belonging to group 1, and MPM[2] is determined to be one of the blocks that do not overlap with MPM[1] belonging to group 1. This is because unless all reference pixels at the boundary of block E are composed of the same pixel value, the prediction blocks may be slightly different (by at least one pixel) for each mode. Here, encoding can be performed by using the existing fixed 5-bit binarization for the prediction mode not selected as the most probable mode (MPM) or by binarization only for the modes in group 1. For example, since there are 8 modes for modes 2 to 9 (group 1) (reduced to 6 by subtracting 2 included in the MPM), encoding can be performed using various binarization methods with bits shorter than the fixed 5 bits.

[0343] When the mode of block E is one of the modes of group 1, MPM[0] is the prediction mode of block E, MPM[1] is DC, and MPM[2] is a mode that does not overlap with the prediction mode of block E of group 1 (since the prediction block generated by DC or the prediction blocks generated by planar, vertical, horizontal, groups 2, 3, and 4 are the same, starting from group 1). In this case, encoding can be performed by binarization as described above.

[0344] Figure 27 It is a diagram showing grouping of prediction modes.

[0345] refer to Figure 27 , it can be assumed that the prediction modes are classified into planar, DC, vertical, horizontal, group 1, group 2, group 3, and group 4. It is possible to allow various settings to group the prediction modes into 2 or more groups, and the following grouping is classified in the case of settings of similar directionality.

[0346] In this case, prediction modes 2 to 9 are for group 1, prediction modes 11 to 17 are for group 2, prediction modes 18 to 25 are for group 3, and prediction modes 27 to 34 are for group 4. In addition, prediction mode 0 may represent planar, prediction mode 1 may represent DC, prediction mode 10 may represent horizontal, and prediction mode 26 may represent vertical.

[0347] Based on this prediction mode, the embodiment will be further described below.

[0348] Figures 28 to 32 is an exemplary view of a case where an MPM is constructed for a prediction mode using neighboring blocks other than an upper block and a left block.

[0349] refer to Figure 28 , the prediction mode of block E can be used by replacing block A, and the prediction mode of block D can be used by replacing block B.

[0350] refer to Figure 29 , the prediction mode of block D can be used by replacing block A, and the prediction mode of block C can be used by replacing block B.

[0351] refer to Figure 30 , the prediction mode of block A is used as it is, and the prediction mode of block E can be used by replacing block B.

[0352] refer to Figure 31 , the prediction mode of block B is used as it is, and the prediction mode of block C can be used by replacing block B.

[0353] refer to Figure 32 , block A may be replaced with the prediction mode of block E, block B may be replaced with the prediction mode of block D, block A may be replaced with the prediction mode of block D, and block B may be replaced with the prediction mode of block C, and block A may be replaced with the prediction mode of block E, and block B may be replaced with the prediction mode of block C. That is, the prediction modes of the available blocks may be replaced in various combinations.

[0354] The above example is described as an example when a block is unavailable due to restricted intra prediction, but it can also be applied when all neighboring blocks are available. If the reference pixels of neighboring blocks are similar or identical (for example, in the case of a computer screen capture), it is also possible to encode by changing the candidate group setting of the most probable mode (MPM). This case will be described below.

[0355] Figure 33 is an exemplary view of a case where a most probable mode (MPM) is constructed as a substitute block when both a current block and neighboring blocks are available.

[0356] refer to Figure 33, black pixels are used as reference pixels, where pixels at the boundaries with blocks A, B, C, and D have no change in pixel value or a similar value of pixel value, and pixels at the boundary of block E are changed.

[0357] In this state, even if you use Figure 27 If any prediction mode in groups 2 to 4 shown in is used, the same or similar prediction blocks are generated, so that redundancy of the most probable mode (MPM) candidate group may occur.

[0358] First, it is possible to classify blocks by the variance σ of the reference pixel values ​​for each block A , σ B , σ C , σ D and σ E Less than the threshold (Th). Assume that the variance σ A , σ B , σ C , σ D and σ E is less than the boundary value, and the variance σ of block E E Greater than Figure 33 When the variance of two or more consecutive blocks is less than the boundary value, redundancy of the most probable mode (MPM) candidate group may occur. For example, if the reference pixels of block B and block C have the same or similar values, group 4 of the prediction mode generated using only block B and block C and the prediction block generated by the vertical mode can be regarded as having the same or similar values. Therefore, group 4 and the vertical mode can perform the redundancy removal process. If the reference pixels of block A and block C are the same or similar, the prediction mode generated using only block A and block C does not exist and does not need to be considered.

[0359] First, since prediction blocks generated from horizontal, group 2, group 3, and vertical of prediction modes generated using reference pixels of blocks A, D, and B have the same and similar values, it can be determined that redundancy occurs.

[0360] Since it is assumed that the prediction blocks generated from the vertical sum group 4 of the prediction mode generated using the reference pixels of blocks B and C have the same and similar values, it can be determined that redundancy occurs in the mode.

[0361] Since it is assumed that the prediction blocks generated in the horizontal, group 2, group 3, vertical, group 4, group 4, and DC planes of the prediction modes generated using the reference pixels of blocks A, D, B, and C have the same prediction blocks, it can be determined that redundancy occurs in the mode. That is, it can be confirmed that the variance of the continuous blocks is calculated (σ ADB , σ BC , σ ADBC ), which is smaller than the boundary value.

[0362] exist Figure 33 In the case of , it is confirmed that the variance of A, D, B, and C is less than the boundary value and leads to the most likely redundancy. Then, the MPM candidate group can be reset by removing the redundancy as described above. The variance is mentioned for the purpose of illustrating the above example, but it is not limited thereto.

[0363] When the prediction mode of block A is different from the prediction mode of block B, refer to Figure 25 , MPM[0] = prediction mode of block A, MPM[1] = prediction mode of block B, MPM[2] = one of planar, DC, or vertical (according to the prediction mode of block A and the prediction mode of block B). If one of the two blocks is in group 1 and the other is in other cases, the prediction mode of block A is one of horizontal, group 2, group 3, vertical, group 4, DC, and planar that are classified by comparing the above-mentioned variance values ​​to generate the same prediction block, and when the mode is one of the modes of group 1, the prediction mode of block B can be constructed so as not to overlap with the prediction mode of block B among MPM[0] = prediction mode of block A, MPM[1] = prediction mode of block B, and MPM[2] = mode of group 1. If both modes belong to group 1, MPM[0] = prediction mode of block A, MPM[1] = prediction mode of block B, and MPM[2] = planar. If both modes do not belong to group 1, then MPM[0] = prediction mode of block A, MPM[1] = one of the modes in group 1, MPM[2] = one of the modes in group 1 that overlaps with MPM[1]. If the prediction mode of block A is the same as the prediction mode of block B, and both modes are greater than mode 2 (not DC and planar) and leave Figure 25 , it can be configured as: MPM[0] = prediction mode of block A and MPM[1] = one of the prediction modes of group 1 and if the two modes are horizontal, one of the modes of groups 2, 3 and vertical that does not overlap with MPM[2] = MPM[1]. Otherwise, it can be configured as MPM[0] = prediction mode of block A, MPM[1] = prediction mode a of block A, MPM[2] = prediction mode of block A+1. If the two modes are the same and both modes are less than mode 2 (one of DC and planar), then MPM[0] = planar, MPM[1] = one of the modes of group 1, and MPM[2] is one of the modes of group 1 that does not overlap with MPM[1].

[0364] Figure 34 is a flowchart illustrating an image encoding method for performing intra-frame prediction according to an embodiment of the present invention.

[0365] refer to Figure 34, an image encoding method for performing intra-frame prediction includes: determining whether each of the left block and the upper block of the current block is available in the reference pixel configuration of the current block (S210); selecting a most probable mode (MPM) candidate group based on the determined result (S220); determining redundancy of a prediction block derived from an intra-frame prediction mode in the MPM candidate group (S230); selecting an MPM group using a prediction mode of an alternative block adjacent to the left block or adjacent to the upper block if redundancy is determined to exist (S240); and performing intra-frame prediction mode encoding for the current block based on the selected MPM group (S250).

[0366] Here, determining whether each block is available (S210) may be determined as unavailable when the left block and the upper block exist outside the image boundary or are encoded using inter prediction, and may be determined as available in other cases.

[0367] Here, the MPM candidate group may include at least one of an intra prediction mode for a block determined to be available among the left block and the upper block, an intra prediction mode predetermined for a block determined to be unavailable, a planar prediction mode, a planar prediction mode, a vertical prediction mode, and a DC prediction mode.

[0368] Here, determining redundancy (S230) may determine redundancy based on a variance value calculated for reference pixel values ​​in two or more consecutive blocks located below the left, to the left, above the left, above the right, and above the current block when both the left block and the upper block are available. For example, two or more consecutive blocks may be selected and grouped, and a variance value of the selected group may be calculated. If the calculated variance value is less than a boundary value, it may be determined that redundancy exists.

[0369] Determining redundancy (S230) may include selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0370] Here, the replacement block may be selected from adjacent blocks located below and to the left, above and to the left, and above and to the right of the current block.

[0371] Here, if the adjacent blocks consist of one or more blocks, the replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0372] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority order.

[0373] An image decoding method for performing intra-frame prediction may include: extracting data on whether a prediction mode of a current block is included in a most probable mode (MPM) group from a received bitstream; determining whether the prediction mode of the current block is included in the MPM group based on the data; extracting MPM group index data of the current block from the received bitstream when it is determined that the prediction mode of the current block is included in the MPM group; confirming the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and performing intra-frame prediction on the current block based on the confirmed prediction mode.

[0374] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct the reference pixels of the current block; selecting the most likely mode MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when redundancy is determined to exist.

[0375] Here, determining whether a block can be used may be determined as unavailable when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction, and may be determined as available in other cases.

[0376] Here, the MPM candidate group may be selected from the intra prediction mode of the block determined to be available among the left block and the above block, the intra prediction mode predetermined for the block determined to be unavailable, the planar prediction mode, the prediction mode, and the DC prediction mode.

[0377] Here, when both the left block and the above block are available, determining the redundancy of the prediction block can be performed by determining the redundancy based on the variance value calculated for the reference pixel values ​​in two or more consecutive blocks among the blocks located to the lower left, left, upper left, above, and upper right of the current block.

[0378] Here, determining the redundancy of the prediction block can be performed by selecting an intra-frame prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0379] Here, the replacement block may be selected from adjacent blocks located below and to the left, above and to the left, and above and to the right of the current block.

[0380] Here, when constructing a neighboring block with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0381] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority order.

[0382] In an image decoding device including one or more processors according to other aspects of the present invention, the one or more processors may: extract data related to whether the prediction mode of the current block is included in the most probable mode (MPM) group from a received bit stream; determine whether the prediction mode of the current block is included in the MPM group based on the data; extract MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group; confirm the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and perform intra-frame prediction on the current block based on the confirmed prediction mode.

[0383] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct the reference pixels of the current block; selecting the MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when redundancy is determined to exist.

[0384] Here, determining whether a block can be used may be performed by determining as unusable when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction, and may be determined as available in other cases.

[0385] Determining the redundancy of the prediction block can be performed by selecting an intra-frame prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0386] According to the above embodiments, it is possible to provide high-performance and efficient image encoding and decoding technology that is generally available in international codecs such as MPEG-2, MPEG-4 and H.264 or other codecs (in which intra-frame prediction is used), and the media and image industries using such codecs.

[0387] In the future, it is expected to be applied to the current High Efficiency Image Coding technology (HEVC) and image processing using standard codecs such as H.264 / AVC and intra prediction.

[0388] While the invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the following claims.

Claims

1. A method for decoding an image for performing inter-frame prediction, the method comprising: determining whether the current picture is included in a reference picture list of the current block; generating a reference picture list 0 and a reference picture list 1 for the current block; specifying a reference picture of the current block based on the reference picture list 0 and the reference picture list 1; performing inter-frame prediction on the current block using the reference picture; and reconstructing the current block based on a prediction block of the current block generated by the inter-frame prediction, wherein the reference picture list 0 and the reference picture list 1 respectively include pictures decoded before the current picture including the current block, According to the determination that the current picture is included in the reference picture list, the reference picture list 0 and the reference picture list 1 also include the current picture, respectively, The reference picture list 0 is generated by sequentially allocating a first reference picture, a second reference picture, and the current picture, The reference picture list 1 is generated by sequentially allocating the second reference picture, the first reference picture and the current picture, In time order, the first reference picture is before the current picture, and the second reference picture is after the current picture.

2. A method for image encoding for performing inter-frame prediction, the method comprising: determining whether the current picture is included in a reference picture list of the current block; generating a reference picture list 0 and a reference picture list 1 for the current block; specifying a reference picture of the current block based on the reference picture list 0 and the reference picture list 1; performing inter-frame prediction on the current block using the reference picture; and encoding information about the current block based on a prediction block of the current block generated by the inter-frame prediction, wherein the reference picture list 0 and the reference picture list 1 respectively include pictures decoded before the current picture including the current block, According to the determination that the current picture is included in the reference picture list, the reference picture list 0 and the reference picture list 1 also include the current picture, respectively, The reference picture list 0 is generated by sequentially allocating a first reference picture, a second reference picture, and the current picture, The reference picture list 1 is generated by sequentially allocating the second reference picture, the first reference picture and the current picture, In time order, the first reference picture is before the current picture, and the second reference picture is after the current picture.

3. A bitstream storage method, comprising: generating a bitstream including information about a current block generated by performing prediction on the current block in a current picture; and storing the bitstream, in, The prediction block of the current block is generated using a reference picture of the current block, The current block is reconstructed based on the prediction block and the information of the current block, The reference picture of the current block is specified based on reference picture list 0 and reference picture list 1, The reference picture list 0 and the reference picture list 1 respectively include pictures decoded before the current picture including the current block, whether the current picture is included in the reference picture list of the current block is determined, and according to the determination that the current picture is included in the reference picture list, the reference picture list 0 and the reference picture list 1 also include the current picture, respectively, The reference picture list 0 is generated by sequentially allocating a first reference picture, a second reference picture, and the current picture, The reference picture list 1 is generated by sequentially allocating the second reference picture, the first reference picture and the current picture, In time order, the first reference picture is before the current picture, and the second reference picture is after the current picture.

Images