Image encoding and decoding method, bit stream storage method

Abstract

The present invention discloses an image coding and decoding method and a bitstream storage method. The image decoding method comprises: decoding current picture reference information indicating whether a current picture of a current block is included in a reference picture list; decoding skip mode flag information indicating whether a skip mode is applied to the current block based at least in part on the current picture reference information; generating reference picture list 0 and reference picture list 1 for the current block; determining a reference picture of the current block based on reference picture list 0 and reference picture list 1; generating a prediction block of the current block using the reference picture; and in response to the skip mode being applied to the current block, reconstructing the current block based on the prediction block of the current block without decoding residual information of the current block.

Description

[0001] This application is a divisional application of the invention patent application with application date of June 7, 2016, application number 201680045604.5, and invention name “Method and device for encoding and decoding intra-frame prediction”. Technical Field

[0002] The present invention relates to image encoding and decoding technology, and more particularly to an encoding and decoding method and apparatus for performing reference pixel construction and most probable mode construction in intra-frame prediction. Background Art

[0003] With the popularization of the Internet and mobile terminals and the development of information and communication technology, the use of multimedia data is rapidly increasing. Therefore, in order to perform various services or tasks through image prediction in various systems, the demand for improving the performance and efficiency of image processing systems has greatly increased, but research and development that can meet this demand is insufficient.

[0004] With the image encoding and decoding methods and apparatuses according to the above-mentioned prior art, there is a need for performance improvement of image processing, especially image encoding or image decoding. Summary of the invention

[0005] Technical issues

[0006] The present invention aims to solve the above-mentioned problems, and an object of the present invention is to provide an image encoding / decoding method that improves the intra-frame prediction mode.

[0007] Another object of the present invention is to provide an image encoding / decoding device with improved intra-frame prediction mode.

[0008] Technical Solution

[0009] In order to achieve the purpose of the present invention, an image encoding method for performing intra-frame prediction according to an aspect of the present invention includes: determining whether boundary pixels within an adjacent block can be used as reference pixels of the current block in consideration of a prediction mode of each of a current block and an adjacent block, obtaining reference pixels of the current block based on the determined result, generating a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decoding the current block using the generated prediction block.

[0010] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may be performed by obtaining a reference pixel composed of a predetermined pixel value.

[0011] Here, determining whether a boundary pixel can be used as a reference pixel may be performed by indicating whether a predefined flag considers a prediction mode of a neighboring block.

[0012] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is usable as a reference pixel of the current picture when the prediction mode of the neighboring block is intra prediction.

[0013] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is not usable as a reference pixel of the current picture when the prediction mode of the neighboring block is inter prediction.

[0014] Here, when the prediction mode of the neighboring block is inter prediction, determining whether the boundary pixel is usable as a reference pixel may be performed by determining whether the boundary pixel is usable as a reference pixel of the current picture in consideration of the reference picture of the neighboring block.

[0015] Here, a reference picture may be selected from List 0 storing data of a reference picture before the current picture or List 1 storing data of a reference picture after the current picture, and the current picture may be selected from List 0 or List 1.

[0016] Here, when the reference picture of the neighboring block is the current picture, the boundary pixels may be determined to be usable as reference pixels of the current picture.

[0017] Here, if the reference picture of the neighboring block is not the current picture, it may be determined that the boundary pixel cannot be used as the reference pixel of the current picture.

[0018] Here, when the reference picture of the neighboring block is an I picture, the boundary pixels may be determined to be usable as reference pixels of the current picture.

[0019] In order to achieve other objects of the present invention, an image decoding method for performing intra-frame prediction according to another aspect of the present invention includes: receiving a bit stream including data on prediction modes of a current block and blocks adjacent to the current block, confirming the prediction modes of the adjacent blocks by extracting data from the received bit stream, determining whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block in consideration of the prediction modes of the adjacent blocks, obtaining reference pixels of the current block according to the determined result, generating a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decoding the current block using the generated prediction block.

[0020] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may be performed by obtaining a reference pixel constructed by a preset pixel value.

[0021] Here, determining whether a boundary pixel can be used as a reference pixel may indicate whether a preset flag considers a prediction mode of a neighboring block.

[0022] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is usable as a reference pixel of the current picture when the prediction mode of the neighboring block is intra prediction.

[0023] Here, determining whether the boundary pixel is usable as a reference pixel may be performed by determining that the boundary pixel is not usable as a reference pixel of the current picture when the prediction mode of the neighboring block is inter prediction.

[0024] Here, considering a reference picture of an adjacent block, determining whether a boundary pixel is usable as a reference pixel may be performed by determining whether the boundary pixel is usable as a reference pixel of a current picture.

[0025] Here, a reference picture may be selected from List 0 storing data of a reference picture before the current picture or List 1 storing data of a reference picture after the current picture, and the current picture may be included in List 0 or List 1.

[0026] Here, when the reference picture of the neighboring block is the current picture, it may be determined that the boundary pixels may be used as reference pixels of the current picture.

[0027] Here, when the reference picture of the neighboring block is not the current picture, it may be determined that the pixels in the neighboring block cannot be used as reference pixels of the current picture.

[0028] Here, when the reference picture of the neighboring block is an I picture, it may be determined that pixels in the neighboring block may be used as a reference picture of the current picture.

[0029] In order to achieve other objectives of the present invention, in an image decoding device including one or more processors according to another aspect of the present invention, the one or more processors can receive a bit stream including data about a prediction mode of a current block and blocks adjacent to the current block, extract data from the received bit stream to confirm the prediction mode of the adjacent block, determine whether boundary pixels within the adjacent block can be used as reference pixels of the current block considering the prediction mode of the adjacent block, obtain reference pixels of the current block based on the determined result, generate a prediction block by performing intra-frame prediction based on the obtained reference pixels, and decode the current block using the generated prediction block.

[0030] In order to achieve the purpose of the present invention, an image encoding method for performing intra-frame prediction according to one aspect of the present invention includes: determining whether each of the left block and the upper block of the current block can be used to construct reference pixels of the current block, selecting a most probable mode (MPM) candidate group based on the determined result, determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group, selecting the MPM group using the prediction mode of an alternative block adjacent to the left block or adjacent to the upper block when it is determined that redundancy exists, and performing intra-frame prediction mode encoding on the current block based on the selected MPM group.

[0031] Here, determining whether each block can be used may be performed by determining as unavailable when the left block and the upper block exist outside the image boundary or are encoded using inter prediction and determining as available in other cases.

[0032] Here, the MPM candidate group may be selected from the intra prediction mode of the block determined to be available among the left block and the upper block, the intra prediction mode predetermined for the block determined to be unavailable, the planar prediction mode, the vertical prediction mode, and the DC prediction mode.

[0033] Here, when both the left block and the upper block are available, determining redundancy may be performed by determining redundancy based on variance values ​​calculated for reference pixel values ​​in two or more consecutive blocks among blocks located to the lower left, left, upper left, above, and upper right of the current block.

[0034] Determining redundancy may include selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0035] Here, the replacement block may be selected from neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0036] Here, when a neighboring block is constructed with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0037] Here, the replacement block may be selected with a clockwise or counterclockwise priority among neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0038] According to another aspect of the present invention, an image decoding method for performing intra-frame prediction may include: extracting data on whether a prediction mode of a current block is included in a most probable mode (MPM) group from a received bit stream; determining whether the prediction mode of the current block is included in the MPM group based on the data; extracting MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group; confirming the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and performing intra-frame prediction on the current block based on the confirmed prediction mode.

[0039] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct reference pixels of the current block; selecting the most likely mode MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of an alternative block adjacent to the left block or to the upper block when it is determined that redundancy exists.

[0040] Here, determining whether a block can be used may be performed by determining as unavailable when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction and determining as available in other cases.

[0041] Here, the MPM candidate group may be selected from the intra prediction mode of the block determined to be available among the left block and the upper block, a predetermined intra prediction mode for the block determined to be unavailable, a planar prediction mode, a vertical prediction mode, and a DC prediction mode.

[0042] Here, when both the left block and the upper block are available, determining the redundancy of the prediction block can be performed by determining the redundancy based on the variance value calculated for the reference pixel values ​​in two or more consecutive blocks among the blocks located below the left, to the left, above the left, above, and above the right of the current block.

[0043] Here, determining the redundancy of the prediction block may be performed by selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0044] Here, the replacement block may be selected from neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0045] When constructing a neighboring block with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0046] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority.

[0047] To achieve the purpose, in an image decoding device including one or more processors according to other aspects of the present invention, the one or more processors may: extract data related to whether the prediction mode of the current block is included in the most probable mode MPM group from a received bit stream; determine whether the prediction mode of the current block is included in the MPM group based on the data; extract the MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group, confirm the prediction mode for the current block in the MPM group based on the extracted MPM group index data, and perform intra-frame prediction on the current block based on the confirmed prediction mode.

[0048] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct reference pixels of the current block; selecting the MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when it is determined that redundancy exists.

[0049] Here, determining whether a block can be used may be performed by determining as unavailable when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction and determining as available in other cases.

[0050] Determining the redundancy of the prediction block may select an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0051] Beneficial Effects

[0052] When the image encoding / decoding method and apparatus according to the embodiments of the present invention are used as described above, the accuracy of prediction can be improved by achieving improvements in reference pixel construction and most probable mode construction in intra prediction.

[0053] In addition, in intra-frame prediction, the encoding efficiency of the picture can be improved by improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0055] Figure 2 is a diagram of a construction of an image encoding device according to an embodiment of the present invention.

[0056] Figure 3 is a diagram of a construction of an image decoding device according to an embodiment of the present invention.

[0057] Figure 4 is an exemplary view illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention.

[0058] Figure 5 is an exemplary view illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

[0059] Figure 6 is an exemplary view illustrating a case where a prediction block is generated in a single direction in an image encoding and decoding method according to an embodiment of the present invention.

[0060] Figure 7 is an exemplary view of construction of a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0061] Figure 8 is an exemplary view illustrating another example of a case where inter prediction is performed from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0062] Fig. 9 is an exemplary view illustrating intra prediction in an image encoding method according to an embodiment of the present invention.

[0063] Fig.10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

[0064] Fig.11 is an exemplary view showing a process of obtaining a prediction block.

[0065] Fig.12 is an exemplary view showing a main procedure of an image encoding method according to an embodiment of the present invention using syntax in encoding units.

[0066] Fig.13 is an exemplary view showing an example of supporting symmetric type partitioning or asymmetric type partitioning as inter prediction when a prediction block is generated by block matching in a current picture.

[0067] Fig.14 is an exemplary view showing that 2N×2N and N×N can be supported in inter prediction.

[0068] Fig.15 is an exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0069] Fig.16ais an exemplary view showing whether to use reference pixels in consideration of encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0070] Fig.16b is an example diagram showing whether to use reference pixels regardless of encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0071] Fig.17 is another exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0072] Fig.18a 1 is an exemplary view showing a case where an encoding mode of a neighboring block is intra prediction in intra prediction according to an embodiment of the present invention.

[0073] Fig.18b 2 is an exemplary view illustrating a case where encoding modes and reference pictures of neighboring blocks are considered in intra prediction according to an exemplary embodiment of the present invention.

[0074] Fig.18c is another exemplary view illustrating a case where encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.

[0075] Fig.19a is an exemplary view showing a method of obtaining a reference pixel when an available neighboring block is located at the lower left end.

[0076] Fig.19b is an exemplary view showing a method of obtaining a reference pixel when an available neighboring block is located at the upper right end.

[0077] Fig.19c is an exemplary view showing a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0078] Fig.19d is an exemplary view showing a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0079] Fig. 20 is a flowchart of an image encoding method according to an embodiment of the present invention.

[0080] Fig.21 The diagram shows an example of an intra prediction mode of HEVC (high efficiency video coding).

[0081] Fig.22a is an exemplary view showing reference pixels used to predict a current block in a reference pixel construction step.

[0082] Figure 22b is an exemplary view showing that additional pixels of a neighboring block are included in a reference pixel candidate group.

[0083] Fig.23a , Figure 23b , Fig.24a and Figure 24b is an exemplary view showing various mode directions of a current block having a wider reference pixel that can be included in a reference pixel construction step.

[0084] Fig.25 is a flow chart illustrating a process of determining the most probable mode for a generic current prediction unit (or prediction block).

[0085] Fig.26 is an exemplary view of determining a prediction mode of a current prediction unit (or block).

[0086] Fig. 27 is an exemplary view of the group prediction mode.

[0087] Figure 28 to Figure 32 is an exemplary view of a case where an MPM is constructed for a prediction mode using neighboring blocks other than an upper block and a left block.

[0088] Fig.33 is an exemplary view of a case where a most probable mode (MPM) is constructed as a substitute block when both a current block and a neighboring block are available.

[0089] Fig.34 is a flowchart illustrating an image encoding method for performing intra-frame prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0090] Although the present invention is capable of various modifications and changes, its specific embodiments are shown in the accompanying drawings by way of example and will be described in detail herein. However, it should be understood that the present invention is not limited to specific embodiments, but includes all modifications, equivalents and replacements falling within the spirit and scope of the present invention.

[0091] Terms such as first, second, A, B, etc. can be used to describe various components, but these components should not be limited by these terms. These terms are only used to distinguish the purpose of one component from another component. For example, without departing from the scope of the present invention, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. The term "and / or" includes any combination of multiple related listed items or any one of multiple related listed items.

[0092] It should be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to another component, but it should be understood that other components can exist between them. On the other hand, when an element is referred to as being "directly connected" or "directly coupled" to another element, it should be understood that there are no other elements between them.

[0093] The terms used herein are only used for the purpose of describing specific embodiments and are not intended to limit the present invention. Unless the context clearly stipulates otherwise, the singular form includes plural indicators. In this application, the terms "including" or "having" are used to specify the presence of stated features, figures, steps, operations, elements, parts or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts or combinations thereof.

[0094] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those skilled in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the contextual meaning of the relevant art, and should not be interpreted as an ideal or excessive form of meaning unless explicitly defined in this application.

[0095] Typically, a motion picture may consist of a series of pictures, and each picture may be divided into predetermined areas such as blocks. In addition, the divided areas may be referred to as various sizes or terms such as coding tree units (CTU), coding units (CU), prediction units (PU), and transform units (TU), as well as blocks. Each unit may be constructed with one luminance block and two color difference blocks, which may be constructed differently according to the color format. In addition, the size of the luminance block and the color difference block may be determined according to the color format. For example, in the case of 4:2:0, the size of the color difference block may have a length of 1 / 2 of the width and height of the luminance block. For these units, reference may be made to items such as existing HEVC or H.264 / AVC. In the present invention, these blocks and items are used in a mixed manner, but they may be understood differently according to standard techniques, and they should be understood as corresponding items or units for encoding and decoding processing according to these standard techniques.

[0096] In addition, a picture, block or pixel referenced when encoding or decoding a current block or current pixel is referred to as a reference picture, reference block or reference pixel. It will be appreciated by those skilled in the art that the term "picture" described below can be used to replace other terms such as image, frame, etc. having equivalent meanings.

[0097] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In order to facilitate a thorough understanding of the present invention, the same reference numerals are used for the same components in the drawings, and repeated descriptions of the same components are omitted.

[0098] Figure 1 is a conceptual diagram of an image encoding and decoding system according to an embodiment of the present invention.

[0099] refer to Figure 1 , the image encoding device 12 and the decoding device 11 can be implemented by a user terminal such as a personal computer (PC), a notebook computer, a personal digital assistant (PDA), a portable multimedia player (PMP), a portable game console (PlayStation Portable, PSP), a wireless communication terminal, a smart phone, a television, etc., or a server terminal such as an application server and a service server, and can include: a communication device such as a communication modem for performing communication with various machines or a wired / wireless communication network; a memory 18 for encoding or decoding an image or storing various programs and data for inter-frame or intra-frame prediction for encoding or decoding; a processor 14 for calculating and controlling via program execution, etc. In addition, the video encoded as a bit stream by the video encoding device 12 can be sent to the video decoding device and then decrypted by the video decoding device to be reconstructed and reproduced as an image in real time or non-real time through a wired or wireless communication network such as the Internet, a local area wireless communication network, a wireless LAN network, a WiBro network, a mobile communication network, etc., or various communication interfaces such as a cable, a universal serial bus (USB), etc.

[0100] Furthermore, an image encoded by a video encoding device may be transmitted as a bit stream from the encoding device to the decoding device through a computer-readable recording medium.

[0101] Figure 2 is a block diagram of an image encoding apparatus according to an embodiment of the present invention. Figure 3 is a block diagram of an image decoding apparatus according to an embodiment of the present invention.

[0102] like Figure 2 As shown in , the image encoding device 20 according to this embodiment includes a prediction unit 200, a subtraction unit 205, a transform unit 210, a quantization unit 215, a dequantization unit 220, an inverse transform unit 225, an addition unit 230, a filter 235, a decoding picture buffer 240 and an entropy encoding unit 245.

[0103] like Figure 3 As shown in , the image decoding device 30 according to the present embodiment includes an entropy decoding unit 305, a prediction unit 310, a dequantization unit 315, an inverse transform unit 320, an addition unit 325, a filter unit 330 and a decoded picture buffer 335.

[0104] The image encoding device 20 and the image decoding device 30 may be configured separately, but they may be combined into one image encoding and decoding device according to an embodiment. In this case, the prediction unit 200, the dequantization unit 220, the inverse transform unit 225, the addition unit 230, the filter 235, and the memory 240 of the image encoding device 20 and the prediction unit 310, the dequantization unit 315, the inverse transform unit 320, the addition unit 325, the filter unit 330, and the memory 335 of the image decoding device 30 are substantially the same technical elements in the disclosed order, and they may include the same structure, or may be implemented to perform at least the same function. In addition, when the entropy encoding part 245 performs its function in reverse, the entropy encoding part 245 may correspond to the entropy decoding unit 305. Therefore, in the following description of the technical elements and their operating principles, the repeated description of the corresponding technical elements will be omitted.

[0105] Since the image decoding device 30 corresponds to a computing device that applies the image encoding method performed in the image encoding device 20 to decoding, the following description will focus on the image encoding device 20.

[0106] The computing device may include a memory storing a program or software module implementing the image encoding method and / or the image decoding method, and a processor coupled to the memory and running the program. The image encoding device may be referred to as an encoder, and the image decoding device may be referred to as a decoder.

[0107] Each component of the image encoding device of this embodiment will be described in more detail below.

[0108] Here, the image encoding device 20 may further include a division unit. The division unit divides the input image into blocks of a predetermined size (M×N). Here, M or N is any natural number greater than 1. In detail, the division unit may be constructed with a picture division unit and a block division unit. The size or type of the block may be determined according to the characteristics and resolution of the image, and the size or type of the block supported by the picture division unit may be an M×N square type (256×256, 128×128, 64×64, 32×32, 16×16, 8×8, 4×4, etc.) or an M×N rectangular type whose horizontal length and vertical length are expressed as a power of 2. For example, for an 8k UHD image with high resolution, the input image may be divided into 256×256, for a 1080p HD image, the input image may be divided into 128×128, and for a WVGA image, the input image may be divided into 16×16.

[0109] Information about the size or type of such a block may be set in units of a sequence, a picture, a slice, etc., and the relevant information may be sent to a decoder. That is, it may be set in units of a sequence parameter set, a picture parameter set, a slice header, or a combination thereof.

[0110] Here, a sequence indicates a constituent unit formed by collecting several related scenes. A picture is referred to as a series of luminance (Y) components or the sum of luminance + chrominance (Y, Cb, Cr) components in one scene or picture, and the range of one picture can be one frame or one domain.

[0111] A slice can refer to an independent slice segment and several subordinate slice segments in the same access unit. An access unit refers to a collection of network abstraction layer (NAL) units associated with a single coded picture. A NAL unit is a syntactic structure of a video compression bitstream constructed in a network-friendly format in the H.264 / AVC and HEVC standards. Typically, a slice unit is constructed as a NAL unit, and a collection of NALs or NALs that constitute a frame is considered an access unit in the system standard.

[0112] Returning to the description of the picture division unit, information related to the size or type (M×N) of the block can be made by explicit marking, and specifically can include block type information, one length information when the block is square, each length information when the block is rectangular, or difference information between width and length. For example, if M and N are composed of powers of k (assuming k is 2) (M=2 m , N = 2 n), the information related to m and n can be encoded by various methods such as unary binarization and truncated unary binarization, so that the relevant information can be sent to the decoding device. Alternatively, the minimum allowable partition size Minblksize supported by the picture partition unit can be sent I×J (for ease of explanation, if I=2 i , J = 2 j , then assume I = J), mI or nj. As another example, when M and N are different from each other, the difference between m and n (|mn|) may be transmitted. Alternatively, the maximum allowable partition size Maxblksize supported by the picture partition unit may be transmitted I×J (for ease of explanation, if I = 2 i , J = 2 j , then assume I=J), information of im or nj.

[0113] In the implicit case, for example, when the syntax of the relevant information exists but cannot be verified by the encoder or decoder, the encoder or decoder can follow the preset default setting. For example, when the relevant syntax cannot be confirmed when confirming the block type information, the block type can be set to a square type, which is a default setting. Alternatively, in the step of confirming the block size information, specifically, in the step of confirming the block size information by the difference with the minimum allowable partition size Minblksize as in the above example, when the syntax related to the difference can be recognized but the syntax related to the minimum allowable partition size Minblksize cannot be recognized, the information can be obtained from the default value related to the preset minimum allowable partition size Minblksize.

[0114] As described above, the size or type of the block in the picture division unit may be explicitly transmitted as relevant information from an encoder or a decoder, or may be implicitly determined according to the characteristics and resolution of an image, and the like.

[0115] As described above, the blocks divided and determined by the picture division unit may be used as basic coding units. The blocks divided and determined by the picture division unit may be the smallest units constituting high-level units such as pictures, slices, and tiles, and may be the largest units of blocks such as coding blocks, prediction blocks, transform blocks, quantization blocks, entropy blocks, and in-loop filtering blocks, but some blocks are not limited thereto and may be exceptions.

[0116] For example, some blocks such as an in-loop filtering block may be applied as a unit larger than the size of the above-mentioned blocks.

[0117] The block partitioning unit performs partitioning on blocks such as coding, prediction, transformation, quantization, entropy, and in-loop filtering. The partitioning unit is included in each component to perform its function. For example, the transform unit 210 may include a transform block partitioning unit, and the quantization unit 215 may include a quantization block partitioning unit. The size or type of the initial block in the block partitioning unit may be determined by the partitioning result of the block at the previous level or higher level. For example, in the case of a coding block, a block obtained by a picture partitioning unit as a previous level may be set as an initial block. Alternatively, in the case of a prediction block, a block obtained by a partitioning process of a higher-level coding block as a prediction block may be set as an initial block. Alternatively, in the case of a transform block, a block obtained by a partitioning process of a higher-level coding block as a transform block may be set as an initial block. The conditions for determining the size or type of the initial block are not always fixed, and parts may be changed or there may be special cases. In addition, the splitting process at the current level (e.g., whether it can be split, the type of divisible blocks, etc.) may also be affected by a combination of at least one factor such as the splitting state of blocks at a previous level or a higher level (e.g., the size of the coding block, the type of the coding block, etc.) and the setting conditions of the current level (e.g., the size of the supported transform blocks, the type of transform blocks, etc.).

[0118] The block partitioning unit can support a quadtree-based partitioning scheme. That is, the block can be divided into four blocks, each of which has a length of 1 / 2 in horizontal and vertical lines in the block before the partition. This means that the partitioning can be repeated until the allowed partition depth limit dep_k is reached (dep_k represents the number of allowed partitions, and the size of the block of the allowed partition depth limit is M>>k, N>>k).

[0119] In addition, a binary tree-based partitioning scheme can be supported. This indicates that one of the horizontal and vertical lines can be divided into two blocks with a length of 1 / 2 compared to the block before the partition. The case of quadtree partitioning and binary tree partitioning can be symmetric partitioning (symmetric partition) or asymmetric partitioning (asymmetric partition), and which partitioning scheme is selected can be determined according to the setting of the encoder or decoder. The symmetric partitioning scheme will be mainly described in the image encoding method of the present invention.

[0120] The division flag (div_flag) may indicate whether each block is divided. If the corresponding value is 1, the division is performed, and if the value is 0, the division is not performed. Alternatively, if the corresponding value is 1, the division is performed and additional division is allowed, and if the value is 0, the division is not performed and additional division is not allowed. Depending on conditions such as the minimum allowable division size and the allowable division depth limit, the flag may only consider whether the division is performed and may not consider whether additional division is performed.

[0121] The partition flag can be used in quadtree partitioning or in binary tree partitioning. In binary tree partitioning, the partition direction can be determined according to the partition depth, coding mode, prediction mode, size, type and category of the block (one of coding, prediction, transform, quantization, entropy, in-loop filtering, etc., or one of brightness and chrominance) and at least one factor such as slice type, partition allowable depth limit, minimum and maximum allowable partition size, or a combination thereof.

[0122] In this case, according to the division direction corresponding to the division flag, the block can be divided by 1 / 2 only in the horizontal direction of the block, or by 1 / 2 only in the vertical direction. For example, when it is assumed that the block supports M×N (M>N) and M is greater than N, horizontal division is supported, and when it is assumed that the current division depth (dep_curr) is less than the allowable division depth limit that allows additional division to be performed, the division flag is assigned 1 bit so that horizontal division is performed when the corresponding value is 1, and no further division is performed when the value is 0. The division depth can be a division depth for quadtree division and binary tree division and each division depth for quadtree division and binary tree division. The allowable division depth limit can be an allowable division depth limit for quadtree division and binary tree division and each allowable division depth limit for quadtree division and binary tree division.

[0123] As another example, if the block is M×N (M>N) and N is equal to a predetermined minimum allowable partition size and horizontal partitioning is not supported, the partition flag is assigned 1 bit so that vertical partitioning is performed when the value is 1, and if the value is 0, no further partitioning is performed.

[0124] And, the flags (div_h_flag, div_v_flag) for horizontal division or vertical division can be supported respectively, and binary division can be supported according to the flag. The horizontal division flag (div_h_flag) or the vertical division flag (div_v_flag) indicates whether the horizontal or vertical division of each block is allowed. When the horizontal division flag (div_h_flag) or the vertical division flag is 1, the horizontal or vertical division is performed, and when the flag is 0, the horizontal or vertical division is not performed. Or when each flag is 1, the horizontal or vertical division is performed, and the additional horizontal or vertical division is allowed. When the value is 0, the horizontal or vertical division is not performed, and further horizontal or vertical division may not be allowed. According to conditions such as the minimum allowable division size and the allowable division depth limit, the flag can consider whether to perform the division, or whether to perform the additional division. Alternatively, the flag (div_flag / h_v_flag) for horizontal division or vertical division can be supported, and binary division can be supported according to the flag. The division flag (div_flag) can indicate whether to perform horizontal or vertical division, and the division direction flag (h_v_flag) can indicate the horizontal or vertical division direction. When the division flag (div_flag) is 1, the division is performed and the horizontal or vertical division is performed according to the division direction flag (h_v_flag), and the horizontal or vertical division is not performed when the value is 0. If the value is 1, the horizontal or vertical division is performed according to the horizontal division flag (h_v_flag), and additional horizontal or vertical division is allowed. When the value is 0, the horizontal or vertical division is not performed, and it is considered that further horizontal or vertical division is not allowed. Depending on conditions such as the minimum allowable division size and the allowable division depth limit, the flag can consider whether to perform the division, and whether to perform additional division.

[0125] These split flags can support horizontal and vertical splits respectively, and also support binary tree splits according to the flags. Also, when the split direction is predetermined, as in the above example, only one of the two split flags can be used, or both split flags can be used.

[0126] For example, when all flags are allowed, the possible types of blocks can be divided into any one of M×N, M / 2×N, M×N / 2, and M / 2×N / 2. In this case, the flags can be encoded as 00, 01, 10, and 11 (in the order of div_h_flag / div_v_flag). The above case is an example of a setting in which the division flags can be overlapped, rather than an example of a setting in which the division flags cannot be overlapped. For example, it is also possible to divide the division block types into M×N, M / 2×N, and M×N / 2, and in this case, the above flags are encoded as 00, 01, 10 (in the order of div_h_flag / div_v_flag), or encoded as 00, 10, and 11 (in the order of div_flag / h_v_flag, h_v_flag is a flag indicating whether the division direction is horizontal or vertical). Here, overlapping means that both horizontal division and vertical division are performed simultaneously. The above-mentioned quadtree division and binary tree division can be used alone or in combination according to the settings of the encoder or decoder. For example, quadtree or binary tree partitioning can be determined according to block size or type. That is, horizontal partitioning is performed when the block type is M×N and M is greater than N, binary partitioning can be supported according to vertical partitioning when the block type is M×N and N is greater than M, and quadtree partitioning can be supported when the block type is M×N and N and M are the same.

[0127] As another example, if the size of the block (M×M) is greater than or equal to the block partition threshold (Thrblksize), binary tree partitioning may be supported; if less than, quadtree partitioning may be supported.

[0128] In another example, quadtree partitioning is supported when M or N of the block (M×N) is greater than or equal to the first minimum allowable partition size (Minblksize1), and binary tree partitioning can be supported when M or N of the block (MxN) is greater than or equal to the second minimum allowable partition size (Minblksize2). When the first partition support range and the second partition support range that can be defined by the maximum allowable partition size and the minimum allowable partition size overlap, the first or second partition method can be given priority according to the setting of the encoder or decoder. In this example, the first partition method is quadtree partitioning, and the second partition method is binary tree partitioning. For example, when the first minimum allowable partition size (Minblksize1) is 16, the second maximum allowable partition size (Maxblksize2) is 64, and the block before partitioning is 64×64, it is possible to allow quadtree partitioning and binary tree partitioning to be allowed within both the first partition support range and the second partition support range. When the first partitioning method (quadtree partitioning in this example) is given priority according to the setting, quadtree partitioning is performed, and additional quadtree partitioning is allowed when the partition flag (div_flag) is 1, and it can be considered that the quadtree partitioning is not performed and the quadtree partitioning is no longer performed when the partition flag (div_flag) is 1. According to such conditions as the minimum allowable partition size and the allowable partition depth limit, the flag can only consider whether to perform the partition, and whether to perform further partitioning can be ignored. When the partition flag (div_flag) is 1, since the block is divided into 4 blocks of a size of 32×32 larger than the first minimum allowable partition size (Minblksize 1), the quadtree partitioning can be continued. If the flag is 0, no further quadtree partitioning is performed, and the current block size (64×64) is within the second partition support range, thereby it is possible to perform binary tree partitioning. If the partition flag (in the order of div_flag / h_v_flag) is 0, no further partitioning is performed. If the split flag is 10 or 11, horizontal splitting or vertical splitting can be performed. If the block before splitting is 32×32 and the split flag (div_flag) is 0, so that no further quadtree splitting is performed, and if the second maximum allowable split size (Maxblksize2) is 16, the size of the current block (32×32) is not within the second split support range, so that splitting is no longer supported. In the above description, the priority of the splitting method can be determined according to at least one or a combination of slice type, coding mode, brightness and chrominance components, etc.

[0129] As another example, various settings can be supported based on luma and chroma components. For example, a quadtree or binary tree partitioning structure determined by the luma component can also be used in the chroma component without encoding or decoding additional information. Alternatively, when independent partitioning of luma and chroma components is supported, quadtree + binary tree partitioning can be supported for the luma component, and quadtree partitioning can be supported for the chroma component. Alternatively, when quadtree + binary tree partitioning is supported in luma and chroma components, the partitioning support range may or may not be the same as or proportional to the luma and chroma components. For example, if the color format is 4:2:0, the partitioning support range of the chroma component may be N / 2 of the partitioning support range of the luma component.

[0130] As another example, different settings may be made according to the type of slice. For example, I slices may support quadtree partitioning, P slices may support binary tree partitioning, and B slices may support quadtree + binary tree partitioning.

[0131] Quadtree partitioning and binary tree partitioning can be set and supported according to various conditions such as in the above examples. The above examples are not limited to the above conditions, and may include situations where the conditions are reversed from each other, may include one or more factors mentioned in the above examples or a combination thereof, and may also be transformed into another example. The allowable partition depth limit can be determined according to at least one factor or a combination thereof in the partitioning method (quadtree, binary tree), slice type, brightness and chrominance components, coding mode, etc. In addition, the partition support range can be determined according to at least one factor or a combination thereof in the partitioning method (quadtree), slice type, brightness and chrominance components, coding mode, etc., and the relevant information that can be represented as the maximum and minimum values ​​of the partition support range. When the information related to it is constructed with a display flag, the length information of each of the maximum and minimum values, or the difference information between the minimum and maximum values ​​can be represented. For example, when the maximum and minimum values ​​are composed of powers of k (assuming k is 2), the power information of the maximum and minimum values ​​can be encoded and sent to the decoding device through various binarizations. Alternatively, the difference between the powers of the maximum and minimum values ​​can be sent. The information sent at this time can be the power information of the minimum value and the difference information of the power.

[0132] According to the above description, information related to a flag may be generated and transmitted in units such as a sequence, a picture, a slice, a tile, a block, and the like.

[0133] The partition flag shown in the above example can represent the block partition information by a quadtree or a binary tree or a mixture of the two tree schemes, and the partition flag can be encoded by various methods such as unary binarization and truncated unary binarization to send the relevant information to the decoding device. The bitstream structure of the partition flag used to represent the partition information of the block can be selected from one or more scanning methods. For example, you can construct the bitstream of the partition flag based on the partition depth order (from dep0 to dep_k), and you can also construct the bitstream of the partition flag based on whether the partition is performed. The method based on the partition depth order allows the partition information of the current level depth to be obtained based on the initial block, and then the partition information is obtained at the next level. The method based on whether the partition is performed indicates that additional partition information is preferably obtained in the partition block based on the initial block, and other scanning methods not shown in the above examples can be included and selected. In addition, according to this embodiment, the block partition unit can generate and represent index information of a block candidate group of predefined types other than the above-mentioned partition flags. The types of block candidates may include M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2, etc., for example, as the types of partition blocks that may be included in the block before partitioning. When the candidate group of the partition block is determined as described above, the index information of the partition block type may be encoded by various methods such as fixed-length binarization, cut-off-truncation binarization, truncation binarization, etc. The candidate group of the partition block may be determined according to at least one factor or a combination thereof such as a partition depth, a coding mode, a prediction mode, a size, a type, a kind (sort) and a slice type, an allowable partition depth limit, and a maximum and minimum allowable partition size. For further description, it is assumed that (M×N, M / 2×N / 2) is candidate list 1, (M×N, M / 2×N, M×N / 2, and M / 2×N / 2) is candidate list 2, (M×N, M / 2×N, M×N / 2) is candidate list 3, and (M×N, M / 2×N, M×N / 2, M / 4×N, 3M / 4×N, M×N / 4, M×3N / 4, M / 2×N / 2) is candidate list 4. For example, in the case of M×N, when (M=N), the partition block candidates of candidate list 2 can be supported, and when (M≠N), the partition block candidates of candidate list 3 can be supported.

[0134] As another example, it is possible to support partition block candidates of candidate list 2 when M or N of M×N is greater than or equal to the boundary value blk_th, and partition block candidates of candidate list 4 when M or N is less than the boundary value blk_th. Alternatively, it is possible to support partition block candidates of candidate list 1 when M or N is equal to or greater than or equal to the first boundary value blk_th_1, partition block candidates of candidate list 1 when M or N is less than the first boundary value blk_th_1 and greater than or equal to the second boundary value blk_th_2, and partition block candidates of candidate list 4 when M or N is less than the second boundary value blk_th_2.

[0135] As another example, when the encoding mode is intra prediction, the partition block candidates of candidate list 2 may be supported, and when the encoding mode is inter prediction, the partition block candidates of candidate list 4 may be supported.

[0136] Even when partition block candidates are supported as described above, the bit construction according to the binarization of each block may be the same or different. For example, as in the case of a partition flag, when the supported partition block candidates are limited according to the size or type of the block, the bit construction according to the binarization of the block candidate may vary. For example, in the case of (M>N), it is possible to support block types according to horizontal partitioning, i.e., M×N, M×N / 2, and M / 2×N / 2, wherein the binary bits of the index of M×N / 2 according to the partition block candidate group M×N, M / 2×N, M×N / 2, M / 2×N / 2, and M×N / 2 of the current condition may be different from each other. Depending on the type of block used for encoding, prediction, transformation, quantization, entropy, in-loop filtering, etc., one of the partition flag method or the partition index method may be used to represent information related to the partition and type of the block. Furthermore, the block size limit and the allowable partition depth limit for supporting the partition and type of the block may differ according to the type of each block.

[0137] The encoding or decoding process of the block unit can be performed according to the process of prediction block determination, transform block determination, quantization block determination, entropy block determination, and in-loop filtering determination after the coding block is determined. The order of the encoding or decoding process is not always fixed, and some orders may be changed or excluded. The size and type of each block are determined according to the encoding cost of each candidate of the size and type of the block, and the image data of each determined block and the division related information of the size and type of each determined block can be encoded.

[0138] The prediction unit 200 can be implemented using a prediction module as a software module, and can generate a prediction block using an intra prediction scheme or an inter prediction scheme for a block to be encoded. Here, a prediction block is understood to be a block that closely matches a block to be encoded in terms of pixel differences, and can be determined by various methods including the sum of absolute difference (SAD) and the sum of square difference (SSD). And, at this time, various syntaxes used when decoding an image block can be generated. The prediction block can be classified into an intra block and an inter block according to the coding mode.

[0139] Intra prediction (intra prediction) is a prediction technique using spatial correlation. It refers to a method of predicting a current block using reference pixels of a block reconstructed by previously being encoded and decoded in the current picture. In other words, intra prediction can use the luminance value reconstructed by prediction and reconstruction as a reference pixel in the encoder and decoder. Intra prediction is effective for flat areas with continuity and areas with constant directionality, and because it uses spatial correlation, it can be used for the purpose of guaranteeing random access and preventing error diffusion.

[0140] Inter-frame prediction uses a compression technique that removes redundancy of data by using time correlation by referring to an image encoded with one or more past or future pictures. That is, inter-frame prediction can generate a prediction signal with high similarity by referring to one or more past or future pictures. The encoder can use inter-frame prediction to search for a block with high correlation with the block to be encoded in the reference picture, and send the position information and residual signal of the selected block to the decoder, and the decoder can use the selected information of the sent image to generate the same prediction block as the encoder, and compensate the sent residual signal to construct a reconstructed image.

[0141] Figure 4 is an exemplary view illustrating inter prediction of a P slice in an image encoding and decoding method according to an embodiment of the present invention. Figure 5 is an exemplary view illustrating inter prediction of a B slice in an image encoding and decoding method according to an embodiment of the present invention.

[0142] In the video encoding method of the present embodiment, since inter-frame prediction generates a prediction block from a previously encoded picture with high temporal correlation, encoding efficiency can be improved. Current (t) may refer to a current picture to be encoded, and may include a first reference picture t-1 having a first temporal distance (t-1) earlier than a picture order count POC of the current picture, and a second reference picture t-2 earlier than the first temporal distance of a temporal stream based on a picture or a picture order count POC.

[0143] like Figure 4 As shown in, in the case of inter-frame prediction that can be adopted in the video encoding method of this embodiment, motion prediction is performed on blocks with high correlation to find the best prediction block from the earlier encoded reference pictures t-1 and t-2 by block matching of the current block of the current picture current (t) with the reference blocks of the reference pictures t-1 and t-2. In order to perform accurate prediction, an interpolation process is performed based on a structure in which at least one sub-pixel is arranged between two adjacent pixels, the best prediction block is found, and then motion compensation is performed to find the result prediction block.

[0144] like Figure 5 As shown in, the inter-frame prediction that can be used in the video encoding method of this embodiment allows the generation of prediction blocks from reference pictures t-1 and t+1 that have been encoded in two directions based on the current picture current (t) in time. In addition, two prediction blocks can be generated from one or more reference pictures.

[0145] When an image is encoded by inter-frame prediction, motion vector information and reference picture information of the best prediction block are encoded. In this embodiment, when a prediction block is generated in a unidirectional or bidirectional direction, the reference block list is constructed differently so that the prediction block can be generated from the corresponding reference picture list. Basically, the reference picture existing before the current picture is assigned to list 0, and the reference picture existing after the current picture is assigned to list 1. When reference picture list 0 may not be filled to the allowable reference picture number of reference picture list 0, a reference picture existing later than the current picture can be assigned. Similarly, when reference picture list 1 cannot be filled to the allowable reference picture number of reference picture list 1, a reference picture existing later than the current picture can be assigned. Similarly, when constructing reference picture list 1, when reference picture list 1 does not satisfy the reference picture list, a reference picture existing before the current picture can be assigned.

[0146] Figure 6 1 is an exemplary view illustrating a case where a prediction block is generated in a single direction in an image encoding and decoding method according to an embodiment of the present invention.

[0147] refer to Figure 6In the image encoding and decoding method according to this embodiment, the prediction block can be found from the previously encoded reference pictures t-1, t-2, and the prediction block can also be found from the area that has been encoded in the current picture current (t).

[0148] That is, the image encoding and decoding method according to the present embodiment can be implemented so that not only a prediction block is generated from previously encoded pictures t-1, t-2 with high temporal correlation, but also a prediction block with high spatial correlation is found. Finding a prediction block with such spatial correlation can correspond to finding a prediction block in an intra-frame prediction manner. In order to perform block matching from an area where encoding is completed in the current picture, the image encoding method of the present embodiment can construct a syntax for information related to prediction candidates mixed with an intra-frame prediction mode.

[0149] For example, when n (n is an arbitrary natural number) intra-frame prediction modes are supported, a mode is added to the intra-frame prediction candidate group to support n+1 modes, and the prediction mode can be encoded using M fixed bits satisfying 2M-1≤n+1≤2M. In addition, one of the candidate groups of prediction modes with high probability, such as the most probable mode MPM of high-efficiency video coding HEVC, can be selected. It can also be preferably encoded in higher-level prediction mode encoding.

[0150] When a prediction block is generated by block matching in the current picture, the video encoding method of this embodiment can construct a syntax for related information mixed with the inter-frame prediction mode. The additional related prediction mode information can be motion or displacement related information. The motion or displacement related information can include the best candidate information among various vector candidates, the difference between the best candidate vector and the actual vector, the reference direction, the reference picture information, etc.

[0151] Figure 7 is an exemplary view of reference picture list construction in an image encoding and decoding method according to an embodiment of the present invention. Figure 8 is an exemplary view showing another example of a case where inter prediction is performed from a reference picture list in an image encoding and decoding method according to an embodiment of the present invention.

[0152] refer to Figure 7 , the image encoding method according to an embodiment of the present invention can perform inter-frame prediction on the current block of the current picture current (t) according to the reference picture list (reference list 0, reference list 1).

[0153] refer to Figure 7 and Figure 8, the reference picture list 0 may be constructed from reference pictures earlier than the current picture (t), where t-1 and t-2 indicate reference pictures having a first temporal distance (t-1) and a second temporal distance (t-2), respectively, earlier than the picture order count POC of the current picture (t). In addition, the reference picture list 1 may be composed of reference pictures later than the current picture (t), where t+1 and t+2 indicate reference pictures having a first temporal distance (t+1) and a second temporal distance (t+2), respectively, later than the POC of the current picture (t).

[0154] The above example of reference picture list construction shows an example in which the reference picture list consists of reference pictures having a difference of 1 in temporal distance (based on POC in this example), but it can be constructed so that the difference in temporal distance between reference pictures is different. This can be confirmed in the example of reference picture list construction described later.

[0155] Prediction can be performed from the reference pictures in the list according to the slice type (I, P or B). When a prediction block is generated via block matching in the current picture (current (t)), the current picture can be added to the reference picture list (reference list 0, reference list 1) to perform inter-frame prediction coding.

[0156] like Figure 8 As shown in , it is possible to add the current picture (t) to the reference picture list 0 or to add the current picture (t) to the reference picture list 1. That is, the reference picture list 0 can be constructed by adding the reference picture of the temporal distance (t) to the reference picture earlier than the current picture (t), and the reference picture list 1 can be constructed by adding the reference picture of the temporal distance (t) to the reference picture later than the current picture (t).

[0157] For example, when constructing reference picture list 0, a reference picture before the current picture may be assigned to reference picture list 0, and then the current picture (t) may be assigned to reference picture list 0. When constructing reference picture list 1, a reference picture may be assigned to reference picture list 1, and then the current picture (t) may be assigned to reference picture list 1. Alternatively, when constructing reference picture list 0, the current picture (t) is assigned, and then the reference picture before the current picture may be assigned. When constructing reference picture list 1, the current picture (t) is assigned, and then the reference picture after the current picture may be assigned. Alternatively, when constructing reference picture list 0, the reference picture before the current picture may be assigned, the reference picture after the current picture may be assigned, and then the current picture (t) may be assigned. Similarly, when constructing reference picture list 1, the reference picture after the current picture may be assigned, the reference picture before the current picture may be assigned, and then the current picture (t) may be assigned. The above examples are not limited to the above cases, but may include cases where the conditions are reversed from each other, and may be transformed into examples of other cases.

[0158] Whether to include the current picture in each reference picture list (e.g., not to add to the list, to add to list 0 only, to add to list 1 only, or to add to list 0 and 1) can be set in the encoder or the decoder, and information related thereto can be transmitted in units of sequence, picture, slice, etc. The information can be encoded by methods such as fixed-length binarization, cut-off-truncated binarization, truncated binarization, etc.

[0159] and Figure 7 Unlike the method of , the image encoding and decoding method of this embodiment allows selecting a prediction block by performing block matching on a current picture (t), configuring a reference picture list including relevant information about the prediction block, and performing encoding and decoding on the image by using the reference picture list.

[0160] In the construction of the reference picture list, the construction order and rule of each list and the allowable number of reference pictures of each list may be set differently, wherein they may be determined according to at least one factor of whether the current picture is included in the list (including the current picture as a reference picture in inter-frame prediction), the slice type, the list reconstruction parameter (which may be applied to lists 0 and 1 respectively, and may be applied to lists 0 and 1 together), the position of the group of pictures (group of picture, GOP), the temporal layer information (temporal id), etc., or a combination of at least one of them, and the relevant information may be explicitly transmitted in units such as sequences, pictures, etc. For example, in the case of a P slice, regardless of whether the current picture is included in the list, the reference picture list 0 may follow the list construction rule A, and in the case of a B slice, the reference picture list 0 including the current picture in the list may follow the list construction rule B, the reference picture list 1 may follow the list construction rule C, and the reference picture list 0 not including the current picture may follow the list construction rule D, and the reference picture list 1 may follow the list construction rule E, wherein the construction of B and D and C and E in the list construction rules may be the same. The list construction rules may be constructed in the same or modified manner as described in the reference picture list construction example. As another example, when the current picture is included in the list, a first allowable reference picture number may be set, and when the current picture is not included in the list, a second allowable reference picture number may be set. The first allowable reference picture number and the second allowable reference picture number may be the same or different from each other, and the difference between the first allowable reference picture number and the second allowable reference picture number may be set to 1 by default. As another example, when the current picture is included in the list and the list reconstruction parameter is applied, all reference pictures in slice A may be a list reconstruction candidate group, and only some reference pictures in slice B may be included in the list reconstruction candidate group. In this case, slice A or B may be classified according to whether the current picture is included in the list, temporal layer information, slice type, position of a group of pictures (GOP), etc. The factors of whether to be included in the candidate group may be determined according to the picture order count (POC) of the reference picture or reference picture index, the reference prediction direction (earlier and later than the current picture), whether there is a current picture, etc.

[0161] According to the above construction, since a reference block encoded with inter prediction can be used in the current picture, inter prediction can be allowed or used even in motion estimation of an I slice.

[0162] Also, when constructing a reference picture list, the index allocation or list construction order may be different depending on the slice type. In the case of an I slice, a low index (e.g., idx=0, 1, 2) is used by increasing the priority level in the current picture (t), and the amount of bits in image encoding can be reduced by binarization (fixed length binarization, cut-off binarization, truncation binarization, etc.) that maximizes the allowable number of reference pictures C of the reference picture list. In the case of a P slice or a B slice, when it is determined that the probability of selecting the reference picture of the current block as a prediction candidate by performing block matching in the current picture is less than the probability of selecting the prediction candidate by other reference pictures, the amount of bits in image encoding can be reduced via various methods, wherein the allowable number of reference pictures of the corresponding reference picture list is set to the maximum by setting the priority of the block matching of the current picture backward using a high index (e.g., idx=C, C-1). In the above example, the priority setting of the current picture can be constructed in the same or modified manner as described in the reference picture list construction example. By not constructing a reference picture list according to a slice type (e.g., I slice), information related to the reference picture can be omitted. For example, a prediction block can be generated by existing inter-frame prediction, but the inter-frame prediction information can be represented by information obtained by excluding reference picture information from motion information in the inter-frame prediction mode.

[0163] The method for performing block matching in the current picture can determine whether block matching is supported according to the slice type. For example, block matching in the current block is supported by I slices, but not P slices or B slices, or other deformations are possible. In addition, the method for supporting block matching in the current picture can determine whether block matching is supported in units of pictures, slices, tiles, etc., or based on the position of the group of pictures (GOP), time layer information (time ID), etc. Such setting information can be sent from the image encoding process or encoder to the decoder in units of sequences, pictures, slices, etc. In addition, even in the case where the above-mentioned setting information or syntax exists in a high-level unit and the setting-related operation is turned on, when the same setting information or syntax exists in a low-level unit, the setting information of the low-level unit can have a priority over the setting information in the high-level unit. For example, when the same or similar setting information is processed in a sequence, picture or slice, the picture unit can have a priority over the sequence unit, and the slice unit can have a priority over the picture unit.

[0164] Fig. 9 is an exemplary view illustrating intra prediction in an image encoding method according to an embodiment of the present invention.

[0165] refer to Fig. 9 As shown, the intra-frame prediction method according to an embodiment of the present invention includes reference sample filling, reference sample filtering, intra-frame prediction and boundary filtering.

[0166] The reference pixel filling step may be an example of a reference pixel construction step, the reference pixel filtering step may be called a reference pixel filter unit, intra-frame prediction may include a prediction block generation step and a prediction mode encoding step, and boundary filtering may be an embodiment of a post-processing filter step.

[0167] That is, the intra prediction performed in the image encoding device of the present embodiment may include a reference pixel construction step, a reference pixel filtering step, a prediction block generation step, a prediction mode encoding step, and a post-processing filtering step. Depending on various environmental factors such as block size, block type, block position, prediction mode, prediction method, quantization parameter, and the order in which the above can be changed to a different order, one or some of the above processes may be omitted and other processes may be added.

[0168] The reference pixel construction step, the reference pixel filtering step, the prediction block generation step, the prediction mode encoding step, and the post-processing filtering step may be implemented so that the software module stored in the memory is executed by the processor connected to the memory. Therefore, in the following description, for the convenience of description, as a functional unit generated by a combination of a software module that implements each step and a processor that executes it or a construction unit that performs the function of the functional unit, the reference pixel construction unit, the prediction block generation unit, the prediction mode encoding unit, and the post-processing filter unit will be referred to as the execution subject of each step.

[0169] More specifically, the reference pixel construction unit constructs reference pixels for predicting the current block by reference pixel filling. When the reference pixel does not exist or is unavailable, the reference pixel filling may perform a method such as copying a value from the nearest available pixel so that the value is used as the reference pixel. The decoded picture buffer (DPB) may be used to copy the value.

[0170] That is, intra prediction uses reference pixels of previously encoded blocks of the current picture to perform prediction. For this reason, in the reference pixel construction step, neighboring pixels of the current block, such as neighboring pixels of blocks to the left, upper left, upper, and upper right, are mainly used as reference pixels. When a scan such as a reverse z-scan is used as a coding order scanning method, the candidate group of neighboring blocks for reference pixels is only an example in which the coding order of the blocks follows a raster scan or a z-scan, and neighboring pixels of blocks other than the above blocks, such as blocks to the right, lower right, and lower, may also be used as reference pixels.

[0171] Furthermore, depending on the implementation, additional pixels other than directly neighboring pixels may be blended with alternative or existing reference pixels according to the step-by-step construction of the intra prediction.

[0172] In addition, when prediction is performed in a mode with a direction among the intra-frame prediction modes, reference pixels in decimal units can be generated by performing linear interpolation on reference pixels in integer units. The mode for performing prediction by reference pixels in the position of integer units includes some modes with vertical, horizontal, 45 degrees and 135 degrees, and there is no need to generate a process of reference pixels in decimal units for the above prediction modes. Prediction modes with other directions other than the above prediction modes are interpolated with interpolation precision of 1 / 2 power such as 1 / 2, 1 / 4, 1 / 8, 1 / 16, 1 / 32 and 1 / 64, and can have precision of multiples of 1 / 2. This is because the interpolation precision can be determined according to the number of supported prediction modes or the prediction direction of the prediction mode. Fixed interpolation precision can always be supported in pictures, slices, tiles, blocks, etc., and adaptive interpolation precision can be supported according to the size of the block, the type of the block, and the prediction direction of the supported mode. At this time, the direction of the prediction mode may be represented by tilt information or angle information based on a specific line (eg, a positive x-axis on a coordinate plane) in a direction indicated by the mode.

[0173] As an interpolation method, linear interpolation can be performed by directly adjacent integer pixels, but other interpolation methods can also be supported. For the purpose of interpolation, one or more filter types and the number of taps can be supported, for example, a 6-tap Wiener filter, an 8-tap Kalman filter, etc., and which interpolation to perform can be determined based on the block size, prediction direction, etc. In addition, relevant information can be sent in units of sequence, picture, slice, block, etc.

[0174] The reference pixel filter unit can perform filtering on the reference pixels after the reference pixels are constructed by reducing the residual degradation in the encoding process to achieve the purpose of improving the prediction efficiency. The reference pixel filter unit can implicitly or explicitly determine the type of filter and whether to apply filtering according to the size, type and prediction mode of the block. That is, even in filters with the same tap, the filter coefficients can be determined differently according to the filter type. For example, 3-tap filters such as [1, 2, 1] / 4 and [1, 6, 1] / 8 can be used.

[0175] In addition, the reference pixel filter unit can determine whether to apply filtering by determining whether to transmit a bit additionally. For example, in the implicit case, the reference pixel filter unit can determine whether to apply filtering based on the characteristics (discreteness, standard deviation, etc.) of the pixels in the peripheral reference block.

[0176] In addition, when the relevant flag satisfies the concealment condition predetermined in the residual coefficient, the intra prediction mode, etc., the reference pixel filter unit can determine whether to apply filtering. For example, the number of taps of the filter is 3 taps such as [1, 2, 1] / 4 for a small block (blk) and 5 taps such as [2, 3, 6, 3, 2] / 16 for a large block (blk), and the number of applications can be determined according to whether filtering is performed, whether filtering is performed once, and whether filtering is performed twice.

[0177] Furthermore, the reference pixel filter unit may basically apply filtering to the reference pixels that are most adjacent to the current block. Additional reference pixels other than the most adjacent reference pixels may also be considered in the filtering process. For example, filtering may be applied to the additional reference pixels by replacing the nearest reference pixels, or filtering may be applied by mixing the additional reference pixels with the nearest reference pixels.

[0178] Filtering can be applied fixedly or adaptively, and this can be determined based on at least one factor or a combination thereof of the size of the current block or the size of the adjacent block, the coding mode of the current block or the adjacent block, the block boundary properties of the current block or the adjacent block (for example, it is the boundary of the coding unit or the boundary of the transform unit), the prediction mode or direction of the current block or the adjacent block, the prediction method of the current block or the adjacent block, the quantization parameter, etc. A determination can be made so that the encoder or the decoder has the same setting (implicitly), or (explicitly) considering the coding cost. Typically, the filter applied is a low-pass filter. Depending on the various factors mentioned above, the number of filter taps, filter coefficients, whether the filter flag is encoded, and the number of filter applications can be determined. Its information is set in units of sequence, slice, block, etc., and the relevant information is sent to the decoder.

[0179] The prediction block generation unit may generate a prediction block via an interpolation method using reference pixels in intra prediction, an interpolation method such as an average value DC of reference pixels or a planar pattern, and a reference pixel duplication method. When duplicating reference pixels, one reference pixel may be duplicated to generate one or more prediction pixels, or one or more reference pixels may be duplicated to generate one or more prediction pixels, and the number of duplicated reference pixels may be equal to or less than the number of prediction pixels.

[0180] Directional prediction methods and non-directional prediction methods can be classified according to the prediction method. In particular, the directional prediction method can be classified into a linear directional method and a curved directional method. The linear directional method uses an extrapolation method so that the pixels of the prediction block are generated by reference pixels located on the prediction direction line. The curved directional method uses an extrapolation method so that the pixels of the prediction block are generated by reference pixels located on the prediction direction line, and refers to a method that allows the partial prediction direction of the pixel unit to be changed by considering the detailed directionality of the block (e.g., edge <edge>). In the case of the directional prediction mode in the image encoding and decoding method of the present invention, the linear directional method will be mainly described. And, in the case of the directional prediction method, the intervals between adjacent prediction modes may be equal or unequal, which can be determined according to the size or type of the block. For example, when the current block obtains a block of size and type M×N by a block partitioning unit, if M and N are the same, the intervals between the prediction modes may be uniform, and if M and N are different, the intervals between the prediction modes may be uneven. As another example, when M is greater than N, a mode with vertical directivity can allow finer intervals to be allocated between prediction modes close to the vertical mode (90 degrees), and wider intervals to be allocated to modes that are not close to the vertical mode. When N is greater than M, a mode with horizontal directivity can allow finer intervals to be allocated between prediction modes close to the horizontal mode (180 degrees), and wider intervals to be allocated to modes that are not close to the horizontal prediction mode. The above examples are not limited to the above cases, but may include cases where the conditions are reversed from each other, and examples of other cases are also allowed. In this case, the intervals between the prediction modes can be calculated based on a numerical value indicating the directionality of each mode, and the directionality of the prediction mode can be numerically expressed by tilt information or angle information of the direction.

[0181] In addition, the prediction block may also be generated by other methods using spatial correlation in addition to the above-mentioned methods. For example, a reference block is generated as a prediction block by using an inter-frame prediction method such as motion search and compensation using a current picture as a reference picture. The prediction block generation step may generate a prediction block using reference pixels according to a prediction method. That is, a prediction block may be generated according to a prediction method by directional prediction or non-directional prediction such as extrapolation, interpolation, replication, and averaging of an existing intra-frame prediction method, an inter-frame prediction method may be used to generate a prediction block, and a prediction block may be generated by other additional methods.

[0182] The intra-frame prediction method may be supported under the same setting of the encoder or decoder, and the intra-frame prediction method may be determined according to the slice type, block size, block type, etc. The intra-frame prediction method may be supported according to at least one of the above prediction methods or a combination thereof. The intra-frame prediction mode may be constructed according to the supported prediction method. The number of supported intra-frame prediction modes may be determined according to the prediction method, slice type, block size, block type, etc. The relevant information may be set and transmitted in units of sequence, picture, slice, block, etc.

[0183] The prediction mode encoding step performed via the prediction mode encoding may determine a mode which is optimal in terms of encoding cost according to the encoding cost of each prediction mode as the prediction mode of the current block.

[0184] As an example, for the purpose of reducing prediction mode bits, the prediction mode encoding unit may use one or more neighboring block modes for predicting the current block mode. The prediction mode may be included in a candidate group of the most probable mode (MPM) that may be the same as the candidate block mode, and the modes of the neighboring blocks may be included in the candidate group. For example, prediction modes of blocks such as the left, upper left, lower left, upper, and upper right blocks of the current block may be included in the above-mentioned candidate group.

[0185] The candidate group of prediction modes can be constructed based on at least one factor or a combination of factors such as the position of adjacent blocks, the priority of adjacent blocks, the priority of partitioned blocks, the size or type of adjacent blocks, a preset specific mode (in the case of chroma blocks), and the prediction mode of luminance blocks, and the relevant information can be sent in units of sequence, picture, slice, block, etc.

[0186] For example, when a block adjacent to a current block is divided into two or more blocks, it is possible to determine which block mode among the divided blocks is included as a mode prediction candidate for the current block under the same setting of the encoder or decoder. For example, the left block of the neighboring block of the current block (M×M) performs quadtree partitioning in the block partitioning unit so that the partitioned block is three blocks, and when the M / 2×M / 2, M / 4×M / 4 and M / 4×M / 4 blocks are included from top to bottom, based on the block size, the prediction mode of the M / 2×M / 2 block can be included as a mode prediction candidate for the current block. As another example, the upper block among the neighboring blocks of the current block (N×N) performs binary tree partitioning in the block partitioning unit so that the partitioned block is three, and when the N / 4×N, N / 4×N and N / 2×N blocks are included from left to right, the prediction mode of the first N / 4×N block from the left can be included as a mode prediction candidate for the current block according to a predetermined order (priority assigned from left to right).

[0187] As another example, when the prediction mode of a block adjacent to the current block is a directional prediction mode, a prediction mode adjacent to the prediction direction of the corresponding mode (in terms of tilt information or angle information of the mode direction) may be included in the mode prediction candidate group of the current block. In addition, according to the prediction mode construction or the combination of adjacent blocks, a predetermined mode (plane, DC, vertical, horizontal, etc.) may be preferably included. In addition, a prediction mode with a high frequency of occurrence among the prediction modes of the adjacent blocks may be preferably included. The priority order indicates not only the possibility of being included in the mode prediction candidate group of the current block, but also the possibility of being assigned a higher priority or index in the candidate group construction (that is, a higher probability of being assigned fewer bits in the binarization process).

[0188] As another example, when the maximum number of mode prediction candidate groups of the current block is k, wherein the left block consists of m blocks whose length is less than the vertical length of the current block, and the upper block consists of n blocks whose length is less than the horizontal length of the current block, if the sum of the divided blocks of the adjacent blocks (m+n) is greater than k, it is possible to fill the candidate group according to a predetermined order (from left to right, from top to bottom), and if the sum of the divided blocks of the adjacent blocks (m+n) is greater than the maximum value k of the candidate group, the prediction mode of the block other than the adjacent block (left block, upper block, etc.) such as the adjacent block in the prediction mode of the adjacent block can be included in the mode prediction candidate group of the current block. The above examples are not limited to the above cases, but may include cases where the conditions are reversed from each other, and examples of other cases are also allowed.

[0189] In this way, the candidate blocks for predicting the current block mode are not limited to a specific block position, but can utilize prediction mode information from at least one of the blocks located to the left, above left, below left, above, and above right. As in the above example, various factors can be considered to construct the prediction mode candidate group for the current block.

[0190] The prediction mode encoding unit may classify the most probable mode (MPM) candidate group (referred to as candidate group 1 in this example) that may be the same as the mode of the current block and the mode candidate group (referred to as candidate group 2 in this example) in other cases. The prediction mode encoding process may be changed according to which candidate group the prediction mode of the current block belongs to. All prediction modes may be composed of the sum of the prediction modes of candidate group 1 and the prediction modes of candidate group 2. The number of prediction modes of candidate group 1 and the number of prediction modes of candidate group 2 are determined according to at least one factor or a combination thereof in the total number of prediction modes, block type, etc. The same binarization or other binarization may be applied according to the candidate group. For example, fixed length binarization may be applied to candidate group 1, and cut-off-truncation binarization may be applied to candidate group 1. In the above description, although the number of candidate groups is illustrated as two, it is possible to expand to, for example, a first mode candidate group with a high probability of being the same as the mode of the current block, a second mode candidate group with a high probability of being the same as the current block, and mode candidate groups in other cases, and deformation thereof is also allowed.

[0191] Considering the high correlation between the reference pixels adjacent to the boundary between the current block and the adjacent block and the pixels in the current block, in the post-processing filtering step performed by the post-processing filter unit, some predicted pixels of the prediction block generated in the previous process can be replaced by values ​​generated by filtering one or more reference pixels and one or more predicted pixels adjacent to the boundary, and the predicted pixels can be replaced by values ​​obtained by applying the values ​​(e.g., the difference in pixel values, tilt information, etc.) obtained by digitizing the features between the reference pixels adjacent to the block boundary to the filtering process, and other methods other than the above methods and having similar purposes (correcting some predicted pixels of the prediction block by reference pixels) can be added. In the post-processing filter unit, a filter and whether to apply the filter can be determined implicitly or explicitly. The position and number of reference pixels and current pixels used in the post-processing filter, a prediction mode applied, etc. can be set in the encoder or decoder, and the relevant information can be sent in units of sequence, picture, slice, etc.

[0192] In addition, in the post-processing filtering step, an additional post-processing step may be performed after the prediction block is generated as in the block boundary filtering. In addition, after obtaining the residual signal, the current block reconstructed by adding the residual signal and the prediction signal obtained through the transformation process, the quantization process, and the dequantization process may be subjected to the post-processing filtering step considering the characteristics of the adjacent predicted pixels similar to the boundary filtering.

[0193] Finally, the prediction block is selected or obtained through the above process. The information obtained from the process may include information related to the prediction mode, and may be sent to the transform unit 210 for encoding the residual signal after obtaining the prediction block.

[0194] Fig.10 is an exemplary view illustrating a prediction principle in a P slice or a B slice in an image encoding method according to an embodiment of the present invention.

[0195] refer to Fig.10 According to the image encoding method of the present embodiment, the image encoding method may include a motion estimation step and an interpolation step. The information about the motion vector, the reference picture index, and the reference direction generated in the motion estimation step may be sent to the interpolation step. In the motion estimation step and the interpolation step, the value stored in the decoded picture buffer (DPB) may be used.

[0196] That is, the image encoding device may perform motion estimation to find a block similar to the current block in a previously encoded picture. In addition, the image encoding device may perform interpolation on the reference picture for prediction with finer precision than the decimal unit. Finally, the image encoding device obtains a predicted block through a predictor, wherein the information obtained in the process includes a motion vector, a reference picture index or a reference index, a reference direction, etc., and then residual signal encoding may be performed.

[0197] In this embodiment, intra prediction is performed in a P slice or a B slice, so that Fig. 9 The combined method shown supports inter-frame prediction and intra-frame prediction.

[0198] Fig.11 is an exemplary view showing a process of obtaining a prediction block.

[0199] refer to Fig.11 , an image encoding method according to an exemplary embodiment of the present invention includes the steps of reference sample filling, reference sample filtering, intra-frame prediction, boundary filtering, motion estimation and interpolation.

[0200] In the case where the image coding device supports block matching in the current picture, the prediction method in the I slice can be Fig.11 Instead of Fig. 9 That is, the image encoding device can use not only the prediction mode in the I slice, but also information such as the motion vector, the reference picture index, and the reference direction that only appear in the P slice or the B slice to generate the prediction block. However, since the reference picture is a feature of the current picture, there may be information that can be partially omitted. For example, when the reference picture is the current picture, the reference picture index and the reference direction can be omitted.

[0201] In addition, in an image encoding device, when interpolation is applied, since blocks matching decimal units may not be necessary due to the nature of artificial images such as computer graphics, whether to perform block matching is set in the encoder, and units such as sequences, pictures, and slices can be set.

[0202] For example, the image encoding device may not perform interpolation on the reference picture used for inter-frame prediction according to the setting of the encoder, and various settings may be performed, such as not interpolating only when block matching is performed in the current picture. That is, it is possible to set the image encoding device of the present embodiment according to whether to perform interpolation of the reference picture. At this time, it can be determined whether to perform interpolation on all or some reference pictures constituting the reference picture list. For example, when it is not necessary to perform block matching in decimal units because the image of the reference block is a feature of an artificial image, the image encoding device does not perform interpolation, and for a specific current block, interpolation is performed when block matching needs to be performed in decimal units because of a natural image.

[0203] Furthermore, the image encoding device may set whether to apply block matching to a reference picture for which interpolation is performed in units of blocks. For example, when a natural image and an artificial image are mixed, interpolation is performed on the reference picture. When the best motion vector can be obtained by searching a portion of the artificial image, the motion vector may be expressed in a specific unit (here, it is assumed to be an integer unit), and when the best motion vector can be obtained by selectively searching a portion of the natural image, the motion vector may be expressed in other specific units (here, it is assumed to be a 1 / 4 unit).

[0204] Fig.12 is an exemplary view showing a main procedure of an image encoding method according to an embodiment of the present invention using syntax in encoding units.

[0205] refer to Fig.12 , curr_pic_BM_enabled_flag indicates a flag that allows block matching in the current picture, and can be defined and sent in units of sequences and pictures. At this time, the process of generating a prediction block by performing block matching on the current picture can represent a case where an operation is performed through inter-frame prediction. It can be assumed that cu_skip_flag, which is an inter-frame technique for not encoding a residual signal, is a flag supported only by P slices or B slices other than I slices. In this case, when curr_pic_BM_enabled_flag is turned on, I slices can also support block matching (block matching, BM) in inter-frame prediction mode.

[0206] That is, the image encoding device of this embodiment can support skipping when generating a prediction block on the current picture by block matching, and can support skipping in the case of screen technology other than block matching. Depending on such conditions, skipping may not be supported in I slices. Whether to perform skipping can be determined according to the encoder setting.

[0207] For example, when the I slice supports skipping, the prediction block can be directly reconstructed as a reconstructed block by block matching without encoding the residual signal through prediction_unit() linked to the prediction unit of if(cu_skip_flag) as a specific flag. In addition, the image encoding device classifies the method of using the prediction block by block matching in the current picture as an inter-frame prediction technology, and such classification can be handled by pred_mode_flag as a specific flag.

[0208] That is, if pred_mode_flag is 0, the image encoding device sets the prediction mode to the inter prediction mode (MODE_INTER), and if pred_mode_flag is 1, the prediction mode can be set to the intra prediction mode (MODE_INTRA). This is an intra-frame technology similar to the prior art, but it can be classified as an intra-frame technology or an inter-frame technology in an I slice to distinguish it from the existing structure. That is, the image encoding device of this embodiment can use a time-related structure without using the time correlation in the I slice. part_mode represents information about the size and type of the block divided in the coding unit.

[0209] Fig.13 is an exemplary view showing an example of supporting symmetric type partitioning or asymmetric type partitioning as inter prediction when a prediction block is generated by block matching in a current picture.

[0210] refer to Fig.13 In the case where a prediction block is generated by block matching in the current picture, as in inter-frame prediction, the image encoding method according to the present exemplary embodiment may support symmetric types such as 2N×2N, 2N×N, and N×2N and asymmetric partitioning such as nLx2N, nRx2N, 2NxnU, and 2NxnD. Various block sizes and types may be determined according to a partitioning method of a block partitioning unit.

[0211] Fig.14 is an exemplary view showing that 2N×2N and N×N can be supported in inter prediction.

[0212] refer to Fig.14, the image encoding method according to the present embodiment can support 2N×2N and N×N as the prediction block types used in conventional intra prediction. This is an example of a square supported by a quadtree partitioning method according to a predefined block candidate group in a block partitioning unit or a partitioning method, and other block types that can be supported by adding a rectangular shape to a binary tree partitioning method or a predefined block candidate group in intra prediction, where the setting can be set in the encoder. The encoder can also set whether to apply skipping only when performing block matching on the current picture, whether to also apply skipping to existing intra predictions, or whether to apply skipping to new intra predictions. Information related thereto can be sent in units of sequences, pictures, slices, etc.

[0213] The subtraction unit 205 may generate a residual block by subtracting the pixel values ​​of the prediction block generated by the prediction unit 200 from the pixel values ​​of the current block to be encoded to derive pixel difference values.

[0214] The transform unit 210 receives a residual block as a difference between the current block and a prediction block generated by intra prediction or inter prediction from the subtraction unit 205, and transforms the residual block to the frequency domain. Through the transform process, each pixel of the residual block corresponds to a transform coefficient of the transform block. The size and type of the transform block may be equal to or smaller than the size of the coding unit. Also, the size and type of the transform block may be the same as or smaller than the prediction unit. The image encoding device may perform the transform process by grouping several prediction units.

[0215] The size or type of the transform block can be determined by the block partitioning unit, and square or rectangular shaped transforms can be supported according to the block partitioning. The block partitioning operation can be affected by the transform related settings (supported transform block size, type, etc.) supported by the encoder or decoder.

[0216] The size and type of each transform block are determined according to the cost of each candidate of the size and type of each transform block, and split information such as the determined image data of each transform block and the determined size and type of each transform block may be encoded.

[0217] The transformation can be performed by a one-dimensional transformation matrix (DCT, DST, etc.). Each transformation matrix can be used adaptively for horizontal and vertical units. Examples of adaptive use can be determined by various factors such as block size, block type, block type (luminance and chrominance), coding mode, prediction mode information, quantization parameter, and coding information of neighboring blocks.

[0218] For example, in the case of intra-frame prediction, when the prediction mode is horizontal, a DCT-based transformation matrix can be used in the vertical direction, and a DST-based transformation matrix can be used in the horizontal direction. When the prediction mode is vertical, a DCT-based transformation matrix can be used in the horizontal direction, and a DST-based transformation matrix can be used in the vertical direction. The transformation matrix is ​​not limited to that described in the above description. Information related thereto can be determined using an implicit or explicit method, and can be determined based on one or more factors or a combination thereof such as the block size, block type, coding mode, prediction mode, quantization parameter, coding information, etc. of the adjacent blocks, and the relevant information can be sent in units of sequences, pictures, slices, blocks, etc.

[0219] Here, considering the case of using an explicit method, when two or more transformation matrices for horizontal and vertical directions are used as candidate groups, information about the transformation matrix used for each direction may be transmitted separately, or information about the transformation matrix used for each of the horizontal and vertical directions and information about the transformation matrix used for the horizontal and vertical directions using two or more pairs as candidate groups may be transmitted.

[0220] In addition, taking into account the characteristics of the image, partial or overall transformations may be omitted. For example, one or both of the horizontal or vertical components may be omitted. Since intra-frame prediction or inter-frame prediction cannot be performed well, when the difference between the current block and the predicted block is large (that is, when the residual component is large), the coding loss due to the transformation may increase. This can be determined based on at least one factor or a combination of coding mode, prediction mode information, block size, block type, block type (luminance and chrominance), quantization parameter, and coding information of adjacent blocks. The above conditions can be expressed using implicit or explicit methods, and information related thereto can be sent in units of sequences, pictures, slices, etc.

[0221] The quantization unit 215 performs quantization of the residual component transformed by the transform unit 210. A quantization parameter is determined on a block basis, and the quantization parameter may be set in units of a sequence, a picture, a slice, a block, or the like.

[0222] For example, the quantization unit 215 may predict the current quantization parameter using one or more quantization parameters derived from neighboring blocks such as left, top left, top, top right, and bottom left of the current block.

[0223] When there is no quantization parameter predicted from an adjacent block, that is, when the block is located at a boundary of a picture, a slice, etc., the quantization unit 215 may output or transmit a difference value from a basic parameter transmitted in units of a sequence, a picture, a slice, etc. When there is a quantization parameter estimated from an adjacent block, the difference value may be transmitted using the quantization parameter of the corresponding block.

[0224] The priority of the block from which the quantization parameter is to be derived may be preset, or may be sent in units of a sequence, a picture, a slice, etc. The residual block may be quantized by dead zone uniform threshold quantization (DZUTQ), a quantization weighting matrix, or an improved scheme. It is possible to use one or more quantization schemes as candidates, and the scheme may be determined by a coding mode, prediction mode information, etc.

[0225] For example, the quantization unit 215 can set a quantization weighting matrix to be applied to inter-frame coding, intra-frame coding units, etc., and can also set different weighting matrices according to the intra-frame prediction mode. Assuming that the size of the block is equal to the quantization block size of size M×N, the quantization weighting matrix can be constructed by changing the quantization coefficient of each position of each frequency component. The quantization unit 215 can select one of various existing quantization methods and can be used under the same settings of the encoder or decoder. Information related thereto can be sent in units of sequence, picture, slice, etc.

[0226] at the same time, Figure 2 and 3 The dequantization units 220 and 315 and the inverse transform units 225 and 320 shown in FIG. 2 can be implemented by inversely performing the processes in the inverse transform unit 210 and the quantization unit 215. That is, the dequantization unit 220 can dequantize the quantized transform coefficients generated by the quantization unit 215, and the inverse transform unit 225 can inversely transform the dequantized transform coefficients to generate a reconstructed residual block.

[0227] Figure 2 and Figure 3 The addition units 230 and 324 shown in FIG. 2 can generate a reconstructed block by adding the pixel values ​​of the prediction block generated by the predictor to the pixel values ​​of the reconstructed residual block. The reconstructed block can be stored in the encoding and decoding picture buffers 240 and 335 and provided to the prediction unit and the filter unit.

[0228] In-loop filters such as deblocking filters, sample adaptive offset (SAO), adaptive loop filters (ALF), etc. can be applied to the reconstructed block. The deblocking filter can filter the reconstructed block to eliminate the distortion between block boundaries that occur during encoding and decoding. SAO is a filter process that reconstructs the difference between the original image and the reconstructed image with an offset on a pixel-by-pixel basis relative to the residual block. ALF can perform filtering to minimize the difference between the predicted block and the reconstructed block. ALF can perform filtering based on a comparison value between the reconstructed block and the current block through a deblocking filter.

[0229] The entropy encoding unit 245 may perform entropy encoding on the transform coefficients quantized by the quantization unit 215. For example, techniques such as context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), syntax-based context adaptive binary arithmetic coding (SBAC), probability interval partitioning entropy (PIPE) encoding, etc. may be performed.

[0230] The entropy coding part 245 may include a bit stream obtained by encoding the quantization coefficients and various information required to decode the encoded bit stream as the encoded data. The encoded data may include a coding block type, a quantization coefficient, a bit stream for encoding the quantization block, and information required for prediction. In the case of the quantization coefficients, the two-dimensional quantization coefficients may be scanned in one dimension. The quantization coefficients may vary according to the characteristics of the image. In particular, in the case of intra-frame prediction, since the distribution of the coefficients may have a specific distribution according to the prediction mode, the scanning method may be set differently.

[0231] In addition, the entropy encoding part 245 may be set differently according to the size of the block to be encoded. The scanning pattern may be preset to have at least one of various patterns such as a zigzag, a diagonal line, and a raster or may be set to a candidate having at least one of various patterns such as a zigzag, a diagonal line, and a raster, may be determined by the encoding mode, the prediction mode information, etc., and may be used under the same setting of the encoder and the decoder. Information related thereto may be transmitted in units of a sequence, a picture, a slice, etc.

[0232] The size of the quantized block (hereinafter referred to as the quantized block) input to the entropy encoding part 245 may be equal to or smaller than the size of the transform block. In addition, the quantized block may be divided into two or more sub-blocks. When the quantized block is divided, the scanning pattern in the divided block may be set to be the same as or different from the scanning pattern of the existing quantized block.

[0233] For example, when the scanning pattern of the existing quantization block is called a zigzag pattern, the zigzag pattern may be applied to all sub-blocks, or the zigzag pattern may be applied to the sub-block located at the upper left of the block including the DC component, and the diagonal pattern may be applied to other blocks. This may also be determined according to the encoding mode, prediction mode information, etc.

[0234] The start position of the scanning pattern in the entropy coding part 245 is basically from the upper left corner, but according to the characteristics of the image, it can start from the upper right corner, the lower right corner or the lower left corner. Information about which one or more candidate groups have been selected can be sent in units of sequence, picture, slice, etc. Entropy coding technology can be used as the coding technology, but is not limited thereto.

[0235] at the same time, Figure 2 and Figure 3 The dequantization of the dequantization unit 220 and the inverse transformation of the inverse transformation unit 225 shown in are constructed inversely to the quantization of the quantization unit 215 and the transformation of the transformation unit 210 , and can be implemented by combining the basic filters 235 and 330 .

[0236] Hereinafter, the formation of the above-mentioned reference pixels will be described in more detail.

[0237] When performing intra prediction according to an embodiment of the present invention, the availability of reference pixels of adjacent blocks must be checked. The availability of reference pixels can be determined based on at least one factor or a combination of the position of the adjacent block, the position of the current block in the partitioned block, the boundary features of the adjacent blocks, etc. For example, when the M×M block is divided into four M / 2×M / 2 blocks (in this example, 0, 1, 2 and 3 indexes are assigned in the raster scanning direction) by the block partitioning unit, and the current block is a number of 3 M / 2×M / 2 blocks, although the adjacent blocks located in a specific direction (in this example, the upper right corner and the lower left corner in this example) belong to the current block in units of pictures, slices or tiles, the corresponding blocks have not been encoded so that it is determined that the blocks are unavailable, and thus the reference pixels of the blocks may not be used. As another example, it can be determined whether to use the reference pixels of the adjacent blocks based on the boundary features between the current block and the adjacent blocks (boundaries of slices, tiles, etc.).

[0238] Another example of using reference pixels in adjacent blocks can support settings for determining whether to use data with a high probability of error propagation in the encoder / decoder in order to avoid error propagation over time in the case of P slices or B slices. According to the setting information, it is possible to determine whether to use data with a high probability of error propagation for generating prediction blocks, and flags such as constrained_intra_pred_flag (CIP) of HEVC can perform a similar role. This flag can determine whether to use adjacent blocks encoded in inter-frame prediction mode with a high probability of error propagation among adjacent blocks referenced in the generation of intra-frame prediction blocks. If the flag is 0, the reference pixels of the block encoded with inter-frame prediction may not be used, and if the flag is 1, the reference pixels of the block encoded with inter-frame prediction may be used.

[0239] The following example shows the case where a prediction block is generated when the flag is not zero. The additional setting of the flag causes the reference pixels to be unavailable, regardless of the error propagation possibility, whereby any reference pixels may not be used for prediction. In this case, since the prediction block may not use any reference pixels, the prediction block may be generated by filling a preset value (e.g., the middle value of the pixel value range of the bit depth). Alternatively, it is possible to generate a prediction block to which edge information is applied based on a predefined value. For example, if the predefined value is A, based on an arbitrary pixel in the M×N block, the right area including the arbitrary pixel is filled with (A+B), and the other area is filled with (AC) to generate a prediction block, where B and C represent parameters for representing edge information.

[0240] In addition, by the additional setting of the flag, the error propagation probability can be determined by the number of adjacent blocks encoded with inter-frame prediction, the number of reference pixels of the adjacent blocks, etc. According to this determination, it is possible to determine whether to use the reference pixels of the block encoded by intra-frame prediction. For example, if less than N blocks of the M adjacent blocks adjacent to the current block are blocks encoded with inter-frame prediction, the reference pixels of the block may not be used to generate the prediction block, and the reference pixels of (MN) blocks encoded with intra-frame prediction may not be used to generate the prediction block. Since the number of blocks as a determination criterion may be affected by the size or type of the block, it can be replaced by the number of reference pixels present at the position to be used to predict the current block, and other factors can be considered.

[0241] The generated prediction block may be affected depending on whether the reference pixels are used. For example, if at least one of the neighboring blocks of the current block is available depending on whether the reference pixels are used, the reference pixels of the unavailable block may be replaced by pixels derived (e.g., copied, averaged, etc.) from the reference pixels of the available blocks. As another example, when there are no available blocks depending on whether the reference pixels are used, the reference pixels of the unavailable blocks may be replaced with predefined values ​​(e.g., middle values ​​of the pixel value range of the bit depth).

[0242] The replacement of the value derived from the available block may be prohibited according to the setting of the encoder or decoder. In this case, the prediction mode derived from the corresponding block may not be used.

[0243] Hereinafter, these contents will be described in more detail with reference to the accompanying drawings.

[0244] Fig.15 is an exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0245] refer to Fig.15, it is possible to determine whether the reference pixels of the corresponding block are used to generate the prediction block according to the encoding mode of the neighboring blocks of the current block. In this case, the neighboring blocks can be in the shape of an M×M square or an M×N rectangle obtained by the block division unit, and can be encoded with one of intra prediction or inter prediction according to the encoding mode. Fig.15 As shown in , when the current block is the middle block, the block located at the upper left is encoded using inter-frame prediction (inter), the two blocks located at the upper end are encoded using intra-frame prediction (intra), and the block located at the upper right is encoded using inter-frame prediction, intra-frame prediction, and intra-frame prediction from the left side.

[0246] Blocks coded with inter-frame prediction can be further classified according to the reference picture. Fig.15 The expression ref=t for an inter-frame prediction coded block indicates that the reference picture is the current picture.

[0247] It will be described under the above assumptions Fig.16a and Fig.16b .

[0248] Fig.16a is an exemplary diagram for explaining whether to use reference pixels when considering encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention. Fig.16b is an exemplary view for explaining whether to use reference pixels without considering encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0249] In the following example, whether to consider the coding mode of the adjacent block will be explained using the CIP flag. Specifically, when the CIP flag indicates 1, a block encoded with a specific coding mode (inter-frame prediction in this example) may not be used to predict the current block, and when the flag indicates 0, a block encoded with a specific coding mode may be used to predict the current block.

[0250] refer to Fig.16a , if the CIP flag is 1 and the neighboring block is coded with inter prediction (inter), the reference pixels of the corresponding block may not be used to predict the current block. The reference blocks of the neighboring blocks of the block coded with intra prediction (intra) indicated by the hatching can be used to predict the current block.

[0251] Here, pixels adjacent to the current block within a block that can be used as reference pixels may be used as reference pixels, and other additional pixels may be included.

[0252] refer to Fig.16b If the CIP flag is 0, the reference pixels of the coding block can be used to predict the current block. That is, the pixels adjacent to the current block in the upper left, upper right, upper left, left, and lower left of the current block can be used as reference pixels.

[0253] Fig.17 is another exemplary view illustrating encoding modes of neighboring blocks in intra prediction according to an embodiment of the present invention.

[0254] refer to Fig.17 , the adjacent block is encoded using one of intra prediction and inter prediction. When the adjacent block is encoded using inter prediction, ref=t indicates that the reference picture is the current picture, ref=t-1 indicates that the reference picture is the picture immediately earlier than the current picture, ref=t-2 indicates that the reference picture is the second earlier than the current picture, and another reference picture can be added according to the reference direction of the inter prediction. The block on the upper right represents a block outside the picture range.

[0255] In such Fig.17 In the coding state shown in Figures 18a to 18c An example of determining availability as a reference pixel is described.

[0256] Fig.18a is an exemplary view for explaining a case where an encoding mode of a neighboring block is intra prediction according to an embodiment of the present invention. Fig.18b is an exemplary view for explaining a case in which encoding modes and reference pictures of neighboring blocks are considered in intra prediction according to an exemplary embodiment of the present invention. Fig.18c is another exemplary view for explaining a case where encoding modes of neighboring blocks are not considered in intra prediction according to an embodiment of the present invention.

[0257] refer to Fig.18a ,exist Fig.17 Neighboring blocks coded with intra prediction in the coding mode may be shown in the shaded area.

[0258] Since the shadow block is encoded using intra prediction, pixels in the corresponding block can be used as reference pixels.

[0259] And in this case, whether to consider the encoding mode of the adjacent block can be indicated according to the CIP flag, and it can be an example of the case where the CIP flag is 1.

[0260] refer to Fig.18b , as mentioned above, in Fig.17 Pixels in a block coded with intra prediction in the coding mode state may be used as reference pixels.

[0261] However, in Fig.18b In the embodiment, even if the adjacent block is encoded with inter-frame prediction, when the reference picture of the adjacent block is the current picture, the pixels in such a block can also be used as reference pixels. Alternatively, when the reference picture of the adjacent block is the current picture, the pixels of the corresponding block may not be used as reference pixels.

[0262] Because when the reference picture of the adjacent block is the current picture, error propagation does not occur temporally between pictures, the pixel can be used. On the contrary, because error propagation may occur when the block referenced in the current picture is a block encoded from a previous or subsequent picture, the pixel may not be used.

[0263] As described above, whether to use reference pixels can be determined based on at least one factor or a combination of the position of the adjacent block, the encoding mode of the adjacent block, the position of the current block in the partition block, the boundary characteristics of the current block and the adjacent block, and the reference picture can be added to the above factors. Specifically, considering the index of the reference picture of the adjacent block, or the picture order count (POC), it is possible to determine whether to use the reference pixels of the corresponding block for predicting the current block. In the above example, although the reference picture is designated as the current picture, it can be extended to a reference picture with a different index or picture order count (POC). In addition, even when the reference direction of the inter-frame prediction is unidirectional prediction (L0, L1) and bidirectional prediction, various settings for the reference picture indexes L0 and L1 can be constructed.

[0264] Therefore, not only can Fig.18a Not only the pixels in the shaded block (intra-prediction block), but also the pixels in the block encoded by inter-prediction can be used as reference pixels.

[0265] And in this case, since whether to consider the encoding mode of the adjacent block can be indicated according to the CIP flag, it can be an example for explaining the case where the flag is 1.

[0266] refer to Fig.18c , in principle, all neighboring pixels in neighboring blocks can be used as reference pixels.

[0267] However, since the area at the upper right of the current block located in the center is an area without image data or an area outside the image, it may correspond to an area that is not available in any case.

[0268] References Figure 15 to Figure 18c The examples described are summarized below.

[0269] First, the availability of reference pixels of neighboring blocks of the current block may be determined.

[0270] When it is determined that reference pixels in all adjacent blocks are unavailable, reference pixels consisting of predefined pixel values ​​may be obtained. For example, the reference pixel may consist of an intermediate value of the representation range of the pixel. More specifically, since a value may be assigned to a reference pixel having 1<<(bit depth-1), and the bit depth represents the number of bits used to represent a pixel, the illustrated value may represent an intermediate value of the representation value of the pixel.

[0271] When it is determined that at least one neighboring block is available, the encoding modes of the neighboring blocks (left, top, left top, right top, left bottom, etc.) may be considered.

[0272] Furthermore, it is possible to use a preset flag to indicate whether the encoding mode is considered.

[0273] For example, all pixels in the adjacent blocks may be used as reference pixels regardless of the encoding mode, and at this time the preset flag may indicate that the encoding mode is not considered.

[0274] Here, if the encoding mode of the neighboring block is inter prediction, the neighboring block is determined to be unavailable, so that pixels in the neighboring block may not be used for reference pixels.

[0275] And, even if the encoding mode of the neighboring block is inter prediction, by checking the reference picture, when the reference picture of the neighboring block is the current picture, the pixel can be used as a reference pixel.

[0276] Here, the reference picture may be selected from List 0 storing data of reference pictures earlier than the current picture or List 1 storing data of reference pictures later than the current picture, and the current picture may be included in List 0 or List 1.

[0277] And, even if the encoding mode of the neighboring block is inter prediction, by checking the reference picture, when the reference picture of the neighboring block is an I picture (other than P or B), the pixel can be used as a reference pixel.

[0278] However, the above constraints should be understood as an example of determining whether a pixel in a neighboring block can be used as a reference pixel in consideration of a specific condition. That is, the coding mode of the neighboring block is inter-frame prediction, and it is possible to determine the availability as a reference pixel by referring to the motion vector of the motion information of the neighboring block, reference picture information, etc. In this case, the above-exemplified conditions can be set in units of sequence, picture, slice, etc.

[0279] The availability determination of reference pixels according to an embodiment of the present invention may be performed according to the following process.

[0280] First, confirm whether the block adjacent to the current block is located outside the boundary of the picture, slice or tile. Then, it can be confirmed whether the adjacent block is encoded. Then, check the slice type of the adjacent block, and if the type is an I slice, the boundary pixels in the adjacent block can be set to be used as reference pixels of the current block. If the type is a P slice, the mode is unidirectional prediction. Therefore, the reference picture is confirmed by checking the reference picture list 0 (ref_idx_l0). When the reference picture is the current picture, the reference pixels of the current picture can be set to available. However, this should be understood as an example. When the current picture is not a reference picture, the reference pixels can be set to unavailable. If the type is a B slice, after confirming whether the current picture is used as a reference picture by confirming whether the prediction mode is unidirectional_L0, unidirectional_L1 or bidirectional, it can be confirmed whether the reference picture is available.

[0281] Fig.19a is an exemplary view for explaining a method of obtaining a reference pixel when an available neighboring block is located at the lower left. Fig.19b is an exemplary view for explaining a method of obtaining a reference pixel when an available neighboring block is located at the upper right end. Fig.19c is an exemplary view for explaining a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end. Fig.19d is an exemplary view for explaining a method of obtaining reference pixels when available neighboring blocks are located at the lower left end and the upper right end.

[0282] refer to Figures 19a to 19d , the shaded pixels may represent neighboring pixels within a block that are available for determining a neighboring block, and the arrow direction may indicate a direction constructed by reference pixels.

[0283] As described above, although the adjacent block may be determined to be available, it may also be determined to be unavailable. Therefore, the pixels in the block determined to be unavailable may not be used as reference pixels, and in this case, how to construct the adjacent pixels in the unavailable block may be problematic.

[0284] refer to Fig.19a and Fig.19b , when only pixels located at the upper right or lower left of the current block can be used as reference pixels, the pixels touching the unavailable area are copied and filled into the pixels adjacent to the current block in the unavailable area.

[0285] refer to Fig.19c When there are available areas both above and below the current block, the pixels can be filled by copying pixels from any available area, by obtaining the average value of pixels on both sides (such as <from bottom to top, from left to right>, <from left to right and from top to bottom> or <from bottom to top and from right to left>) or by linear interpolation.

[0286] refer to Fig.19d When an area available as a reference pixel is mixed, the pixel can be filled with a pixel value obtained by copying in one direction (left direction, downward direction); when there is an available area on both sides (such as the upper end of the current block), the pixel can be filled with a value generated by using pixels on both sides such as interpolation and averaging; and when there is an available area on one side of the current block such as the left side or the lower left end, the pixel can be filled with a value obtained by copying.

[0287] More specifically, refer to Fig.19d , pixels located on the right, including P(-1, -1) and P(blk_size, -1) located on the upper right of the current block, may be available. Values ​​from P(0, -1) to P(blk_size-1, -1) may not be available. In this case, if an average value is assigned, it is possible to assign it as P(x, y) {x is 0 to blk_size-1, y is 1} = {P(-1, -1) + P(blk_size, -1)} + 1 / 2. At this time, blk_size can represent the block size.

[0288] Pixels located at the bottom including P(-1, 0) may not be available. At this time, P(x, y) {x is 1, y is 0 to 2*blk_size-1} may be allocated with P(-1, -1).

[0289] In addition, it is possible to determine whether to perform interpolation based on the position of the reference pixel. Fig.19c As shown in , even if there are available areas on both sides, interpolation may be limited when the positions of the available areas are not connected by a vertical line. Specifically, when comparing the coordinates of both ends of the available area, when both x and y do not match (such as P(-1, a) and P(b, -1)), interpolation may be limited. When one of x and y matches each other (such as P(a, -1), P(b, -1)), interpolation can be performed.

[0290] In addition, even in Fig.19d There are available areas on both sides shown in , and interpolation can be performed when the positions of the available areas are located on vertical lines.

[0291] Fig. 20 is a flowchart of an image encoding method according to an embodiment of the present invention.

[0292] refer to Fig. 20, an image encoding method for performing intra-frame prediction includes: considering the prediction mode of each of the current block and the adjacent blocks, determining whether the boundary pixels within the adjacent blocks can be used as reference pixels of the current block (step S110); obtaining the reference pixels of the current block according to the determined result (S120); generating a prediction block through intra-frame prediction based on the obtained reference pixels (S130); and encoding the current block using the generated prediction block (S140).

[0293] A description of each step in this case can be found in Figure 15 to Figure 18c , and therefore its detailed description will be omitted.

[0294] An image decoding method for performing intra-frame prediction according to an embodiment of the present invention includes the following steps: receiving a bit stream including data on a prediction mode of a current block and blocks adjacent to the current block; extracting data from the received bit stream to confirm the prediction mode of the adjacent block; determining whether boundary pixels within the adjacent block can be used as reference pixels of the current block in consideration of the prediction mode of the adjacent block; obtaining reference pixels of the current block according to the determined result; generating a prediction block for intra-frame prediction based on the obtained reference pixels; and decoding the current block using the generated prediction block.

[0295] Here, when the boundary pixel is determined to be unusable as a reference pixel of the current block, obtaining the reference pixel may obtain a reference pixel constructed by a preset pixel value.

[0296] Here, determining whether a boundary pixel can be used as a reference pixel may indicate whether a preset flag considers a prediction mode of a neighboring block.

[0297] Here, when the prediction mode of the neighboring block is intra prediction, determining whether the boundary pixel can be used as a reference pixel may determine whether the boundary pixel can be used as a reference pixel of the current picture.

[0298] Here, when the prediction mode of the adjacent block is inter prediction, determining whether the boundary pixel can be used as a reference pixel may determine that the boundary pixel cannot be used as a reference pixel of the current picture.

[0299] Here, when the prediction mode of the neighboring block is inter prediction, determining whether the boundary pixel can be used as a reference pixel may determine whether the boundary pixel can be used as a reference pixel of the current picture in consideration of the reference picture of the neighboring block.

[0300] Here, the reference picture may be selected from List 0 storing data of reference pictures earlier than the current picture or List 1 storing data of reference pictures later than the current picture, and the current picture may be included in List 0 or List 1.

[0301] Here, when the reference picture of the neighboring block is the current picture, it may be determined that the boundary pixels may be used as the reference picture of the current picture.

[0302] Here, when the reference picture of the neighboring block is not the current picture, it may be determined that the pixels in the neighboring block cannot be used as the reference picture of the current picture.

[0303] Here, when the reference picture of the neighboring block is an I picture, it may be determined that pixels in the neighboring block may be used as a reference picture of the current picture.

[0304] In an image decoding device including one or more processors according to an embodiment of the present invention, the one or more processors may: receive a bit stream including data on a prediction mode of a current block and blocks adjacent to the current block; and confirm the prediction mode of the adjacent blocks by extracting data from the received bit stream; determine whether boundary pixels within the adjacent blocks can be used as reference pixels of the current block in consideration of the prediction mode of the adjacent blocks; obtain reference pixels of the current block based on the determined result; generate a prediction block for intra-frame prediction based on the obtained reference pixels; and decode the current block using the generated prediction block.

[0305] Fig.21 is a diagram showing an example of the intra prediction mode of HEVC.

[0306] refer to Fig.21 , there are a total of 35 prediction modes for the luma component (Intra_fromLuma, 35). There may be prediction modes with various directions including planar (Intra_planar, 0) and average (Intra_DC, 1). For each prediction mode, the direction of each arrow may indicate the direction in which a prediction block is constructed using reference pixels.

[0307] Fig.22a is an example diagram for explaining reference pixels used for current block prediction in the reference pixel construction step.

[0308] refer to Fig.22a, pixels (A0~P0, X0) of blocks to the left, upper left, lower left, upper and upper right of the current block may be included in the reference pixel candidate group. For example, in the case of an M×N block, pixels of p[0,-1]~p[(2×M)-1,-1], p[-1,0]~p[(2×N)-1] and p[-1,-1] based on the upper left coordinates may be included in the candidate group. Here, p[0,-1] corresponds to A0, p[2×M-1,-1] corresponds to H0, and p[-1,-1] corresponds to X0. In the above example, 2×(M+N)+1 reference pixels may be included in the candidate group, and may be determined based on at least one factor such as the size of the block, the type of the block, the prediction mode, the prediction method, the slice type, and a combination thereof. For example, according to the prediction mode, some reference pixels may be added to or excluded from the candidate group. As another example, according to the prediction scheme, some reference pixels may be added to or excluded from the candidate group.

[0309] Figure 22b is an exemplary view for explaining that additional pixels of a neighboring block are included in a reference pixel candidate group.

[0310] refer to Figure 22b For example, in the case of an M×N block, pixels of p[-1, -2] to p[(2×M)-1, -2], p[-2, -1] to p[-2, (2×N)-1], and p[-1, -1] based on the upper left coordinate of the block may be included in the new reference pixel candidate group. Pixels may be classified into a first reference pixel line (A0 to P0, X0) and additional reference pixel lines to represent differences from existing reference pixels, and other additional reference pixel lines may be included as new reference pixel candidate groups. The order of the reference pixel lines may be determined based on the distance between the reference pixel and the current block.

[0311] The reference pixel line may be commonly applied to the pixels of the blocks at the left, upper left, lower left, upper, and upper right, and may be constructed by one adjacent block or two or more adjacent block units. For example, when a pixel line is constructed relative to the pixels of the block at the left, the pixels of the corresponding block may be constructed on the reference pixel line L. When a pixel line is constructed relative to the blocks at the upper left, upper, and upper right, the pixels of the corresponding block may be constructed in the reference pixel line U. This example will be described based on the reference pixel line commonly applied to the adjacent blocks.

[0312] Whether to add additional reference pixel lines other than existing reference pixel lines can be determined based on one or more factors of the block, block type, prediction mode and prediction method, or a combination thereof. In addition, whether to use existing reference pixel lines and whether to use additional reference pixel lines can have various constructed combinations. For example, existing reference pixel lines are always used, it can be determined whether to use additional pixel rows, and it can be determined separately whether to use each existing reference pixel line and whether to use each of the additional pixel rows. In addition, when there are more than two additional pixel rows, various combinations are possible relative to the above settings. And, when reference pixel lines are not applied to adjacent blocks in common but partially, it is possible to have various combinations related to the above settings.

[0313] In addition, the allowable range of the reference pixel line can be set. That is, information about how much the reference pixel line will be added (maximum value) can be set in the encoder / decoder, and this can be determined based on one factor or a combination of the type of the current or adjacent block, the size of the current or adjacent block, etc. If the order of the reference pixel line is a small number, the priority may be higher, but the assumption thereon may not lead to a binarization process of the relevant information. The index information or identifier information of the reference pixel line can be generated under predetermined conditions. The reference pixel line index or identifier information can be binarized by various methods such as unary binarization, cutting off the unary binarization, and the binarization can be determined based on the allowable maximum value information in the reference pixel line.

[0314] When adjacent blocks are used together for reference pixel line construction or other, various combinations relative to the above settings are possible. Information related to reference pixel lines can be sent in units of sequences, pictures, slices, etc. The encoded / decoded information can be included in the prediction mode information in the prediction mode encoding step.

[0315] After the reference pixel candidate group is constructed through the above process, a process of checking whether to use the reference pixel of the adjacent block is performed. Whether to use the reference pixel can be determined by at least one factor or a combination thereof such as the position of the adjacent block, the coding mode of the adjacent block, the reference image of the adjacent block, the position of the current block in the partition block, and the boundary characteristics of the current block and the adjacent block. The prediction of the current block can be performed using the pixel values ​​of the replacement of the pixel positions determined to be available by the process and the pixel positions determined to be unavailable by the process among the reference pixel candidate group.

[0316] In the case of prediction in a directional mode among intra prediction modes, reference pixels can be generated in decimal units by using various interpolation methods for reference pixels of integer pixels. When reference pixel lines are added as in the above example, reference pixels of two or more reference pixel lines can be supported in the interpolation process. Not only one-dimensional filtering that interpolates fractional pixels between two existing pixels can be applied, but also two-dimensional filtering that interpolates fractional pixels between four pixels can be applied.

[0317] In the reference pixel filtering step, a process of applying filtering to the reference pixels obtained from the reference pixel construction step will be described based on a case with wider reference pixels.

[0318] Fig.23a , Figure 23b , Fig.24a and Figure 24b 1 is a diagram showing various mode directions for a current block with a wider reference pixel that may be included in the reference pixel construction step. This is for the purpose of explaining the directions of the various prediction modes described above and is not limited thereto. In the case where the first reference pixel line (A0-P0, X0) and the second reference pixel line (A1-P1, X1, X2) are assumed to be 10 , X 01 , X 11 ) is constructed as a reference pixel candidate group for explanation.

[0319] When only the first reference pixel line is included in the reference pixel candidate group, filtering may be applied to pixels included in the reference pixel candidate group, that is, the most adjacent pixels of the current block. When additional reference pixel lines are included in the reference pixel candidate group, filtering that takes into account the additional reference pixels may be applied.

[0320] Fig.23a An exemplary view of a pattern (vertical) of reference pixels of a second reference pixel line, more specifically, reference pixels A1, B1, C1, and D1 located in the second row from the top of the current block are referenced so that pixels generated by applying filtering to the reference pixels are applied for prediction, and an exemplary view of a pattern of reference pixels of a first reference pixel line, specifically, reference pixels A0, B0, C0, and D0 located in the first row so that pixels generated by applying filtering to the reference pixels are applied for prediction.

[0321] Figure 23bAn exemplary view of a pattern (horizontal) of reference pixels of the second reference pixel line, specifically, reference pixels I1, J1, K1, and L1 of the second row located to the left of the current block are referenced so that pixels generated by applying the generated reference pixels are applied to prediction, and an exemplary view of a pattern of reference pixels of the first reference pixel line, specifically, reference pixels I0, J0, K0, and L0 so that pixels generated by applying filtering to the reference pixels are applied to prediction.

[0322] Fig.24a An exemplary view of a pattern of reference pixels of a second reference pixel line (a directional pattern toward the lower left end) is shown, specifically, reference pixels C1, D1, E1, F1, G1, and H1 located on the second row from the upper right end of the current block are referenced so that pixels generated by applying filtering to the reference pixels are applied to prediction, and an exemplary view of a pattern of reference pixels of a first reference pixel line, more specifically, reference pixels B0, C0, D0, E0, F0, G0, and H0 located on the first row from the upper right end of the current block are also referenced so that pixels generated by applying filtering to the reference pixels are applied to prediction.

[0323] Figure 24b An exemplary view of a pattern of reference pixels of the second reference pixel line (a directional pattern leading to the lower right end) is shown, specifically, reference pixels J1, I1, X01, X11, X10, A1 and B1 are referenced so that pixels generated by applying filtering to the reference pixels are applied to prediction, and an exemplary view of a pattern of reference pixels of the first reference pixel line, specifically, reference pixels K0, J0, I0, X0, A0, B0 and C0 located at the upper left of the current block so that pixels generated by applying filtering to the reference pixels are applied to prediction.

[0324] When filtering is applied to the reference pixels of the first reference pixel line, adaptive filtering may be performed according to the mode of the current block. When the reference pixels of the second reference pixel line are also applied for filtering, not only is adaptive filtering performed according to the mode of the current block, but the reference pixels applied for filtering may also be adaptive. That is, filtering on the neighboring pixels of the current block may be independent of the prediction mode direction, and when the reference pixels of the second reference pixel line are also included in the filtering process, the filtering may be performed according to the mode of the current block. Fig.22a , Figure 22b , Fig.23a and Figure 23b Pixels on the directional lines of the pattern shown in are applied to the filtering.

[0325] In the prediction block generation step, a process of applying filtering to the reference pixels obtained in the reference pixel construction step will be described based on a case with wider reference pixels.

[0326] In the case of a directional prediction method using the first reference pixel line as a reference pixel candidate group, the most adjacent reference pixel may be used to generate a prediction block. If additional reference pixel lines other than the first reference pixel line are supported, the reference pixels used to generate the prediction block may be determined. For example, depending on the prediction mode, a prediction block may be generated as a first reference pixel, a prediction block may be generated as a second reference pixel, and a prediction block may be generated as a prediction pixel of the first reference pixel line and the second reference pixel line.

[0327] In the following, the prediction mode encoding step will be described in more detail.

[0328] In this case, the following examples will be described in the HEVC standard, and the relevant settings (number of prediction modes, prediction mode direction, etc.) are not limited to the examples described below. Since there are 35 modes in the case of HEVC, after determining the best mode, the effective mode information needs to be encoded. Generally, when an image is divided into several blocks, adjacent blocks usually have similar characteristics. Therefore, when the mode of the current block is encoded, the adjacent block information is utilized. HEVC encodes the prediction mode of the current block based on the prediction modes of the left and top blocks.

[0329] Fig.25 is a flowchart showing an optimal mode determination process for a general current prediction unit (or prediction block). Here, the most probable mode (MPM) is a prediction mode having a high probability of being a prediction mode of the current prediction unit. Since prediction units around the current prediction unit may have a mode similar to that of the current prediction unit, a mode may be selected from modes of peripheral blocks.

[0330] refer to Fig.25 , based on the current prediction unit (PU), the prediction mode of the prediction unit on the left is referred to as left_PU_MPM, and the prediction mode of the prediction unit on the upper side is represented as Above_PU_MPM. Specifically, when the prediction mode of the left prediction unit is different from the prediction mode of the upper prediction unit, the first most probable mode MPM[0] is determined as the prediction mode of the left prediction unit, the second most probable mode MPM[1] is determined as the prediction mode of the upper prediction unit, and the third most probable mode MPM[2] is determined as one of the non-overlapping plane, DC, and vertical modes. On the other hand, when the prediction mode of the left prediction unit is the same as the prediction mode of the upper prediction unit, it is determined whether the same prediction mode is less than mode 2. If the prediction mode is less than mode 2, the plane, DC, and vertical modes are set as the most probable modes, and when the prediction modes are larger or the same, the prediction mode of the left prediction unit, a mode smaller than it, and a mode larger than it can be set to the first, second, and third most probable modes (MPMs), respectively.

[0331] In short, if Fig.25 As shown in , the mode with high probability as the prediction mode of the current block can be classified into the most probable mode (MPM) set or group and encoded. When the current block has the same characteristics as the adjacent block, so that the prediction mode of the adjacent block is the same or similar to the prediction mode of the current block, the possibility of selecting a mode of the MPM group is high, so that short bits can be allocated, thereby performing effective encoding. If this does not happen, the prediction mode of the current block can be encoded by binarizing the remaining modes except the most probable mode (MPM) among all modes and selecting one of them. In the case of HEVC, the number of modes in the group of the best mode (MPM) is three, so that one or two bits are used to perform encoding. If not selected in the most probable mode (MPM) group, the encoding is performed using a fixed length of 5 bits. According to an embodiment of the present invention, the use of the mode as the most probable mode (MPM) for the left and upper prediction units is not limited, and the mode information of one or more blocks from the left, upper left and upper, upper right and lower left blocks can be utilized. Furthermore, in the case where one of the blocks is divided into two or more blocks, it is possible to set a priority for information on which block among the divided blocks is referenced. For example, in the case where the current block is 2N×2N and two divided blocks of N×N are located on the left, according to the setting rule, the upper N×N mode may be used, or the lower N×N mode may be used.

[0332] The number of modes selected by the most probable mode (MPM) may also be preset in the encoding / decoding device, or may be sent in units of a sequence, a picture, a slice, etc. In addition, the binarization of the number of modes may be represented by using various binarization methods such as a fixed length or a variable length. In addition, various binarization methods may be used for the binarization of modes that are not classified as the most probable mode (MPM). In addition, in order to effectively binarize a mode that is not classified as the most probable mode (MPM), an MPM candidate group may be set. For example, M-2 of the M mode may be selected. N Can be binarized into the most probable mode (MPM) group, and 2N modes can be binarized into candidate groups that are not classified as the most probable mode (MPM). For example, if there are 19 modes, N can be selected as one of 1, 2, 3, 4, and the modes in the most probable mode (MPM) group can be represented as variable length (if N is 4, it can be three), and the candidate group that is not classified as the most probable mode (MPM) can be represented as a fixed length (4 bits).

[0333] Hereinafter, a specific example of determining the most probable mode (MPM) of the current prediction unit (or block) according to an embodiment of the present invention will be described. For ease of explanation, it is assumed that a block having the same block type and size as the current block is located in a neighboring block. It is also assumed that the mode of the block determined to be unavailable according to the availability of reference pixels of the neighboring blocks is DC.

[0334] First, in order to obtain the mode information of the adjacent blocks, it is necessary to check the availability of the boundary pixels of the corresponding block as reference pixels. For example, it is necessary to check whether the mode is located on the boundary line of the picture or slice, or whether the encoding method is the same. In this example, regardless of the reference picture, if the encoding mode is an inter-frame prediction mode, the reference pixels of the corresponding block are set to unavailable. (As in HEVC, the left and upper modes are used as MPM) When the left block generates a prediction block in the current picture by block matching and is encoded using an inter-frame prediction method, the reference pixels of the corresponding block can be determined to be unavailable. When the upper block is encoded using intra-frame prediction, the mode (vertical) of the corresponding block can be included in the MPM candidate. Since the reference pixels of the left block are unavailable, the mode of the corresponding block can be set to DC and included in the MPM candidate.

[0335] Available reference pixels and unavailable reference pixels can be classified by a process of checking the availability of reference pixels of neighboring blocks. Reference pixels determined to be unavailable can be filled with predefined values ​​(e.g., the middle value of the range of pixel values ​​of the bit depth) or substitute values ​​derived from reference pixels determined to be available. A prediction block can be generated by using the pixel values ​​of reference pixels available for intra prediction and substitute values ​​of unavailable reference pixels.

[0336] Fig.26 is an exemplary view for determining a prediction mode of a current prediction unit (or block).

[0337] refer to Fig.26 , available reference pixels can be displayed in black, and unavailable reference pixels can be displayed in no color. Unavailable reference pixels can be replaced by predefined values ​​or values ​​derived from available pixels. Fig.26 In the above, the pixel values ​​of the unavailable reference pixels of blocks A, B, C, and D may be filled with values ​​obtained by copying the pixel value in the upper right of block E. That is, all the reference pixels of blocks A, B, C, and D may have the same value as the pixel in the upper right of block E. For example, all modes that generate prediction blocks using blocks B and C as reference pixels may construct all the prediction pixels of the prediction blocks as the same pixel value (the pixel value in the upper right of block E).

[0338] Here, since block A or block B is determined to be unavailable, the block may have a DC mode as previously assumed. In this case, since the prediction mode of block A is set to DC and the mode of block B is also set to DC, it is possible to reduce the possibility that the mode determined as the most probable mode (MPM) (two DCs appear) is the prediction mode of the current block. Therefore, the mode can be used by replacing it with the prediction mode of the available block adjacent to the current block.

[0339] refer to Fig.26 , the mode of block A can replace the prediction mode of block E. By using DC as the prediction mode of block E and the prediction mode of block B, it is possible to increase the possibility that the prediction mode of the current block is selected as a mode of MPM. Fig.25 The flowchart of may set MPM[0]=planar mode, MPM[1]=DC mode, and MPM[2]=vertical mode, which is a general method. Assuming that the pixels in the right boundary of block E are available (black cells in the figure), and the pixel values ​​on top of the boundary pixels are copied to the boundary pixels of the above-mentioned unavailable blocks to perform intra-frame prediction, all the same prediction blocks are generated from plane, DC and vertical, whereby mode redundancy may occur. Therefore, as described above, the prediction mode using the alternative block can have a more favorable effect.

[0340] Reference again Fig.26 , if constrained intra prediction is performed, then Fig.26 The upper right pixel value of the boundary pixel of block E shown in black in FIG. 1 is copied to the boundary pixel of block A to perform intra prediction. It is also assumed that the mode of block E is not DC. Under this assumption, if the prediction block is a vertical, horizontal, group 2, group 3, and group 4 including plane and DC (refer to the following description for each prediction mode group) Fig. 27 ), it is possible to generate a prediction block in which the pixel values ​​of all prediction blocks generated by the prediction mode are one value (the upper right pixel value of block E). This may be because all are copied to the same pixel value. Therefore, reference Fig.25 , when the prediction mode of block E is selected by replacing block A and encoding according to the most probable mode (MPM), the candidate group can be classified into left_PU_MPM as the prediction mode of block E, and Above_PU_MPM as the prediction mode of block B (i.e., DC). Since Left_PU_MPM and Above_PU_MPM are different from each other, MPM[0] = prediction mode of block E, MPM[1] = DC, and MPM[2] = plane or vertical (determined according to the mode of block E).

[0341] When the prediction mode of block E is planar, DC, vertical, horizontal, and one of groups 2 to 4, MPM[0] is planar, DC, vertical, horizontal, and one of groups 2 to 4, and MPM[1] is DC, then if the mode of block E is not planar, MPM[2] is planar, and if the mode of block E is planar, MPM[2] is vertical. However, when a prediction block is generated with the modes of planar, DC, vertical, horizontal, and groups 2, 3, and 4 as described above, the same prediction block is generated (if the reference pixel is constructed with a pixel value of 100, all pixel values ​​are 100 regardless of which mode is used to generate the prediction block), so that the MPM is managed in an overlapping mode.

[0342] Therefore, when the prediction mode of block E is one of plane, DC, vertical, horizontal, and groups 2 to 4 (groups 2, 3, 4), MPM[0] is determined to be one of plane, DC, vertical, horizontal, groups 2 to 4, MPM[1] is determined to be one of the modes belonging to group 1, and MPM[2] is determined to be one of the blocks that do not overlap with MPM[1] belonging to group 1. This is because unless the reference pixels at the boundary of block E are all composed of the same pixel value, the prediction blocks may be slightly different (at least one pixel) for each mode. Here, encoding can be performed via binarization with an existing fixed 5 bits for the prediction mode that is not selected as the most probable mode (MPM) or via binarization only for the modes in group 1. For example, since there are 8 modes (reduced to 6 by subtracting 2 included in the MPM) for modes 2 to 9 (group 1), encoding can be performed via various binarization methods with bits shorter than the fixed 5 bits.

[0343] When the mode of block E is one of the modes of group 1, MPM[0] is the prediction mode of block E, MPM[1] is DC, and MPM[2] is a mode that does not overlap with the prediction mode of block E of group 1 (starting from group 1 because the prediction block generated by DC or the prediction block generated by plane, vertical, horizontal, group 2, 3, and 4 is the same). In this case, encoding can be performed by binarization as described above.

[0344] Fig. 27 It is a diagram showing grouping of prediction modes.

[0345] refer to Fig. 27 , it can be assumed that the prediction modes are classified into planar, DC, vertical, horizontal, group 1, group 2, group 3, and group 4. It is possible to allow various settings to group the prediction modes into 2 or more groups, and the following grouping is classified in the case of settings of similar directionality.

[0346] In this case, prediction modes 2 to 9 are for group 1, prediction modes 11 to 17 are for group 2, prediction modes 18 to 25 are for group 3, and prediction modes 27 to 34 are for group 4. In addition, prediction mode 0 may represent planar, prediction mode 1 may represent DC, prediction mode 10 may represent horizontal, and prediction mode 26 may represent vertical.

[0347] Under the premise of this prediction mode, the embodiment will be further described below.

[0348] Figure 28 to Figure 32 is an exemplary view of a case where an MPM is constructed for a prediction mode using neighboring blocks other than an upper block and a left block.

[0349] refer to Fig.28 , the prediction mode of block E can be used by replacing block A, and the prediction mode of block D can be used by replacing block B.

[0350] refer to Fig.29 , the prediction mode of block D can be used by replacing block A, and the prediction mode of block C can be used by replacing block B.

[0351] refer to Fig.30 , the prediction mode of block A is used as is, and the prediction mode of block E can be used by replacing block B.

[0352] refer to Fig.31 , the prediction mode of block B is used as is, and the prediction mode of block C can be used by replacing block B.

[0353] refer to Fig.32 , block A may be replaced with the prediction mode of block E, block B may be replaced with the prediction mode of block D, block A may be replaced with the prediction mode of block D, and block B may be replaced with the prediction mode of block C, and block A may be replaced with the prediction mode of block E, and block B may be replaced with the prediction mode of block C. That is, the prediction modes of the available blocks may be replaced by various combinations.

[0354] The above example is described as an example when a block is not available due to restricted intra prediction, but it can also be applied when all neighboring blocks are available. If the reference pixels of the neighboring blocks are similar or identical (for example, in the case of a computer capture screen), it is also possible to encode by changing the candidate group setting of the most probable mode (MPM). This case will be described below.

[0355] Fig.33 is an exemplary view of a case where a most probable mode (MPM) is constructed as a substitute block when both a current block and a neighboring block are available.

[0356] refer to Fig.33, black pixels are used as reference pixels, where pixels at the boundaries with blocks A, B, C, and D have no change in pixel value or a similar value of pixel value, and pixels at the boundary of block E are changed.

[0357] In this state, even if you use Fig. 27 If any prediction mode in groups 2 to 4 shown in FIG. 1 is used, the same or similar prediction block is generated, so that redundancy of the most probable mode (MPM) candidate group may occur.

[0358] First, it is possible to classify blocks by the variance σ of the reference pixel values ​​of each block A , σ B , σ C , σ D and σ E Less than the threshold (Th). Assuming the variance σ A , σ B , σ C , σ D and σ E is less than the boundary value, and the variance σ of block E E Greater than Fig.33 When the variance of two or more consecutive blocks is less than the boundary value, redundancy of the most probable mode (MPM) candidate group may occur. For example, if the reference pixels of block B and block C have the same or similar values, the group 4 of the prediction mode generated using only block B and block C and the prediction block generated by the vertical mode can be regarded as having the same or similar values. Therefore, the group 4 and the vertical mode can perform the redundancy removal process. If the reference pixels of block A and block C are the same or similar, the prediction mode generated using only block A and block C does not exist and therefore does not need to be considered.

[0359] First, since prediction blocks generated from horizontal, group 2, group 3, and vertical of prediction modes generated using reference pixels of blocks A, D, and B have the same and similar values, it can be determined that redundancy occurs.

[0360] Since it is assumed that the prediction blocks generated in the vertical sum group 4 of the prediction mode generated using the reference pixels of the blocks B and C have the same and similar values, it can be determined that redundancy occurs in the mode.

[0361] Since it is assumed that the prediction blocks generated in the horizontal, group 2, group 3, vertical, group 4, group 4, and DC planes of the prediction mode generated using the reference pixels of blocks A, D, B, and C have the same prediction blocks, it can be determined that redundancy occurs in the mode. That is, it can be confirmed that the variance of the continuous blocks is calculated (σ ADB , σ BC , σ ADBC ), which is smaller than the boundary value.

[0362] exist Fig.33 In the case of , it is confirmed that the variance of A, D, B, C is less than the boundary value and causes the most likely redundancy. Then, the MPM candidate group can be reset by removing the redundancy as described above. The variance is mentioned to illustrate the above example, but it is not limited to this.

[0363] When the prediction mode of block A is different from the prediction mode of block B, refer to Fig.25 , MPM[0]=prediction mode of block A, MPM[1]=prediction mode of block B, MPM[2]=one of plane, DC or vertical (according to the prediction mode of block A and the prediction mode of block B). If one of the two blocks is in group 1 and the other is in other cases, the prediction mode of block A is classified by comparing the above variance values ​​to generate one of horizontal, group 2, group 3, vertical, group 4, DC, plane of the same prediction block, and when the mode is one of the modes of group 1, the prediction mode of block B can be constructed not to overlap with the prediction mode of block B among MPM[0]=prediction mode of block A, MPM[1]=prediction mode of block B, and MPM[2]=mode of group 1. If both modes belong to group 1, MPM[0]=prediction mode of block A, MPM[1]=prediction mode of block B, and MPM[2]=plane. If neither mode belongs to group 1, then MPM[0] = prediction mode of block A, MPM[1] = one of the modes in group 1, MPM[2] = one of the modes in group 1 that overlaps with MPM[1]. If the prediction mode of block A is the same as the prediction mode of block B, and both modes are greater than mode 2 (not DC and planar) and leave Fig.25 , it can be configured as: MPM[0] = prediction mode of block A and MPM[1] = one of the prediction modes of group 1 and if both modes are horizontal, one of the modes of groups 2, 3 and vertical that does not overlap with MPM[2] = MPM[1]. Otherwise, it can be configured as MPM[0] = prediction mode of block A, MPM[1] = prediction mode a of block A, MPM[2] = prediction mode of block A + 1. If the two modes are the same and both modes are less than mode 2 (one of DC and planar), then MPM[0] = planar, MPM[1] = one of the modes of group 1, and MPM[2] is one of the modes of group 1 that does not overlap with MPM[1].

[0364] Fig.34 is a flowchart illustrating an image encoding method for performing intra-frame prediction according to an embodiment of the present invention.

[0365] refer to Fig.34, an image encoding method for performing intra-frame prediction includes: determining whether each of the left block and the upper block of the current block is available in the reference pixel configuration of the current block (S210); selecting a most probable mode MPM candidate group based on the determined result (S220); determining redundancy of a prediction block derived from an intra-frame prediction mode in the MPM candidate group (S230); selecting an MPM group using a prediction mode of an alternative block adjacent to the left block or adjacent to the upper block if redundancy is determined to exist (S240); and performing intra-frame prediction mode encoding for the current block based on the selected MPM group (S250).

[0366] Here, determining whether each block is available (S210) may be determined as unavailable when the left block and the upper block exist outside the image boundary or are encoded with inter prediction, and may be determined as available in other cases.

[0367] Here, the MPM candidate group may include at least one of an intra prediction mode for a block determined to be available among the left block and the upper block, an intra prediction mode predetermined for a block determined to be unavailable, a plane prediction mode, a plane prediction mode, a vertical prediction mode, and a DC prediction mode.

[0368] Here, determining redundancy (S230) may determine redundancy based on variance values ​​calculated for reference pixel values ​​in two or more continuous blocks among blocks located at the lower left, left, upper left, upper, and upper right of the current block when both the left block and the upper block are available. For example, two or more continuous blocks are selected and grouped, and variance values ​​of the selected groups are calculated. If the calculated variance value is less than a boundary value, it may be determined that redundancy exists.

[0369] Determining redundancy (S230) may include selecting an intra prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0370] Here, the replacement block may be selected from neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0371] Here, if the neighboring blocks consist of one or more blocks, the replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0372] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority.

[0373] An image decoding method for performing intra-frame prediction may include: extracting data on whether a prediction mode of a current block is included in a most probable mode (MPM) group from a received bit stream; determining whether the prediction mode of the current block is included in the MPM group based on the data; extracting MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group; confirming the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and performing intra-frame prediction on the current block based on the confirmed prediction mode.

[0374] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct reference pixels of the current block; selecting the most likely mode MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of an alternative block adjacent to the left block or to the upper block when it is determined that redundancy exists.

[0375] Here, it is determined whether a block can be used when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction, and can be determined to be available in other cases.

[0376] Here, the MPM candidate group may be selected from the intra prediction mode of the block determined to be available among the left block and the above block, the intra prediction mode predetermined for the block determined to be unavailable, the planar prediction mode, the prediction mode, and the DC prediction mode.

[0377] Here, when both the left block and the upper block are available, determining the redundancy of the prediction block can be performed by determining the redundancy based on the variance value calculated for the reference pixel values ​​in two or more consecutive blocks among the blocks located below the left, to the left, above the left, above, and above the right of the current block.

[0378] Here, determining the redundancy of the prediction block can be performed by selecting an intra-frame prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0379] Here, the replacement block may be selected from neighboring blocks located at the lower left, upper left, and upper right of the current block.

[0380] Here, when a neighboring block is constructed with one or more blocks, a replacement block may be selected with a clockwise or counterclockwise priority among the one or more blocks.

[0381] Here, the replacement block may be selected from among neighboring blocks located at the lower left, upper left, and upper right of the current block in a clockwise or counterclockwise priority.

[0382] In an image decoding device including one or more processors according to other aspects of the present invention, the one or more processors may: extract data related to whether a prediction mode of a current block is included in a most probable mode (MPM) group from a received bit stream; determine whether the prediction mode of the current block is included in the MPM group based on the data; extract MPM group index data of the current block from the received bit stream when it is determined that the prediction mode of the current block is included in the MPM group; confirm the prediction mode of the current block in the MPM group based on the extracted MPM group index data; and perform intra-frame prediction on the current block based on the confirmed prediction mode.

[0383] Here, the MPM group can be selected by: determining whether the left block and the upper block of the current block can be used to construct reference pixels of the current block; selecting the MPM candidate group based on the determined result; determining the redundancy of the prediction block derived from the intra-frame prediction mode in the MPM candidate group; and using the prediction mode of the alternative block adjacent to the left block or the upper block when it is determined that redundancy exists.

[0384] Here, determining whether a block can be used may be performed by determining to be unusable when each of the left block and the upper block exists outside the image boundary or is encoded with inter prediction, and may be determined to be available in other cases.

[0385] Determining the redundancy of the prediction block can be performed by selecting an intra-frame prediction mode constructed by referring only to boundary pixels between the unavailable block and the current block when at least one of the left block and the upper block is unavailable, and determining that redundancy exists when at least two of the selected prediction modes are included in the MPM candidate group.

[0386] According to the above-mentioned embodiments, it is possible to provide high-performance and efficient image encoding and decoding technology that is generally available in international codecs such as MPEG-2, MPEG-4 and H.264 or other codecs (in which intra-frame prediction is used), and the media and image industries using such codecs.

[0387] In the future, it is expected to be applied to the current High Efficiency Image Coding Technology (HEVC) and image processing using standard codecs such as H.264 / AVC and intra prediction.

[0388] While the invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined by the following claims.

Claims

1. An image decoding method, the method comprising: Decoding the current picture reference information indicating whether the current picture including the current block is included in the reference picture list; decoding skip mode flag information indicating whether a skip mode is applied to the current block based at least in part on the current picture reference information; generating a reference picture list 0 and a reference picture list 1 for the current block; Determine a reference picture of the current block based on the reference picture list 0 and the reference picture list 1; generating a prediction block of the current block using the reference picture; and In response to the skip mode being applied to the current block, reconstructing the current block based on the prediction block of the current block without decoding residual information of the current block, wherein the reference picture list 0 and the reference picture list 1 respectively include at least one of pictures decoded before the current picture, The reference picture list 0 and the reference picture list 1 also include the current picture according to the current picture reference information, The reference picture list 0 is generated by sequentially allocating a reference picture before the current picture, a reference picture after the current picture, and the current picture, The reference picture list 1 is generated by sequentially allocating a reference picture after the current picture, a reference picture before the current picture, and the current picture, and The current picture reference information is decoded in units of pictures.

2. A method for encoding an image, the method comprising: Generate a reference picture list 0 and a reference picture list 1 for the current block; Determine a reference picture of the current block based on the reference picture list 0 and the reference picture list 1; performing prediction on the current block using the reference picture; encoding current picture reference information indicating whether a current picture of the current block is included in a reference picture list; encoding skip mode flag information indicating whether a skip mode is applied to the current block based at least in part on the current picture reference information; and In response to the skip mode flag information indicating that the skip mode is not applied to the current block, encoding residual information of the current block, wherein the reference picture list 0 and the reference picture list 1 respectively include at least one of pictures decoded before the current picture, The reference picture list 0 and the reference picture list 1 also include the current picture according to the current picture reference information, The reference picture list 0 is generated by sequentially allocating a reference picture before the current picture, a reference picture after the current picture, and the current picture, The reference picture list 1 is generated by sequentially allocating a reference picture after the current picture, a reference picture before the current picture, and the current picture, The current picture reference information is encoded in picture units, and In response to the skip mode flag information indicating that the skip mode is applied to the current block, encoding of the residual information of the current block is skipped.

3. A bitstream storage method, comprising: generating a bitstream including current picture reference information indicating whether a current picture including a current block is included in a reference picture list and skip mode flag information indicating whether a skip mode is applied to the current block; and storing the bitstream, in, The current block is predicted using a reference picture, The reference picture of the current block is determined based on at least one of a reference picture list 0 and a reference picture list 1, The reference picture list 0 and the reference picture list 1 respectively include at least one of pictures decoded before the current picture, The reference picture list 0 and the reference picture list 1 also include the current picture according to the current picture reference information, The reference picture list 0 is generated by sequentially allocating a reference picture before the current picture, a reference picture after the current picture, and the current picture, The reference picture list 1 is generated by sequentially allocating a reference picture after the current picture, a reference picture before the current picture, and the current picture, The current picture reference information is encoded in picture units, and When the skip mode flag information indicates that the skip mode is applied to the current block, the bitstream further includes residual information of the current block.

Images