Image decoding / encoding method, method of transmitting bitstream, and recording medium

Abstract

This invention provides an image decoding / encoding method, a method for transmitting bit streams, and a recording medium. The image encoding method of this invention includes: a first segmentation step of dividing a current image into multiple blocks; and a second segmentation step of dividing a segmentation target block containing the boundary of the current image into multiple lower-level blocks, wherein the second segmentation step is recursively executed with lower-level blocks containing the boundary of the current image as the segmentation target blocks, until there are no more lower-level blocks containing the boundary of the current image among the lower-level blocks.

Description

[0001] This application is a divisional application of the invention patent application filed on October 12, 2017, with application number 201780063610.8 and entitled "Image Encoding Method / Apparatus, Image Decoding Method / Apparatus and Recording Medium Containing Bit Stream". Technical Field

[0002] This invention relates to an image encoding / decoding method and apparatus. More particularly, it relates to an image boundary processing method and apparatus during the image encoding / decoding process. Specifically, it relates to a method and apparatus for effectively processing image boundaries in image filling and tree structures. Background Technology

[0003] In recent years, the demand for multimedia data such as video on the Internet has been increasing dramatically. However, the current rate of development of channel bandwidth is insufficient to fully meet the rapidly increasing volume of multimedia data. As part of this trend, international standardization organizations, namely the Video Coding Expert Group (VCEG) of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and the Moving Picture Expert Group (MPEG) of the International Organization for Standardization / International Electrotechnical Commission (ISO / IEC), are working tirelessly through collaborative research to develop more efficient video compression standards.

[0004] Video compression generally consists of intra-frame prediction, inter-frame prediction, transform, quantization, entropy coding, and in-loop filtering. Intra-frame prediction refers to the technique of generating a predicted block for the current block using the reconstructed pixels surrounding the current block. Because the encoding process is performed on a block-by-block basis, if the image size is not a multiple of the block size, it becomes difficult to encode in blocks. Summary of the Invention

[0005] The purpose of this invention is to provide an image encoding / decoding method and apparatus that can improve compression efficiency during the image encoding / decoding process.

[0006] Furthermore, the present invention aims to provide a filling method and apparatus that can align image sizes in block units during the encoding / decoding process of images.

[0007] Furthermore, the purpose of this invention is to provide a block segmentation method and apparatus that can improve the efficiency of image boundary processing during image encoding / decoding.

[0008] Furthermore, the present invention aims to provide an image encoding / decoding method and apparatus that can improve compression efficiency by effectively utilizing information from surrounding blocks when encoding / decoding the current block during the image encoding / decoding process.

[0009] Furthermore, the object of the present invention is to provide a computer-readable recording medium capable of storing a bitstream generated by an image encoding method / apparatus applicable to the present invention.

[0010] The image encoding method applicable to the present invention includes: a first segmentation step of segmenting a current image into multiple blocks; and a second segmentation step of segmenting a segmentation object block containing the boundary of the current image into multiple lower-level blocks; wherein the second segmentation step can be recursively executed with the lower-level blocks containing the boundary of the current image as the segmentation object blocks, until there are no more lower-level blocks containing the boundary of the current image in the lower-level blocks.

[0011] In the image encoding method to which this invention is applied, the first segmentation step may be a step of segmenting the current image into multiple maximum blocks of the same size.

[0012] In the image encoding method to which this invention is applied, the second segmentation step may be a step of performing quadtree segmentation or binary tree segmentation on the segmentation target block.

[0013] In the image encoding method to which this invention is applied, when the segmentation object block is of a size suitable for performing the quadtree segmentation, the quadtree segmentation can be performed on the segmentation object block in the second segmentation step; and when the segmentation object block is of a size unsuitable for performing the quadtree segmentation, the binary tree segmentation can be performed on the segmentation object block in the second segmentation step.

[0014] In the image encoding method to which the present invention is applied, when the quadtree segmentation or the binary tree segmentation is performed on the segmentation object block in the second segmentation step, the first segmentation information used to indicate whether the segmentation object block has been segmented by a quadtree or the second segmentation information used to indicate whether the segmentation object block has been segmented by a binary tree is not encoded.

[0015] In the image encoding method to which this invention is applied, when the binary tree segmentation is performed on the segmentation object block in the second segmentation step, the segmentation direction of the binary tree segmentation can be determined based on the shape of the encoded object block that is contained in the segmentation object block and also corresponds to the boundary region of the current image.

[0016] In the image encoding method to which this invention is applied, when the binary tree segmentation is performed on the segmented object block in the second segmentation step, the segmentation direction information used to indicate the segmentation direction of the binary tree segmentation can be omitted from encoding.

[0017] In the image encoding method to which the present invention is applied, the second segmentation step may further include: a step of comparing the size of the remaining region that is included in the segmentation object block and is also equivalent to the boundary region outside the current image with a specific threshold value; wherein, one of the quadtree segmentation and the binary tree segmentation may be performed on the segmentation object block based on the comparison structure.

[0018] In the image encoding method to which this invention is applied, the size of the remaining region is the smaller of the horizontal length and the vertical length of the remaining region. The second segmentation step can perform the quadtree segmentation on the segmentation object block when the size of the remaining region is greater than the specific threshold value, and perform the binary tree segmentation on the segmentation object block when the size of the remaining region is less than the specific threshold value.

[0019] In the image encoding method to which this invention is applied, the second segmentation step is a step of performing binary tree segmentation on the segmentation object block. The segmentation direction of the binary tree segmentation can be determined based on the shape of the remaining region that is included in the segmentation object block and also corresponds to the boundary region outside the current image.

[0020] In the image encoding method to which this invention is applied, the segmentation direction of the binary tree segmentation can be vertical when the remaining region is a vertically elongated block, and horizontal when the remaining region is a horizontally elongated block.

[0021] The image decoding method applicable to the present invention includes: a first segmentation step of segmenting a current image into multiple blocks; and a second segmentation step of segmenting a segmentation object block containing the boundary of the current image into multiple lower-level blocks; wherein the second segmentation step can be recursively executed with the lower-level blocks containing the boundary of the current image as the segmentation object blocks until there are no more lower-level blocks containing the boundary of the current image in the lower-level blocks.

[0022] In the image decoding method to which this invention is applied, the second segmentation step may be a step of performing quadtree segmentation or binary tree segmentation on the segmentation target block.

[0023] In the image decoding method to which this invention is applied, when the segmentation object block is of a size suitable for performing the quadtree segmentation, the quadtree segmentation can be performed on the segmentation object block in the second segmentation step; and when the segmentation object block is of a size unsuitable for performing the quadtree segmentation, the binary tree segmentation can be performed on the segmentation object block in the second segmentation step.

[0024] In the image decoding method to which the present invention is applied, when the quadtree segmentation or the binary tree segmentation is performed on the segmentation object block in the second segmentation step, it is possible to derive the information from a specific value instead of decoding the first segmentation information indicating whether the segmentation object block has been segmented by a quadtree or the second segmentation information indicating whether the segmentation object block has been segmented by a binary tree from the bitstream.

[0025] In the image decoding method to which this invention applies, when the binary tree segmentation is performed on the segmentation object block in the second segmentation step, the segmentation direction of the binary tree segmentation can be determined based on the shape of the decoded object block that is contained in the segmentation object block and also corresponds to the boundary region of the current image.

[0026] In the image decoding method to which this invention is applied, when the binary tree segmentation is performed on the segmentation object block in the second segmentation step, the segmentation direction information used to indicate the segmentation direction of the binary tree segmentation can be derived from a specific value instead of decoding it from the bitstream.

[0027] In the image decoding method to which the present invention is applied, the second segmentation step may further include: a step of comparing the size of the remaining region that is included in the segmentation object block and is also equivalent to the boundary region outside the current image with a specific threshold value; wherein, one of the quadtree segmentation and the binary tree segmentation may be performed on the segmentation object block based on the comparison structure.

[0028] In the image decoding method to which this invention applies, the size of the remaining region is the smaller of the horizontal length and the vertical length of the remaining region. The second segmentation step can perform the quadtree segmentation on the segmentation object block when the size of the remaining region is greater than the specific threshold value, and perform the binary tree segmentation on the segmentation object block when the size of the remaining region is less than the specific threshold value.

[0029] In the image decoding method to which this invention is applied, the second segmentation step is a step of performing binary tree segmentation on the segmentation object block. The segmentation direction of the binary tree segmentation can be determined based on the shape of the remaining region that is included in the segmentation object block and also corresponds to the boundary region outside the current image.

[0030] In the image decoding method to which this invention is applied, the segmentation direction of the binary tree segmentation can be vertical when the remaining region is a vertically elongated block, and horizontal when the remaining region is a horizontally elongated block.

[0031] The image decoding method applicable to the present invention can include: a step of determining the segmentation pattern of a parent block including the current block; a step of constructing a reference candidate list of the current block based on the segmentation pattern of the parent block; a step of deriving prediction information of the current block based on the reference candidate list; and a step of generating a prediction block of the current block based on the prediction information; wherein the reference candidate list can contain at least one candidate block or prediction information of the at least one candidate block.

[0032] In the image decoding method of the present invention, when the segmentation form of the upper-level block is a binary tree segmentation, it may further include: a step of comparing the segmentation depth of the block preceding the current block in the decoding order with the segmentation depth of the current block; and a step of determining whether the upper-level block of the preceding block is the same as the upper-level block of the current block; wherein, when the segmentation depth of the preceding block is the same as the segmentation depth of the current block and the upper-level block of the preceding block is the same as the upper-level block of the current block, the preceding block may be excluded from the at least one candidate block.

[0033] The image decoding method to which the present invention is applied may further include the step of including a replacement block in the reference candidate list when the previous block is excluded from the at least one candidate block.

[0034] In the image decoding method to which this invention is applied, the alternative block can be a block that is not adjacent to the current block and is divided from the block of the upper-level block of the current block.

[0035] In the image decoding method to which this invention is applied, when the current block uses intra-frame prediction, the reference candidate list can be an MPM list, and when the current block uses inter-frame prediction, the reference candidate list can be a candidate list that includes spatial candidates.

[0036] In the image decoding method of the present invention, when the segmentation form of the upper-level block is quadtree segmentation, it may further include: a step of determining whether the current block is the last sub-block in the decoding order of the four sub-blocks generated by performing quadtree segmentation on the upper-level block; and a step of determining whether the remaining three sub-blocks other than the current block have the same prediction information when the current block is the last sub-block; wherein, when the remaining three sub-blocks have the same prediction information, the remaining three sub-blocks can be excluded from the at least one candidate block.

[0037] In the image decoding method to which this invention is applied, the at least one candidate block can be included in the reference candidate list according to a specific priority.

[0038] In the image decoding method to which this invention is applied, the priority can be determined based on at least one of whether the current block is adjacent to the candidate block and the position of the candidate block.

[0039] In the image decoding method to which this invention is applied, candidate blocks adjacent to the current block can be assigned a higher priority than candidate blocks that are not adjacent to the current block.

[0040] In the image decoding method to which this invention is applied, a candidate block located to the left or top of the current block can be assigned a higher priority than other candidate blocks.

[0041] In the image decoding method to which this invention is applied, the priority can be determined based on at least one of the segmentation depth of the current block and the segmentation depth of the candidate block.

[0042] In the image decoding method to which this invention is applied, a higher priority can be assigned as the difference between the segmentation depth of the current block and the segmentation depth of the candidate block is smaller.

[0043] In the image decoding method to which this invention is applied, candidate blocks with larger segmentation depths can be assigned a higher priority than candidate blocks with smaller segmentation depths.

[0044] In the image decoding method to which this invention is applied, the priority can be determined based on the segmentation order of the candidate blocks.

[0045] In the image decoding method to which this invention is applied, candidate blocks with earlier segmentation order can be assigned a higher priority than candidate blocks with later segmentation order.

[0046] In the image decoding method to which this invention is applied, the priority can be determined based on the relationship between the prediction information of the candidate blocks.

[0047] In the image decoding method to which this invention is applied, when the prediction information of the first candidate block in the candidate blocks is determined based on the prediction information of the second candidate block in the candidate blocks, the first candidate block can be assigned a lower priority than the second candidate block.

[0048] The image decoding method applicable to the present invention may include: a step of deriving prediction information of a current block; a step of generating a prediction block of the current block based on the prediction information; a step of determining the segmentation pattern of an upgraded block containing the current block; a step of constructing a reference candidate list of the current block based on the segmentation pattern of the upgraded block; and a step of encoding the prediction information of the current block based on the reference candidate list; wherein the reference candidate list may contain at least one candidate block or prediction information of the at least one candidate block.

[0049] In the image coding method of the present invention, when the segmentation form of the upper-level block is a binary tree segmentation, the method further includes: a step of comparing the segmentation depth of the block preceding the current block in the coding order with the segmentation depth of the current block; and a step of determining whether the upper-level block of the preceding block is the same as the upper-level block of the current block; wherein, when the segmentation depth of the preceding block is the same as the segmentation depth of the current block and the upper-level block of the preceding block is the same as the upper-level block of the current block, the preceding block is excluded from the at least one candidate block in the image coding method.

[0050] In the image coding method of the present invention, when the segmentation form of the upper-level block is quadtree segmentation, it may further include: a step of determining whether the current block is the last sub-block in the coding order of the four sub-blocks generated by performing quadtree segmentation on the upper-level block; and a step of determining whether the remaining three sub-blocks other than the current block have the same prediction information when the current block is the last sub-block; wherein, when the remaining three sub-blocks have the same prediction information, the remaining three sub-blocks can be excluded from the at least one candidate block.

[0051] In the image coding method to which this invention is applied, the at least one candidate block can be included in the reference candidate list according to a specific priority.

[0052] In the image coding method to which this invention is applied, the priority can be determined based on at least one of whether the current block is adjacent to the candidate block and the position of the candidate block.

[0053] In the image coding method to which this invention is applied, the priority can be determined based on at least one of the segmentation depth of the current block and the segmentation depth of the candidate block.

[0054] In the image coding method to which this invention is applied, the priority can be determined based on the segmentation order of the candidate blocks.

[0055] In the image coding method to which this invention is applied, the priority can be determined based on the relationship between the prediction information of the candidate blocks.

[0056] The computer-readable recording medium to which this invention is applied is capable of storing bitstreams generated by the image encoding method to which this invention is applied.

[0057] This invention provides an image encoding / decoding method and apparatus that can improve compression efficiency.

[0058] Furthermore, this invention provides a filling method and apparatus that aligns image sizes in block units during the encoding / decoding process of images.

[0059] Furthermore, this invention provides a block segmentation method and apparatus that can improve the efficiency of image boundary processing during image encoding / decoding.

[0060] Furthermore, this invention provides an image encoding / decoding method and apparatus that can improve compression efficiency by effectively utilizing information from surrounding blocks when encoding / decoding the current block.

[0061] Furthermore, this invention provides a computer-readable recording medium capable of storing bitstreams generated by the image encoding method / apparatus to which this invention is applicable.

[0062] By applying the filling method and apparatus or the block segmentation method and apparatus of the present invention, the compression efficiency of images can be improved. Attached Figure Description

[0063] Figure 1 This is a block diagram illustrating a simplified image encoding apparatus according to one embodiment of the present invention.

[0064] Figure 2 This is a schematic diagram illustrating the types of intra-frame prediction modes defined by the image encoding / decoding device.

[0065] Figure 3 This is a schematic diagram illustrating an intra-frame prediction method based on planar patterns.

[0066] Figure 4This is a schematic diagram illustrating the reference pixels during intra-frame prediction.

[0067] Figure 5 This is a schematic diagram illustrating intra-frame prediction methods based on horizontal and vertical modes.

[0068] Figure 6 This is a block diagram illustrating a simplified image decoding apparatus according to one embodiment of the present invention.

[0069] Figure 7 This is a schematic diagram illustrating the filling process to which one embodiment of the present invention applies.

[0070] Figure 8a And 8b is a schematic diagram illustrating the filling process to which the present invention is applicable.

[0071] Figure 9 It is used for Figure 1 A schematic diagram illustrating the operation of the image segmentation unit 101.

[0072] Figure 10 This is a schematic diagram illustrating the operation of the decoder to which the present invention is applied.

[0073] Figure 11 This is an illustrative diagram illustrating the case where the size of the current image is not equal to a multiple of the maximum block size.

[0074] Figures 12a to 12e This is an example diagram used to illustrate an image segmentation method to which one embodiment of the present invention is applicable.

[0075] Figures 13a to 13e This is an example diagram used to illustrate an image segmentation method to which another embodiment of the present invention is applicable.

[0076] Figure 14 This is a schematic diagram illustrating the method for generating motion information candidate groups in skip mode and merge mode.

[0077] Figure 15 This is a schematic diagram illustrating the method for deriving candidate groups of motion information for motion prediction in the AMVP model.

[0078] Figure 16 It is a diagram used to illustrate the positions of spatial and temporal supplementary blocks for the current block.

[0079] Figure 17 This is a schematic diagram illustrating the method for determining motion information for time candidates.

[0080] Figure 18This is a diagram illustrating the priority of combinations that can be used when generating bidirectional motion information candidates.

[0081] Figure 19 This is a diagram illustrating the generation of the MPM candidate list.

[0082] Figure 20 This is a schematic diagram illustrating the encoding method for optimal prediction information in skip mode, merge mode, AMVP mode, and intra-frame prediction mode.

[0083] Figure 21 This is a schematic diagram used to illustrate the decoding method for optimal prediction information.

[0084] Figure 22 This is a schematic diagram illustrating an embodiment of constructing a reference candidate list using the surrounding blocks of the current block.

[0085] Figure 23 This is a schematic diagram illustrating an instance of constructing a reference candidate list for the current block based on the block segmentation pattern.

[0086] Figure 24 This is a schematic diagram illustrating another embodiment of constructing a reference candidate list using the surrounding blocks of the current block.

[0087] Figure 25 This is a schematic diagram illustrating another embodiment for constructing a reference candidate list of the current block based on the block segmentation pattern.

[0088] Figure 26 This is a diagram illustrating the method for changing the priority of neighboring blocks of the current block that are included in the reference candidate list.

[0089] Figure 27 This is a schematic diagram used to illustrate the method for determining the priority of surrounding blocks based on segmentation depth.

[0090] Figure 28 This is a schematic diagram used to illustrate the method for determining the priority of neighboring blocks based on the segmentation order.

[0091] Figure 29 This is a schematic diagram used to illustrate a method for determining the priority of surrounding blocks based on the causal relationship of predictive information.

[0092] Figure 30 This is a schematic diagram used to illustrate the method for determining the priority of surrounding blocks when both segmentation depth and segmentation order are taken into account. Detailed Implementation

[0093] This invention is capable of various modifications and has many different embodiments. Specific embodiments will be illustrated and described in detail below with reference to the accompanying drawings. However, the following description is not intended to limit the invention to specific implementations, but should be understood to include all modifications, equivalents, and substitutions within the scope of the invention's concept and technology. Similar reference numerals are used for similar constituent elements in the description of the various drawings.

[0094] In describing different constituent elements, terms such as "first" and "second" may be used, but the constituent elements are not limited by these terms. These terms are merely used to distinguish one constituent element from others. For example, without departing from the scope of the claims, a first constituent element can also be named a second constituent element, and similarly, a second constituent element can also be named a first constituent element. The term "and / or" includes a combination of multiple related descriptions or one of multiple related descriptions.

[0095] When a component is described as being "connected" or "in contact" with other components, it should be understood that it can not only be directly connected or in contact with the other components, but also that other components can exist between them. Conversely, when a component is described as being "directly connected" or "directly in contact" with other components, it should be understood that no other components exist between them.

[0096] The terminology used in this application is for illustrative purposes only and is not intended to limit the invention. Singular statements also have plural meanings unless the context clearly indicates otherwise. In this invention, terms such as “~comprises~” or “~consisting of~” are merely to indicate the presence of features, numbers, steps, actions, constituent elements, components, or combinations thereof described in the specification, and should not be construed as excluding the possibility of the presence or addition of one or more other features, numbers, steps, actions, constituent elements, components, or combinations thereof.

[0097] The embodiments to which the present invention applies will now be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals will be used for the same constituent elements in the drawings, and repeated descriptions of the same constituent elements will be omitted.

[0098] Figure 1 This is a block diagram illustrating an image encoding apparatus to which one embodiment of the present invention is applied.

[0099] See Figure 1The image encoding apparatus 100 may include an image segmentation unit 101, an intra-frame prediction unit 102, an inter-frame prediction unit 103, a subtraction unit 104, a transform unit 105, a quantization unit 106, an entropy encoding unit 107, an inverse quantization unit 108, an inverse transform unit 109, an addition unit 110, a filtering unit 111, and a memory 112.

[0100] exist Figure 1 The various components are illustrated separately to show the different special functions of the image encoding apparatus, but this does not mean that each component is composed of separate hardware or software units. That is, although the various components are listed and described for ease of explanation, it is possible to combine at least two components into one component, or to divide one component into multiple components and make them perform corresponding functions. The embodiments in which the various components are integrated and the embodiments in which they are separated, as described above, are included within the scope of the claims of this invention without departing from the essence of this invention.

[0101] Furthermore, some constituent elements may not be essential for performing the essential functions of this invention, but are merely optional elements used to improve performance. This invention can include only the constituent parts necessary for realizing the essence of the invention, excluding those merely used to improve performance, and structures that include only the essential constituent elements, excluding the optional elements used to improve performance, are also included within the scope of the claims of this invention.

[0102] The image segmentation unit 100 can segment an input image into at least one block. The input image can be of various shapes and sizes, such as a picture, slice, tile, or segment. A block can refer to a coding unit (CU), prediction unit (PU), or transform unit (TU). The segmentation can be performed based on at least one of a quadtree or a binary tree. A quadtree divides a parent block into four child blocks, each half the width and height of the parent block. A binary tree divides a parent block into two child blocks, each half the width or height of the parent block. Through segmentation based on the binary tree, blocks can be segmented not only into squares but also into non-square shapes.

[0103] In the embodiments where the present invention is applied, the encoding unit can be used not only as the meaning of encoding execution unit, but also as the meaning of decoding execution unit.

[0104] Prediction units 102 and 103 may include an inter-frame prediction unit 103 for performing inter-frame prediction and an intra-frame prediction unit 102 for performing intra-frame prediction. After determining whether to perform inter-frame or intra-frame prediction on a prediction unit, specific information (e.g., intra-frame prediction mode, motion vector, reference image, etc.) can be determined based on the different prediction methods. In this case, the processing unit performing the prediction can be different from the processing unit that determines the prediction method and specific content. For example, the prediction method and prediction mode can be determined at the prediction unit level, while the prediction can be performed at the transformation unit level.

[0105] The residual value (residual block) between the generated prediction block and the original block can be input to the transform unit 105. Furthermore, information used during prediction, such as prediction mode information and motion vector information, can be encoded together with the residual value by the entropy coding unit 107 and then transmitted to the decoder. When using a specific coding mode, it is also possible to directly encode the original block and transmit it to the decoding unit without generating the prediction block through the prediction units 102 and 103.

[0106] The intra-prediction unit 102 can generate a prediction block based on pixel information within the current image, i.e., reference pixel information surrounding the current block. When the prediction mode of the blocks surrounding the current block performing intra-prediction is inter-prediction, reference pixels contained in the surrounding blocks to which inter-prediction has been applied can be replaced with reference pixels in other surrounding blocks to which intra-prediction has been applied. That is, if reference pixels are unavailable, they can be used after replacing the unavailable reference pixel information with at least one of the available reference pixels.

[0107] In intra-frame prediction, prediction modes can include directional prediction modes that use reference pixel information based on the prediction direction, and non-directional modes that do not use directional information when performing prediction. The mode used to predict luminance information can be different from the mode used to predict chrominance information. When predicting chrominance information, the intra-frame prediction mode information used in the process of predicting luminance information or the predicted luminance signal information can be used.

[0108] The intra-prediction unit 102 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is used to filter the reference pixels of the current block, and its application can adaptively determine whether to apply the filter based on the prediction mode of the current prediction unit. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0109] When the intra-prediction mode of the prediction unit is a prediction unit that performs intra-prediction based on pixel values ​​interpolated from reference pixels, the reference pixel interpolation unit of the intra-prediction unit 102 can generate reference pixels at fractional unit positions by interpolating the reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, interpolation of reference pixels is not required. When the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks by filtering.

[0110] It can generate residual blocks that contain residual information, namely the difference between the prediction unit generated in prediction units 102 and 103 and the original block of the prediction unit. The generated residual blocks can be input into transformation unit 130 for transformation.

[0111] Figure 2 This is a schematic diagram used to illustrate an example of intra-frame prediction mode. Figure 2 The intra-prediction modes shown include a total of 35 modes. Mode 0 refers to Planar mode, mode 1 refers to DC mode, and modes 2 through 34 refer to Angular mode.

[0112] Figure 3 This is a schematic diagram used to illustrate intra-frame prediction in planar mode.

[0113] Figure 3 This is a schematic diagram used to illustrate the planar mode. To generate the predicted value of the first pixel P1 in the current block, linear interpolation is performed between the reconstructed pixel at the same position on the Y-axis and the reconstructed pixel T located at the upper right of the current block, as shown in the diagram. Similarly, to generate the predicted value of the second pixel P2, linear interpolation is performed between the reconstructed pixel at the same position on the X-axis and the reconstructed pixel L located at the lower left of the current block, as shown in the diagram. The average value of the two predicted pixels P1 and P2 becomes the final predicted pixel. In planar mode, the predicted block of the current block is generated by deriving the predicted pixels as described above.

[0114] Figure 4 This is a schematic diagram used to illustrate the DC mode. After calculating the average value of the reconstructed pixels surrounding the current block, this average value is used as the predicted value for all pixels within the current block.

[0115] Figure 5 It is used for utilization Figure 2This diagram illustrates an example of generating prediction blocks using Mode 10 (horizontal mode) and Mode 26 (vertical mode). When using Mode 10, the prediction block for the current block is generated by copying each reference pixel adjacent to the left side of the current block to the right. Similarly, when using Mode 26, the prediction block for the current block is generated by copying each reference pixel adjacent to the top side of the current block to the bottom.

[0116] Revisit Figure 1 The inter-frame prediction unit 103 can predict prediction units based on information from at least one of the previous or next images of the current image, and in some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 103 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0117] The reference image interpolation unit can receive reference image information from memory 112 and generate pixel information in integer pixel units or smaller from the reference image. For luminance pixels, in order to generate pixel information in integer pixel units or smaller, a DCT-based 8-tap interpolation filter with varying filter coefficients can be used. For chrominance signals, in order to generate pixel information in integer pixel units or smaller, a DCT-based 4-tap interpolation filter with different filter coefficients can be used.

[0118] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). Motion vectors can be calculated using motion vector values ​​in 1 / 2 or 1 / 4 pixel units based on the interpolated pixels. The motion prediction unit can predict the current prediction unit using different motion prediction methods. Various motion prediction methods can be used, such as skipping, merging, and AMVP (Advanced Motion Vector Prediction).

[0119] The subtraction unit 104 generates a residual block of the current block by performing a subtraction operation between the current block to be encoded and the prediction block generated in the intra-frame prediction unit 102 or the inter-frame prediction unit 103.

[0120] In the transform unit 105, a transform block can be generated by transforming the residual block, which is the difference between the original block and the prediction block. The transform block can be the smallest unit for performing the transform and quantization processes. The transform unit 105 can generate a transform block containing transform coefficients by transforming the residual signal into a frequency region. To transform the residual block containing residual data into a frequency region, transform methods such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT (Karhunen-Loeve Transform) can be used. By transforming the residual signal into a frequency region using the aforementioned transform methods, transform coefficients can be generated. To facilitate the execution of the transform, matrix operations using basis vectors can be performed. Depending on the prediction mode when encoding the prediction block, multiple different transform methods can be mixed during matrix operations. For example, the transform method can be determined based on the intra-frame prediction mode of the prediction unit used when generating the residual block. For example, it is possible to use DCT in the horizontal direction and DST in the vertical direction based on the intra-frame prediction mode.

[0121] The quantization unit 106 is capable of quantizing the values ​​transformed into frequency regions in the transform unit 105. That is, the quantization unit 106 can quantize the transform coefficients of the transform block generated from the transform unit 105 to generate a quantized transform block with quantized transform coefficients. As a quantization method, methods such as Dead Zone Uniform Threshold Quantization (DZUTQ) or a quantization weighted matrix can be used. Alternatively, various quantization methods, such as modified quantization methods, can be employed. The quantization coefficients can vary depending on the importance of the block or image. The values ​​calculated in the quantization unit 106 can be provided to the inverse quantization unit 108 and the entropy coding unit 107.

[0122] The transformation unit 105 and / or quantization unit 106 can be selectively included in the image encoding apparatus 100. That is, the image encoding apparatus 100 can perform at least one of transformation or quantization on the residual data of the residual block, or it can simultaneously skip transformation and quantization while encoding the residual block. Even if the image encoding apparatus 100 does not perform either transformation or quantization, or neither transformation nor quantization is performed, the block input to the entropy encoding unit 107 is generally referred to as a transformed block.

[0123] The entropy encoding unit 107 performs entropy encoding on the input data. The entropy encoding unit 107 can encode the quantized transform blocks and output a bitstream. That is, the entropy encoding unit 107 can encode the quantized transform coefficients of the quantized transform blocks output from the quantization unit 106 using various encoding techniques such as entropy encoding. Furthermore, the entropy encoding unit 107 can also encode additional information (e.g., prediction mode information and quantization coefficients) required for decoding the corresponding blocks in the image decoding apparatus described later. When performing entropy encoding, various encoding methods can be used, such as Exponential Golomb codes, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0124] The entropy coding unit 107 can encode various types of information from the prediction units 102 and 103, such as residual coefficient information of the coding unit, block type information, prediction mode information, segmentation unit information, prediction unit information, transmission unit information, motion vector information, reference frame information, block interpolation information, and filtering information. In the entropy coding unit 107, the coefficients of the transform block can be encoded using various types of flags, such as those indicating non-zero coefficients, coefficients with absolute values ​​greater than 1 or 2, and the sign of the coefficients, based on partial block units within the transform block. For coefficients that cannot be encoded solely based on these flags, they can be encoded based on the absolute value of the difference between the coefficient encoded using the flags and the coefficients of the actual transform block. In the inverse quantization unit 108 and the inverse transform unit 109, the values ​​quantized in the quantization unit 106 are inverse quantized, and the values ​​transformed in the transform unit 105 are inversely transformed. A reconstructed block can be generated by merging the residual values ​​generated in the inverse quantization unit 108 and the inverse transform unit 109 with the prediction units predicted by the motion estimation unit, motion compensation unit, and intra-frame prediction unit 102 included in the prediction units 102 and 103. The adder 110 can generate a reconstructed block by performing an addition operation on the prediction block generated in the prediction units 102 and 103 and the residual block generated by the inverse transform unit 109.

[0125] The filtering unit 111 may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0126] Deblocking filters eliminate block distortion in reconstructed images caused by boundaries between blocks. To determine whether deblocking is necessary, the decision can be based on the pixels contained in several columns or rows of the block. When applying a deblocking filter to a block, a strong or weak filter can be used depending on the required deblocking intensity. Furthermore, during the application of a deblocking filter, horizontal and vertical filtering can be performed in parallel.

[0127] The offset correction unit can perform offset correction between the deblocked image and the original image on a pixel-by-pixel basis. To perform offset correction on a specific image, methods can be used such as dividing the pixels contained in the image into a certain number of regions, determining the regions where offset needs to be performed, and applying the offset to the corresponding regions, or applying the offset while considering the edge information of each pixel.

[0128] ALF (Adaptive Loop Filtering) can be performed based on a comparison between the filtered reconstructed image and the original image. It can also determine which filter to apply to the appropriate group after dividing the pixels in the image into specific groups, and perform different filtering on different groups. Information related to whether ALF is applied can be transmitted in coding units (CUs), and the shape and filtering coefficients of the ALF filters applied in each block can differ. Furthermore, it is possible to apply ALF filters of the same shape (fixed shape) regardless of the characteristics of the target block.

[0129] The memory 112 can store the reconstructed blocks or images calculated by the filtering unit 111, and the stored reconstructed blocks or images can be provided to the prediction units 102 and 103 when performing inter-frame prediction.

[0130] The intra-frame prediction unit 102 and the inter-frame prediction unit 103 can be collectively referred to as the prediction unit. The prediction unit can generate prediction blocks using the surrounding pixels of the current block or a previously reconstructed reference image. One or more prediction blocks can be generated within the current block. When there is only one prediction block within the current block, the prediction block can have the same shape as the current block. After generating the prediction blocks, residual blocks corresponding to the difference between the current block and the prediction blocks can be generated. By applying various techniques, such as rate-distortion optimization (RDO), to the generated residual blocks, the optimal prediction mode can be determined. For example, Formula 1 shown below can be used when calculating RDO.

[0131] [Formula 1]

[0132]

[0133] In Formula 1, D(), R(), and J() represent the distortion caused by quantization, the bit rate of the compressed data stream, and the RD cost, respectively. Φ represents the encoding mode. λ represents the Lagrange multiplier, used as a scaling factor to match the unit of error to bit quantity. To be selected as the optimal encoding mode during the encoding process, j(), i.e., the RD cost, should be less when using the phase mode than when using other modes. When calculating the RD cost, both bit rate and error can be considered simultaneously.

[0134] Figure 6 This is a block diagram illustrating an image decoding apparatus 600 to which one embodiment of the present invention is applied.

[0135] See Figure 6 The image decoding device 600 may include an entropy decoding unit 601, an inverse quantization unit 602, an inverse transform unit 603, an addition unit 604, a filtering unit 605, a memory 606, and prediction units 607 and 608.

[0136] When the image bitstream generated by the image encoding device 100 is input to the image decoding device 600, the input bitstream can be decoded in the reverse order of the process performed by the image encoding device 100.

[0137] The entropy decoding unit 601 can perform entropy decoding according to the reverse steps of entropy encoding performed in the entropy encoding unit 107 of the image encoding apparatus 100. For example, it can be compatible with various methods such as Exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding), corresponding to the methods performed in the image encoder. In the entropy decoding unit 601, the coefficients of the transform block can be decoded in partial block units within the transform block based on various types of flags, such as those indicating non-zero coefficients, coefficients with absolute values ​​greater than 1 or 2, and the sign of the coefficients. For coefficients that cannot be represented solely by these flags, decoding can be performed based on the sum of the coefficients represented by the flags and the signaling coefficients.

[0138] In the entropy decoding unit 601, information related to intra-frame prediction and inter-frame prediction performed in the encoder can be decoded.

[0139] The inverse quantization unit 602 generates a transform block by performing inverse quantization on the quantized transform block. According to... Figure 1 The inverse quantization unit 108 in the middle works in essentially the same way.

[0140] The inverse quantization unit 603 generates a residual block by performing an inverse transform on the transform block. At this time, the transform method can be determined based on information related to the prediction method (inter-frame or intra-frame prediction), the block size and / or shape, and the intra-frame prediction mode. Figure 1 The inverse transformation unit 109 in the middle works in the same way.

[0141] The addition unit 604 generates a reconstruction block by performing an addition operation on the prediction block generated in the intra-frame prediction unit 607 or the inter-frame prediction unit 608 and the residual block generated by the inverse transform unit 603. Figure 1The addition unit 110 in the middle works in essentially the same way.

[0142] The filter unit 605 is used to reduce various types of noise that appear in the reconstructed blocks.

[0143] The filtering unit 605 may include a deblocking filter, an offset correction unit, and an ALF.

[0144] The image encoding device 100 can receive information on whether a deblocking filter is applied to a corresponding block or image, and if so, whether strong or weak filtering is applied. The image decoding device 600's deblocking filter can perform deblocking filtering on the corresponding block within the image decoding device 600 after receiving the deblocking filter information provided by the image encoding device 100.

[0145] The offset correction unit can perform offset correction on the reconstructed image based on the offset correction type and offset value information applicable to the image during encoding.

[0146] ALF can be applied to the encoding unit based on ALF applicability information and ALF coefficient information provided from the image encoding device 100. The ALF information described above can be included in a specific parameter set. The filtering unit 605 follows the same procedure as... Figure 1 The filtering section 111 in the middle works in essentially the same way.

[0147] The memory 606 is used to store the reconstructed blocks generated by the addition unit 604. According to... Figure 1 The memory 112 in the middle works in essentially the same way.

[0148] Prediction units 607 and 608 can generate prediction blocks based on prediction blocks provided by entropy decoding unit 601 and previously decoded blocks or image information provided by memory 606.

[0149] Prediction units 607 and 608 may include an intra-frame prediction unit 607 and an inter-frame prediction unit 608. Although not shown separately, prediction units 607 and 608 may also include a prediction unit determination unit. The prediction unit determination unit can receive various types of information input from the entropy decoding unit 601, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method, and distinguish prediction units from the current decoding unit, thereby determining whether the prediction unit performs inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 608 can perform inter-frame prediction on the current prediction unit based on information contained in at least one of the previous or next images of the current image containing the current prediction unit, using the information required for inter-frame prediction of the current prediction unit provided by the image coding apparatus 100. Alternatively, it can perform inter-frame prediction based on information of a reconstructed portion of the current image containing the current prediction unit.

[0150] To perform inter-frame prediction, the motion prediction method for the prediction unit contained in the corresponding coding unit can be determined based on the coding unit as a reference, which method is Skip Mode, Merge Mode, or Advanced Motion Vector Prediction Mode (AMVP Mode).

[0151] The intra-frame prediction unit 607 generates prediction blocks using pixels located around the block to be encoded in the current period and pixels that have already been reconstructed.

[0152] The intra-frame prediction unit 607 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is used to filter the reference pixels of the current block, and its application can adaptively determine whether to apply the filter based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided from the image coding apparatus 100 and the AIS filter information. When the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0153] When the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, the reference pixel interpolation unit of the intra-frame prediction unit 607 can generate reference pixels at fractional unit positions by interpolating reference pixels. The generated reference pixels at fractional unit positions can be used as predicted pixels for pixels within the current block. When the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, interpolation of reference pixels is not required. When the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks by filtering.

[0154] Intra-frame prediction unit 607 according to and Figure 1 The intra-prediction unit 102 in the middle works in essentially the same way.

[0155] The inter-frame prediction unit 608 generates inter-frame prediction blocks using reference images stored in memory 606 and motion information. The inter-frame prediction unit 608 then generates inter-frame prediction blocks according to... Figure 1 The inter-frame prediction unit 103 in the middle works in essentially the same way.

[0156] This invention relates in particular to filling and image boundary processing, and various different embodiments to which this invention is applied will be described in detail below with reference to the accompanying drawings.

[0157] In one embodiment of the present invention, it is possible to input an encoded object image into... Figure 1 A preprocessing process is performed before the image segmentation unit 101. The image to be encoded can have various horizontal and vertical sizes in pixel units. However, since image encoding and decoding are performed in arbitrary block units rather than pixel units, a padding process is required to match the size of the image to the block units.

[0158] Figure 7 This is a schematic diagram illustrating the filling process to which one embodiment of the present invention applies.

[0159] exist Figure 7 In the example shown, the encoded object image includes block regions and non-block regions. By performing the preprocessing procedure on the encoded object image, the filled regions can be added. After adding the filled regions, the encoded object image can be input into... Figure 1 In the image segmentation unit 101.

[0160] exist Figure 7 In the example shown, the smallest block unit can be 4×4. The horizontal or vertical length of the block can be 2n. Therefore, for example, when the horizontal and vertical lengths of the encoded object image are both 9, the rightmost column and the bottom row of the encoded object image can become non-block regions not included in the block region. As described above, when the encoded object image contains non-block regions, padding can be performed taking into account the size of the smallest block. By performing padding taking into account the block size as described above, the padded encoded object image will no longer contain non-block regions. By inputting the padded encoded object image into... Figure 1 The image segmentation unit 101 is divided into multiple blocks, and can be encoded in block units.

[0161] Figure 8a And 8b is a schematic diagram illustrating the filling process to which the present invention is applicable.

[0162] First, it can perform filling along the horizontal direction using the pixel closest to the pixel to be filled. Figure 8a In the example shown, the top three fill object pixels can be filled using pixel A of the non-block region adjacent to the left.

[0163] Next, the filling can be performed along the vertical direction using the pixel closest to the pixel to be filled. Figure 8b In the example shown, the leftmost three fill object pixels can be filled using pixels I from the non-block region adjacent to the top.

[0164] In the example described, the filling process from the horizontal to the vertical direction was illustrated; however, the invention is not limited to this, and filling from the vertical to the horizontal direction is also possible. The size of the minimum block can be a combination of... Figure 9 The block unit used for explanation can also be a sub-block unit.

[0165] Figure 9 It is used for Figure 1 A schematic diagram illustrating the operation of the image segmentation unit 101.

[0166] The input image can be divided into multiple maximum blocks. The size of the maximum block can be preset or signaled via a bitstream. For example, when the size of the input image is 128×129 and the size of the maximum block is 64×64, the input image can be divided into 4 maximum blocks.

[0167] Each largest block can be input to Figure 1 The image segmentation unit 101 divides the image into multiple blocks. Input images can also be directly input into the image segmentation unit 101 without being segmented into multiple maximum blocks. The minimum size of the segmented block and / or the maximum size of a block that can be segmented into sub-blocks can be preset or signaled via the block's parent header. The parent header can be at least one of video, sequence, picture, slice, tile, or largest coding unit (LCU).

[0168] The current block can be divided into 4 sub-blocks or 2 sub-blocks.

[0169] The process of dividing the current block into four sub-blocks is called quad-tree partitioning. The current block can be partitioned into four sub-blocks of the same size using quad-tree partitioning. The first partition information indicates whether the current block has been partitioned using quad-tree partitioning. The first partition information can be a 1-bit flag.

[0170] The process of dividing the current block into two sub-blocks is called binary-tree partitioning. The current block can be partitioned into two sub-blocks of the same or different sizes using quadtree partitioning. Secondary partitioning information indicates whether the current block has undergone binary-tree partitioning. This secondary partitioning information can be a 1-bit flag. When performing binary-tree partitioning on the current block, further signaling can be provided regarding the partitioning direction. This partitioning direction information indicates whether the partition is horizontal or vertical. This partitioning direction information can also be a 1-bit flag. When the current block is partitioned into two sub-blocks of different sizes, further signaling can be provided regarding the partitioning pattern. This partitioning pattern information indicates the partitioning ratio of the two sub-blocks.

[0171] The current block can be partitioned recursively using quadtree partitioning and binary tree partitioning. For example, after partitioning the current block into 4 sub-blocks using quadtree partitioning, each sub-block can be partitioned again recursively using quadtree or binary tree partitioning, or it can choose not to partition further. After partitioning the current block into 2 sub-blocks using binary tree partitioning, each sub-block can be partitioned again recursively using quadtree or binary tree partitioning, or it can choose not to partition further. Alternatively, it can choose not to perform quadtree partitioning on the sub-blocks generated by binary tree partitioning.

[0172] The encoder can determine whether to perform quadtree segmentation on the input current block. In step S901, the first segmentation information can be encoded based on the determination. When performing quadtree segmentation on the current block, that is, when the determination result in step S902 is yes, in step S903, the current block can be divided into 4 blocks by quadtree segmentation. In steps S904 and S916, the 4 blocks generated by quadtree segmentation can be re-inputted into step S901 in a specific order. The specific order can be a Z-scan order. In step S904, the first block among the 4 blocks can be specifically designated according to the specific order and input as the current block into step S901.

[0173] When a quadtree partition is not performed on the current block, i.e., when the determination result in step S902 is negative, the encoder can decide whether to perform a binary tree partition on the current block. In step S905, the second partitioning information can be encoded based on the determination. When a binary tree partition is performed on the current block, i.e., when the determination result in step S906 is positive, in step S907, the encoder can decide whether to perform a horizontal or vertical partition on the current block and encode the partitioning direction information accordingly. When a horizontal partition is performed on the current block, i.e., when the determination result in step S908 is positive, the current block can be partitioned horizontally in step S909. Otherwise, i.e., when the determination result in step S908 is negative, the current block can be partitioned vertically in step S910. When the binary tree partition of the current block is asymmetric, the partitioning ratio can be determined and the partitioning shape information can be further encoded. In this case, steps S909 and S910 can be executed while taking into account the segmentation morphology information. In steps S911 and S914, the two sub-blocks generated by the binary tree segmentation can be re-inputted into step S905 in a specific order. The specific order can be from left to right or from top to bottom. In step S911, the first sub-block of the two sub-blocks can be specifically designated according to the specific order and input as the current block into step S905. When quadtree segmentation can be performed again on the sub-blocks generated by the binary tree segmentation, the two sub-blocks generated by the binary tree segmentation can be re-inputted into step S901 in a specific order.

[0174] When a binary tree partition is not performed on the current block, i.e., when the determination result in step S906 is negative, the current block or the current sub-block can be encoded in step S912. The encoding in step S912 can include prediction, transformation, and quantization.

[0175] When the sub-block encoded in step S912 is not the last sub-block generated by binary tree partitioning, that is, when the determination result in S913 is negative, in step S914, the next sub-block generated by binary tree partitioning can be specifically specified and input as the current block into step S905.

[0176] When the sub-block encoded in step S912 is the last sub-block generated by binary tree segmentation, that is, when the determination result in S913 is yes, in step S915, it can be determined whether the block to which the encoded sub-block belongs is the last block among the blocks generated by quadtree segmentation. If it is the last block, that is, when the determination result in step S915 is yes, the encoding of the largest block or current image input in step S901 can be ended. If it is not the last block, that is, when the determination result in step S915 is no, in step S916, the next block generated by quadtree segmentation can be specifically specified and input as the current block into step S901.

[0177] Figure 10 This is a schematic diagram illustrating the operation of the decoder to which the present invention is applied.

[0178] In step S1001, the decoder decodes the first segmentation information of the input current block. When performing quadtree segmentation on the current block, i.e., when the determination result in step S1002 is yes, in step S1003, the current block can be divided into four blocks using quadtree segmentation. In steps S1004 and S1016, the four blocks generated by quadtree segmentation can be re-inputted into step S1001 in a specific order. This specific order can be a Z-scan order. In step S1004, the first block among the four blocks can be specifically designated according to the specific order and input as the current block into step S1001.

[0179] When a quadtree partition is not performed on the current block, i.e., when the determination result in step S1002 is negative, the decoder can decode the second partition information of the current block in step S1005. When a binary tree partition is performed on the current block, i.e., when the determination result in step S1006 is positive, the decoder can decode the partition direction information in step S1007. When a horizontal partition is performed on the current block, i.e., when the determination result in step S1008 is positive, the current block can be partitioned horizontally in step S1009. Otherwise, i.e., when the determination result in step S1008 is negative, the current block can be partitioned vertically in step S1010. When the binary tree partition of the current block is asymmetric, the partition shape information can be further decoded. In this case, steps S1009 and S1010 can be executed while considering the partition shape information. In steps S1011 and S1014, the two sub-blocks generated by the binary tree partitioning can be re-inputted into step S1005 in a specific order. This specific order can be from left to right or from top to bottom. In step S1011, the first sub-block of the two sub-blocks can be specifically designated according to the specific order and input as the current block into step S1005. When a quadtree partitioning can be performed again on the sub-blocks generated by the binary tree partitioning, the two sub-blocks generated by the binary tree partitioning can be re-inputted into step S1001 in a specific order.

[0180] When a binary tree partition is not performed on the current block, i.e., when the determination result in step S1006 is negative, the current block or the current sub-block can be encoded in step S1012. Decoding in step S1012 can include operations such as prediction, inverse quantization, and inverse transform.

[0181] When the sub-block being decoded in step S1012 is not the last sub-block generated by binary tree partitioning, that is, when the determination result in S1013 is negative, in step S1014, the next sub-block generated by binary tree partitioning can be specifically specified and input as the current block into step S1005.

[0182] When the sub-block being decoded in step S1012 is the last sub-block generated by binary tree segmentation, that is, when the determination result in S1013 is yes, in step S1015, it can be determined whether the block to which the decoded sub-block belongs is the last block among the blocks generated by quadtree segmentation. If it is the last block, that is, when the determination result in step S1015 is yes, the decoding of the largest block or current image input in step S1001 can be ended. If it is not the last block, that is, when the determination result in step S1015 is no, in step S1016, the next block generated by quadtree segmentation can be specifically specified and input as the current block into step S1001.

[0183] Figure 11 This is an illustrative diagram illustrating the case where the size of the current image is not equal to a multiple of the maximum block size.

[0184] like Figure 11 As shown, when the current image is divided into multiple maximum blocks, a region equivalent to a portion of the maximum block size will remain on the right or bottom of the current image. That is, in maximum block 2, maximum block 5, maximum block 6, maximum block 7, or maximum block 8, only a portion of the region contains pixels from the current image.

[0185] Next, we will discuss the effective segmentation methods such as... Figure 11 The image segmentation method of the present invention applicable to the current image shown will be described.

[0186] Figures 12a to 12e This is an example diagram used to illustrate an image segmentation method to which one embodiment of the present invention is applicable. Next, it will be discussed in conjunction with... Figures 12a to 12e right Figure 11 The method for segmenting the largest block 2 in the data and / or the encoding of the segmentation information are explained. This is combined with... Figures 12a to 12e In the following description, it is assumed that the current image size is 146×146, the maximum block size is 64×64, the minimum block size for quadtree partitioning is 16×16, and the minimum sub-block size is 2×2.

[0187] Figure 12a This is the result of magnifying the largest block 2, where the area enclosed by the thick line represents the region within the current image. The largest block 2 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12aAs shown, the rightmost and bottommost positions in the largest block 2 are not completely contained within the current image area. Furthermore, the size of the largest block 2 is 64×64, which is a size suitable for quadtree segmentation. Therefore, in step S903, quadtree segmentation can be performed on the largest block 2. That is, step S901 can be omitted for the largest block 2. Next, segmentation and encoding can be performed on the first block, block A0, of the four blocks (A0, A1, A2, and A3) generated by quadtree segmentation of the largest block 2. Because blocks A1 and A3 are not completely contained within the current image area, segmentation and encoding can be omitted on them. Block A2 is the same as block A0 and can be segmented and encoded according to the present invention.

[0188] Figure 12b Yes Figure 12a The result of magnifying block A0 in the image, where the area enclosed by the thick line represents the region within the current image. Block A0 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12b As shown, the rightmost position B1 and the bottommost position B3 in block A0 are not completely contained within the current image area. Furthermore, block A0 is 32×32 in size, which is a size suitable for quadtree segmentation. Therefore, in step S903, quadtree segmentation can be performed on block A0. That is, step S901 can be omitted for block A0. Next, the first block, block B0, can be segmented and encoded from the four blocks (B0, B1, B2, and B3) generated by quadtree segmentation of block A0. Block B0 can be combined with... Figure 9 The method described herein performs segmentation and encoding. That is, block B0 can be segmented using quadtree segmentation and / or binary tree segmentation, or encoded without segmentation. Block B2 is the same as block B0 and can be segmented and encoded according to the present invention. Because blocks B1 and B3 are not completely contained within the current image region, they can be left unsegmented and unencoded. Block B2 can be segmented and encoded using the same method as block B0. That is, step S901 cannot be omitted for blocks B0 and B2. However, step S901 can be omitted for blocks B1 and B3 as described below.

[0189] Figure 12c Yes Figure 12b The result of magnifying block B1 in the image, where the area enclosed by the thick line represents the region within the current image. Block B1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12cAs shown, the rightmost and bottommost positions in block B1 are not completely contained within the current image area. Furthermore, block B1 is 16×16 in size, which is too small for quadtree segmentation. Therefore, quadtree segmentation cannot be performed on block B1, but binary tree segmentation is possible. Additionally, since the encoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S901 to S907 can be omitted for block B1, and binary tree segmentation in step S909 or S910 can be performed based on the shape of the encoding target block. Next, segmentation and encoding can be performed on the first block, block C1, of the two blocks (C1 and C2) generated by performing binary tree segmentation on block B1. Since block C2 is not completely contained within the current image area, segmentation and encoding can be omitted from it.

[0190] Figure 12d Yes Figure 12c The result of magnifying block C1 in the image, where the area enclosed by the thick line represents the region within the current image. Block C1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12d As shown, the rightmost and bottommost positions in block C1 are not completely contained within the current image area. Furthermore, since binary tree segmentation was applied to the parent block B1 of block C1, binary tree segmentation can also be performed on block C1. Moreover, since the encoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S905 to S907 can be omitted for block C1, and the binary tree segmentation in step S909 or S910 can be performed according to the shape of the encoding target block. Next, segmentation and encoding can be performed on the first block, block D1, of the two blocks (D1 and D2) generated by performing binary tree segmentation on block C1. Since block D2 is not completely contained within the current image area, segmentation and encoding can be omitted from it.

[0191] Figure 12e Yes Figure 12d The result of magnifying block D1 in the image, where the area enclosed by the thick line represents the region within the current image. Block D1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12eAs shown, the rightmost and bottommost positions in block D1 are not completely contained within the current image area. Furthermore, since binary tree segmentation was applied to the parent block C1 of block D1, binary tree segmentation can also be performed on block D1. Moreover, since the encoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S905 to S907 can be omitted for block D1, and the binary tree segmentation in step S909 or S910 can be performed according to the shape of the encoding target block. Next, segmentation and encoding can be performed on the first block, block E1, of the two blocks (E1 and E2) generated by performing binary tree segmentation on block D1. The horizontal size of block E1 is equal to the assumed minimum size of the sub-block, which is 2. Therefore, block E1 cannot be segmented vertically. That is, block E1 can be encoded without segmentation or after horizontal segmentation. In this case, signaling can be performed only for information related to whether or not the block is segmented for block E1. Because block E2 is not completely contained within the current image region, it is possible to avoid performing segmentation and encoding on it.

[0192] Figure 11 The segmentation information omission methods for the largest block 2 and the largest block 5 shown can be the same. Furthermore, the segmentation information omission methods for the largest blocks 6 and 7 are identical to those for the largest blocks 2 and 5, except for differences in horizontal or vertical direction. For the largest block 8, the segmentation information can be omitted in the same way as for other largest blocks using pre-defined horizontal or vertical standards.

[0193] Next, we will combine Figures 12a to 12e right Figure 11 The method for segmenting the largest block 2 in the data and / or the decoding of the segmentation information are explained.

[0194] Figure 12a This is the result of magnifying the largest block 2, where the area enclosed by the thick line represents the region within the current image. The largest block 2 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12aAs shown, the rightmost and bottommost positions in the largest block 2 are not completely contained within the current image area. Furthermore, the size of the largest block 2 is 64×64, which is a size suitable for quadtree segmentation. Therefore, in step S1003, quadtree segmentation can be performed on the largest block 2. That is, step S1001 can be omitted for the largest block 2. Next, segmentation and decoding can be performed on the first block, block A0, of the four blocks (A0, A1, A2, and A3) generated by quadtree segmentation of the largest block 2. Because blocks A1 and A3 are not completely contained within the current image area, segmentation and decoding can be omitted on them. Block A2 is the same as block A0 and can be segmented and decoded according to the present invention.

[0195] Figure 12b Yes Figure 12a The result of magnifying block A0 in the image, where the area enclosed by the thick line represents the region within the current image. Block A0 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12b As shown, the rightmost and bottommost positions in block A0 are not completely contained within the current image area. Furthermore, block A0 is 32×32 in size, which is a size suitable for quadtree segmentation. Therefore, in step S1003, quadtree segmentation can be performed on block A0. That is, step S1001 can be omitted for block A0. Next, segmentation and decoding can be performed on the first block, block B0, from the four blocks (B0, B1, B2, and B3) generated by quadtree segmentation of block A0. Block B0 can be segmented and decoded according to... Figure 9 The method described herein performs segmentation and decoding. That is, block B0 can be segmented using quadtree segmentation and / or binary tree segmentation, or decoded without segmentation. Block B2 is the same as block B0, and can be segmented and decoded according to the present invention. Because blocks B1 and B3 are not completely contained within the current image region, segmentation and decoding can be omitted for them. Block B2 can be segmented and decoded using the same method as block B0. That is, step S1001 cannot be omitted for blocks B0 and B2. However, step S1001 can be omitted for blocks B1 and B3 as described below.

[0196] Figure 12c Yes Figure 12b The result of magnifying block B1 in the image, where the area enclosed by the thick line represents the region within the current image. Block B1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12cAs shown, the rightmost and bottommost positions in block B1 are not completely contained within the current image area. Furthermore, block B1 is 16×16 in size, which is not a suitable size for quadtree segmentation. Therefore, quadtree segmentation cannot be performed on block B1, but binary tree segmentation can be performed. Additionally, since the decoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S1001 to S1007 can be omitted for block D1, and binary tree segmentation in step S1009 or S1010 can be performed based on the shape of the encoding target block. Next, segmentation and decoding can be performed on the first block, block C1, of the two blocks (C1 and C2) generated by performing binary tree segmentation on block B1. Since block C2 is not completely contained within the current image area, segmentation and decoding can be omitted from it.

[0197] Figure 12d Yes Figure 12c The result of magnifying block C1 in the image, where the area enclosed by the thick line represents the region within the current image. Block C1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12d As shown, the rightmost and bottommost positions in block C1 are not completely contained within the current image area. Furthermore, since binary tree segmentation was applied to the parent block B1 of block C1, binary tree segmentation can also be performed on block C1. Additionally, since the decoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S1005 to S1007 can be omitted for block C1, and the binary tree segmentation in step S1009 or S1010 can be performed according to the shape of the encoding target block. Next, segmentation and decoding encoding can be performed on the first block, block D1, of the two blocks (D1 and D2) generated by performing binary tree segmentation on block C1. Since block D2 is not completely contained within the current image area, segmentation and decoding can be omitted from it.

[0198] Figure 12e Yes Figure 12d The result of magnifying block D1 in the image, where the area enclosed by the thick line represents the region within the current image. Block D1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 12eAs shown, the rightmost and bottommost positions in block D1 are not completely contained within the current image area. Furthermore, since binary tree segmentation was applied to the parent block C1 of block D1, binary tree segmentation can also be performed on block D1. Moreover, since the decoding target block is a long rectangle, the segmentation direction can be determined to be horizontal. Therefore, steps S1005 to S1007 can be omitted for block D1, and the binary tree segmentation in step S1009 or S1010 can be performed according to the shape of the encoding target block. Next, segmentation and decoding can be performed on the first block, block E1, from the two blocks (E1 and E2) generated by performing binary tree segmentation on block D1. The horizontal size of block E1 is equal to the assumed minimum size of the sub-block, which is 2. Therefore, block E1 cannot be segmented vertically. That is, block E1 can be decoded without segmentation or after horizontal segmentation. At this point, signaling can be applied to block E1 only regarding whether or not it is segmented. Since block E2 is not fully contained within the current image region, segmentation and decoding can be omitted.

[0199] In the embodiment described in conjunction with Figure 12, binary tree segmentation can be performed after quadtree segmentation of the segmented object block, provided that a specific condition is met. In this embodiment, the specific condition is whether the size of the segmented object block is the size at which quadtree segmentation is possible. This specific condition can also be set using the minimum size and / or a threshold value of the block at which quadtree segmentation and binary tree segmentation are possible.

[0200] When using a threshold, one of quadtree segmentation or binary tree segmentation can be performed based on the comparison between the size of the region in the segmented object block that is not completely contained in the current image (hereinafter referred to as the "remaining region") and the threshold value. In the following description, the threshold value is assumed to be 32.

[0201] For example, in Figure 12a The size of the encoded object block is 18×64, while the size of the remaining region is 46×64. Because the remaining region is a long rectangle, its horizontal length can be compared to a threshold. Since the horizontal length of the remaining region, 46, is greater than the threshold of 32, quadtree partitioning can be performed on the 64×64 segmented object block. Figure 12bThe horizontal length 14 of the remaining region is less than the threshold value 32, therefore binary tree partitioning can be performed on the 32×32 segmented object block. That is, even if the size of the segmented object block is the size at which quadtree partitioning can be performed, binary tree partitioning can be performed without quadtree partitioning according to the aforementioned condition. In this case, the partitioning direction of the binary tree is based on the smaller of the horizontal and vertical dimensions of the 18×32 encoded object block. Therefore, Figure 12b Block A0 can be divided into two vertically longer 16×32 blocks. Then, vertical binary tree partitioning can be repeatedly performed until the block size is too large for further binary tree partitioning.

[0202] The threshold value can be signaled via the upper-level header of the block as described above. The threshold value should be a value between the maximum and minimum size of a block capable of quadtree partitioning. Furthermore, the threshold value can be set to be less than the maximum size of a block capable of binary tree partitioning. When the threshold value is set to be greater than the maximum size of a block capable of binary tree partitioning, the threshold value can be reset to the maximum size of a block capable of binary tree partitioning.

[0203] Figures 13a to 13e This is an example diagram used to illustrate an image segmentation method to which another embodiment of the present invention is applicable.

[0204] Figure 13a This is the result of magnifying the largest block 2, where the area enclosed by the thick line represents the region within the current image. The largest block 2 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13a As shown, the rightmost and bottommost positions in the largest block 2 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13a The remaining region is a 46×64 block with a relatively long vertical dimension, so a vertical binary tree partition can be performed on the largest block 2. That is, steps S901 to S907 can be omitted for the largest block 2, and steps S909 or S910 can be performed according to the shape of the remaining region.

[0205] Figure 13b Yes Figure 13a The result of magnifying block A0 in the image, where the area enclosed by the thick line represents the region within the current image. Block A0 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13b As shown, the rightmost and bottommost positions in block A0 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13bThe remaining area is a 14×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block A0. Since block B0 is a block within the current image area, it can be encoded using self-block segmentation. Steps S905 to S908 cannot be omitted for block B0.

[0206] Figure 13c Yes Figure 13b The result of magnifying block B1 in the image, where the area enclosed by the thick line represents the region within the current image. Block B1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13c As shown, the rightmost and bottommost positions in block B1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13c The remaining region is a 14×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block B1. That is, steps S901 to S907 can be omitted for block B1, and steps S909 or S910 can be performed according to the shape of the remaining region. Since block C2 is not completely contained within the current image region, it can be left unsegmented and unencoded.

[0207] Figure 13d Yes Figure 13c The result of magnifying block C1 in the image, where the area enclosed by the thick line represents the region within the current image. Block C1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13d As shown, the rightmost and bottommost positions in block C1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13c The remaining region is a 6×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block C1. That is, steps S901 to S907 can be omitted for block C1, and steps S909 or S910 can be performed according to the shape of the remaining region. Since block D2 is not completely contained within the current image region, it can be left unsegmented and unencoded.

[0208] Figure 13e Yes Figure 13d The result of magnifying block D1 in the image, where the area enclosed by the thick line represents the region within the current image. Block D1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13e As shown, the rightmost and bottommost positions in block D1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13c The remaining area is a 2×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block D1. That is, steps S901 to S907 can be omitted for block B1, and steps S909 or S910 can be performed according to the shape of the remaining area. The horizontal size of block E1 is equal to the assumed minimum size of the sub-block, which is 2. Therefore, block E1 cannot be vertically segmented. That is, block E1 can be encoded without segmentation or after horizontal segmentation. In this case, signaling can be performed on block E1 only for information related to whether or not it is segmented. Because block E2 is not completely contained within the current image area, it can be left unsegmented and unencoded.

[0209] Next, we will combine Figures 13a to 13e right Figure 11 The method for segmenting the largest block 2 in the data and / or the decoding of the segmentation information are explained.

[0210] Figure 13a This is the result of magnifying the largest block 2, where the area enclosed by the thick line represents the region within the current image. The largest block 2 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13a As shown, the rightmost and bottommost positions in the largest block 2 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13a The remaining region is a 46×64 block with a relatively long vertical dimension, so a vertical binary tree partition can be performed on the largest block 2. That is, steps S1001 to S1007 can be omitted for the largest block 2, and steps S1009 or S1010 can be performed according to the shape of the remaining region.

[0211] Figure 13b Yes Figure 13a The result of magnifying block A0 in the image, where the area enclosed by the thick line represents the region within the current image. Block A0 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13b As shown, the rightmost and bottommost positions in block A0 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13b The remaining area is a 14×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block A0. Since block B0 is a block within the current image area, it can be decoded using self-block segmentation. Steps S1005 to S1008 cannot be omitted for block B0.

[0212] Figure 13c Yes Figure 13b The result of magnifying block B1 in the image, where the area enclosed by the thick line represents the region within the current image. Block B1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13c As shown, the rightmost and bottommost positions in block B1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13c The remaining region is a 14×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block B1. That is, steps S1001 to S1007 can be omitted for block B1, and steps S1009 or S1010 can be performed according to the shape of the remaining region. Since block C2 is not completely contained within the current image region, segmentation and decoding can be omitted for it.

[0213] Figure 13d Yes Figure 13c The result of magnifying block C1 in the image, where the area enclosed by the thick line represents the region within the current image. Block C1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13d As shown, the rightmost and bottommost positions in block C1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13c The remaining region is a 6×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block C1. That is, steps S1001 to S1007 can be omitted for block C1, and steps S1009 or S1010 can be performed according to the shape of the remaining region. Since block D2 is not completely contained within the current image region, segmentation and decoding can be omitted for it.

[0214] Figure 13e Yes Figure 13d The result of magnifying block D1 in the image, where the area enclosed by the thick line represents the region within the current image. Block D1 can be combined with... Figure 9 The segmented object blocks are explained. For example... Figure 13e As shown, the rightmost and bottommost positions in block D1 are not completely contained within the current image area. In this case, the block segmentation method can be selected based on the shape of the remaining area. Because... Figure 13cThe remaining region is a 2×64 block with a relatively long vertical dimension, so a vertical binary tree segmentation can be performed on block D1. That is, steps S1001 to S1007 can be omitted for block D1, and steps S1009 or S1010 can be performed according to the shape of the remaining region. The horizontal size of block E1 is equal to the assumed minimum size of the sub-block, which is 2. Therefore, block E1 cannot be segmented vertically. That is, block E1 can be decoded without segmentation or after horizontal segmentation. In this case, signaling can be performed only for information related to whether or not it is segmented for block E1. Because block E2 is not completely contained within the current image region, segmentation and decoding can be omitted for it.

[0215] In the embodiment described in conjunction with FIG13, only binary tree segmentation is performed on the segmented object block, and the direction of the binary tree segmentation can be determined according to the shape of the remaining region.

[0216] In combination Figure 9 as well as Figure 10 In the illustrated embodiment, image boundary segmentation processing can be performed on a per-block or block-unit basis. In this case, signaling can be applied to information related to the unit used in performing image boundary segmentation processing. For example, this information can be signaled from the encoder to the decoder via the block's parent header as described above.

[0217] As mentioned above, when performing inter-frame prediction for the current block, modes such as skip mode, merge mode, and AMVP mode can be used.

[0218] When using the skip mode, the optimal prediction mode for the current block is determined using motion information from the reconstructed areas. Specifically, the image encoding device can construct motion information candidate groups within the reconstructed areas and use the candidate with the lowest RD cost in the corresponding candidate group as the prediction information to generate the prediction information for the current block. The method for constructing the motion information candidate groups will be discussed in detail later. Figure 14 Please provide an explanation.

[0219] When using the merge mode, the process of determining the optimal prediction mode for the current block using motion information from the reconstructed region is the same as in the skip mode. However, in the skip mode, motion information with a prediction error of 0 is searched from the motion information candidate set, while in the motion mode, a prediction error of non-zero is allowed. Similar to the skip mode, the image coding device can construct motion information candidate sets within the reconstructed region and use the candidate with the minimum RD cost in the corresponding candidate set as the prediction information to generate the prediction information for the current block.

[0220] Figure 14 This is a schematic diagram illustrating the method for generating motion information candidate groups in skip mode and merge mode.

[0221] A motion information candidate group can contain up to N candidates. N can be a predefined positive integer in the image encoding and decoding apparatus. Alternatively, information related to N can be signaled via a bitstream. For example, information related to N can be signaled via the parent header of a block. The parent of the block can be at least one of video, sequence, picture, slice, tile, or largest coding unit (LCU). Alternatively, N can be determined in a variable manner based on the size and / or shape of the current block. Figure 14 In the following embodiments, N is assumed to be equal to 5.

[0222] In step S1401, spatial candidates can be selected. A spatial candidate can refer to a spatial candidate block within the image to which the current block belongs, or the motion information of that spatial candidate block. Up to four spatial candidates can be selected from five spatial candidate blocks within the current image. When a spatial candidate block does not contain motion information, such as when the spatial candidate block is intra-prediction or PCM encoded, that spatial candidate block cannot be selected as the spatial candidate for the current block. Furthermore, spatial candidates can be selected in a manner where the motion information does not overlap.

[0223] Figure 16 It is a diagram used to illustrate the positions of spatial and temporal supplementary blocks for the current block.

[0224] exist Figure 16 In the example shown, the spatial candidate block can include at least one of blocks A1, A2, A3, A4, and A5. Except... Figure 16 In addition to the spatial candidate blocks shown, any block within the reconstructed region of the current image can also be used as a spatial candidate block. When selecting a spatial candidate from the spatial candidate blocks, blocks A1, A2, A3, A4, and A5 can be considered in that order. Based on the stated priority, available spatial candidate blocks or their motion information can be selected as spatial candidates. When the motion information of available spatial candidate blocks is not repeated, the spatial candidate block with the lowest priority can be disregarded.

[0225] In step S1402, a time candidate can be selected. A time candidate can refer to a time candidate block within another image (e.g., a collocated picture) outside the current block's image, or motion information of that time candidate block. One time candidate can be selected from two time candidate blocks within the collocated picture. When the time candidate block does not contain motion information, for example, when the time candidate block is intra-prediction or PCM encoded, that time candidate block cannot be selected as the time candidate for the current block.

[0226] like Figure 16 As shown, the position of the temporal candidate block can be determined based on the position of the current block. That is, the temporal candidate block can be determined based on a block in a juxtaposed image at the same position as the current block within the current image. The juxtaposed image can be either a reconstructed image or a reference image. The juxtaposed image can be selected using a method predefined in the image encoding and decoding apparatus. For example, a reference image at any position (e.g., the first position) within a list of reference images can be selected as the juxtaposed image. Alternatively, signaling can be applied to information related to the selection of the juxtaposed image.

[0227] A temporal candidate block can be a block within a juxtaposed image that contains samples at the same position as any position within the current block, or a block adjacent to it. The "any position" within the current block can be the top-left pixel position, the center pixel position, or the bottom-right pixel position of the current block. Figure 16 In the example shown, the temporal candidate blocks within the juxtaposed image can be blocks B1 and B2. When selecting a temporal candidate from these temporal candidate blocks, they can be considered, for example, in the order of blocks B1 and B2. When considering blocks B1 and B2 in this order, if a higher-priority candidate block can be used as a temporal candidate, then a lower-priority temporal candidate block can be disregarded.

[0228] Figure 17 This is a schematic diagram illustrating the method for determining motion information for time candidates.

[0229] The motion information of temporal candidate blocks within a juxtaposed image can refer to predicted blocks contained in reference image B. Temporal candidate blocks can reference different reference images. However, for ease of explanation below, the reference image of the temporal candidate block will be referred to as reference image B. The motion vectors of the temporal candidate blocks can be considered... Figure 17 The scaling is performed under the premise of TB and TD. TB represents the distance between the current image and the reference image A, and TD represents the distance between the juxtaposed image and the reference image B. The scaled temporal candidate motion vector can be used as the motion information of the current block's temporal candidate. The scaling can be performed using Equation 2 as shown below.

[0230] [Formula 2]

[0231]

[0232] In Formula 2, MV represents the motion vector of the temporal candidate block, and MVscale represents the scaled motion vector. Furthermore, reference image A and reference image B can be the same reference image. It is possible to determine whether the scaled motion vector MVscale, as described above, is used as the motion vector of the temporal candidate. Furthermore, it is possible to determine whether the reference image information of the temporal candidate is used as the reference image of the current image.

[0233] When the sum of the spatial and temporal candidates derived through steps S1401 and S1402 is less than N, step S1403 can be executed. In step S1403, the candidates derived in the previous steps can be combined, and new bidirectional motion information candidates can be added to the motion information candidate group. Bidirectional motion information candidates refer to new candidates generated by retrieving and combining historical or future direction motion information derived in previous steps one by one.

[0234] Figure 18 This is a diagram illustrating the priority of combinations that can be used when generating bidirectional motion information candidates. For example, it can be prioritized according to... Figure 18 The priority shown first considers the combination of candidate index (historical) 0 and candidate index (future) 1. Figure 18 The combinations of priorities and indices shown are for illustrative purposes only and are not intended to be limiting. For example, it is also possible to use, except... Figure 18 Additional combinations beyond those shown, or changes to the priority.

[0235] If the total number of motion candidate information after adding bidirectional motion information candidates in step S1403 is still less than N, step S1404 can be executed. In step S1404, new motion information candidates with zero motion vectors and different reference images in the prediction direction can be added to the motion information candidate group.

[0236] When using the AMVP mode, optimal motion information can be determined by performing motion inference on each reference image along the prediction direction. The prediction direction can be a single direction using only one of the historical / future directions, or it can be both historical and future directions. Motion compensation can be performed using the optimal motion information determined by motion inference to generate motion blocks. At this point, candidate sets of motion information for motion inference can be derived for each reference image along the prediction direction. These candidate sets of motion information can be used as the starting position for motion inference.

[0237] Although the predicted directions are described as historical and future directions in this specification, this is not a limitation. For example, they can also be described as forward and backward. Alternatively, they can be described as the first direction and the second direction. For example, when using a list of N reference images for the current block, there can be predicted directions corresponding to the reference image lists. In this case, the predicted directions can be described as the first direction to the Nth direction. N can be an integer greater than 2.

[0238] Figure 15 This is a schematic diagram illustrating the method for deriving candidate groups of motion information for motion prediction in the AMVP model.

[0239] and combination Figure 15 Similarly, the maximum number of candidates that can be included in the motion information candidate group can be determined by the same value in the image encoding device and the image decoding device, or by signaling through the upper-level header of the block, or by a variable method based on the size and / or shape of the current block.

[0240] In steps S1501 and S1502, spatial candidates and temporal candidates can be selected. Steps S1501 and S1502 can respectively follow the... Figure 14 Steps S1401 and S1402 are performed in a similar manner. However, the number of spatial candidates selected in steps S1501 and S1401 can be different. For example, two spatial candidates can be selected in step S1501. Furthermore, the priorities among the spatial candidate blocks used to select spatial candidates can also be different.

[0241] In step S1503, duplicate motion information in the currently derived candidates can be removed. Step S1504 can be performed in a similar manner to step S1404. After deriving motion information candidates, the motion information candidate with the lowest RD cost can be selected as the optimal motion information. By performing motion inference based on the selected motion information, the optimal motion information for the AMVP mode can be derived.

[0242] Intra-prediction modes used to perform intra-prediction as described above can include DC mode, planar mode, and N-direction orientation mode. The N-direction orientation mode generates a prediction block for the current block by dividing the reconstructed pixels surrounding the current block into N directions and mapping the surrounding pixels to the prediction pixels in each direction. After determining the optimal intra-prediction mode, an MPM candidate list can be generated for encoding it.

[0243] Figure 19 This is a diagram illustrating the generation of the MPM candidate list.

[0244] The number of MPM candidates can be determined to be P (where P is a positive integer). The method used to determine MPM candidates is not limited to combining... Figure 19 The method of explanation. In Figure 19 In this context, L represents the intra-prediction mode information of the surrounding blocks to the left of the current block, and A represents the intra-prediction mode information of the surrounding blocks above the current block. Figure 19 The conditions shown can generate an MPM candidate list containing 3 MPM candidates.

[0245] Figure 20 This is a schematic diagram illustrating the encoding method for optimal prediction information in the skip mode, merge mode, AMVP mode, and intra-frame prediction mode as described above.

[0246] In step S2001, skip mode action information can be encoded. Skip mode action information can be information indicating whether a skip mode applies to the current block. In step S2002, it can be determined whether a skip mode is selected based on the skip mode action information. If it is a skip mode, that is, when the determination result in step S2002 is yes, the process can end after proceeding to step S2007 to encode the merge candidate index.

[0247] When the mode is not skipped, i.e., when the determination result in step S2002 is negative, the prediction mode information can be encoded in step S2003. The prediction mode information can be information used to indicate whether inter-frame prediction or intra-frame prediction is performed on the current block. In step S2004, it can be determined whether the prediction mode for the current block is inter-frame prediction mode or intra-frame prediction mode based on the prediction mode information. When the prediction mode for the current block is inter-frame prediction mode, i.e., when the determination result in step S2004 is positive, the process can proceed to step S2005; and when the prediction mode for the current block is intra-frame prediction mode, i.e., when the determination result in step S2004 is negative, the process can proceed to step S2017.

[0248] In step S2005, merge mode action information can be encoded. Merge mode action information can be information indicating whether a merge mode is applicable to the current block. In step S2006, a determination can be made based on the merge mode action information to determine whether a merge mode is applicable to the current block. If a merge mode is applicable to the current block, that is, if the determination result in step S2006 is yes, then in step S2007, the process can end after encoding the merge index information.

[0249] If the determination result in step S2006 is that the merge mode is not applicable to the current block, the process can proceed to step S2008. In step S2008, the predicted direction can be encoded as historical, future, or bidirectional. If the determination result in step S2009 is that the predicted direction is not a future direction, step S2010 can be executed. If the predicted direction is a future direction, step S2013 can be executed.

[0250] Next, the reference image index information for the historical direction is encoded in step S2010, the MVD (Motion Vector Difference) information for the historical direction is encoded in step S2011, and the MVP (Motion Vector Predictor) information for the historical direction is encoded in step S2012. Here, MVD refers to the difference vector between the optimal motion vector of the predicted block and the optimal candidate motion vector determined in the AMVP candidate list. Furthermore, MVP information refers to the index information of the determined optimal AMVP candidate motion information.

[0251] In step S2013, it is determined whether the predicted direction is a historical direction. If it is a historical direction, the process can end. Figure 20 The steps in step S2013 are as follows. When the determination result in step S2013 is that the predicted direction is not the historical direction, step S2014 can be executed. Next, the reference image index information of the future direction can be encoded in step S2014, the MVD information of the future direction can be encoded in step S2015, and the MVP information of the future direction can be encoded in step S2016.

[0252] When applying intra-frame prediction to the current block, i.e., when the determination result in step S2004 is negative, the MPM flag information can be encoded in step S2017. The MPM flag information can be information used to indicate whether the optimal intra-frame prediction mode of the current block is included in the MPM candidate list. In step S2018, it can be determined whether the MPM flag information is true. When the MPM flag information is true, i.e., when the determination result in step S2018 is positive, step S2019 can be executed. In step S2019, the index information used to indicate the candidate in the MPM candidate list that is the same as the optimal intra-frame prediction mode can be encoded.

[0253] When the determination result in step S2018 is that the MPM flag information is false, step S2020 can be executed. In step S2020, information indicating which intra-prediction mode is the optimal intra-prediction mode among the remaining intra-prediction modes other than those included in the MPM candidate list can be encoded.

[0254] Figure 21 This is a schematic diagram used to illustrate the decoding method for optimal prediction information.

[0255] In step S2101, the skip mode action information can be decoded. In step S2102, it can be determined whether it is a skip mode based on the skip mode action information. If it is a skip mode, that is, when the determination result in step S2102 is yes, the process can end after moving to step S2107 to decode the merge candidate index.

[0256] When the mode is not skipped, i.e., when the determination result in step S2102 is negative, the prediction mode information can be decoded in step S2103. In step S2104, it is possible to determine whether the prediction mode for the current block is inter-frame prediction mode or intra-frame prediction mode based on the prediction mode information. When the prediction mode for the current block is inter-frame prediction mode, i.e., when the determination result in step S2104 is positive, the process can proceed to step S2105; and when the prediction mode for the current block is intra-frame prediction mode, i.e., when the determination result in step S2104 is negative, the process can proceed to step S2117.

[0257] In step S2105, the merge mode action information can be decoded. In step S2106, it can be determined whether the merge mode of the current block is applicable based on the merge mode action information. When the merge mode is applicable to the current block, that is, when the determination result in step S2106 is yes, the process can end after decoding the merge index information in step S2107.

[0258] If the determination result in step S2106 is that the merge mode is not applicable to the current block, the process can proceed to step S2108. In step S2108, the predicted direction can be decoded as historical, future, or bidirectional. If the determination result in step S2109 is that the predicted direction is not a future direction, step S2110 can be executed. If the predicted direction is a future direction, step S2113 can be executed.

[0259] Next, the reference image index information of the historical direction can be decoded in step S2110, the MVD information of the historical direction can be decoded in step S2115, and the MVP information of the historical direction can be decoded in step S2112.

[0260] In step S2113, it is determined whether the predicted direction is a historical direction. If it is a historical direction, the process can end. Figure 21The steps in step S2113 are as follows. When the determination result in step S2113 is that the predicted direction is not the historical direction, step S2114 can be executed. Next, the reference image index information of the future direction can be decoded in step S2114, the MVD information of the future direction can be decoded in step S2115, and the MVP information of the future direction can be decoded in step S2116.

[0261] When applying intra-frame prediction to the current block, i.e., when the determination result in step S2104 is negative, the MPM flag information can be decoded in step S2117. In step S2118, it can be determined whether the MPM flag information is true. When the MPM flag information is true, i.e., when the determination result in step S2118 is positive, step S2119 can be executed. In step S2119, the index information used to indicate the candidates in the MPM candidate list that are the same as the optimized intra-frame prediction mode can be decoded.

[0262] When the determination result in step S2118 is that the MPM flag information is false, step S2120 can be executed. In step S2120, the information used to indicate which intra-prediction mode is the optimal intra-prediction mode among the remaining intra-prediction modes other than those included in the MPM candidate list can be decoded.

[0263] Next, several different embodiments will be described for using the prediction information of the surrounding blocks of the current block when encoding / decoding the prediction information used to generate the prediction block.

[0264] When segmenting blocks using binary tree partitioning, it is possible to remove the prediction information of specific blocks surrounding the current block. That is, when constructing the MPM candidate list for intra-frame prediction, or deriving spatial candidates for skip, merge, or AMVP for inter-frame prediction, it is possible to remove some of the surrounding blocks from the candidates while considering the block segmentation structure. The intra-frame prediction MPM candidate list or the candidate list containing inter-frame prediction spatial candidates is then referred to as the "reference candidate list."

[0265] Figure 22 This is a schematic diagram illustrating an embodiment of constructing a reference candidate list using the surrounding blocks of the current block.

[0266] In step S2201, it can be determined whether the partitioning pattern of the parent block of the current block is a binary tree partition. That is, in step S2201, it can be determined whether the current block is a block generated by binary tree partitioning. When the partitioning pattern of the parent block is not a binary tree partition, it can be determined that there are no surrounding blocks that need to be excluded when constructing the reference candidate list and the process ends. Figure 22 The processing in the process. When the segmentation form of the upper-level block is a binary tree segmentation, that is, when the current block is a block generated by binary tree segmentation, step S2202 can be executed.

[0267] In step S2202, it is possible to determine whether the depth of the current block is the same as the depth of the previous block and whether the current block is the same as the parent block of the previous block. The previous block can refer to a block that completed encoding / decoding one step earlier than the current block in the encoding order. The parent block refers to a block that was segmented into the current block or the previous block. Conversely, a lower-level block can refer to a block generated by segmenting the current block. The process can end when the segmentation depth of the current block is different from the depth of the previous block and / or the current block is different from the parent block of the previous block. Figure 22 The processing in step S2202 is as follows: When the determination result in step S2202 is negative, the usable surrounding blocks can be directly added to the reference candidate list as candidates without excluding any surrounding blocks as in this embodiment during the construction of the reference candidate list.

[0268] When the determination result in step S2202 is that the current block and the previous block have the same segmentation depth and the parent block of the current block is the same as the parent block of the previous block, in step S2203, the previous block can be either not included as a candidate in the reference candidate list of the current block or excluded from the reference candidate list. When the parent block is segmented into the current block and the previous block through binary tree segmentation, the current block is more likely to be dissimilar to the previous block. In this case, the likelihood of using the prediction information of the previous block as the prediction information of the current block is also lower. Therefore, even for blocks that are peripheral to the current block, prediction efficiency and compression rate can be improved by excluding the previous block from the candidate group of the current block.

[0269] If the previous block is not included in the reference candidate list in step S2203, in step S2204, an alternative candidate can be searched to replace the excluded reference candidate.

[0270] Figure 23 This is a schematic diagram illustrating an instance of constructing a reference candidate list for the current block based on the block segmentation pattern.

[0271] exist Figure 23In the example shown in (a), when the current block is block number 4, the previous block in the encoding order is block number 3. Furthermore, the parent block of the current block and the previous block is a block that contains both blocks number 3 and 4. Because the parent block containing blocks number 3 and 4 is partitioned by a horizontal binary tree, the determination result in step S2201 is yes. Furthermore, because blocks number 3 and 4 have the same depth and the same parent block, the determination result in step S2202 is yes. Therefore, according to step 2203, the previous block, i.e., block number 3, will not be included in the reference candidate list of the current block.

[0272] Similarly, in Figure 23 In the example shown in (a), when the current block is block number 7, the previous block, block number 6, is excluded from the current block's reference candidate list. Similarly, in Figure 23 In the example shown in (b), when the current block is block number 11, the prediction information of the previous block, namely block number 10, will not be used as a reference candidate for the current block.

[0273] Such as combination Figure 23 As illustrated in embodiments (a) and (b), when the previous block of the current block is not used as a reference candidate, in step S2204, other candidates can be searched to replace the prediction information of neighboring blocks excluded from the reference candidate list. The replacement candidates can be determined using various methods. As an example, prediction information of blocks segmented from the upper-level depth block of the current block that do not have adjacent blocks to the current block can be used. For example, in... Figure 23 In the example shown in (a), when the current block is block number 4, the parent deep block of block number 4 can be a block that merges blocks number 2, 3, and 4. In this case, since block number 2 is a block segmented from the parent deep block that is not adjacent to block number 4, the prediction information of block number 3 can be replaced by appending the prediction information of block number 2 to the reference candidate list of block number 4. As another example for determining replacement candidates, arbitrary prediction information determined in the parent header (such as video, sequence, image, stripe, parallel block, CTU, etc.) can be used to replace the prediction information of surrounding blocks excluded from the reference candidate list. For example, when performing intra-prediction of the current block using an intra-prediction mode, the arbitrary prediction information can be a specific intra-prediction mode (e.g., planar mode or DC mode). For example, when performing intra-frame prediction on the current block using an inter-frame prediction mode, the arbitrary prediction information can be specific prediction information (e.g., {(0,0) motion vector, reference pixel index 0, historical direction}). Furthermore, step S2204 can be omitted, and alternative candidates can be not searched.

[0274] When the current block is a block generated by partitioning a quadtree, it is also possible to not use the prediction information of specific blocks surrounding the current block.

[0275] Figure 24 This is a schematic diagram illustrating another embodiment of constructing a reference candidate list using the surrounding blocks of the current block.

[0276] Figure 25 This is a schematic diagram illustrating another embodiment for constructing a reference candidate list of the current block based on the block segmentation pattern.

[0277] In step S2401, it can be determined whether the partitioning pattern of the parent block of the current block is a quadtree partition. That is, in step S2401, it can be determined whether the current block is a block generated by quadtree partitioning. When the partitioning pattern of the parent block is not a quadtree partition, it can be determined that there are no surrounding blocks that need to be excluded when constructing the reference candidate list and the process ends. Figure 24 The processing in the process. When the partitioning form of the upper-level block is a quadtree partition, that is, when the current block is a block generated by partitioning a quadtree, step S2402 can be executed.

[0278] That is, in step S2402, it can be determined whether the current block is the last sub-block in the encoding order among the four sub-blocks generated by quadtree partitioning. Figure 25 In the example shown, the last sub-block can be sub-block D. If the current block is not sub-block D, it can be determined that there are no surrounding blocks that need to be excluded when constructing the reference candidate list, and the process ends. Figure 24 The processing in the process. When the current block is sub-block D, step S2403 can be executed.

[0279] In step S2403, it is possible to... Figure 24 The system determines whether the prediction information for sub-blocks A, B, and C is completely identical. If the prediction information for sub-blocks A, B, and C is not completely identical, it can be determined that there are no surrounding blocks that need to be excluded when constructing the reference candidate list, and the process ends. Figure 24 In the processing, when the prediction information of sub-blocks A, B, and C are all the same, in step S2404, the prediction information of sub-blocks A, B, and C can be excluded from the reference candidate list when constructing the reference candidate list of sub-block D of the current block. When the parent block is divided into 4 sub-blocks by quadtree partitioning, there is a high probability that at least one of the 4 sub-blocks is dissimilar to the remaining sub-blocks. In this case, the probability of using the prediction information of dissimilar sub-blocks as the prediction information of the current block is also low. For example, in Figure 25In the example shown, when the prediction information of sub-blocks A, B, and C is the same, there is a high probability that the prediction information of sub-block D is different or dissimilar to the prediction information of sub-blocks A, B, and C. Therefore, even if the sub-blocks are peripheral blocks of the current block, the prediction efficiency and compression rate can be improved by excluding the sub-blocks A, B, and C from the candidate group of sub-block D.

[0280] In step S2405, alternative candidates can be searched to replace the excluded reference candidates. The alternative candidates can be determined using various methods. For example, as described above, pre-defined prediction information (DC or planar mode in intra-frame prediction mode, zero motion information in inter-frame prediction mode, etc.) can be used. Another example is the use of prediction information from blocks that were encoded in the most recent time before the current block and do not have adjacent blocks. Alternatively, step S2405 can be omitted.

[0281] Figure 26 This is a diagram illustrating the method for changing the priority of neighboring blocks of the current block that are included in the reference candidate list.

[0282] In combination Figure 26 In the illustrated embodiment, when constructing the reference candidate list for the current block, the priority of the available neighboring blocks of the current block can be adaptively determined.

[0283] In the first method to which this invention is applied, priority can be determined based on whether the current block is adjacent to surrounding blocks and / or the position of the surrounding blocks. For example, higher priority can be assigned to surrounding blocks adjacent to the current block, and lower priority can be assigned to surrounding blocks that are not adjacent. Figure 26 In the example shown, surrounding blocks A, B, F, and G are blocks adjacent to the current block. Therefore, higher priority can be assigned to surrounding blocks A, B, F, and G. Furthermore, the priority between adjacent blocks can be determined according to a specific method. For example, a higher priority can be assigned to a block at a specific location. This specific location could be, for example, the left and / or top edge of the current block. According to this criterion, [the following can be done]... Figure 26 The priority of the surrounding blocks shown is ultimately determined by the block order A→B→F→G→C→D→E. Alternatively, it is also possible to assign lower priority to blocks adjacent to the current block and higher priority to non-adjacent blocks. In this case, the priority can be determined by the block order C→D→E→A→B→F→G.

[0284] In the second method to which this invention is applied, different priorities can be set based on the block segmentation depth information of the current block and surrounding blocks. For example, priorities can be assigned based on the similarity between the segmentation depth of the current block and the segmentation depth of surrounding blocks. For example, a higher priority can be assigned to surrounding blocks with a segmentation depth similar to that of the current block. Among surrounding blocks with the same segmentation depth, priorities can be assigned according to any criterion. For example, a higher priority can be assigned to surrounding blocks at specific positions such as the left or top of the current block.

[0285] Figure 27 This is a schematic diagram used to illustrate the method for determining the priority of surrounding blocks based on segmentation depth.

[0286] exist Figure 27 In the example shown, the segmentation depth of the current block is X, the segmentation depths of surrounding blocks A and F are X, the segmentation depths of surrounding blocks B and G are X+1, the segmentation depths of surrounding blocks D and E are X+2, and the segmentation depth of surrounding block C is X+3. Using the second method of the present invention, a higher priority can be assigned to surrounding blocks with a similar segmentation depth to the current block, considering the current block's segmentation depth, thus determining the block order as A→F→G→B→D→E→C. Alternatively, a higher priority can be assigned to surrounding blocks with a higher segmentation depth without considering the current block's segmentation depth. For example, considering only the segmentation depth of surrounding blocks, the priority of the surrounding blocks can be determined as C→D→E→A→F→G→B. Alternatively, a lower priority can be assigned to surrounding blocks with a similar segmentation depth to the current block, or a higher priority can be assigned to surrounding blocks with a lower segmentation depth.

[0287] In the third method to which this invention is applied, the priority of surrounding blocks can be determined based on the block segmentation order. For example, using the current block as a benchmark, higher priority can be assigned to the most recently segmented and encoded / decoded surrounding blocks. Figure 28 This is a schematic diagram illustrating a method for determining the priority of neighboring blocks based on segmentation order. Figure 28 In the example shown, the order in which the surrounding blocks are divided is marked on the boundaries between the blocks. Figure 28 In the example shown, considering the segmentation order of surrounding blocks, the priority of the surrounding blocks can be determined as the block order E→G→B→C→F→A→D. Alternatively, the current block can be used as a benchmark to assign lower priority to the most recently segmented and encoded / decoded surrounding blocks.

[0288] In the fourth method to which this invention is applied, the priority of a neighboring block can be determined based on the causal relationship between the prediction information of neighboring blocks. For example, when the first neighboring block of the current block is encoded / decoded using prediction information from a second neighboring block that is different from it, a lower priority can be assigned to the first neighboring block. Figure 29 This is a schematic diagram used to illustrate a method for determining the priority of surrounding blocks based on the causal relationship of predictive information. Figure 29 (a) illustrates the position of the current block relative to its surrounding blocks. Figure 29 (b) illustrates the respective prediction information for the surrounding blocks. Figure 29 In the example shown, block B's prediction information is determined by the prediction information of block G. Similarly, block C's prediction information is determined by the prediction information of block B, block D's by the prediction information of block A, and block G's by the prediction information of block E. That is, for example, since the prediction information of block B is determined by the prediction information of block G, it can be determined that there is a causal relationship between the prediction information of block G and block B. Figure 29 In the example shown in (a), the priority of the surrounding blocks can be in any order, such as the block order A→B→C→D→E→F→G, and can also be considered. Figure 29 The priority can be changed based on the causal relationship between the prediction information shown in (b). For example, since blocks B, C, D, and G are encoded / decoded with reference to the prediction information of other surrounding blocks, a lower priority can be assigned. Ultimately, considering the causal relationship between the prediction information of surrounding blocks, the priority of the surrounding blocks can be determined as the block order A→E→F→B→C→D→G. Alternatively, a lower priority can be assigned to the second surrounding block if the first surrounding block references the prediction information of the second surrounding block.

[0289] The content describes four methods for determining the priority of surrounding blocks when constructing a reference candidate list using the surrounding blocks of the current block. Methods 1 through 4 can be applied independently. Alternatively, at least two of methods 1 through 4 can be combined. For example, the priority of surrounding blocks used to construct the reference candidate list can be determined by simultaneously applying methods 2 and 3. Figure 30 This is a schematic diagram illustrating a method for determining the priority of neighboring blocks when both segmentation depth and segmentation order are considered. Figure 30The diagram illustrates the segmentation depths X, X+1, X+2, and X+3 of the surrounding blocks, as well as the segmentation orders 1 to 5. The second and third methods can also be applied simultaneously in various ways. For example, it's possible to assign higher priority to surrounding blocks with segmentation depths similar to the current block, while simultaneously assigning higher priority to surrounding blocks with earlier segmentation orders. In this case, priorities can be sorted first based on the segmentation order. Figure 30 In the example shown, blocks can be sorted according to the partitioning order in the order E→G→B→C→F→A→D. Next, neighboring blocks with partitioning depths similar to the current block can be assigned higher priority. For example, because block G's partitioning depth X+1 is closer to the current block's partitioning depth X than block E's partitioning depth X+2, they can be reordered in the order G→E→B→C→F→A→D. Furthermore, because blocks A and F's partitioning depth X is closer to the current block's partitioning depth X than block C's partitioning depth X+3, they can be reordered in the order E→G→B→F→A→C→D. As described above, the priority of neighboring blocks in the reference candidate list used for the current block can be determined by applying two or more criteria.

[0290] The exemplary methods in this disclosure are described as a sequence of actions for clarity of explanation, but this is not intended to limit the order in which the steps are executed. The steps can be executed simultaneously or in different orders if necessary. To implement the methods in this disclosure, additional steps can be added to the example steps, or only the remaining steps (excluding some steps) can be included, or additional steps can be added after excluding some steps.

[0291] The various embodiments described herein are not a list of all possible combinations, but are merely illustrative of representative forms of the disclosure. The matters described in the various embodiments may apply independently or in combination of two or more.

[0292] Furthermore, the various embodiments described herein can be implemented using hardware, firmware, software, or a combination thereof. When implemented in hardware, they can be implemented using one or more ACICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0293] The scope of this disclosure includes software, device-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that enable actions in methods of various different embodiments to be executed on an apparatus or computer, and a device- or computer-executable non-transitory computer-readable medium storing said software or instructions, etc.

[0294] Industry availability

[0295] This invention can be used to encode / decode images.

Claims

1. An image decoding method, characterized in that, include: Perform the first segmentation step, which divides the current image into multiple blocks; as well as, Perform the second segmentation step, which divides the segmented object block into multiple sub-blocks. The current image includes the filled region. The filled region is formed by horizontally copying adjacent pixels within the filled region. The second segmentation step uses either quadtree segmentation or binary tree segmentation based on specific conditions. The specific condition is determined by comparing at least one of the sizes of blocks that can be partitioned into quadtrees and blocks that can be partitioned into binary trees with the size of the block to be partitioned. The number of sub-blocks is 2 or 4.

2. The image decoding method according to claim 1, characterized in that, The first segmentation step divides the current image into multiple largest blocks of the same size.

3. An image encoding method, characterized in that, include: Perform the first segmentation step, which divides the current image into multiple blocks; as well as, Perform the second segmentation step, which divides the segmented object block into multiple sub-blocks. The current image includes the filled region. The filled region is formed by horizontally copying adjacent pixels within the filled region. The second segmentation step uses either quadtree segmentation or binary tree segmentation based on specific conditions. The specific condition is determined by comparing at least one of the sizes of blocks that can be partitioned into quadtrees and blocks that can be partitioned into binary trees with the size of the block to be partitioned. The number of sub-blocks is 2 or 4.

4. A method for transmitting a bit stream, characterized in that, include: Bitstreams are generated using image encoding methods; as well as Transmit the bit stream, The image encoding method includes: Perform the first segmentation step, which divides the current image into multiple blocks; Perform the second segmentation step, which divides the segmented object block into multiple sub-blocks. The current image includes the filled region. The filled region is formed by horizontally copying adjacent pixels within the filled region. The second segmentation step uses either quadtree segmentation or binary tree segmentation based on specific conditions. The specific condition is determined by comparing at least one of the sizes of blocks that can be partitioned into quadtrees and blocks that can be partitioned into binary trees with the size of the block to be partitioned. The number of sub-blocks is 2 or 4.

Images