Video signal decoding and encoding method, bitstream transmission method

Abstract

The present invention relates to a video signal decoding and encoding method, a bitstream transmission method. A video signal processing method for performing partitioning and encoding on an input image in a block unit according to the present invention includes a partitioning determination for determining whether to perform partitioning on a current block, partitioning the current block into a plurality of sub-blocks according to the determination result, generating block partitioning information related to the partitioning of the current block, and performing encoding on the block partitioning information, the current block, or the sub-blocks. The block partitioning according to the present invention includes a quadtree structure, a binary tree structure, and / or a ternary tree structure, and further includes partitioning using an n-ary tree structure.

Description

[0001] This application is a divisional application of the invention patent application filed on June 23, 2017, with application number 201780039316.3 and entitled "Video Signal Processing Method and Apparatus". Technical Field

[0002] This invention relates to a video signal processing method and apparatus. Background Technology

[0003] Recently, various application fields have seen an increasing demand for high-resolution, high-quality images such as HD (High Definition) and UHD (Ultra High Definition) images. As the resolution and quality of image data increase, the data volume also increases significantly compared to traditional image data. Therefore, the transmission and storage costs increase when using traditional wired or wireless broadband networks for transmission or storage. To address the aforementioned problems arising from the increasing resolution and quality of image data, efficient image compression techniques can be employed.

[0004] Image compression techniques include various techniques such as inter-frame prediction, which predicts the pixel values ​​contained in the current image based on the previous or subsequent images, intra-frame prediction, which predicts the pixel values ​​contained in the current image using pixel information within the current image, and entropy coding, which assigns shorter codes to values ​​that occur more frequently and longer codes to values ​​that occur less frequently. Using the image compression techniques described above, image data can be effectively compressed before transmission or storage.

[0005] Furthermore, with the increasing demand for high-resolution imaging, the demand for stereoscopic imaging content, as a new type of imaging service, is also increasing. Therefore, video compression technologies for effectively delivering high-resolution and ultra-high-resolution stereoscopic imaging content are being actively discussed. Summary of the Invention

[0006] The purpose of this invention is to provide a method and apparatus for encoding / decoding images.

[0007] Furthermore, the present invention aims to provide a method and apparatus for encoding / decoding input images based on adaptive segmentation of input images.

[0008] Furthermore, the present invention aims to provide a method and apparatus capable of signaling adaptive segmentation of input images.

[0009] Furthermore, the present invention aims to provide a method and apparatus capable of performing transformation and / or filtering on adaptively segmented blocks of an input image.

[0010] Furthermore, the present invention aims to provide a video signal processing method and apparatus capable of effectively detecting noise generated at the corners of blocks contained in a video signal decoded in block units, and effectively compensating (filtering) the detected noise.

[0011] The video signal decoding method performed by a video signal decoding device according to the present invention includes the following steps: decoding block segmentation information of a current block; dividing the current block into multiple sub-blocks based on the block segmentation information; and decoding the sub-blocks, wherein, in the segmentation step, at least one of a quadtree structure, a binary tree structure, and a ternary tree structure is used to segment the current block, wherein the quadtree structure is used preferentially over the binary tree structure and the ternary tree structure, and the sub-block is an encoding unit.

[0012] The video signal encoding method performed by a video signal encoding device according to the present invention includes the following steps: determining whether to segment the current block; based on the determination, segmenting the current block into multiple sub-blocks; generating block segmentation information for the segmentation of the current block; and encoding the block segmentation information and the sub-blocks, wherein, in the segmentation step, the current block is segmented using at least one of a quadtree structure, a binary tree structure, and a ternary tree structure, wherein the quadtree structure is used preferentially over the binary tree structure and the ternary tree structure, and the sub-block is an encoding unit.

[0013] The present invention relates to a method for transmitting a bitstream of a video signal received by a video signal decoding device and used for decoding a video signal, wherein the bitstream includes block segmentation information of a current block; the block segmentation information is used to divide the current block into multiple sub-blocks, wherein the current block is segmented using at least one of a quadtree structure, a binary tree structure, and a ternary tree structure, wherein the quadtree structure is used preferentially over the binary tree structure and the ternary tree structure, and the sub-blocks are encoding units.

[0014] The video signal processing method of the present invention, which performs segmentation and encoding of input images in block units, includes: a segmentation determination step of determining whether to perform segmentation on the current block; a block segmentation step of dividing the current block into multiple sub-blocks based on the determination result; a step of generating block segmentation information related to the segmentation of the current block; and an encoding step of encoding the block segmentation information, the current block, or the sub-blocks.

[0015] In the video signal processing method to which the present invention is applied, the above-described block segmentation step can utilize two or more tree structures to segment the current block.

[0016] In the video signal processing method to which the present invention is applied, the block segmentation step described above can utilize one or more of the two or more tree structures as the main segmentation structure and the remainder as the secondary segmentation structure to perform segmentation on the block.

[0017] In the video signal processing method to which the present invention is applied, the block segmentation information includes main segmentation information for indicating whether the block is segmented using the main segmentation structure. When the main segmentation information indicates that the block is segmented using the main segmentation structure and there are multiple main segmentation structures, the block segmentation information may also include information for specifying one of the multiple main segmentation structures.

[0018] In the video signal processing method to which the present invention is applied, the block segmentation information includes main segmentation information indicating whether the block is segmented using the main segmentation structure, and when the main segmentation information indicates that the block is not segmented using the main segmentation structure, the block segmentation information may further include secondary segmentation information indicating whether the block is segmented using a secondary segmentation structure.

[0019] In the video signal processing method to which the present invention is applied, when the sub-segmentation information indicates that the block is segmented using the sub-segmentation structure and there are multiple sub-segmentation structures, the block segmentation information may further include information for specifying one of the multiple sub-segmentation structures.

[0020] In the video signal processing method to which the present invention is applied, when the main segmentation information indicates that the block is not segmented using the main segmentation structure and the sub-segmentation information indicates that the block is not segmented using the sub-segmentation structure, the current block can be set as an encoding unit.

[0021] In the video signal processing method to which the present invention is applied, the block segmentation information includes first information indicating whether to perform segmentation on the block, and when the first information indicates that the block is to be segmented and multiple segmentation structures are used to perform segmentation on the block, the block segmentation information may further include second information for specifying one of the multiple segmentation structures.

[0022] In the video signal processing method to which the present invention is applied, information related to the main segmentation structure or the sub-segmentation structure described above can be encoded at at least one of the following levels: sequence level, image level, strip level, parallel block level, and block level.

[0023] In the video signal processing method to which this invention is applied, the block segmentation is not performed for blocks smaller than a certain size, and the information related to the certain size can be encoded at at least at the sequence level, image level, strip level, parallel block level, and block level.

[0024] In the video signal processing method of the present invention, the encoding step of encoding the current block or the sub-block includes at least one of prediction, transformation, and quantization, wherein the transformation includes a non-square transformation, and the transformation is capable of being passed through Y = AXB. T (Where, X is a residual signal block of size m×n, A is a one-dimensional n-point transformation in the horizontal direction, B...) T (This is a one-dimensional m-point transformation in the vertical direction, where Y is the transformation block obtained by transforming X.)

[0025] In the video signal processing method to which this invention is applied, A and B can be different transformations.

[0026] The video signal processing method of the present invention, which performs segmentation and decoding of input images in block units, can include: a step of decoding block segmentation information of the current block; a block segmentation step of dividing the current block into multiple sub-blocks based on the block segmentation information; and a step of decoding the current block or the sub-blocks.

[0027] The video signal processing apparatus for segmenting and encoding input images in block units according to the present invention includes: a segmentation determination unit for determining whether to segment the current block; a block segmentation unit for segmenting the current block into multiple sub-blocks based on the determination result; a block segmentation information generation unit for generating block segmentation information related to the segmentation of the current block; and an encoding unit for encoding the block segmentation information, the current block, or the sub-blocks.

[0028] The video signal processing method of the present invention, which performs segmentation and decoding of input images in block units, can include: a block segmentation information decoding unit that decodes the block segmentation information of the current block; a block segmentation unit that divides the current block into multiple sub-blocks based on the block segmentation information; and a decoding unit that performs decoding on the current block or the sub-blocks.

[0029] Furthermore, the video signal processing method of the present invention, which performs segmentation and encoding of input images in block units, is characterized by: generating residual blocks related to the current block, encoding the residual blocks, decoding the encoded residual blocks, reconstructing the current block using the decoded residual blocks, and filtering the reconstructed image containing the reconstructed current block, wherein the filtering is performed based on the shape or size of two blocks adjacent to the block boundary.

[0030] The video signal processing method applicable to this invention is characterized by determining the number of filtered pixels or the filtering intensity based on the shape or size of two blocks adjacent to the block boundary.

[0031] The video signal processing method applicable to the present invention is characterized in that: when at least one of two blocks adjacent to the block boundary is not square, filtering is performed on more pixels in the larger block of the two blocks.

[0032] The video signal processing method applicable to the present invention is characterized in that: when at least one of two blocks adjacent to the block boundary is not square, a stronger filter is applied to the larger of the two blocks.

[0033] The video signal processing method applicable to the present invention is characterized in that: when two blocks adjacent to the block boundary are of different sizes, filtering is performed on more pixels in the larger block of the two blocks.

[0034] The video signal processing method applicable to the present invention is characterized in that: when two blocks adjacent to the block boundary are of different sizes, a stronger filter is applied to the larger of the two blocks.

[0035] The video signal processing method of the present invention, which performs segmentation and decoding of input images in block units, is characterized by: decoding residual blocks related to the current block from the bit stream, reconstructing the current block using the decoded residual blocks, and filtering the reconstructed image containing the reconstructed current block, wherein the filtering is performed based on the shape or size of two blocks adjacent to the block boundary.

[0036] The video signal processing apparatus for segmenting and encoding an input image in block units according to the present invention is characterized by comprising: a residual block generation unit for generating residual blocks related to a current block; a residual block decoding unit for encoding the residual blocks; a residual block decoding unit for decoding the encoded residual blocks; a current block reconstruction unit for reconstructing the current block using the decoded residual blocks; and a filtering unit for filtering the reconstructed image containing the reconstructed current block; wherein the filtering unit performs filtering on the block boundary based on the shape or size of two blocks adjacent to the block boundary.

[0037] The video signal processing apparatus for segmenting and decoding an input image in block units according to the present invention is characterized by comprising: a residual block decoding unit that decodes residual blocks related to the current block from a bitstream; a current block reconstruction unit that reconstructs the current block using the decoded residual blocks; and a filtering unit that filters the reconstructed image containing the reconstructed current block; wherein the filtering unit filters the block boundary based on the shape or size of two blocks adjacent to the block boundary.

[0038] Furthermore, the video signal processing method applicable to the present invention is characterized in that: when the corner points of the four blocks contained in the video signal decoded in block units are adjacent to each other at an intersection point, a corner pixel is selected from the four corner pixel pixels adjacent to the intersection point as a corner outlier and filtering is performed on the corner outlier. The selection of the corner outlier uses the difference value between the pixel values ​​of the four corner pixel pixels adjacent to the intersection point and a first threshold.

[0039] The video signal processing method applicable to the present invention is characterized in that: the first threshold is determined based on the quantization parameters of the above four blocks.

[0040] The video signal processing method applicable to the present invention is characterized in that: the similarity between the pixel containing the same block as the selected corner outlier and the pixel adjacent to the corner outlier is further determined, and the filtering is performed based on the similarity determination result.

[0041] The video signal processing method applicable to the present invention is characterized in that: the similarity determination uses the difference value between the pixel value of the corner outlier that is contained in the same block as the corner outlier and is adjacent to the corner outlier, and the pixel value of the corner outlier, as well as a second threshold.

[0042] The video signal processing method applicable to the present invention is characterized in that the second threshold is determined based on the quantization parameters of the four blocks.

[0043] The video signal processing method applicable to the present invention is characterized in that: it further determines whether the block boundary adjacent to the selected corner outlier is an edge of the image region, and the filtering is performed based on the determination result of whether the block boundary is an edge of the image region.

[0044] The video signal processing method applicable to the present invention is characterized in that: the process of determining whether the block boundary is an edge of an image region includes, as pixels in the block adjacent to the corner outlier, a first edge determination using the change in pixel values ​​between pixels adjacent to the block boundary and a third threshold.

[0045] The video signal processing method applicable to the present invention is characterized in that the third threshold is determined based on the quantization parameters of the four blocks.

[0046] The video signal processing method applicable to the present invention is characterized in that: the process of determining whether the block boundary is an edge of an image region includes a second edge determination using the difference value between the pixel value of the corner point that is horizontally or vertically adjacent to the corner outlier and the pixel value of the corner outlier, and a fourth threshold.

[0047] The video signal processing method applicable to the present invention is characterized in that the fourth threshold is determined based on the quantization parameters of the four blocks.

[0048] The video signal processing method applicable to the present invention is characterized in that: the filtering is performed by setting the weighted average of the four corner pixels adjacent to the intersection point as the filtered pixel value of the corner outlier.

[0049] The video signal processing method applicable to the present invention is characterized in that: the filtering includes filtering of pixels that are contained in the same block as the corner outlier and that are adjacent to the corner outlier.

[0050] The video signal processing apparatus applicable to the present invention is characterized by comprising: a corner outlier filter, wherein when the corners of four blocks contained in a video signal decoded in block units are adjacent to each other at an intersection, a corner pixel is selected from the four corner pixel pixels adjacent to the intersection as a corner outlier and filtering is performed on the corner outlier; wherein the selection of the corner outlier uses the difference value between the pixel values ​​of the four corner pixel pixels adjacent to the intersection and a first threshold.

[0051] The video signal processing apparatus to which this invention is applied is characterized in that the first threshold is determined based on the quantization parameters of the four blocks.

[0052] The video signal processing apparatus to which the present invention is applied is characterized in that: the corner outlier filter further determines the similarity between the pixel contained in the same block as the selected corner outlier and the pixel adjacent to the corner outlier, and the corner outlier itself; the filtering is performed based on the similarity determination result.

[0053] The video signal processing apparatus to which the present invention is applied is characterized in that: the similarity determination is based on the difference between the pixel value of the corner outlier and the pixel value of the corner outlier, which are contained in the same block as the corner outlier and are adjacent to the corner outlier, and a second threshold.

[0054] The video signal processing apparatus to which this invention is applied is characterized in that the second threshold is determined based on the quantization parameters of the four blocks.

[0055] The video signal processing apparatus applicable to the present invention is characterized in that: the corner outlier filter further determines whether the block boundary adjacent to the selected corner outlier is an edge of the image region, and the filtering is performed based on the determination result of whether the block boundary is an edge of the image region.

[0056] The video signal processing apparatus to which the present invention is applied is characterized in that: the process of determining whether the block boundary is an edge of an image region includes, as pixels in a block adjacent to the corner outlier, a first edge determination using the change in pixel values ​​between pixels adjacent to the block boundary and a third threshold.

[0057] The video signal processing apparatus to which this invention is applied is characterized in that the third threshold is determined based on the quantization parameters of the four blocks.

[0058] The video signal processing apparatus to which the present invention is applied is characterized in that: the process of determining whether the block boundary is an edge of an image region includes a second edge determination using the difference between the pixel value of the corner point that is horizontally or vertically adjacent to the corner outlier and the pixel value of the corner outlier, and a fourth threshold.

[0059] The video signal processing apparatus to which this invention is applied is characterized in that the fourth threshold is determined based on the quantization parameters of the four blocks.

[0060] The video signal processing apparatus to which the present invention is applied is characterized in that: the filtering is performed by setting the weighted average of the four corner pixels adjacent to the intersection point as the filtered pixel value of the corner outlier.

[0061] The video signal processing apparatus to which the present invention is applied is characterized in that: the filtering includes filtering of pixels that are contained in the same block as the corner outlier and that are adjacent to the corner outlier.

[0062] The present invention provides a method and apparatus for encoding / decoding images.

[0063] Furthermore, this invention enables adaptive segmentation of blocks based on various tree structures, including quadtree, binary tree, and / or ternary tree structures, thereby improving coding efficiency.

[0064] Furthermore, this invention enables effective signaling of block segmentation information when an input image is adaptively segmented, thereby improving coding efficiency.

[0065] Furthermore, this invention enables the effective transformation and / or filtering of blocks with arbitrary shapes through adaptive segmentation of the input image, thereby improving coding efficiency.

[0066] Furthermore, this invention enables the effective detection of noise generated at the corners of blocks contained in a video signal decoded in block units.

[0067] Furthermore, this invention can effectively compensate for noise generated at the corners of blocks in a video signal decoded in block units.

[0068] Furthermore, this invention enables the effective detection and compensation of noise generated at the corners of units, i.e., blocks, during encoding / decoding processing, allowing the corresponding blocks to be used as references for inter-frame prediction and / or intra-frame prediction, thereby preventing noise from propagating to other blocks or other images. Attached Figure Description

[0069] Figure 1 This is a block diagram illustrating an image encoding apparatus to which one embodiment of the present invention is applied.

[0070] Figure 2 This is a block diagram illustrating an image decoding apparatus to which one embodiment of the present invention is applied.

[0071] Figure 3a This is a schematic diagram illustrating an example of the structure of basic blocks in an input image segmented using a quadtree structure.

[0072] Figure 3b This is a schematic diagram illustrating, as an example, the structure of basic blocks of an input image segmented using a quadtree and / or binary tree structure.

[0073] Figure 4This is a schematic diagram illustrating an example of how a quadtree structure can be used to segment blocks contained in an input image.

[0074] Figure 5 This is a schematic diagram illustrating an example of how a binary tree structure can be used to segment blocks contained in an input image.

[0075] Figure 6 This is a schematic diagram illustrating an example of how a triple tree structure can be used to segment blocks contained in an input image.

[0076] Figure 7a This is an exemplary illustration of a structure that uses QT cross segmentation as the main segmentation structure and BT vertical 1:1 and / or BT horizontal 1:1 segmentation as the secondary segmentation structure to divide the blocks contained in the input image into multiple sub-blocks.

[0077] Figure 7b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 7a The diagram illustrates the block segmentation information related to the block segmentation structure shown in the figure.

[0078] Figure 7c According to another embodiment of the present invention, a tree structure is used to pair with... Figure 7a The diagram illustrates the block segmentation information related to the block segmentation structure shown in the figure.

[0079] Figure 8 This is a schematic diagram illustrating the various sizes and shapes of the sub-blocks that can be segmented when the blocks contained in the input image are divided into multiple sub-blocks using QT cross segmentation as the main segmentation structure and BT vertical 1:1 and / or BT horizontal 1:1 segmentation as the secondary segmentation structure.

[0080] Figure 9a This is an exemplary illustration of a structure that uses QT cross segmentation as the main segmentation structure and BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1 and / or TT horizontal 1:2:1 segmentation as secondary segmentation structures to divide the blocks contained in the input image into multiple sub-blocks.

[0081] Figure 9b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 9a The diagram illustrates the block segmentation information related to the block segmentation structure shown in the figure.

[0082] Figure 10a This is an exemplary illustration of a structure that uses QT cross segmentation as the main segmentation structure and BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3 and / or BT vertical 3:1 segmentation as secondary segmentation structures to divide the blocks contained in the input image into multiple sub-blocks.

[0083] Figure 10b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 10a The diagram illustrates the block segmentation information related to the block segmentation structure shown in the figure.

[0084] Figure 11 This is a schematic diagram illustrating the various sizes and shapes of the sub-blocks that can be used when dividing the blocks contained in the input image into multiple sub-blocks using QT cross segmentation as the main segmentation structure and BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3 and / or BT vertical 3:1 segmentation as the secondary segmentation structure.

[0085] Figure 12 This is a schematic diagram illustrating, as an example, the real base of DCT-II that can be used in the transformation, and the integer base obtained by multiplying the real base by a specific value.

[0086] Figure 13 This is a schematic diagram illustrating, as an example, the real base of DST-VII that can be used in the transformation, and the integer base obtained by multiplying the real base by a specific value.

[0087] Figure 14 This is a flowchart illustrating a filtering method applicable to one embodiment of the present invention.

[0088] Figure 15 For the purpose of execution Figure 14 The diagram illustrates the filtering process using two blocks adjacent to the block boundary and their internal pixels.

[0089] Figure 16 This is a schematic diagram illustrating the structure of blocks segmented using quadtree and / or binary tree structures to which the present invention applies, and the block boundaries to which filtering is applied.

[0090] Figure 17 This is a flowchart illustrating filtering to which another embodiment of the present invention is applicable.

[0091] Figure 18 This is a schematic diagram illustrating an example of a pixel subjected to strong filtering according to an embodiment of the present invention.

[0092] Figure 19 This is a schematic diagram illustrating an example of a pixel subjected to weak filtering according to an embodiment of the present invention.

[0093] Figure 20 This is a schematic diagram illustrating, as an example, the pixel range to which the present invention applies when weak filtering is performed.

[0094] Figure 21 (a) is a schematic diagram illustrating a corner outlier point that is the filtering object of a corner outlier filter applicable to one embodiment of the present invention.

[0095] Figure 21 (b) is to Figure 21 The diagram illustrates the pixel values ​​of the pixels in the 2×2 region centered on the intersection of (a).

[0096] Figure 21 (c) is a schematic diagram illustrating the indexes of the pixels used to indicate the positions of pixels when detecting and filtering outliers at corner points.

[0097] Figure 22 This is a schematic diagram illustrating the operation of a corner outlier filter applicable to one embodiment of the present invention. Detailed Implementation

[0098] This invention is capable of various modifications and has many different embodiments. Specific embodiments will be illustrated and described in detail below. However, the following content is not intended to limit the invention to specific implementations, but should be understood to include all modifications, equivalents, and substitutions within the scope of the invention's concept and technology. Similar reference numerals are used for similar constituent elements in the description of the various figures.

[0099] In describing different constituent elements, terms such as "first" and "second" may be used, but the constituent elements are not limited by these terms. These terms are merely used to distinguish one constituent element from others. For example, without departing from the scope of the claims of this invention, a first constituent element may also be named a second constituent element, and similarly, a second constituent element may also be named a first constituent element. The term "and / or" includes a combination of multiple related descriptions or one of multiple related descriptions.

[0100] When a constituent element is described as being "connected" or "in contact" with other constituent elements, it should be understood that it can not only be directly connected or in contact with the aforementioned other constituent elements, but also that other constituent elements can exist between the two. Conversely, when a constituent element is described as being "directly connected" or "directly in contact" with other constituent elements, it should be understood that no other constituent elements exist between the two.

[0101] The terminology used in this application is for illustrative purposes only and is not intended to limit the invention. Singular statements also have plural meanings unless the context clearly indicates otherwise. In this invention, terms such as “~comprises~” or “~consisting of~” are merely to indicate the presence of features, numbers, steps, actions, constituent elements, components, or combinations thereof described in the specification, and should not be construed as excluding the possibility of one or more other features, numbers, steps, actions, constituent elements, components, or combinations thereof being present or added.

[0102] Next, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the following description, the same reference numerals will be used for the same constituent elements in the drawings, and repeated descriptions of the same constituent elements will be omitted.

[0103] Figure 1 This is a block diagram illustrating an image encoding apparatus to which one embodiment of the present invention is applied.

[0104] like Figure 1 As shown, the image encoding apparatus 100 may include: an image segmentation unit 110; prediction units 120 and 125; a transformation unit 130; a quantization unit 135; a reordering unit 160; an entropy encoding unit 165; an inverse quantization unit 140; an inverse transformation unit 145; a filtering unit 150; and a memory 155.

[0105] Figure 1 The various components illustrated are shown separately to illustrate different features and functions of the image encoding apparatus, and do not represent that each component is composed of separate hardware or software units. That is, although the various components are listed for ease of explanation, it is possible to combine at least two components into one component, or to divide one component into multiple components to perform corresponding functions. The embodiments where the various components are integrated or separated as described above are included within the scope of the claims of this invention without departing from the essence of the invention.

[0106] Furthermore, some constituent elements may not be essential for performing the essential functions of this invention, but rather optional elements used to improve performance. This invention can include only the constituent parts essential for realizing the essence of the invention, excluding those merely used to improve performance, and structures including essential constituent elements, excluding optional elements used to improve performance, are also included within the scope of the claims of this invention.

[0107] The image segmentation unit 110 is capable of segmenting the input image into at least one processing unit. This processing unit can be a prediction unit (PU), a transformation unit (TU), or a coding unit (CU). In the image segmentation unit 110, an image can be segmented into a combination of multiple coding units, prediction units, and transformation units, and a combination of coding units, prediction units, and transformation units can be selected according to a specific criterion (e.g., a cost function) to encode the image.

[0108] For example, an image can be segmented into multiple coding units. To segment coding units in an image, a recursive tree structure, such as a quadtree structure, can be used. A coding unit, with an image or the largest coding unit as the root, can be segmented into other coding units, each with child nodes corresponding to the number of coding units segmented. Coding units that are not further segmented under certain constraints become leaf nodes. That is, assuming that only one coding unit can be squarely segmented, a coding unit can be segmented into a maximum of four other coding units.

[0109] To segment coded units in an image, a tree structure can be used. The tree structure can include at least one of a quadtree, a binary tree structure, and / or a triple tree structure. Segmentation can be performed using an image or the largest coded unit as the root according to the tree structure. For the segmented blocks, the tree structure can be applied again recursively or hierarchically. The tree structure used to further segment the segmented blocks can be different from the previously used tree structure. Blocks that are not further segmented are leaf nodes, which can serve as units for prediction, transformation, and / or quantization. When segmenting blocks using a tree structure, leaf nodes can be not only squares but also non-squares.

[0110] In the following embodiments of the present invention, the encoding unit can be used as both a unit for performing encoding and a unit for performing decoding.

[0111] A prediction unit can be formed by dividing a coding unit into at least one square or rectangle of the same size, or it can be formed by dividing a coding unit into prediction units in a way that the shape and / or size of one prediction unit is different from that of another prediction unit.

[0112] When generating prediction units that perform intra-frame prediction based on coding units, if they are not the smallest coding units, intra-frame prediction can be performed without splitting them into multiple prediction units N×N.

[0113] Prediction units 120 and 125 can include an inter-frame prediction unit 120 for performing inter-frame prediction and an intra-frame prediction unit 125 for performing intra-frame prediction. It can determine whether to use inter-frame prediction or perform intra-frame prediction for a prediction unit, and determine specific information related to each prediction method (e.g., intra-frame prediction mode, motion vector, reference image, etc.). In this case, the processing unit for performing prediction can be different from the processing unit for determining the prediction method and specific content. For example, the prediction method and prediction mode can be determined at the prediction unit level, while prediction execution can be performed at the transformation unit level. The residual value (residual block) between the generated prediction block and the original block can be input to the transformation unit 130. Furthermore, the prediction mode information, motion vector information, etc., used during prediction can be encoded together with the residual value in the entropy coding unit 165 and then transmitted to the decoder. When using a specific coding mode, the original block can also be encoded directly without generating a prediction block through prediction units 120 and 125 and then transmitted to the decoding unit.

[0114] The inter-frame prediction unit 120 can predict prediction units based on information from at least one of the previous or subsequent images of the current image, and in some cases, it can also predict prediction units based on information from a portion of the encoded region within the current image. The inter-frame prediction unit 120 may include a reference image interpolation unit, a motion prediction unit, and a motion compensation unit.

[0115] The reference image interpolation unit can receive reference image information from memory 155 and generate pixel information in the reference image down to the integer pixel level. For luminance pixels, in order to generate pixel information down to the integer pixel level in 1 / 4 pixel units, an 8-tap DCT-based interpolation filter with different filtering coefficients can be used. For chrominance signals, in order to generate pixel information down to the integer pixel level in 1 / 8 pixel units, a 4-tap DCT-based interpolation filter with different filtering coefficients can be used.

[0116] The motion prediction unit can perform motion prediction based on a reference image interpolated by the reference image interpolation unit. Various methods can be used to calculate motion vectors, such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm). The motion vector can have a motion vector value in 1 / 2 or 1 / 4 pixel units based on the interpolated pixels. The motion prediction unit can predict the current prediction unit using different motion prediction methods. Various motion prediction methods can be used, such as skipping, merging, AMVP (Advanced Motion Vector Prediction), and intra-block copying.

[0117] The intra-prediction unit 125 can generate prediction units based on pixel information within the current image, i.e., reference pixel information surrounding the current block. If the surrounding block of the current prediction unit is a block that has already undergone inter-frame prediction, and therefore its reference pixels are pixels that have undergone inter-frame prediction, then the reference pixels contained in the block that has undergone inter-frame prediction can be replaced with the reference pixel information of the surrounding blocks that have undergone intra-frame prediction. That is, when reference pixels are unavailable, at least one of the available reference pixels can be used to replace the unavailable reference pixel information before use.

[0118] In intra-frame prediction, the prediction mode can include a directional prediction mode that uses reference pixel information based on the prediction direction and a non-directional mode that does not use directional information when performing prediction. The mode used to predict luminance information can be different from the mode used to predict chromatic difference information, and in order to predict chromatic difference information, the intra-frame prediction mode information used when predicting luminance information or the predicted luminance signal information can be used.

[0119] If the size of the prediction unit is the same as the size of the transform unit when performing intra-prediction, intra-prediction can be performed based on the pixels to the left of the prediction unit, the pixels at the top left, and the pixels at the top. However, if the size of the prediction unit is different from the size of the transform unit, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using N×N segmentation can be performed only on the smallest coding unit.

[0120] Intra-prediction methods generate prediction blocks based on a prediction pattern after applying an AIS (Adaptive IntraSmoothing) filter to a reference pixel. The type of AIS filter applied to the reference pixel can be different. To perform intra-prediction, the intra-prediction pattern of the current prediction unit can be predicted using the intra-prediction patterns of neighboring prediction units. When predicting the prediction pattern of the current prediction unit using pattern information predicted from neighboring units, if the intra-prediction patterns of the current prediction unit and its neighboring units are the same, specific marker information can be used to transmit information indicating that the prediction patterns of the current prediction unit and its neighboring units are the same. If the prediction patterns of the current prediction unit and its neighboring units are different, entropy coding can be used to encode the prediction pattern information of the current block.

[0121] Furthermore, it is also possible to generate residual blocks containing residual information, which is the difference between the prediction unit and the original block of the prediction unit, based on the prediction unit generated in prediction units 120 and 125. The generated residual blocks can be input to transformation unit 130.

[0122] The transform unit 130 can perform transformations on the original block and the residual block containing the residual information of the prediction units generated by the prediction units 120 and 125 using transformation methods such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), and KLT (Caronan-Loy Transform). Whether to apply DCT, DST, or KLT when performing the transformation on the residual block can be determined based on the intra-frame prediction mode information of the prediction unit used when generating the residual block.

[0123] The quantization unit 135 is capable of quantizing the values ​​that have been transformed into the frequency domain in the transformation unit 130. The quantization coefficients can be changed according to the importance of the block or image. The values ​​calculated by the quantization unit 135 can be provided to the inverse quantization unit 140 and the reordering unit 160.

[0124] The reordering unit 160 is capable of reordering the coefficient values ​​of the quantized residual values.

[0125] The reordering unit 160 can convert the coefficients of a two-dimensional block shape into a one-dimensional vector shape using a coefficient scanning method. For example, the reordering unit 160 can use a zig-zag scan method to scan from DC coefficients to high-frequency domain coefficients and convert them into a one-dimensional vector shape. Depending on the size of the transformation unit and the intra-frame prediction mode, a vertical scan that scans the coefficients of the two-dimensional block shape along the column direction or a horizontal scan that scans the coefficients of the two-dimensional block shape along the row direction can be used instead of the zig-zag scan. That is, the choice between the zig-zag scan, the vertical scan, and the horizontal scan can be determined based on the size of the transformation unit and the intra-frame prediction mode.

[0126] The entropy coding unit 165 can perform entropy coding based on the value calculated by the reordering unit 160. Entropy coding can use various coding methods such as Exponential Golomb code, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding).

[0127] The entropy coding unit 165 can encode various information from the reordering unit 160 and the prediction units 120 and 125, such as residual coefficient information of the coding unit, block type information, prediction mode information, segmentation unit information, prediction unit information and transmission unit information, motion vector information, reference frame information, block interpolation information, filtering information, etc.

[0128] The entropy encoding unit 165 is capable of entropy encoding the coefficient values ​​of the encoding units input from the reordering unit 160.

[0129] The inverse quantization unit 140 and the inverse transform unit 145 perform inverse quantization on the value quantized by the quantization unit 135 and inverse transform on the value transformed by the transform unit 130. The residual value generated in the inverse quantization unit 140 and the inverse transform unit 145 can be merged with the prediction unit predicted by the motion prediction unit, motion compensation unit and intra-frame prediction unit included in the prediction units 120 and 125 to generate a reconstructed block.

[0130] The filtering unit 150 may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter).

[0131] Deblocking filters can eliminate block distortion caused by the boundaries between blocks in a reconstructed image. To determine whether deblocking should be performed, the application of a deblocking filter to the current block can be based on the pixels contained in several columns or rows within the block. When applying a deblocking filter to a block, a strong filter or a weak filter can be used depending on the required deblocking intensity. Furthermore, when applying a deblocking filter, horizontal and vertical filtering can be performed in parallel.

[0132] When performing block filtering, adaptive filtering can be performed based on the shape, size, and / or characteristics of the two blocks P and Q adjacent to the block boundary. For example, when the two blocks P and Q are of different sizes, filtering can be performed on more pixels in the larger block compared to the smaller block. Furthermore, adaptive filtering can be performed based on whether at least one of the two blocks P and Q is a non-square block. For example, in the case where block P is an 8×8 block and block Q is an 8×16 block, when filtering the boundary between adjacent blocks P and Q, filtering can be performed on more pixels in block Q compared to block P.

[0133] When two adjacent blocks P and Q have different sizes or at least one is a non-square block, it is possible to apply the same number of filtered pixels to both blocks P and Q, but with different filtering intensities. Alternatively, it is also possible to apply different numbers of filtered pixels and different filtering intensities to the two blocks P and Q.

[0134] The offset correction unit can correct the offset between the deblocked image and the original image on a pixel-by-pixel basis. To perform offset correction on a specific image, a method can be used that divides the pixels contained in the image into a certain number of regions, determines the region to be offset, and applies the offset to the corresponding region, or a method can be used that applies the offset while taking into account the edge information of each pixel.

[0135] At this point, ALF (Adaptive Loop Filtering) can be performed based on the comparison between the filtered reconstructed image and the original image. After dividing the pixels in the image into specific groups, it is possible to determine which filter should be applied to the corresponding group, and then perform different filtering on different groups. Information related to the applicability of ALF can be transmitted according to each coding unit (CU), and the shape and filtering coefficients of the applicable ALF filter can differ depending on the block. Furthermore, it is also possible to apply ALF filters of the same shape (fixed shape) without considering the characteristics of the target block.

[0136] The memory 155 can store the reconstructed blocks or images calculated by the filtering unit 150, and the stored reconstructed blocks or images can be provided to the prediction units 120 and 125 when performing inter-frame prediction.

[0137] Figure 2 This is a block diagram illustrating an image decoding apparatus to which one embodiment of the present invention is applied.

[0138] like Figure 2 As shown, the image decoder 200 may include an entropy decoding unit 210, a reordering unit 215, an inverse quantization unit 220, an inverse transform unit 225, a prediction unit 230, 235, a filtering unit 240, and a memory 245.

[0139] When an image bitstream is input from an image encoder, the input bitstream can be decoded by following the reverse steps of the image encoder.

[0140] The entropy decoding unit 210 can perform entropy decoding in the reverse order of the entropy encoding process performed in the entropy encoding unit of the image encoder. For example, it can apply various methods such as Exponential Golomb code, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) depending on the method used in the entropy encoder.

[0141] The entropy decoding unit 210 is capable of decoding information related to intra-frame prediction and inter-frame prediction performed in the encoder.

[0142] The reordering unit 215 can reorder the bitstream entropy-decoded in the decoding unit 210 based on the reordering method performed in the encoding unit. That is, it can reconstruct coefficients in one-dimensional vector form into coefficients in two-dimensional block form and reorder them. The reordering unit 215 can perform reordering by receiving information related to the coefficient scan performed in the encoding unit and performing a reverse scan based on the scan order performed in the corresponding encoding unit.

[0143] The inverse quantization unit 220 can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values ​​of the reordered blocks.

[0144] The inverse transform unit 225 can perform the inverse transforms of DCT, DST, and KLT (i.e., inverse DCT, inverse DST, and inverse KLT) performed by the transform unit on the quantization results executed in the image encoder. The inverse transform can be performed based on the transmission unit determined in the image encoder. The inverse transform unit 225 of the image decoder can selectively perform transform methods (e.g., DCT, DST, KLT) based on various information such as the prediction method, the size of the current block, and the prediction direction.

[0145] Prediction units 230 and 235 can generate prediction blocks based on prediction block generation information provided by entropy decoding unit 210 and previously decoded blocks or image information provided by memory 245.

[0146] As described above, if the size of the prediction unit is the same as the size of the transform unit when performing intra-prediction in the same manner as in the image encoder, intra-prediction can be performed based on the pixels to the left of the prediction unit, the pixels at the upper left, and the pixels at the top. However, if the size of the prediction unit is different from the size of the transform unit when performing intra-prediction, intra-prediction can be performed using reference pixels based on the transform unit. Furthermore, intra-prediction using N×N segmentation can also be performed only on the smallest coding unit.

[0147] Prediction units 230 and 235 may include a prediction unit determination unit, an inter-frame prediction unit, and an intra-frame prediction unit. The prediction unit determination unit can receive various information input from the entropy decoding unit 210, such as prediction unit information, prediction mode information of the intra-frame prediction method, and motion prediction-related information of the inter-frame prediction method. It distinguishes prediction units from the current decoding unit area and determines whether the prediction unit should perform inter-frame prediction or intra-frame prediction. The inter-frame prediction unit 230 can utilize the information required for inter-frame prediction of the current prediction unit provided by the image encoder, and perform inter-frame prediction for the current prediction unit based on information contained in at least one of the previous or subsequent images of the current image containing the current prediction unit. Alternatively, it can perform inter-frame prediction within the current image containing the current prediction unit based on information from a reconstructed portion of the region.

[0148] In order to perform inter-frame prediction, it is possible to determine, based on the coding unit, which of the following motion prediction methods is used for the prediction units contained in the corresponding coding unit: Skip Mode, Merge Mode, Advanced Motion Vector Prediction Mode (AMVP Mode), or Intra-frame Block Copy Mode.

[0149] The intra-prediction unit 235 can generate prediction blocks based on pixel information within the current image. When the prediction unit is a prediction unit that has already undergone intra-prediction, intra-prediction can be performed based on the intra-prediction mode information of the prediction unit provided by the image encoder. The intra-prediction unit 235 may include an AIS (Adaptive Intra Smoothing) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is used to filter the reference pixels of the current block, and its application is determined based on the prediction mode of the current prediction unit. AIS filtering can be performed on the reference pixels of the current block using the prediction mode of the prediction unit provided by the image encoder and the AIS filter information. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

[0150] When the prediction mode of the prediction unit is a prediction unit that performs intra-frame prediction based on pixel values ​​interpolated from reference pixels, the reference pixel interpolation unit can generate reference pixels with pixel units smaller than integer values ​​by interpolating the reference pixels. When the prediction mode of the current prediction unit is a prediction mode that generates prediction blocks without interpolating reference pixels, interpolation of reference pixels is not required. When the prediction mode of the current block is DC mode, the DC filter can generate prediction blocks by filtering.

[0151] The reconstructed blocks or image can be provided to the filtering unit 240. The filtering unit 240 may include a deblocking filter, an offset correction unit, and an ALF.

[0152] The image encoder can receive information regarding the applicability of deblocking filters for corresponding blocks or images, and information regarding whether strong or weak filtering is applicable when applying deblocking filters. The image decoder's deblocking filter can receive the deblocking filter information provided by the image encoder and perform deblocking filtering on the corresponding blocks in the image decoder.

[0153] The offset correction unit can perform offset correction on the reconstructed image based on the offset correction type and offset value information applicable to the image during encoding.

[0154] ALF can be applied to encoding units based on ALF applicability information and ALF coefficient information provided by the encoder. The aforementioned ALF information can be provided in a way that includes specific parameter groups.

[0155] Memory 245 can save the reconstructed image or block and use it as a reference image or reference block, and can also provide the reconstructed image to the output unit.

[0156] As described above, in the following embodiments of the present invention, the term "coding unit" is used for ease of explanation, but it can refer to either a unit that performs encoding or a unit that performs decoding.

[0157] Figure 3a This is a schematic diagram illustrating an example of a structure that uses a quadtree structure to divide a basic block of an input image into multiple sub-blocks, according to one embodiment of the present invention.

[0158] Figure 3b This is an exemplary illustration of a structure that uses a quadtree and / or a binary tree to divide a basic block of an input image into multiple sub-blocks.

[0159] To enable efficient encoding, the input image to be encoded can be segmented into basic block units before encoding. The basic block of this invention can be defined using a Largest Coding Unit (LCU) or a Coding Tree Unit (CTU). The basic block can take the form of a rectangle or square with a size of M×N. M and N can both be 2. n(n is an integer greater than 1), where M represents the horizontal length of the block and N represents the vertical length of the block. The LCU or CTU can also be a square of size 64×64 or 128×128. For more efficient image compression, additional segmentation can be performed on the basic blocks as described above.

[0160] To effectively perform image compression, it is advisable to segment the image into homogeneous regions based on its homogeneity. A homogeneous region refers to a region where the brightness and / or color difference values ​​of the samples within it do not vary, or the variations are below a specific threshold. In other words, a homogeneous region consists of samples with homogeneous values, and the degree of homogeneity can be determined according to specific criteria. By dividing the basic block into multiple homogeneous sub-blocks while considering image homogeneity, the prediction residual signal energy of the sub-blocks can be more effectively concentrated, thereby improving the compression efficiency during transformation and quantization.

[0161] To divide a basic block within an input image into multiple sub-blocks based on homogeneity, structures such as binary trees, quadtrees, ternary trees, octrees, and / or general N-ary tree structures can be used. By utilizing at least one of these tree structures, a basic block can be divided into multiple sub-blocks.

[0162] Figure 3a It is an instance that uses only a quadtree structure to divide a basic block into multiple sub-blocks. The quadtree structure can divide the block to be divided into 4 blocks of the same size, and the 4 blocks can be further divided according to the quadtree structure.

[0163] Figure 3b This is an instance where a basic block is divided into multiple sub-blocks using a quadtree and / or binary tree structure. The letters labeled within each sub-block are indices representing homogeneity. For example, sub-blocks labeled with the letter 'a' represent regions with the same homogeneity. Furthermore, in... Figure 3b In the diagram, the attached drawing number marked inside the block indicates the corresponding block, while the attached drawing number marked on the block boundary indicates the block divided by the corresponding boundary.

[0164] like Figure 3a , Figure 3b As shown, a basic block can be divided into multiple sub-blocks. Sub-blocks that are not further divided can be designated as coding units (coding cells). Figure 3a , Figure 3bIn this context, the encoding unit can be a unit of prediction, transformation, and / or quantization. Alternatively, the encoding unit can be further segmented to perform prediction, transformation, and / or quantization. For example, to predict the encoding unit, segmentation using a quadtree structure, segmentation using a binary tree structure, or asymmetric segmentation can be performed, and segmentation in various non-square or non-rectangular shapes can also be considered. Furthermore, the segmentation methods used in the prediction described above can also be applied to transform and / or quantize the encoding unit.

[0165] When segmenting basic blocks of an input image, information related to the use of quadtree and / or binary tree structures can be signaled via a bitstream. Information related to the segmentation structure of the basic blocks of the input image can be signaled at units such as sequence units, image units, strip units, parallel block units, and / or basic block units. For example, when the above information is signaled at the image unit level, it can instruct the use of both quadtree and binary tree structures, or only quadtree structures, for all or some basic blocks contained in the corresponding image. If it is determined that only quadtree structures are used, the block segmentation information of the basic blocks can include only segmentation information using quadtree structures and exclude segmentation information using binary tree structures.

[0166] As mentioned above, a quadtree structure can divide a block into four blocks of the same size. Furthermore, a binary tree structure can divide a block into two blocks of the same size. When using a binary tree structure to divide a block, it is also necessary to encode / decode the information related to the division direction, which indicates whether the division is horizontal or vertical. The method for encoding / decoding the block division information will be explained later.

[0167] like Figure 3b As shown, for example, a quadtree structure can be used to divide a basic block 300 into four sub-blocks. When performing block segmentation using a tree structure, the depth of each block can be determined based on the depth of the tree structure. The basic block 300 corresponds to a block with a depth of 0 in the tree structure. The sub-blocks obtained by segmenting the basic block 300 correspond to blocks with a depth of 1. The four sub-blocks (depth = 1) obtained by segmentation can have the same size. It is possible not to perform further segmentation on the upper left sub-block (depth = 1) 301 of the four sub-blocks. For example, when it is determined that the upper left sub-block (depth = 1) 301 is a homogeneous region, it is possible not to perform further segmentation on sub-block 301. The sub-block (depth = 1) 301 that is no longer segmented can be set as the encoding unit.

[0168] For example, in the four sub-blocks obtained by segmenting the basic block (depth = 0) 300 using a quadtree structure, the upper-right sub-block (depth = 1) 302 can be recursively or hierarchically segmented into four sub-blocks (depth = 2) using a quadtree structure. These sub-blocks (depth = 2) can be set as encoding units. Alternatively, the upper-left sub-block (depth = 2) 302-1 of the upper-right sub-block (depth = 1) 302 of the basic block (depth = 0) 300 can be segmented again using a binary tree structure. When it is impossible to segment the resulting blocks again or when they are determined to belong to homogeneous regions that do not require segmentation, they can be set as encoding units.

[0169] like Figure 3b As shown, when performing segmentation on basic blocks using a quadtree structure and / or a binary tree structure, compared to performing segmentation only using a quadtree structure... Figure 3a Compared to the situation shown, it can more adaptively divide blocks into homogeneous regions of different sizes.

[0170] As mentioned above, it is also possible to further segment the encoding unit for prediction, transformation, and / or quantization. However, in situations such as... Figure 3b The method shown can define highly accurate homogeneous regions as encoding units when performing segmentation. This eliminates the need to segment the encoding units again for prediction, transformation, and / or quantization. That is, the encoding unit itself can serve as a prediction unit and / or a transformation unit. Because the encoding unit can be used directly as a prediction unit and / or transformation unit, the cost of segmenting the encoding unit into prediction and / or transformation units is saved. In particular, because the segmentation information related to the shape of the encoding unit being segmented into prediction and / or transformation units does not need to be encoded into syntax elements, compression efficiency is improved. Furthermore, in the combination described above… Figure 3b In the block segmentation method described, since each sub-block, which is set as the encoding unit, is a homogeneous region with extremely high accuracy, the energy concentration of the residual signal is very efficient, thereby improving the compression efficiency during transformation and / or quantization.

[0171] In this invention, a basic block can be divided into multiple coding units using a quadtree structure and / or a binary tree structure. The quadtree and binary tree structures can be selected and used appropriately in any order as needed. Alternatively, a quadtree structure can be used as the primary segmentation structure and a binary tree structure as the secondary segmentation structure. Or, a binary tree structure can be used as the primary segmentation structure and a quadtree structure as the secondary segmentation structure. When one is used as the primary segmentation structure and the other as the secondary segmentation structure, segmentation can be performed first using the primary segmentation structure. Upon reaching the lead node of the primary segmentation structure, the lead node becomes the root node of the secondary segmentation structure, and segmentation is performed using the secondary segmentation structure.

[0172] When basic blocks are segmented and encoded using a specific combination of tree structures during the encoding process, it is necessary to signal information related to the tree structure, segmentation shape, direction, and / or proportion used in segmenting the basic blocks (hereinafter referred to as "block segmentation information"). The decoder can decode the block segmentation information of the basic blocks based on the information contained in the bitstream or the information induced during the decoding of the bitstream, and then perform decoding on the basic blocks based on the block segmentation information.

[0173] When a tree-structured block is segmented and no further segmentation is performed, the node where no further segmentation is performed is equivalent to a leaf node. Leaf nodes can serve as units for prediction, transformation, and / or quantization, for example, they can be equivalent to coding units (CUs) as defined in this specification. The size of a coding unit equivalent to a leaf node can be 2. n ×2 n 2 n ×2 m Or 2n×2m (n and m are integers greater than 1).

[0174] Next, a method for performing encoding / decoding by segmenting basic blocks using at least one or more combinations of quadtrees, binary trees, and / or ternary trees to form corresponding block segmentation information will be described. However, the tree structure used for segmenting blocks in this invention is not limited to the quadtrees, binary trees, and / or ternary trees described above, and n-ary tree structures can also be widely applied to block segmentation as described above.

[0175] Figure 4 This is a schematic diagram illustrating an example of a structure that uses a quadtree structure to divide a block contained in an input image into multiple sub-blocks, according to one embodiment of the present invention.

[0176] When splitting the current block using a quadtree structure, the current block can be divided into 4 sub-blocks.

[0177] For example, such as Figure 4 As shown in (a), four sub-blocks can be generated by splitting the current block using two intersecting lines. In this specification, the splitting pattern described above can be defined as a "QT cross". Here, "QT" can represent the meaning of a quadtree. In the above case, the horizontal and vertical lengths of the split sub-blocks can correspond to half of the block before the split.

[0178] Or, such as Figure 4 As shown in (b), four sub-blocks can be generated by dividing the current block using three horizontal lines. In this specification, the division pattern described above can be defined as "QT level". In the above case, the horizontal length of the divided sub-blocks is the same as that of the block before division, while the vertical length can correspond to 1 / 4 of the block before division.

[0179] Or, such as Figure 4 As shown in (c), four sub-blocks can be generated by dividing the current block using three vertical lines. In this specification, the division pattern described above can be defined as "QT Vertical". In the above case, the vertical length of the divided sub-blocks is the same as that of the block before division, while the horizontal length can correspond to 1 / 4 of the block before division.

[0180] Block partitioning using a quadtree structure is not limited to Figure 4 (a) to Figure 4 (c) can also define various different ratios. For example, it can also split blocks in ratios such as 1:1:1:2, 1:2:2:4, etc. That is, block splitting using a quadtree structure can include all forms of dividing an object block into 4 sub-blocks in any ratio.

[0181] Figure 5 This is a schematic diagram illustrating an example of a structure that uses a binary tree structure to divide a block contained in an input image into multiple sub-blocks, according to one embodiment of the present invention.

[0182] When splitting the current block using a binary tree structure, the current block can be split into 2 sub-blocks.

[0183] For example, such as Figure 5As shown in (a), two sub-blocks can be generated by dividing the current block in a 1:3 ratio using vertical lines. In this specification, the division pattern described above can be defined as "BT Vertical 1:3". Here, "QT" can represent the meaning of a binary tree. The vertical length of the two sub-blocks is the same as that of the original block, while the horizontal length ratio of the two sub-blocks is 1:3.

[0184] Or, such as Figure 5 As shown in (b), two sub-blocks can be generated by dividing the current block at a 1:1 ratio using vertical lines. In this specification, the division pattern described above can be defined as "BT Vertical 1:1". The vertical length of the two sub-blocks is the same as that of the block before the division, while the ratio of the horizontal lengths of the two sub-blocks is 1:1.

[0185] Or, such as Figure 5 As shown in (c), two sub-blocks can be generated by dividing the current block using a vertical line at a ratio of 3:1. In this specification, the division pattern described above can be defined as "BT Vertical 3:1". The vertical length of the two sub-blocks is the same as that of the block before the division, while the ratio of the horizontal lengths of the two sub-blocks is 3:1.

[0186] Or, such as Figure 5 As shown in (d), two sub-blocks can be generated by dividing the current block at a ratio of 1:3 using a horizontal line. In this specification, the division pattern described above can be defined as "BT horizontal 1:3". The horizontal length of the two sub-blocks is the same as that of the block before the division, while the ratio of the vertical lengths of the two sub-blocks is 1:3.

[0187] Or, such as Figure 5 As shown in (e), two sub-blocks can be generated by dividing the current block at a 1:1 ratio using a horizontal line. In this specification, the division pattern described above can be defined as "BT horizontal 1:1". The horizontal length of the two sub-blocks is the same as that of the block before the division, while the ratio of the vertical lengths of the two sub-blocks is 1:1.

[0188] Or, such as Figure 5 As shown in (f), two sub-blocks can be generated by dividing the current block using a horizontal line at a ratio of 3:1. In this specification, the division pattern described above can be defined as "BT horizontal 3:1". The horizontal length of the two sub-blocks is the same as that of the block before the division, while the ratio of the vertical lengths of the two sub-blocks is 3:1.

[0189] Block partitioning using a binary tree structure is not limited to Figure 5 (a) to Figure 5 The (f) property can also define various different ratios. For example, it can also split blocks at ratios such as 1:2, 1:4, 1:5, etc. That is, block splitting using a binary tree structure can include all forms of dividing an object block into two sub-blocks at any ratio.

[0190] Figure 6 This is a schematic diagram illustrating an example of a structure that uses a ternary tree structure to divide a block contained in an input image into multiple sub-blocks, according to one embodiment of the present invention.

[0191] When splitting the current block using a ternary tree structure, the current block can be divided into 3 sub-blocks.

[0192] For example, such as Figure 6 As shown in (a), three sub-blocks can be generated by dividing the current block using two vertical lines in a 1:1:2 ratio. In this specification, the division pattern described above can be defined as "TT vertical 1:1:2". Here, "TT" can represent the meaning of a Triple Tree. The vertical length of the three sub-blocks is the same as that of the original block, while the horizontal length ratio of the three sub-blocks is 1:1:2.

[0193] Or, such as Figure 6 As shown in (b), three sub-blocks can be generated by dividing the current block using two vertical lines in a 1:2:1 ratio. In this specification, the division pattern described above can be defined as "TT vertical 1:2:1". The vertical length of the three sub-blocks is the same as that of the block before division, while the horizontal length ratio of the three sub-blocks is 1:2:1.

[0194] Or, such as Figure 6 As shown in (c), three sub-blocks can be generated by dividing the current block using two vertical lines in a 2:1:1 ratio. In this specification, the division pattern described above can be defined as "TT vertical 2:1:1". The vertical length of the three sub-blocks is the same as that of the block before division, while the horizontal length ratio of the three sub-blocks is 2:1:1.

[0195] Or, such as Figure 6As shown in (d), three sub-blocks can be generated by dividing the current block using two horizontal lines in a 1:1:2 ratio. In this specification, the division pattern described above can be defined as "TT horizontal 1:1:2". The horizontal length of the three sub-blocks is the same as that of the original block, while the vertical length ratio of the three sub-blocks is 1:1:2.

[0196] Or, such as Figure 6 As shown in (e), three sub-blocks can be generated by dividing the current block using two horizontal lines in a 1:2:1 ratio. In this specification, the division pattern described above can be defined as "TT horizontal 1:2:1". The horizontal length of the three sub-blocks is the same as that of the block before the division, while the vertical length ratio of the three sub-blocks is 1:2:1.

[0197] Or, such as Figure 6 As shown in (f), three sub-blocks can be generated by dividing the current block using two horizontal lines in a 2:1:1 ratio. In this specification, the division pattern described above can be defined as "TT horizontal 2:1:1". The horizontal length of the three sub-blocks is the same as that of the block before the division, while the vertical length ratio of the three sub-blocks is 2:1:1.

[0198] Block partitioning using a ternary tree structure is not limited to Figure 6 (a) to Figure 6 The (f) property can also define various different ratios. For example, it can also split blocks in ratios such as 1:2:2, 1:2:4, 1:2:5, etc. That is, block splitting using a ternary tree structure can include all forms of dividing an object block into 3 sub-blocks in any ratio.

[0199] As described above, in conjunction with Figure 3 to... Figure 6 The description demonstrates that a block can be divided into multiple sub-blocks using a quadtree, binary tree, and / or ternary tree structure. Furthermore, the quadtree, binary tree, and / or ternary tree structure can be used again to recursively or hierarchically divide the multiple sub-blocks obtained through the division.

[0200] like Figure 4 The illustrated embodiment demonstrates that QT's block segmentation can achieve three segmentation patterns. Furthermore, as... Figure 5 as well as Figure 6The illustrated embodiments demonstrate that block partitioning using BT or TT can have six partitioning patterns respectively. Therefore, the block partitioning information needs to include information representing the tree structure used when partitioning the current block (tree structure information). Furthermore, the block partitioning information also needs to include information representing one of the multiple partitioning patterns of the selected tree structure (partitioning pattern information).

[0201] Based on the number of tree structures that can be used in block partitioning, the number of bits required to encode tree structure information can be determined. Furthermore, based on the number of partition patterns contained in a tree structure, the number of bits required to encode partition pattern information can be determined.

[0202] The type of tree structure that can be used in block segmentation can be determined in advance in the encoder and decoder. Alternatively, the type of tree structure that can be used in block segmentation can also be encoded in the encoder and then transmitted to the decoder via bitstream. Information related to the type of tree structure can be transmitted after being encoded at at least at the sequence level, image level, stripe level, parallel block level, and basic block level. For example, when a block is segmented using only QT in a stripe, information indicating that QT was used when segmenting the block can be signaled through the stripe header, etc. For example, when a block is segmented using QT, BT, or TT in a stripe, information indicating that QT, BT, or TT was used when segmenting the block can be signaled through the stripe header, etc. If the encoder and decoder determine the default segmentation structure to be used in advance, signaling can be performed on the tree structure used at the corresponding level even without transmitting the corresponding information. Alternatively, the type of tree structure that can be used at a lower level can be limited to some or all of the types of tree structures that can be used at a higher level. For example, when signaling information using QT or BT via the sequence header, it can be specified that only QT, BT, or a combination of QT and BT can be used in the images contained in the corresponding sequence, and TT cannot be used. For example, when no information related to the tree structure that can be used in block segmentation is transmitted, signaling information from the higher level can be directly inherited.

[0203] In this specification, signaling information may include not only explicit signaling information transmitted via bit stream, but also implicit signaling information.

[0204] As described above, when a tree structure or a combination of tree structures that can be used at the current level is selected, signaling can be applied to the corresponding information. The blocks contained in the current level can be segmented using one of the aforementioned available tree structures. When three tree structures can be used at the current level, the encoding of the tree structure information requires at least 2 bits. For example, when the tree structure information represents a segmentation using QT, it can be expressed using 1 bit. For example, when the tree structure information represents a segmentation using either BT or TT, it can be expressed using 2 bits.

[0205] After specifying the tree structure used in the segmentation of the specified block, it is necessary to further specify one of the segmentation methods of the specified tree structure. For example, ... Figure 6 As shown, since there are 6 possible partitioning patterns for TT, when partitioning the current block using TT, it is also necessary to specify the partitioning pattern information for one of the 6 partitioning patterns. The number of bits required to encode the partitioning pattern information can be determined considering that there are 6 available partitioning patterns. For example, the required number of bits can be set to 3 bits using the calculation result of ceil(log2(6)). Here, ceil() represents the floor function.

[0206] Even when the tree structure used in block partitioning is determined, it is not necessary to use all the partitioning patterns contained in the corresponding tree structure. For example, it is possible to use only... Figure 6 The diagram illustrates a subset of the six segmentation patterns, thereby reducing the number of bits required to encode segmentation pattern information. Information representing the segmentation patterns used in the segmentation of a corresponding tree structure can be transmitted after being encoded at at least one level: sequence level, image level, strip level, parallel block level, and basic block level.

[0207] like Figures 4 to 6 As shown, some segmentation patterns can also be derived from other segmentation patterns. For example, Figure 5 The segmentation pattern of (d) can be obtained by... Figure 4 The result is obtained by merging the bottom three blocks in the partitioning pattern (b). Therefore, it is not necessary to use... Figures 4 to 6 All segmentation patterns illustrated in the figure. For example, it is possible to encode or decode at least one tree structure and / or multiple segmentation patterns used in a corresponding sequence, image, strip, parallel block, or basic block, using tree structure information and segmentation pattern information.

[0208] When segmenting blocks using two or more tree structures, the applicable order between the tree structures can be determined in advance. The order between the tree structures can be determined in advance in the encoder / decoder, or transmitted after encoding at at least one level: sequence level, image level, strip level, parallel block level, or basic block level.

[0209] For example, a segmentation using QT can be used as the primary segmentation, and a segmentation using BT / TT can be used as the secondary segmentation. In this case, the segmentation using QT will be performed first, and then the segmentation using BT / TT will be performed on the QT leaf nodes that are no longer subject to QT segmentation. When segmenting blocks using the primary and secondary segmentation structures, the number of bits required to encode the tree structure information and / or segmentation pattern information as described above can be further reduced. At this time, if there are two or more tree structures that can be used as secondary segmentation structures, segmentation can be performed using a single secondary segmentation structure without specifying the order. Alternatively, the block can be segmented after determining the order among multiple secondary segmentation structures. For example, a hierarchical structure can also be adopted, using QT as the primary segmentation structure, using BT to segment the leaf nodes of QT, and then using TT to segment the leaf nodes of BT.

[0210] The number of bits required to express tree structure information and / or segmentation morphology information can vary depending on the circumstances.

[0211] Information related to the presence or absence of a specific applicable order among multiple sub-segments and the tree structure that needs to be used as the main segmentation structure in multiple tree structures can be determined in advance in the encoder / decoder, or transmitted after being encoded at at least at the sequence level, image level, strip level, parallel block level, or basic block level.

[0212] Figure 7a This is an exemplary illustration of a structure that uses QT cross segmentation as the primary segmentation structure and BT vertical 1:1 and / or BT horizontal 1:1 segmentation as secondary segmentation structures to divide blocks contained in an input image into multiple sub-blocks. Figure 7a In the diagram, the attached drawing number marked inside the block indicates the corresponding block, while the attached drawing number marked on the block boundary indicates the block divided by the corresponding boundary.

[0213] Figure 7b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 7a The diagram illustrates, for example, the block segmentation information related to the block segmentation structure shown in the figure. The block segmentation information may include tree structure information and / or segmentation pattern information.

[0214] Figure 7cAccording to another embodiment of the present invention, a tree structure is used to pair with... Figure 7a The diagram illustrates, for example, the block segmentation information related to the block segmentation structure shown in the figure. The block segmentation information may include tree structure information and / or segmentation pattern information.

[0215] exist Figure 7a In , b, and c, block partitioning using QT cross-splitting is represented by solid lines, while block partitioning using BT vertical 1:1 and / or BT horizontal 1:1 partitioning is represented by dashed lines.

[0216] like Figure 7a As shown, QT cross-splitting divides the basic block (depth = 0) 700 into four sub-blocks (depth = 1) of the same size. At this point, the depth of each block represents its depth within the tree structure when performing splitting using a tree structure. Each of the four sub-blocks (depth = 1) can be further split using QT cross-splitting. Sub-blocks that are no longer split using QT cross-splitting are equivalent to the terminal nodes of a quadtree structure. The terminal node of a quadtree can become the root node of a binary tree. For the root node of a binary tree, splitting using a binary tree structure (BT vertical 1:1 and / or BT horizontal 1:1 splitting) can be performed. Alternatively, splitting using a binary tree structure can be omitted. When splitting using a binary tree structure is not performed on the terminal node of the quadtree, the corresponding terminal node can be set as the encoding unit.

[0217] As mentioned above, the terminal node of a quadtree can be further partitioned using a binary tree structure. That is, the terminal node of a quadtree can become the root node of a binary tree. For example, in... Figure 7a In the process of dividing the basic block (depth = 0) 700 into four sub-blocks, the lower right sub-block (depth = 1) 701 is not further divided using a quadtree structure. The lower right sub-block (depth = 1) 701, as the root node of a binary tree, can then be divided using a binary tree structure. For example... Figure 7a As shown, the lower right sub-block (depth = 1) 701 can be divided into two sub-blocks (depth = 2) by performing a BT vertical 1:1 split. Furthermore, the left sub-block (depth = 2) 701-1 can be further divided into two sub-blocks (depth = 3) 701-1a and 701-1b by performing a BT vertical 1:1 split again. These two sub-blocks (depth = 3) 701-1a and 701-1b can correspond to the terminal nodes of the binary tree. The two sub-blocks (depth = 3) 701-1a and 701-1b that are no longer split can be set as encoding units.

[0218] As described above, this invention can utilize a quadtree structure (QT cross-segmentation) as the primary segmentation structure and a binary tree structure (BT vertical 1:1 segmentation and / or BT horizontal 1:1 segmentation) as the secondary segmentation structure to segment the basic block 700. In this case, the basic block 700 can be set as the root node of the quadtree. The root node of the quadtree can be further decomposed using the quadtree structure in a recursive or hierarchical manner before reaching the end node of the quadtree. The end node of the quadtree can become the root node of a binary tree. The root node of the binary tree can be further decomposed using the binary tree structure in a recursive or hierarchical manner before reaching the end node of the binary tree. When no further segmentation, including segmentation using the quadtree structure and / or segmentation using the binary tree structure, is performed on the current block, the current block can be set as an encoding unit.

[0219] Figure 7b According to one embodiment of the present invention, for use with respect to Figure 7a The diagram illustrates the block segmentation information encoded / decoded within the block segmentation structure shown in the image. This block segmentation information can be composed of a combination of tree structure information and / or segmentation pattern information.

[0220] exist Figure 7b , Figure 7c In the tree structure, the attached diagrams inside the parentheses are numbered as follows: Figure 7a The block number shown in the figure is an example of block segmentation information associated with the corresponding block, and the information represented by 0 and / or 1 is an instance of block segmentation information.

[0221] Next, the information used to indicate whether a block was split using a quadtree structure will be called the "quadtree split information". The quadtree split information can be encoded with a first bit. The first bit can be 1 bit. When splitting the current block using a quadtree structure, the quadtree split information for the current block can be encoded as "1". When splitting the current block without using a quadtree structure, the quadtree split information can be encoded as "0". That is, the information indicating whether a quadtree structure was used to split the current block can be encoded using 1 bit of quadtree split information.

[0222] Next, the block segmentation information used to represent the binary tree structure will be referred to as "binary segmentation information". Binary segmentation information can include at least one of the following: information indicating whether the block is segmented using a binary tree structure and information related to the segmentation direction using the binary tree structure. Binary segmentation information can be encoded with a second bit length. The second bit length can be 1 bit or 2 bits. When the current block is a segmentation target using a binary tree structure, the 2-bit binary segmentation information described below can be used. When the current block is segmented using a binary tree structure, the first bit of the aforementioned 2-bit binary segmentation information can be encoded as "1". When the current block is not segmented using a binary tree structure, the first bit of the aforementioned 2-bit binary segmentation information can be encoded as "0". In the block segmentation embodiment shown in FIG7, the block segmentation patterns using a binary tree structure include BT vertical 1:1 segmentation and BT horizontal 1:1 segmentation. This allows for the appending encoding of information related to the segmentation direction (or segmentation pattern). That is, when the partitioning using a binary tree structure is a BT horizontal 1:1 partition, the second bit of the aforementioned 2-bit binary partitioning information can be encoded as "0". When the partitioning using a binary tree structure is a BT vertical 1:1 partition, the second bit of the aforementioned 2-bit binary partitioning information can be encoded as "1". When the current block is partitioned without using a binary tree structure, the second bit of the aforementioned 2-bit binary partitioning information can be omitted.

[0223] The length of the first bit and the length of the second bit can also be the same, and are not limited to the embodiment described above. Furthermore, in the above embodiments, the meanings of the bit values ​​in the quadrilateral segmentation information and the binary segmentation information can be defined in opposite ways. For example, using "0" in the first bit of the quadrilateral segmentation information and / or the binary segmentation information can indicate that segmentation using the corresponding structure is performed, and using "1" can indicate that segmentation is not performed. Alternatively, the case where the second bit of the binary segmentation information is "0" can be defined as a vertical segmentation, and the case where it is "1" can be defined as a horizontal segmentation.

[0224] The block segmentation information of the current block can be encoded using the first bit length of the quadruple segmentation information and the second bit length of the binary segmentation information as shown in Table 1 below.

[0225] Table 1

[0226]

[0227] In Table 1 above, the quadtree partitioning information can be "0" or "1", and the binary partitioning information can be encoded as "0", "10", or "11". Block partitioning information is used to indicate whether a block has been partitioned, the partitioning type (or tree structure information), and the partitioning direction (or partitioning shape information). It can be information combining quadtree partitioning information and binary partitioning information, or information representing a combination of quadtree partitioning information and binary partitioning information. In Table 1 above, the first bit of the quadtree partitioning information and / or binary partitioning information can correspond to the aforementioned tree structure information. In Table 1 above, the second bit of the binary partitioning information can correspond to the aforementioned partitioning shape information. In the embodiment illustrated in Figure 7, since the quadtree partitioning shape only includes QT cross partitioning, independent partitioning shape information is not required. In the embodiment illustrated in Figure 7, since the binary tree partitioning shape includes BT vertical 1:1 and BT horizontal 1:1 partitioning, partitioning shape information is needed to distinguish them. In this case, the length of the partitioning shape information can be as short as 1 bit.

[0228] In Table 1 above, block segmentation information "00" indicates that neither quadtree nor binary tree segmentation is performed on the current block. Block segmentation information "010" indicates that quadtree segmentation is not performed on the current block (first bit = 0), but binary tree segmentation is performed (second bit = 1), and the binary tree segmentation is a BT horizontal 1:1 segmentation (third bit = 0). Block segmentation information "011" indicates that quadtree segmentation is not performed on the current block (first bit = 0), but binary tree segmentation is performed (second bit = 1), and the binary tree segmentation is a BT vertical 1:1 segmentation (third bit = 1). Block segmentation information "1" indicates that quadtree segmentation is performed on the current block.

[0229] In Table 1 above, when a quadtree structure is used as the primary segmentation structure and a binary tree structure is used as the secondary segmentation structure, once a block segmented using a quadtree structure is further segmented using a binary tree structure, no further segmentation using a quadtree structure will be performed. Therefore, for blocks obtained by segmenting using a binary tree structure, the quadtree segmentation information used to indicate whether quadtree segmentation was performed is no longer required to be encoded / decoded. In this case, the block segmentation information of blocks obtained by segmenting using a binary tree structure can contain only binary segmentation information. That is, binary segmentation information can be used directly as block segmentation information without containing quadtree segmentation information. Block segmentation information "10" indicates that a binary tree structure segmentation was performed on the current block (first bit = 1), and the binary tree segmentation is a BT horizontal 1:1 segmentation (second bit = 0). Block segmentation information "11" indicates that a binary tree structure segmentation was performed on the current block (first bit = 1), and the binary tree segmentation is a BT vertical 1:1 segmentation (second bit = 1). Block segmentation information "0" indicates that the current block obtained by segmenting using a binary tree structure will not be segmented using a binary tree structure again.

[0230] Figure 7b It utilizes the block segmentation information encoding method shown in Table 1 above, using a tree structure to encode the information. Figure 7a The diagram illustrates the block segmentation information related to the block segmentation structure shown in the image. The depth of the tree structure and... Figure 7a The depth correspondence of the block segmentation structure in the text. Figure 7b The information marked in each node represents the block segmentation information of the block corresponding to the aforementioned node. For example, in Figure 7b The "1" marked in the root node is the block corresponding to the above node, i.e. Figure 7a The block segmentation information for basic block 700 in the code indicates that basic block 700 is divided into 4 sub-blocks using a QT cross-segmentation method with a quadtree structure. When a node has multiple child nodes... Figure 7b The sorting order of child nodes in the data should follow the rules. Figure 7a The diagram illustrates the raster scan order (or Z-shaped scan order) of the block segmentation structure. Specifically, when the basic block (depth = 0) 700 is divided into four sub-blocks (depth = 1) using a quadtree structure, for example, the raster scan order of the lower right sub-block (depth = 1) 700 is the last of the four sub-blocks (depth = 1). Therefore, in Figure 7b In the tree structure, the "011" marked at the last node position in the child nodes at depth 1 is... Figure 7a The block segmentation information corresponds to the lower right sub-block (depth = 1) 701.

[0231] As shown in Table 1, block segmentation information can be encoded by combining or integrating quadtree and binary segmentation information. Alternatively, quadtree and binary segmentation information can be encoded separately using independent syntax elements. Furthermore, information within the binary segmentation information indicating whether a binary tree structure was used to segment the block, as well as information related to the segmentation direction using the binary tree structure, can be encoded separately using independent syntax elements. Alternatively, information indicating whether a segmentation using a quadtree structure or a binary tree structure is used can be encoded using one syntax element, and segmentation pattern information can be encoded using another syntax element when a segmentation using a binary tree structure is used.

[0232] Figure 7c According to one embodiment of the present invention, for use with respect to Figure 7a The diagram illustrates the block segmentation information encoded by the block segmentation structure shown in the figure.

[0233] In one embodiment of the invention, quadruple segmentation information requires 1 bit while binary segmentation information requires 2 bits. That is, because additional information related to the segmentation pattern is needed when signaling binary segmentation information, binary segmentation information requires more bits compared to quadruple segmentation information. In combination with... Figure 7c In the block segmentation information encoding method described below, the information indicating whether the segmentation method applicable to the current block is a quadtree structure segmentation or a binary tree structure segmentation is encoded in the manner shown in Table 2 below, taking into account the number of bits of the block segmentation information.

[0234] Table 2

[0235]

[0236] In Table 2 above, block partitioning information "0" indicates that no partitioning using a quadtree structure or a binary tree structure is performed on the current block. That is, it represents a block that does not require partitioning. Block partitioning information "1" indicates that a partitioning using a quadtree structure is performed on the current block. Block partitioning information "10" indicates that the partitioning using a binary tree structure on the current block is a BT horizontal 1:1 partition. Block partitioning information "11" indicates that the partitioning using a binary tree structure on the current block is a BT vertical 1:1 partition.

[0237] The block segmentation information encoding method shown in Table 2 signals the tree structure information (in the embodiment shown in Figure 7, this information indicates whether a quadtree or binary tree structure is used) based on the number of bits in the block segmentation information. A bit count of 1 indicates whether a block has been segmented and / or whether a segmentation is performed using a quadtree structure. A bit count of 2 indicates that a segmentation is performed using a binary tree structure. Therefore, the block segmentation information can be encoded with fewer bits compared to the block segmentation information encoding method shown in Table 1.

[0238] Figure 7c It utilizes the block segmentation information encoding method shown in Table 2 above, using a tree structure to encode the information. Figure 7a The diagram illustrates the block segmentation structure and related block segmentation information. Figure 7b as well as Figure 7c In addition to using different block segmentation information encoding methods as shown in Tables 1 and 2, they have the same tree structure. Therefore, for Figure 7b The explanations, especially those related to the sorting order of child nodes, can also be applied to... Figure 7c .

[0239] The encoding method for block partitioning information is not limited to the methods shown in Tables 1 and 2. For example, the methods shown in Tables 1 and / or 2 can be used in combination or after omitting some of them. Block partitioning information refers to all forms of information, such as whether partitioning of the current block has been performed, information indicating whether the partitioning applicable to the current block is a quadtree partition or a binary tree partition, and / or, when a binary tree partition is applied, information related to the partitioning pattern indicating whether it is a BT vertical 1:1 partition or a BT horizontal 1:1 partition.

[0240] Figure 8 This is a schematic diagram illustrating, for example, the various sizes and shapes of the sub-blocks that can be used when a basic block is divided into multiple sub-blocks using a quadtree structure and / or a binary tree structure (BT vertical 1:1 partitioning and / or BT horizontal 1:1 partitioning).

[0241] When the size of the encoding unit is too small, it may actually lead to a decrease in encoding efficiency (prediction, transform, and / or quantization). Furthermore, the amount of data that needs to be transmitted when encoding block segmentation information may increase. Therefore, it is necessary to limit the size of blocks that can be divided into smaller blocks. For example, segmentation can be stopped when the length (horizontal and / or vertical) of the segmented block is below a certain value. This specific value can be set to any size, such as 4, 8, or 16. This specific value can be signaled via a bitstream. This specific value can be adaptively signaled using sequence units, image units, stripe units, parallel block units, or basic block units. Alternatively, this specific value can be set to a value pre-agreed upon by the encoder and decoder.

[0242] Alternatively, when the length of only one side of the block, either vertically or horizontally, is below the aforementioned specific value, a binary tree structure can be used to perform partitioning only in that direction. For example, when the horizontal length of the block is below the specific value, preventing further partitioning, but the vertical length of the block exceeds the specific value, a binary tree partitioning can be performed only in the horizontal direction. Specifically, when the minimum length for partitioning a block is 4, for example... Figure 6 The 4×32, 4×16, and 4×8 blocks shown can only be partitioned horizontally (BT) at a 1:1 ratio using a binary tree structure. Since the block sizes (4×32, 4×16, 4×8) predetermine that only horizontal BT partitioning is possible, the block partitioning information can be encoded as "10" or "1". If no further partitioning is performed, the block partitioning information can be encoded as "0". Similarly, for 32×4, 16×4, and 8×4 blocks, the binary tree structure can only be partitioned vertically (BT) at a 1:1 ratio. Again, since the block sizes (32×4, 16×4, 8×4) predetermine that only vertical BT partitioning is possible, the block partitioning information can be encoded as "11" or "1". If no further partitioning is performed, the block partitioning information can be encoded as "0".

[0243] Alternatively, the maximum depth of executable block segmentation can be limited. For example, segmentation can be stopped when a certain depth is reached during recursive or hierarchical block segmentation. The aforementioned setting of the specific depth and encoding method can be applied to the aforementioned setting and encoding method of the minimum block size for executable block segmentation.

[0244] Figure 9a This is an exemplary illustration of a structure that uses QT cross segmentation as the primary segmentation structure and BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1, and / or TT horizontal 1:2:1 segmentation as secondary segmentation structures to divide blocks contained in the input image into multiple sub-blocks. Figure 9a In the diagram, the attached drawing number marked inside the block indicates the corresponding block, while the attached drawing number marked on the block boundary indicates the block divided by the corresponding boundary.

[0245] Figure 9b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 9a The diagram illustrates, for example, the block segmentation information related to the block segmentation structure shown in the figure. The block segmentation information may include tree structure information and / or segmentation pattern information.

[0246] exist Figure 9a In b, block partitioning using QT cross-segmentation is represented by solid lines, while block partitioning using BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1 and / or TT vertical 1:2:1 is represented by dashed lines.

[0247] like Figure 9a As shown, the basic block (depth = 0) 900 can be divided into four sub-blocks (depth = 1) of the same size using QT cross-splitting. In this case, the depth of each block represents its depth within the tree structure when performing the split. Each of the four sub-blocks (depth = 1) can be further split using QT cross-splitting. For example, the upper right sub-block (depth = 1) 901 and the lower left sub-block (depth = 1) 902 of the basic block (depth = 0) 900 can be split using QT cross-splitting. Sub-blocks that are no longer split using QT cross-splitting are equivalent to the terminal nodes of a quadtree. The terminal nodes of a quadtree can become the root nodes of a binary tree and / or a ternary tree. For the root node of a binary tree and / or ternary tree, partitioning can be performed using the binary tree structure (BT vertical 1:1 and / or BT horizontal 1:1 partitioning) and / or partitioning can be performed using the ternary tree structure (TT horizontal 1:2:1 and / or TT vertical 1:2:1 partitioning). Alternatively, partitioning using the binary tree structure and / or ternary tree structure can be omitted. When partitioning using the binary tree structure and ternary tree structure is not performed on the terminal node of a quadtree, the corresponding terminal node can be set as the encoding unit.

[0248] As described above, it is possible to further partition the terminal nodes of a quadtree structure using binary and / or ternary tree structures. That is, the terminal node of a quadtree can become the root node of a binary and / or ternary tree. For example, in... Figure 9aOf the four sub-blocks obtained by splitting the basic block (depth = 0) 900, the upper-right sub-block (depth = 1) 901 can be recursively or hierarchically split again using QT cross-splitting. For the four sub-blocks (depth = 2) 901-1 to 901-4 obtained by splitting the sub-block (depth = 1) 901 using QT cross-splitting, QT cross-splitting is not required. The four sub-blocks (depth = 2) 901-1 to 901-4 that do not undergo QT cross-splitting can each correspond to the terminal node of a quadtree. Furthermore, the four sub-blocks (depth = 2) 901-1 to 901-4 can each correspond to the root node of a binary tree and / or ternary tree. For blocks 901-1, 901-2, and 901-4 of the above four sub-blocks, BT splitting and / or TT splitting are not required. In the above case, sub-blocks (depth=2) 901-1, 901-2, and 901-4 can be set as encoding units respectively.

[0249] For block 901-3 of the above four sub-blocks, BT partitioning and / or TT partitioning can be performed. Figure 9a In the example, sub-block (depth = 2) 901-3 can be divided into two sub-blocks (depth = 3) through BT vertical 1:1 partitioning. For the two sub-blocks (depth = 3) obtained through BT vertical 1:1 partitioning, BT partitioning and / or TT partitioning can be performed again in a hierarchical and / or recursive manner. For example, as... Figure 9a As shown, in the two sub-blocks (depth = 3) obtained by performing a BT vertical 1:1 split on sub-block (depth = 2)901-3, the left block can be split into a TT horizontal 1:2:1 split.

[0250] Blocks can be split in a hierarchical and / or recursive manner using the methods described above, and the sub-blocks corresponding to leaf nodes that are no longer split can be set as encoding units.

[0251] Whether or not a block should be segmented can be determined in the decoder. The decoder can determine whether or not to segment a block, taking into account factors such as image characteristics, homogeneous regions, encoder and / or decoder complexity, and / or the number of bits required to signal the block segmentation information. Furthermore, as mentioned above, the smallest block size that will no longer be segmented can be predetermined or signaled.

[0252] As described above, this invention can utilize a quadtree structure (QT cross-segmentation) as the primary segmentation structure and a binary tree structure and / or a ternary segmentation structure (BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1, and / or TT vertical 1:2:1 segmentation) as secondary segmentation structures to segment the basic block 900. At this time, the basic block 900 can be set as the root node of the quadtree. The root node of the quadtree can be further decomposed using the quadtree in a recursive or hierarchical manner before reaching the end node. The end node of the quadtree can become the root node of a binary tree and / or a ternary tree. The root node of the binary tree and / or ternary tree can be further decomposed using the binary tree structure and / or ternary tree structure in a recursive or hierarchical manner before reaching the end node. When segmentation, including segmentation using quadtree, binary tree, and / or ternary tree structures, is no longer performed on the current block, the current block can be set as an encoding unit.

[0253] Figure 9b According to one embodiment of the present invention, for use with respect to Figure 9a The diagram illustrates the block segmentation information encoded / decoded within the block segmentation structure shown in the image. This block segmentation information can be composed of a combination of tree structure information and / or segmentation pattern information.

[0254] exist Figure 9b In the tree structure, the attached diagrams inside the parentheses are numbered as follows: Figure 9a The block number shown in the figure is an example of block segmentation information associated with the corresponding block, and the information represented by 0 and / or 1 is an instance of block segmentation information.

[0255] In the following description relating to the embodiment illustrated in FIG9, the information used to indicate whether a block is segmented using a quadtree structure is referred to as "primary segmentation information". The primary segmentation information can be encoded with a 3-bit length. The 3-bit length can be 1 bit. When the current block is segmented using a quadtree structure, the primary segmentation information of the current block can be encoded as "1". When the current block is not segmented using a quadtree structure, the primary segmentation information can be encoded as "0". That is, the information regarding whether the current block is segmented using a quadtree structure can be encoded using 1 bit of primary segmentation information.

[0256] Next, the block segmentation information used to represent the use of binary tree and / or ternary tree structures will be referred to as "sub-segmentation information." Sub-segmentation information can include at least one of the following: information indicating whether a sub-segmentation using a binary tree and / or ternary tree structure has been performed; information indicating which tree structure (binary or ternary) is used (tree structure information); and segmentation pattern information representing one or more block segmentation patterns among the various tree structures. Sub-segmentation information can be encoded with a 4-bit length. The 4-bit length can be 1 bit, 2 bits, or 3 bits. When the current block is a segmentation object using a sub-segmentation structure, 3-bit binary segmentation information as described below can be used. When the current block is split using a sub-segmentation structure based on a binary tree and / or ternary tree (BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1, and / or TT vertical 1:2:1 split), the sub-segmentation information can include information indicating the execution of the sub-segmentation, information indicating the tree structure (binary or ternary tree) used in the sub-segmentation, and / or information indicating the splitting pattern, each of which can be expressed in 1 bit. When the current block is split without using a sub-segmentation structure, the above sub-segmentation information can be expressed in only 1 bit, without needing to transmit information related to the tree structure and splitting pattern used in the sub-segmentation. Alternatively, information specifying one of all splitting patterns used in the sub-segmentation (e.g., BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1, and / or TT vertical 1:2:1 split) can be encoded, thereby using a single syntax element to encode information related to the tree structure and splitting pattern.

[0257] The block segmentation information of the current block can be encoded using the aforementioned main segmentation information of length 3 bits and the secondary segmentation information of length 4 bits, as shown in Tables 3 and 4 below. Table 3 shows the block segmentation information that blocks included in the main segmentation structure can have. Table 4 shows the block segmentation information that blocks included in the secondary segmentation structure can have. Blocks that can be included in both the main segmentation structure and the secondary segmentation structure can have the block segmentation information shown in Table 3.

[0258] Table 3

[0259]

[0260] As shown in Table 3 above, when performing a primary split (QT cross split) on the current block, it is not necessary to transmit information related to the secondary split. Therefore, the block split information of the block undergoing the primary split can be expressed as "1".

[0261] In the block segmentation information of blocks where a main segment is not executed, the information indicating whether a main segment has been executed can be expressed as "0". This information can be expressed using, for example, the first bit in the block segmentation information. However, this is merely one embodiment, and all embodiments in which the block segmentation information includes information indicating whether a main segment has been executed should be included within the scope of this invention.

[0262] In the block partitioning information of blocks where a primary partition is not performed, information indicating whether a secondary partition has been performed can also be included. For example, in the embodiment shown in Table 3, the execution of a secondary partition is expressed using the second bit in the block partitioning information. However, the embodiment shown in Table 3 is only one embodiment included in this invention, and the invention is not limited thereto.

[0263] In the embodiments shown in Table 3, when the block segmentation information of a block is "00", it indicates that the corresponding block is a block that will no longer be segmented.

[0264] In the block partitioning information of a block where a primary partition is not performed but a secondary partition is performed, information indicating whether a primary partition is performed and information indicating whether a secondary partition is performed can be included. For example, in the embodiment shown in Table 3, the first two bits of the block partitioning information are used to express this. That is, by setting the first two bits of the block partitioning information to "01", it can be indicated that a primary partition is not performed but a secondary partition is performed for the corresponding block. However, the embodiment shown in Table 3 is only one embodiment included in the present invention, and the present invention is not limited thereto.

[0265] For blocks undergoing secondary partitioning, information is needed to specify whether the partition is horizontal or vertical. In the embodiments shown in Table 3, this can be expressed using a third bit. For example, the third bit in the block partitioning information can be set to "0" for horizontal partitioning and to "1" for vertical partitioning. However, the embodiments shown in Table 3 are merely one embodiment included in this invention, and the invention is not limited thereto.

[0266] For blocks undergoing secondary partitioning, information is needed to specify whether a BT partition or a TT partition is required. In the embodiments shown in Table 3, this can be expressed using the fourth bit. For example, in a BT partition, the fourth bit in the block partitioning information can be set to "0", while in a TT partition, the fourth bit in the block partitioning information can be set to "1".

[0267] Alternatively, for example, in the embodiment illustrated in Figure 9, since the partitioning patterns used in the sub-partitions include four types—BT vertical 1:1, BT horizontal 1:1, TT horizontal 1:2:1, and / or TT vertical 1:2:1—the tree structure and partitioning pattern can be specified using the 2 bits calculated by ceil(log2(4)). That is, it can be seen that in the case described above, a tree structure and partitioning pattern need to be specified using 2 bits, and the information allocated for different block partitioning patterns can correspond to the third and fourth bits in the block partitioning information shown in Table 3.

[0268] The embodiment described in conjunction with Table 3 is only one of the many embodiments included in this invention, and the invention is not limited thereto. For example, the block segmentation information applicable to this invention only needs to include information indicating whether a main segmentation has been executed, information indicating whether a sub-segmentation has been executed, information specifying one of a plurality of sub-segmentation structures, and / or information specifying one of a plurality of segmentation forms. Therefore, the encoding order of the above information in the bitstream, or the order of its appearance in the bitstream, or the induced order from the bitstream, is not limited to the embodiment described in conjunction with Table 3. For example, the positions of the information specifying one of a plurality of sub-segmentation structures and the information specifying one of a plurality of segmentation forms can also be interchanged.

[0269] Furthermore, the present invention is not limited to expressing the case of performing a segmentation as "1" and the case of not performing a segmentation as "0", and bit values ​​can also be allocated and used in the opposite manner.

[0270] Furthermore, the present invention is not limited to expressing the vertical segmentation as "1" and the horizontal segmentation as "0", and bit values ​​can also be allocated and used in the opposite manner.

[0271] Furthermore, in the embodiments shown in Figure 9 and Table 3, the segmentation patterns of BT and TT are each limited to two types, but the segmentation patterns of BT and / or TT can also include combinations. Figure 5 and / or Figure 6 This describes some or all of the various segmentation patterns. For example, when there are three or more segmentation patterns, the segmentation pattern information can be expressed using more than 2 bits.

[0272] Table 4

[0273]

[0274] Table 4 above describes an embodiment of the block segmentation information that blocks included in the sub-segmentation structure may have. Since a primary segmentation (QT cross-segmentation) is not performed on blocks included in the sub-segmentation structure, the block segmentation information does not need to include information indicating whether a segmentation using the primary segmentation structure has been performed. Therefore, the bits other than the first bit "0" in the block segmentation information that blocks included in the primary segmentation structure (block segmentation information in Table 3) can constitute the block segmentation information that blocks included in the sub-segmentation structure may have (block segmentation information in Table 4).

[0275] The block segmentation information shown in Table 4 is only one embodiment of the block segmentation information applicable to the present invention. Various embodiments described in conjunction with Table 3 are also applicable to Table 4.

[0276] When dividing a basic block into multiple sub-blocks using the block segmentation method described in conjunction with Figure 9, Table 3, and Table 4, sub-blocks that are no longer segmented can be set as encoding units. The size and shape of each sub-block set as an encoding unit are as follows: Figure 8 As shown, it can be 2 n ×2 m A square or rectangle of a certain size.

[0277] Figure 10a This is an exemplary illustration of a structure that uses QT cross segmentation as the primary segmentation structure and BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1 segmentation as secondary segmentation structures to divide blocks contained in the input image into multiple sub-blocks. Figure 10a In the diagram, the attached drawing number marked inside the block indicates the corresponding block, while the attached drawing number marked on the block boundary indicates the block divided by the corresponding boundary.

[0278] Figure 10b According to one embodiment of the present invention, a tree structure is used to connect with... Figure 10a The diagram illustrates, for example, the block segmentation information related to the block segmentation structure shown in the figure. The block segmentation information may include tree structure information and / or segmentation pattern information.

[0279] exist Figure 10a as well as Figure 10b In the diagram, block partitioning using QT cross-splitting is represented by solid lines, while block partitioning using BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3 and / or BT vertical 3:1 is represented by dashed lines.

[0280] Except for the addition of BT horizontal ratios of 1:3, BT horizontal ratios of 3:1, BT vertical ratios of 1:3, and / or BT vertical ratios of 3:1 to the basis of Figure 7, the embodiment illustrated in Figure 10 is the same as the embodiment illustrated in Figure 7. Furthermore, except for the addition of BT horizontal ratios of 1:3, BT horizontal ratios of 3:1, BT vertical ratios of 1:3, and / or BT vertical ratios of 3:1 to the basis of Figure 9, replacing the TT segmentation, the embodiment illustrated in Figure 10 is the same as the embodiment illustrated in Figure 9. Therefore, in the description of Figure 10, parts that can be easily understood by referring to the descriptions in Figures 7 and 9 will be omitted.

[0281] In the embodiment illustrated in Figure 10, the main segmentation information can be encoded with a 5th bit length. The 5th bit length can be 1 bit. When the current block is segmented using a quadtree structure, the main segmentation information of the current block can be encoded as "1". When the current block is not segmented using a quadtree structure, the main segmentation information can be encoded as "0". That is, the information on whether the current block is segmented using a quadtree structure can be encoded using 1 bit of main segmentation information.

[0282] In the embodiment illustrated in Figure 10, the block segmentation information using BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1 is referred to as "sub-segmentation information." The sub-segmentation information can include at least one of the following: information indicating whether sub-segmentation was performed, and information indicating which segmentation form was used among BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1. The sub-segmentation information can be encoded with a 6th bit length. The 6th bit length can be one to four bits. When the current block is a segmentation object using a sub-segmentation structure, 4 bits of sub-segmentation information as described below can be used. When a block is partitioned using one of the following sub-partitioning structures: BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1, the sub-partitioning information can include information indicating the sub-partitioning and / or partitioning format information. The partitioning format information can include information indicating the direction (vertical or horizontal) of the sub-partition, information indicating the ratio (1:1 or 1:3), and / or, when the ratio is 1:3, information indicating whether it is 1:3 or 3:1. Each piece of information can be expressed using 1 bit.

[0283] The block segmentation information of the current block can be encoded using the aforementioned 5-bit main segmentation information and the aforementioned 6-bit secondary segmentation information as shown in Tables 5 and 6 below. Table 5 shows the block segmentation information that blocks included in the main segmentation structure can have. Table 6 shows the block segmentation information that blocks included in the secondary segmentation structure can have. Blocks that can be included in both the main and secondary segmentation structures can have the block segmentation information shown in Table 5.

[0284] Table 5

[0285]

[0286]

[0287] As shown in Table 5 above, when performing a primary split (QT cross split) on the current block, it is not necessary to transmit information related to the secondary split. Therefore, the block split information of the block undergoing the primary split can be expressed as "1".

[0288] In the block partitioning information of blocks where a main partition is not performed, the information indicating whether a main partition was performed can be expressed as "0". The information indicating whether a main partition was performed can be expressed using, for example, the first bit in the block partitioning information.

[0289] The block segmentation information for blocks that do not undergo primary segmentation can also include information indicating whether secondary segmentation has been performed. For example, in the embodiment shown in Table 5, the second bit in the block segmentation information is used to express whether secondary segmentation has been performed. Therefore, when the block segmentation information of a block is "00", it indicates that the corresponding block is a block that will no longer undergo segmentation.

[0290] In the block partitioning information of a block where a primary partition is not performed but a secondary partition is performed, information indicating whether a primary partition is performed and information indicating whether a secondary partition is performed can be included. For example, in the embodiment shown in Table 5, the first two bits of the block partitioning information are used to express this. That is, by setting the first two bits of the block partitioning information to "01", it can be indicated that a primary partition is not performed but a secondary partition is performed for the corresponding block.

[0291] For blocks that undergo secondary splitting, information is also needed to indicate whether the split is horizontal or vertical. In the embodiments shown in Table 5, this can be expressed using a third bit. For example, the third bit in the block splitting information can be set to "1" for vertical splitting and to "0" for horizontal splitting.

[0292] Next, information is needed to specify whether the horizontal or vertical segmentation ratio is 1:1 or 1:3. For example, in the embodiment shown in Table 5, the fourth bit in the block segmentation information can be set to "0" for a 1:1 segmentation, while the fourth bit in the block segmentation information can be set to "1" for a 1:3 segmentation.

[0293] When the segmentation ratio is 1:3, information is also needed to indicate whether it is a 1:3 segmentation or a 3:1 segmentation. For example, in the embodiment shown in Table 5, the fifth bit in the block segmentation information can be set to "0" in a 1:3 segmentation, while the fifth bit in the block segmentation information can be set to "1" in a 3:1 segmentation.

[0294] Alternatively, for example, in the embodiment illustrated in Figure 10, since the partitioning patterns used in the sub-partitions include six types such as BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1 partitioning, the tree structure and partitioning pattern can be specified using the 3 bits calculated by ceil(log2(6)). That is, it can be seen that in the case described above, a tree structure and partitioning pattern need to be specified using 3 bits, and the information allocated for different block partitioning patterns can correspond to the third, fourth, and fifth bits in the block partitioning information shown in Table 5.

[0295] The embodiment described with reference to Table 5 is only one of the many embodiments included in this invention, and the invention is not limited thereto. For example, the block segmentation information applicable to this invention only needs to include information indicating whether a main segment has been executed, information indicating whether a sub-segment has been executed, and / or information specifying one of a plurality of sub-segment structures. Therefore, the encoding order of the above information in the bitstream, or the order of its appearance in the bitstream, or the induced order from the bitstream, is not limited to the embodiment described with reference to Table 5. For example, the positions of the information specifying the direction of the sub-segment and the information specifying the proportion of the sub-segment can also be interchanged.

[0296] Furthermore, the present invention is not limited to expressing the case of performing a segmentation as "1" and the case of not performing a segmentation as "0", and bit values ​​can also be allocated and used in the opposite manner.

[0297] Furthermore, the present invention is not limited to expressing the vertical segmentation as "1" and the horizontal segmentation as "0", and bit values ​​can also be allocated and used in the opposite manner.

[0298] Furthermore, the present invention is not limited to expressing a 1:1 ratio as "0" and a 1:3 ratio as "1", and bit values ​​can also be allocated and used in the opposite manner.

[0299] Furthermore, the present invention is not limited to expressing a 1:3 ratio as "0" and a 3:1 ratio as "1", and bit values ​​can also be allocated and used in the reverse manner.

[0300] Table 6

[0301]

[0302]

[0303] Table 6 above describes an embodiment of the block segmentation information that blocks included in the sub-segmentation structure of the embodiment illustrated in FIG10 may have. Since no main segmentation (QT cross-segmentation) is performed on the blocks included in the sub-segmentation structure, the block segmentation information does not need to include information indicating whether or not a segmentation using the main segmentation structure has been performed. Therefore, the bits other than the first bit "0" in the block segmentation information that blocks included in the main segmentation structure (block segmentation information in Table 5) can constitute the block segmentation information that blocks included in the sub-segmentation structure may have (block segmentation information in Table 6).

[0304] The block segmentation information shown in Table 6 is only one embodiment of the block segmentation information applicable to the present invention. Various embodiments described in conjunction with Table 5 are also applicable to Table 6.

[0305] When dividing a basic block into multiple sub-blocks using the block segmentation method described in conjunction with Figure 10, Table 5, and Table 6, sub-blocks that are no longer segmented can be set as encoding units. The encoding unit determined according to the embodiment described in conjunction with Figure 10 can be a square or rectangle of size 2n × 2m (n and m are integers greater than 1). When assuming the minimum segment size for which segmentation can be performed is 4 × 3, such as... Figure 11 As shown, it can generate shapes such as 4×4, 8×4, 4×8, 12×4, etc., with a horizontal or vertical length not exceeding 2. n The encoding unit.

[0306] In the segmentation method described with reference to Figure 7, QT cross segmentation is used as the primary segmentation structure, and BT vertical 1:1 and / or BT horizontal 1:1 segmentation is used as the secondary segmentation structure. In the segmentation method described with reference to Figure 9, QT cross segmentation is used as the primary segmentation structure, and BT vertical 1:1, BT horizontal 1:1, TT vertical 1:2:1 and / or TT horizontal 1:2:1 segmentation are used as the secondary segmentation structures. Figure 11In the segmentation method described, QT cross-segmentation is used as the primary segmentation structure, and BT vertical 1:1, BT horizontal 1:1, BT horizontal 1:3, BT horizontal 3:1, BT vertical 1:3, and / or BT vertical 3:1 segmentation are used as secondary segmentation structures. Next, the method of segmenting blocks using the primary and secondary segmentation structures based on QT, BT, and / or TT will be further explained, along with the information and number of bits required to be included in the block segmentation information.

[0307] like Figures 4 to 6 As shown, QT segmentation can include three segmentation patterns. BT segmentation can include six segmentation patterns. TTT segmentation can also include six segmentation patterns. However, this is only based on... Figures 4 to 6 The diagrams shown are based on the basic segmentation patterns, but the number of segmentation patterns is not limited to this. For example, when more ratios are set, the number of segmentation patterns for QT, BT, and TT will also increase. For example, it is possible to set various ratios such as BT 1:4, BT 1:5, BT 2:5, TT 1:3:1, TT 1:4:1, and TT 1:2:2.

[0308] exist Figures 4 to 6 The diagram illustrates a total of 15 segmentation patterns, one of which can be used as the primary segmentation structure. For example, QT cross splitting can be used as the primary segmentation structure, but the primary segmentation structure is not limited to QT cross splitting.

[0309] When QT cross-splitting is used as the primary splitting structure, all or some of the remaining 14 splitting patterns can be used as secondary splitting structures.

[0310] Information relating to which segmentation form to use as the primary segmentation structure and / or which segmentation form to use as the secondary segmentation structure can be transmitted through at least one of the following: sequences, images, strips, parallel blocks, and basic blocks, as described above. Alternatively, it can be determined in advance in the encoder and decoder. Or, it can be induced based on encoding parameters and / or internal variables induced during encoding / decoding.

[0311] As mentioned above, 1 bit is needed to indicate whether a segmentation using the main segmentation structure has been performed.

[0312] As mentioned above, 1 bit is needed to indicate whether a segmentation using the sub-segmentation structure has been performed.

[0313] Assuming there are n possible segmentation patterns that can be used as secondary segmentation structures, the information indicating which segmentation pattern to use among the n available segmentation patterns can be expressed using ceil(log2(n)). Here, ceil() represents the floor function.

[0314] Therefore, the block segmentation information of a block belonging to the main segmentation structure can include information indicating whether the main segmentation has been executed (e.g., 1 bit of information), information indicating whether the sub-segmentation has been executed when the main segmentation has not been executed (e.g., 1 bit of information), and / or information indicating one of the n available segmentation forms when the sub-segmentation has been executed (e.g., ceil(log2(n)) bits of information). This information is shown in Table 7 below.

[0315] Table 7

[0316]

[0317]

[0318] Furthermore, the block segmentation information of blocks belonging to the sub-segment structure can include information indicating whether a sub-segment has been executed (e.g., 1 bit of information) and / or information indicating one of the n available segmentation forms when a sub-segment is executed (e.g., ceil(log2(n)) bits of information). This information is shown in Table 8 below.

[0319] Table 8

[0320]

[0321] The above is combined with Figures 7 to 7. Figure 11The various embodiments described are also applicable to general segmentation methods utilizing primary and / or secondary segmentation structures. Furthermore, as other embodiments of segmentation methods utilizing primary and / or secondary segmentation structures, limitations can be imposed on the number of primary segmentations, their depth, and / or block sizes (maximum and / or minimum). Additionally, limitations can also be imposed on the number of secondary segmentations, their depth, and / or block sizes (maximum and / or minimum). For example, it can be limited to a method where only one primary segmentation is applied initially, and only secondary segmentations are performed in subsequent processes. Alternatively, it can be limited to a method where multiple primary segmentations can be applied, but only one secondary segmentation can be applied. Alternatively, it can be limited to a method where only n primary segmentations and only m secondary segmentations can be applied. In this case, n and m can be integers greater than or equal to 1. Information related to the above-mentioned additional limitations on primary and secondary segmentation can be transmitted through at least one of the following units: sequence, image, stripe, parallel block, and basic block. Alternatively, information related to the above-mentioned additional limitations can be determined in advance in the encoder and decoder. Alternatively, information related to the aforementioned additional restrictions can be determined through encoding parameters and / or internal parameters used during the encoding / decoding process.

[0322] The coding parameters and / or internal parameters can include information related to the block size, the block segmentation depth, the luminance and / or chrominance components, the inter-frame mode, the intra-frame mode, coded block markers, quantization variables, motion vectors, information related to the reference image, and / or information related to whether or not PCM mode is used for coding. Furthermore, the aforementioned coding parameters and / or internal variables can include not only information related to the current block but also information related to adjacent blocks.

[0323] The segmentation method that uses a tree structure to hierarchically divide blocks can include segmentation methods that distinguish between primary and secondary segments, as well as segmentation methods that distinguish between primary and secondary segments of a unit.

[0324] A segmentation method that distinguishes between primary and secondary segments can be defined as a multi-level segmentation approach. For example, it can be defined using the methods shown in Figure 7 to... Figure 11Examples of various segmentation methods will be given below. In a multi-level segmentation approach, the primary segmentation structure can determine one segmentation structure, and the secondary segmentation structure can determine more than one segmentation structure. The secondary segmentation structure can include multiple segmentation structures, and different tree structures can be used to perform secondary segmentation. For example, secondary segmentation structures using BT and / or TT can be employed. In this case, BT and TT can be used as secondary segmentation structures in any order. That is, a segmentation using TT can be performed after a segmentation using BT, followed by another segmentation using BT. Alternatively, the order of application between BT and TT can be specified. For example, BT can be applied preferentially to the leaf nodes of the primary segmentation structure, followed by TT to the leaf nodes of the secondary segmentation structure using BT. In this case, if the secondary segmentation using BT is not performed on the leaf nodes of the primary segmentation structure, then the secondary segmentation using TT can be applied.

[0325] A segmentation method that does not distinguish between primary and secondary segmentations can be defined as a single segmentation hierarchy. For example, it can combine... Figures 4 to 6 Some or all of the various segmentation patterns described are used as the block segmentation structure in any order. In a single segmentation hierarchy using n segmentation patterns, 1 to ceil(log2(n))+1 bits are needed to represent the block segmentation information of each node in the tree structure. Therefore, the number of bits required to encode the block segmentation information may increase when using a single segmentation hierarchy compared to using a multi-segmentation hierarchy.

[0326] Various methods for performing block segmentation and various embodiments for encoding block segmentation information have been described above. In the above embodiments, the information to be transmitted from the encoder to the decoder can be transmitted at least at one of the following levels: sequence level, image level, strip level, parallel block level, and basic block level. The encoded information can include information related to whether a single segmentation hierarchy or a multi-segmentation hierarchy is applicable. Furthermore, the encoded information can include information related to the segmentation pattern that can be used as a primary segmentation structure and / or as a secondary segmentation structure. In addition, the encoded information can include information related to the number / depth / block size of primary and / or secondary segments that can be performed. The above-mentioned encoded information can also be set in advance in the encoder and decoder. Alternatively, the above information can also be induced by other encoding parameters or internal variables.

[0327] As described above, by segmenting the basic block using a segmentation structure including a quadtree, binary tree, and / or ternary tree structure, multiple sub-blocks that will no longer be segmented can be identified. These sub-blocks, which will not be segmented again, can be used as units for prediction, transformation, and / or quantization by setting them as coding units. In the coding step, the prediction signal can be obtained by performing inter-frame prediction or intra-frame prediction on each coding unit. The residual signal can be calculated by the difference between the obtained prediction signal and the original signal of the coding unit. For the calculated residual signal, a transformation can be performed to concentrate energy.

[0328] Because the encoding units applicable to this invention can be squares or rectangles of various sizes, such as 4×4, 4×8, 8×4, 8×8, 16×4, and 4×16, it is necessary to define square transformations and non-square transformations in order to transform the residual signals of the encoding units applicable to this invention. The formulas used in the transformation process are shown below.

[0329] [Formula 1]

[0330] Y = AXB T

[0331] X is a two-dimensional residual signal of size m×n, A represents a one-dimensional n-point transformation in the horizontal direction, and B... T This represents a one-dimensional m-point transformation in the vertical direction. (B) T Let m represent the transpose of B. m and n can have different sizes or the same size. Furthermore, A and B can have the same transform basis or different transform bases. Y represents the transform block obtained by transforming the residual signal block X.

[0332] The formula used in the inverse transformation of the transformed block Y is shown below.

[0333] [Formula 2]

[0334] X = A T YB

[0335] In Equations 1 and 2 above, the vertical and horizontal transformations yield similar results regardless of their execution order. However, when the transform coefficients have a limited bit precision, such as 16 bits, the vertical and horizontal transformations should be performed in the same order in both the encoder and decoder. This is because limited bit precision can lead to data loss during computation. To prevent potential mismatches in the encoder / decoder, the vertical and horizontal transformations should be performed in the same order in both systems.

[0336] For the transformation formula in Equation 1 and the inverse transformation formula in Equation 2 to hold, the transformation basis needs to satisfy the separation property and orthogonality. These constraints are necessary because the computational cost will increase from O(n^2) to O(n^2). 4 Reduced to O(n) 3 And satisfy A T =A -1 .

[0337] Types of basis vectors (kernels) that can be used as transform basis vectors include DCT-II (Discrete Cosine Transform type-II), DCT-V, DCT-VIII, DST-I (Discrete Sine Transform type-I), and DST-VII. In practical encoders / decoders, to improve computational speed and accuracy, the transform basis can be approximated and rounded.

[0338] When the size of the coding unit is m×n, a one-dimensional n-point transformation and a one-dimensional m-point transformation need to be performed based on the separation property. Equation 3 below is an example of a one-dimensional DCT-II transformation basis applicable to all coding unit sizes when 4<=m, n<=64.

[0339] [Formula 3]

[0340]

[0341] To be used in the encoder / decoder transformation process, it is possible to multiply by each element of the real-valued transformation basis. Then, the integer transformation basis is generated by rounding the integers. Figure 12 This is a schematic diagram illustrating, as an example, the real basis of DCT-II that can be used in the transformation, and the integer basis obtained by multiplying the real basis by a specific value. Figure 12 The diagram illustrates the integer transform basis obtained by applying a K value of 64. While increasing the K value can improve transform accuracy, it also increases memory consumption in the encoder / decoder; therefore, the K value should be chosen carefully based on the specific situation. Conversely, the inverse transform should not be divided by 64. To reduce computation by using shift operations, we can use K=2. K (K is an integer greater than 1).

[0342] Figure 13 This is a schematic diagram illustrating, as an example, the real number transformation basis of DST-VII that can be used in transformations, and the integer basis obtained by multiplying the real number transformation basis by a specific value. (See attached diagram.) Figure 12 The explanation provided states that when the size of the encoding unit is m×n, a one-dimensional n-point transformation and a one-dimensional m-point transformation need to be performed. For example... Figure 13 As shown, when 4 <= m and n <= 64, the one-dimensional DST transformation can be applied to encoding units of all sizes.

[0343] Figure 14 This is a flowchart illustrating an in-loop filter applicable to one embodiment of the present invention. The in-loop filter may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). Alternatively, the in-loop filter may include filtering suitable for reconstructed samples and / or predicted samples.

[0344] Figure 15 For the purpose of execution Figure 14 The diagram illustrates the use of two blocks adjacent to the block boundary and their internal pixels for in-loop filtering.

[0345] In-loop filtering is a method used to reduce blocking artifacts that occur at block boundaries during block-based transformations and quantization. In-loop filtering allows for horizontal filtering of vertical block boundaries of any size followed by vertical filtering, or vice versa. Because the result of the first filter becomes the input for subsequent filters, the filtering order in the encoder / decoder should be identical. Different filtering orders can lead to mismatches between filtered pixels during encoding / decoding.

[0346] The size of the in-loop filter can be preset in the encoder / decoder or determined using information from the bitstream signaling. For example, the size can be 4×4, 8×8, 16×16, etc., and the horizontal and vertical lengths can also be different.

[0347] In-loop filtering can be performed selectively, thus allowing for a preliminary determination of whether or not to perform it. For example, the determination of whether or not to perform filtering can be based on at least one of the following: information indicating whether or not filtering is performed, the BS (Boundary Strength) value, and at least one variable value related to the amount of change in adjacent pixels at the block boundary. The information indicating whether or not filtering is performed can be signaled through at least one of the following: sequence, image, strip, parallel block, and block level.

[0348] In such Figure 14 In the method shown, to determine whether filtering should be performed, the encoding information of at least one block adjacent to the boundary of the filtered block can be used. The encoding information can include the block's prediction mode, motion information, and / or transform coefficients, etc.

[0349] In step S1401, a Boundary Strength (BS) value representing the strength of the block boundary can be calculated for the boundaries of the Prediction Unit (PU) and / or Transform Unit (TU) (S1401). The Prediction Unit and / or Transform Unit represent two types of blocks adjacent to the block boundary, and the block type used to calculate the BS value is not limited to these. For example, the BS value can also be calculated for the boundaries of the Coding Unit (CU) in the manner described later.

[0350] For example, when at least one of two blocks adjacent to a block boundary is coded in intra-prediction mode, the BS value can be set to a first value. The first value can be, for example, a constant of 2 or greater than 2. When both blocks adjacent to a block boundary are coded in inter-prediction mode, the BS value can be set to a second value based on information such as the motion vector values ​​of the two blocks, the number of motion vectors, whether the reference picture is the same, and / or whether at least one of the two blocks has a non-zero quantized residual coefficient. The second value can be a constant (e.g., 0, 1, etc.) less than the first value. The BS value is not limited to the first and second values; for example, it can have multiple levels of values ​​such as a third value, a fourth value, etc., by subdividing the BS value setting criteria. Next, an embodiment with a BS value of 0, 1, or 2 will be described based on this. However, as mentioned above, since the BS value can also be greater than the above values, the following embodiment is not limited to this one.

[0351] In step S1402, it can be determined whether the BS value is 0. If the BS value is 0, filtering can be omitted (S1403).

[0352] When the BS value is not 0, step S1404 can be executed. In step S1404, a delta value can be calculated to measure the pixel change of pixels adjacent to the block boundary. The delta value can be calculated in a specific block unit. At this time, all or part of the lines (rows or columns) belonging to the specific block unit can be used. The position and / or number of the aforementioned part of the lines can be fixed values ​​agreed upon in advance in the encoder / decoder, or values ​​that vary according to the size / shape of the block. In addition, one or more pixels located on the lines can be used, where the multiple pixels can be consecutively arranged pixels or non-consecutive pixels spaced at a certain interval. For example, the delta value can be used in every 4×4 block unit. Figure 15 The pixel changes in the areas represented by light gray within the block are calculated.

[0353] exist Figure 15 In the diagram, the thicker vertical line in the center represents the boundary between blocks P and Q on the left and right sides. Block P and each grid cell within block P represent a pixel. The symbols marked inside each grid cell are indices used to indicate the pixel corresponding to that grid cell.

[0354] When calculating the delta value, it can be based on the change in brightness value of pixels surrounding the block boundary. This change in brightness value can be calculated based on the change in brightness value of pixels centered on the block boundary and located to its left and / or right (or above and / or below) it. For example, when calculating the delta value, it is possible to utilize... Figure 15 The pixels of blocks P and Q shown in the diagram are calculated according to Formula 4 below. Formula 4 is a formula used to calculate the edge strength on the corresponding block boundaries.

[0355] [Formula 4]

[0356] dp0=abs(p2,0-2*p1,0+p0,0)

[0357] dp3=abs(p2,3-2*p1,3+p0,3)

[0358] dq0=abs(q2,0-2*q1,0+q0,0)

[0359] dq3=abs(q2,3-2*q1,3+q0,3)

[0360] dp = dp0 + dp3

[0361] dq = dq0 + dq3

[0362] dpq0=dp0+dq0

[0363] dpq3=dp3+dq3

[0364] delta = dpq0 + dpq3

[0365] Furthermore, in step S1404, two critical values, namely the β value and t, can be induced. C Values. β value and t C The value can be derived based on the quantization parameter (QP) and Table 3 below. This quantization parameter can be derived from the quantization-related parameters of two blocks adjacent to the block boundary, or at least one of the two blocks. The Q values ​​in Table 9 are derived from the quantization parameter. After deriving the Q value from the quantization parameter, β' and t can be determined with reference to Table 9. C 'Value. β value and t C The value can be determined based on the set β' and t. C 'Value is the basis for decision.'

[0366] Table 9

[0367]

[0368] β value and / or t C The value can be used to determine whether filtering is performed. Alternatively, the β value and / or t... C The value can also be used to select the filter type when filtering is determined to be performed. The filter type can vary depending on the range of pixels to which the filter is applied, the filter strength, etc. For example, the filter type can include strong filtering and weak filtering. However, the filter type is not limited to this, and more than three filter types with different filter strengths can be used, for example. Alternatively, as described later, when different filters are applied based on the shape, size, and / or characteristics of two blocks adjacent to the block boundary, independent filter types can be set separately. Alternatively, the β value and / or t C The value can be used to clip the filtered pixels during the filtering process.

[0369] In step S1405, the delta value and the β value can be compared. Based on the comparison result in step S1405, it can be determined whether or not filtering should be performed. If the delta value is not less than the β value, filtering can be not performed (S1403). If the delta value is less than the β value, it can be determined that filtering should be performed and the appropriate filter type for the corresponding block boundary can be determined. Alternatively, it can be determined that filtering should not be performed if the delta value is less than the β value, and that filtering should be performed if the delta value is not less than the β value. Figure 14 In the illustrated embodiment, the filtering type includes two types: strong filtering and weak filtering. In step S1406, it can be determined which type of filtering, strong filtering or weak filtering, should be performed. This selection can be determined by comparing the change in pixel values ​​within two blocks adjacent to the block boundary with a specific threshold value. The change in pixel values ​​within the block used in this case can be the change used in the delta value calculation. Alternatively, the change in pixel values ​​between pixels within each block adjacent to the block boundary can be compared with a specific threshold value. Alternatively, the change in pixel values ​​between two or more adjacent pixels centered on the block boundary can be compared with a specific threshold value. This specific threshold value can be a predefined constant value, a value obtained through bitstream signaling, a value determined based on characteristics such as the block shape and size, the aforementioned β value, and t. C At least one or more of the values. Alternatively, it is also possible to use the values ​​of β and / or h or t. C The value is scaled. Alternatively, the change between the aforementioned pixel values ​​can be compared with the aforementioned threshold after scaling using the threshold. For example, whether to perform strong or weak filtering can be determined based on whether the conditions of Equation 5 and / or Equation 6 below are met.

[0370] [Formula 5]

[0371] Condition A: 2*dpq0<β / 4

[0372] Condition B: abs(p3.0-p0.0)+abs(q3.0-q0.0)<β / 8

[0373] Condition C: abs(p0.0-q0.0)<(5*t) C +1) / 2

[0374] [Formula 6]

[0375] Condition A: 2*dpq3<β / 4

[0376] Condition: abs(p3.3-p0.3)+abs(q3.3-q0.3)<β / 8

[0377] Condition C: abs(p0.3-q0.3)<(5*t) C +1) / 2

[0378] Formula 5 above is related to Figure 15 The formula related to the first column in the above formula 6 is... Figure 15 The formula related to the fourth column in the code. In step S1407, strong filtering can be performed on the corresponding block boundary when the conditions of formula 5 and / or formula 6 above are met simultaneously. Otherwise, weak filtering can be performed in step S1408. As explained above regarding the calculation of the delta value, when determining the filtering type, all or part of the lines (rows or columns) belonging to a specific block unit adjacent to the block boundary can be used. The position and / or number of the aforementioned part of the lines can be fixed values ​​agreed upon in advance in the encoder / decoder, or values ​​that vary according to the size / shape of the block. In addition, one or more pixels located on the line can be used, where the multiple pixels can be consecutively arranged pixels or non-consecutive pixels spaced at a certain interval. When the shapes and / or sizes of two blocks adjacent to the block boundary are different, the indices (e.g., p0,0, p1,0, p2,0, etc.) used in this specification to specify the position of the pixels can be adaptively changed.

[0379] When performing strong filtering (S1407), two blocks adjacent to the block boundary can be used simultaneously, and filtering can be performed on m pixels (where m is a constant of 3 or greater than 3) in each block. When performing weak filtering (S1408), two blocks adjacent to the block boundary can be used simultaneously, or one block can be used to perform filtering on n pixels (where n is a constant less than m) in each block. Figure 14 The filtering application range illustrated in the lower part of steps S1407 and S1408 is only an illustrative example for the case where m is 3 and n is 2; m and n are not limited to this. Furthermore, Figure 14 The filtering method illustrated in the figure is not limited to this. That is, it is possible to skip individual steps as needed, or to merge two or more steps into one step, or to split one step into two or more steps. In addition, it is also possible to change the execution order of some steps.

[0380] When dividing basic blocks using a quadtree structure and / or a binary tree structure according to the present invention, such as Figure 3b , Figure 7a As shown, the shape and size of the block, which serves as the encoding unit, can be changed arbitrarily, and it can be divided into encoding units containing rectangular shapes.

[0381] When dividing blocks using a quadtree and / or binary tree structure according to the present invention, blocks that are no longer divided can serve as encoding units for performing prediction, transformation, and / or quantization. That is, basic blocks can be divided into encoding units of various sizes with square or rectangular shapes, and each encoding unit does not need to be divided again for performing prediction or transformation; it can be used directly as a prediction unit and / or transformation unit.

[0382] Because the encoding unit can be directly used as the transform unit, the transform unit can have various sizes when the size of the encoding unit changes. Consequently, the blockiness occurring at the boundaries of the transform unit will vary more depending on the shape, size, and / or characteristics of the block. Therefore, the strength and / or applicability of the in-loop filter need to be adjusted based on the shape, size, and / or characteristics of the blocks adjacent to the block boundary.

[0383] Figure 16 This is a schematic diagram illustrating the structure of blocks segmented using quadtree and / or binary tree structures to which this invention applies, and the block boundaries where in-loop filtering is applied. Figure 16 The block boundaries marked with thick lines are exemplary illustrations of block boundaries for which in-loop filtering is applicable.

[0384] Considering the complexity of the encoder and decoder, filtering can be omitted for blocks with a width or height of 4 (e.g., 4×8, 8×4, 4×4, etc.). That is, for blocks in... Figure 16 Block boundaries marked with black lines in the middle can be exempted from filtering.

[0385] Figure 17 This is a flowchart illustrating filtering to which another embodiment of the present invention is applicable. The following will be combined with... Figure 17 According to Figure 16 The method shown is applicable to the modification of loop filtering methods in images containing heterogeneous blocks. Here, heterogeneous blocks refer to blocks with different widths and heights.

[0386] In the Figure 17 In explaining each step, and in relation to Figure 14 The same parts of the descriptions of each step will be omitted. The following will describe... Figure 17 The methods and Figure 14 The differences between the methods described are explained in detail. Figure 14 In step S1401, the BS value was calculated based on PU and TU, but in Figure 17In step S1701, the BS can be calculated based on two coding units (CUs) adjacent to the block boundary.

[0387] The BS value, as described above, can be the first value and / or the second value or a larger value. In step S1702, filtering can be not performed (S1703) or filtering can be performed according to step S1704, depending on whether BS is 0.

[0388] In step S1704, the delta value, β value, and t can be calculated / induced. C Value. In step S1405, the delta value and the β value can be compared. When it is determined in step S1705 that the delta value is less than the β value, it can be confirmed whether at least one block adjacent to the block boundary is a heterogeneous block (S1706). When both blocks adjacent to the block boundary are square blocks, it can be determined according to Figure 14 Steps S1406 to S1408 in the process perform filtering (S1707). When at least one of the two blocks adjacent to the block boundary is a heterogeneous block, the process after S1708 can be performed according to another embodiment of the present invention. Alternatively, when the two blocks adjacent to the block boundary have different shapes and / or sizes, the process after S1708 can also be performed according to another embodiment of the present invention. In step S1708, a filtering type can be selected according to a specific criterion. The filtering type can include, for example, strong filtering or weak filtering, but as mentioned above, it is not limited to this. When strong filtering is selected, strong filtering will be performed in step S1709, and when weak filtering is selected, weak filtering will be performed in step S1710. As illustrated in the exemplary diagrams in steps S1709 and S1710, the range of filtering applicable to the two blocks adjacent to the block boundary can be different. The applicable range of filtering will be explained in detail later. Figure 17 The filtering method illustrated in the figure is not limited to this. That is, it is possible to skip individual steps as needed, merge two or more steps into one step, or split one step into two or more steps. Furthermore, the execution order of some steps can be changed. Moreover, without combining... Figure 17 Within the scope of conflicts described in the embodiments, the same combination is also applicable. Figure 14 Explanation of the procedure.

[0389] Figure 18 According to Figure 17 The schematic diagram illustrating step S1709 in the example demonstrates the use of strongly filtered pixels. For example... Figure 18As shown, the two blocks adjacent to the block boundary (marked by red lines) can be 8×8 and 16×8 respectively. The 16×8 block is a non-homogeneous block. (This is in conjunction with...) Figure 14 The explanation provided demonstrates the ability to determine whether to apply strong or weak filtering to block boundaries based on the quantization parameters of two blocks or at least one of the two blocks. When at least one of the two blocks adjacent to the block boundary is a non-homogeneous block, the width and / or height of the two adjacent blocks can be considered simultaneously. Specifically, considering the width and height of the two adjacent blocks, namely block P and block Q, filtering can be performed on more pixels around the block boundary in the larger block. This is because the blockiness caused by transformation and quantization is not only related to the quantization parameters but also affected by the size of the two blocks adjacent to the block boundary. For example, the AC component that excludes the DC component of a block in the relatively larger block among the two blocks adjacent to the block boundary may be quantized and deleted due to the quantization step (Qstep). In this case, the corresponding position where the AC part is deleted within the relatively larger block may be adjacent to the DC part of the adjacent block. This can lead to severe blockiness. A filtering method applicable to another embodiment of the present invention can perform filtering while taking into account the shape, size and / or characteristics of the blocks in order to reduce the mismatch phenomenon described above.

[0390] The processes of inducing the boundary strength (BS) of block boundaries, determining whether to perform filtering, and selecting strong or weak filtering can be performed based on the size of the two blocks adjacent to the block boundary, or they can be performed by combining... Figure 14 The method described is to be executed.

[0391] like Figure 18 As shown, when two adjacent blocks are 8×8 and 16×8 in size, respectively, and strong filtering is applied to the corresponding block boundaries, more pixels in the larger block can be filtered. For example, filtering can be applied to three pixels (b, c, d) in the 8×8 block and to four pixels (e, f, g, h) in the relatively larger 16×8 block. The number of pixels filtered is not limited to this; in the relatively larger block, more pixels can be filtered compared to the relatively smaller block. Figure 18 The filtering of the target pixels illustrated in the figure can be performed using the average and / or weighted average of the pixel values ​​within two adjacent blocks, or by calculating an intermediate value based on this. For example, filtering can be performed using the following formula 7.

[0392] [Formula 7]

[0393] b' = (2a + 2b + c + d + c + f) / 8

[0394] c' = (b + c + d + e + f) / 5

[0395] d' = (b + 3c + 5d + 3e + f + 8) / 16

[0396] e′=(c+3d+5e+3f+g+8) / 16

[0397] f′=(c+d+c+f+g) / 5

[0398] g′=(c+d+e+f+g+2g+2h) / 8

[0399] h' = (2d + 2e + f + g + h + i + 4) / 8

[0400] In Formula 7 above, the parts marked with letters a, b, c, d, e, f, g, h, i represent the reconstructed pixel values ​​obtained by adding the prediction signal and the decoded residual signal, while the letters marked with b', c', d', e', f', g', h' represent the modified pixel values ​​after filtering using the filtering coefficients.

[0401] Figure 19 According to Figure 17 The schematic diagram illustrating step S1710 in the example uses pixels with strong filtering. For example... Figure 19 As shown, when the sizes of two blocks adjacent to the block boundary are 8×8 and 16×8 respectively, and weak filtering is performed on the corresponding block boundaries, for example, filtering can be performed on two pixels (c, d) in the 8×8 block and filtering can be performed on three pixels (e, f, g) in the relatively larger 16×8 block.

[0402] The filtering process can be performed by calculating a Δ value based on the difference between pixel values ​​within two blocks adjacent to the block boundary. In this case, a weighted value with a larger difference between the two pixel values ​​adjacent to the block boundary can be used to calculate the Δ value. In this case, the difference between the two pixel values ​​adjacent to the block boundary is directly proportional to the Δ value. Alternatively, the Δ value can also be calculated using the difference between pixel values ​​not directly adjacent to the block boundary. In this case, the difference between the pixel values ​​not directly adjacent to the block boundary is inversely proportional to the Δ value. The Δ value can be calculated using one of the aforementioned differences between pixel values, and the calculated value can be scaled using a predefined constant, a value determined based on characteristics such as the block shape or size, and / or a signaling value transmitted via the bitstream. For example, the Δ value can be calculated using Formula 8 below. The filtered pixel value can be calculated by adding or subtracting the calculated Δ value from the pixel value being filtered.

[0403] [Formula 8]

[0404] Δ=(9*(q0-p0)-3*(q1-p1)+8)>>4

[0405] Weak filtering can be performed on a single block or on two blocks simultaneously. When applying weak filtering to the block boundary of two adjacent blocks where at least one block is a heterogeneous block, the pixels at the block boundary, i.e., d and e, can be filtered by adding or subtracting their respective pixel values ​​from the difference value Δ calculated using Equation 8. For pixels (f, g) located inside a heterogeneous block, the average and / or weighted average of the pixel values ​​inside the two adjacent blocks can be used, or an intermediate value can be calculated based on this. For example, filtering can be performed using the peripheral pixel values ​​of the block boundary using Equation 9 below.

[0406] [Formula 9]

[0407] f'=(e+5f+2g)>>3

[0408] g'=(2f+4g+2h)>>3

[0409] exist Figure 19 In the case where the left side of the block boundary is surrounded by a 16×8 block, the above formula 9 can be applied to both pixels b and c simultaneously.

[0410] Filtering according to another embodiment of the present invention can be performed when at least one of the two blocks located at the block boundary is a heterogeneous block. Furthermore, filtering according to another embodiment of the present invention can be performed when the two blocks located at the block boundary are of different sizes. When the two blocks adjacent to the block boundary are of the same size or both blocks are square, combining can be performed. Figure 14 The filtering is explained.

[0411] Figure 20 This is a schematic diagram illustrating, as an example, the pixel range to which the filter is applied when weak filtering is performed according to the present invention. Figure 20 In the diagram, thick lines represent block boundaries. The grid cells inside the thick lines represent the pixels within each block. Figure 20 For example, the smallest block size for which filtering is performed is 8×8. Therefore, filtering can be omitted for the boundaries of blocks with a horizontal and / or vertical size of 4. Figure 20 In the middle, the parts marked in dark gray represent applicable combinations. Figure 14 The pixel range of the filter is explained. Figure 20In the diagram, the areas marked in light gray represent the internal regions of a non-homogeneous block (16×8), indicating applicable combinations. Figure 17 The range of pixels for which the filter is described.

[0412] The filtering unit 150 of the video encoding apparatus and the filtering unit 240 of the video decoding apparatus to which the present invention is applied may further include a corner outlier filter for filtering corner outliers, i.e., the objects to which the present invention is applied. The corner outlier filter may be located before or after the deblocking filter, before or after the offset correction unit, or before or after the ALF (Adaptive Loop Filter). Alternatively, the filtering to which the present invention is applied may be performed as part of in-loop filtering, or may be performed on pixels used as references for intra-frame prediction or inter-frame prediction.

[0413] Figure 21 (a) is a schematic diagram illustrating a corner outlier point that is the filtering object of a corner outlier filter applicable to one embodiment of the present invention. Figure 21 (b) is to Figure 21 The diagram illustrates the pixel values ​​of the pixels in the 2×2 region centered on the intersection of (a). Figure 21 (c) is a schematic diagram illustrating the indexes of pixels used to indicate the positions of pixels when detecting and filtering outliers at corner points. Figure 21 of (a), Figure 21 (b) and Figure 21 In (c), the parts marked with thick lines represent the boundaries between blocks, and each grid corresponds to a pixel.

[0414] like Figure 21 As shown in (a), in an image decoded in block units, the corner points of four blocks 2101, 2102, 2103, and 2104 can intersect around a single intersection point 2100. The sizes of the four blocks 2101, 2102, 2103, and 2104 can be different from each other, or they can be partially or completely the same. The four blocks 2101, 2102, 2103, and 2104 can be unit blocks for prediction, quantization, or transformation, respectively. The quantization parameters used in the quantization (or inverse quantization) of the four blocks 2101, 2102, 2103, and 2104 can be different from each other, or they can be partially or completely the same.

[0415] The decoded image described above can contain image regions of various shapes, and the boundaries (edges) of these image regions may not coincide with the boundaries of the encoding / decoding unit, i.e., the block. For example, in Figure 21In (a), the image region 2105 of the diagonal section can exist simultaneously in multiple blocks 2101, 2102, 2103, and 2104, while one of the blocks 2101 can exist only in the corner portion. At this time, since the four corner pixels centered at the intersection point 2100 are adjacent pixels contained in the same image region 2105, it is possible to predict that they have similar pixel values.

[0416] However, because encoding / decoding processes such as prediction, quantization, and transformation are performed in block units, significant pixel value differences may occur between the aforementioned four adjacent corner pixels within the same image region 2105 of the decoded image. For example, in Figure 21 In (a), the pixel value of the corner pixel belonging to the upper left block 2101 among the four corner pixels adjacent to the intersection point 2100 may be significantly smaller or larger than the pixel value of the remaining three corner pixels.

[0417] The corner outlier filter of the present invention, when four blocks 2101, 2102, 2103, and 2104 in a decoded image intersect at a central intersection point 2100, can detect corner pixels whose pixel values ​​differ significantly from those of other corner pixels among the four corner pixels adjacent to the intersection point 2100, and filter them accordingly. That is, a corner outlier refers to a corner pixel containing noise that, due to quantization error or prediction error, causes a significant difference between the corner pixel value of a block in the reconstructed image and the corner pixel values ​​of its adjacent blocks. Furthermore, the corner outlier filter of the present invention can include pixels whose pixel values ​​differ significantly from those of their surrounding adjacent pixels, as well as the aforementioned surrounding pixels.

[0418] Figure 21 (b) is to Figure 21 An enlarged view of the 2×2 region centered at intersection 2100 in (a). Figure 21 In (b), the numbers contained in each grid exemplarily represent the pixel values ​​of the pixels contained in the block.

[0419] like Figure 21 As shown in (b), the pixel values ​​of the four corner pixels adjacent to intersection point 2100 can be 120, 61, 44, and 29, respectively. As described above, it can be observed that among the four adjacent corner pixels, the corner pixel with a pixel value of 120 (the diagonal portion) exhibits a significant difference from the pixel values ​​of the other three corner pixels (61, 44, and 29). Therefore, the corner pixel with a pixel value of 120 can be the target of the filter applicable to this invention, i.e., a corner outlier.

[0420] Figure 21 (c) is a schematic diagram illustrating the indexes of the pixels in the aforementioned 2×2 region used to indicate the positions of pixels in the 2×2 region when detecting and filtering outliers at corner points. Figure 21 As shown in (c), the positions of the corner pixels adjacent to the intersection point 2100 are marked with uppercase letters A, B, C, and D, respectively, while the positions of the pixels adjacent to each corner pixel are marked with a combination of lowercase letters and numbers a1, a2, b1, b2, c1, c2, d1, and d2. The input to the corner outlier filter used for corner outlier detection and filtering is not limited to... Figure 21 The pixels in the region shown in (c) can be pixels within regions that are vertically, horizontally, and / or diagonally adjacent, centered on the boundaries between blocks. For example, pixels in square regions such as 3×3 or 4×4 can be used, as well as pixels in rectangular regions such as 1×2, 2×1, 1×3, 3×1, 1×4, 4×1, 2×3, 3×2, 2×4, 4×2, 3×4, and 4×3. The pixels used as input to the corner outlier filter can be pixels at known locations in the encoder / decoder. Alternatively, signaling can be performed by including information related to the location of the aforementioned pixels in the bitstream.

[0421] Next, we will combine Figure 21 The pixel location-related index information illustrated in (c) explains the operation of the corner outlier filter. Furthermore, in Figure 21 In the pixel position-related index information illustrated in (c), it is assumed that the pixel value of corner pixel B is significantly different from the pixel values ​​of its neighboring corner pixels A, C, and D.

[0422] Figure 22 This is a schematic diagram illustrating the operation of a corner outlier filter applicable to one embodiment of the present invention.

[0423] As input to the corner outlier filter, the pixel values ​​of pixels contained in four adjacent blocks 2101, 2102, 2103, and 2104 centered at a crossroads 2100 can be used. For example, it is possible to... Figure 21 The pixel values ​​of the pixels in the 2×2 region shown in (c) are used as input to the corner outlier filter.

[0424] In the corner outlier selection step (S2201), when four blocks 2101, 2102, 2103, and 2104 are adjacent to each other with an intersection point 2100 as the center, the corner pixel that shows a large difference in pixel value from other adjacent corner pixels among the four corner pixels A, B, C, and D adjacent to the intersection point is selected as the corner outlier.

[0425] The selection of corner pixels can be performed using the difference between the pixel values ​​of the corner pixels adjacent to the aforementioned intersection points and a first threshold. The difference between the pixel values ​​can be the difference between the pixel values ​​of horizontally, vertically, and / or diagonally adjacent pixels. The first threshold can be set based on quantization parameters. For example, the first threshold can be one of the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104, or it can be the maximum, minimum, most frequent, mode, median, average, weighted average, and / or a value induced by scaling the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104 using a specific constant value. The specific constant value can be a fixed value or a variable value, or it can be obtained based on information contained in the bitstream for signaling. However, the first threshold is not limited to this; it can also use predefined values, different values ​​set according to the characteristics of the image, or values ​​signaled via bitstream.

[0426] In one embodiment of the present invention, the corner pixel whose pixel value differs significantly from that of the other adjacent corner pixels among the four corner pixels A, B, C, D adjacent to the above intersection point can be selected as the corner outlier point by the following formula 10.

[0427] [Formula 10]

[0428] If (|AC|>|BD|)

[0429] If (|AB|>|CD|)

[0430] If (|AB|>threshold &&|AC|>threshold &&|AD|>threshold)

[0431] Return A

[0432] otherwise

[0433] If (|CA|>threshold &&|CB|>threshold &&|CD|>threshold)

[0434] Return to C

[0435] otherwise

[0436] If (|BC|>|AD|)

[0437] If (|BA|>threshold &&|BC|>threshold &&|BD|>threshold)

[0438] Return to B

[0439] otherwise

[0440] If (|DA|>threshold &&|DB|>threshold &&|CD|>threshold)

[0441] Return to D

[0442] Using Formula 10 above, we can first select corner pixels that show a large difference in pixel value from other adjacent corner pixels, based on the difference between the pixel values ​​of the four corner pixels.

[0443] Specifically, it can determine whether corner pixels A or C, or corner pixels B or D, contain outliers by comparing the differences between the pixel values ​​of corner pixels A and C, and the differences between the pixel values ​​of corner pixels B and D. For example, if the difference between the pixel values ​​of corner pixels A and C is less than the difference between the pixel values ​​of corner pixels B and D, it can be determined that either corner pixel B or D contains an outlier.

[0444] After determining that corner pixel B or corner pixel D contains a corner outlier, the difference between the pixel values ​​of corner pixel B and corner pixel C can be compared with the difference between the pixel values ​​of corner pixel A and corner pixel D. If the difference between the pixel values ​​of corner pixel B and corner pixel C is greater than the difference between the pixel values ​​of corner pixel A and corner pixel D, then it can be determined that corner pixel B or corner pixel C contains a corner outlier.

[0445] In the above example, after determining that corner pixel B or corner pixel D contains a corner outlier through the first comparison process (if (|AC|>|BD|), and determining that corner pixel B or corner pixel C contains a corner outlier through the second comparison process (if (|BC|>|AD|), it can be confirmed through the above two comparison processes that the pixel value of corner pixel B shows a large difference from the pixel values ​​of other corner pixels A, C, and D. Therefore, corner pixel B can be determined to be a corner outlier.

[0446] As described above, by comparing the differences between the pixel values ​​of adjacent corner pixels, corner pixels that exhibit significant differences in pixel values ​​compared to the other three adjacent corner pixels can be selected from four adjacent corner pixels A, B, C, and D. However, in addition to the method described in Formula 10, various other methods can be used to select corner pixels that exhibit significant differences in pixel values ​​compared to the other four adjacent corner pixels.

[0447] In Formula 10 above, after selecting corner pixel B, which exhibits a significant difference in pixel value from the other three adjacent corner pixels A, B, C, and D through two comparison processes, the difference in pixel value between the selected corner pixel B and the other three adjacent corner pixels A, C, and D can be compared with a first threshold. This first threshold can be set to 1 / 3 of the average quantization parameter of the four adjacent blocks, i.e., QP / 3. However, the first threshold is not limited to this; different values ​​set according to image characteristics or values ​​signaled via bitstream can also be used.

[0448] If the difference between the pixel values ​​of the corner pixel B selected in Formula 10 and the pixel values ​​of its three adjacent corner pixels A, C, and D is greater than the first threshold, then the selected corner pixel B can be selected as a corner outlier. If the difference between the pixel values ​​of the corner pixel B selected in Formula 10 and the pixel values ​​of one or more of its three adjacent corner pixels A, C, and D is less than the first threshold, then the corner outlier can be ignored. In this case, the corner outlier filtering operation associated with the corresponding corner outlier filter input can be terminated.

[0449] After selecting the corner outlier in step S2201, the similarity between the selected corner outlier and the pixels that are contained in the same block as it and are adjacent to it can be determined (S2202). In some cases, step S2202 can be omitted, for example, based on the characteristics of the image or the information of the signaling.

[0450] For example, when selecting using formula 10 Figure 21 When the corner pixel B in (c) is a corner outlier, it is possible to determine the similarity between the corner pixel B and the horizontally adjacent pixel b1 and / or the vertically adjacent pixel b2 contained in the same block 2101 as the corner pixel B.

[0451] The similarity determination described above can be performed based on the difference between the pixel values ​​of pixels within the same block and the pixel values ​​of corner pixel B. Pixels within the same block can be pixels located on the same horizontal and / or vertical line as corner pixel B. Pixels within the same block can be one or more pixels continuously adjacent to corner pixel B, or one or more pixels located at a specific distance from it. For example, this can be performed by comparing the difference between the pixel values ​​of horizontally and / or vertically adjacent pixels b1 and b2 within the same block and the pixel values ​​of corner pixel B with a second threshold. The second threshold can be set based on quantization parameters. For example, the second threshold can be one of the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104, or it can be the maximum, minimum, most frequent, mode, median, average, weighted average, and / or the value induced by scaling the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104 using a specific constant value. The specific constant value can be a fixed value or a variable value, or it can be obtained based on information included in the signaling within the bitstream. In one embodiment of the invention, the second threshold can be set to 1 / 6 of the average value of the quantization parameters of the four adjacent blocks, i.e., QP / 6. However, the second threshold is not limited to this; different values ​​set according to the characteristics of the image or values ​​signaled via the bitstream can also be used.

[0452] In one embodiment of the present invention, the similarity between adjacent pixels b1 and corner pixels B within the same block can be determined using the following formula 11.

[0453] [Formula 11]

[0454]

[0455] In Formula 11 above, the difference between the pixel value of corner pixel B and its horizontally adjacent pixel b1 is compared with a second threshold, QP / 6. If the difference between the pixel values ​​of corner pixel B and pixel b1 is less than QP / 6, then corner pixel B can be determined to be similar to pixel b1. The similarity determination between corner pixel B and pixel b2 can also be performed using the same method.

[0456] Based on the structure of the similarity judgment above, when it is determined that the adjacent pixels b1 and b2 in the same block are not similar to the corner pixel B, the corner outlier filtering operation of the corner pixel B selected as the corner outlier can be terminated.

[0457] Based on the structure of the similarity judgment above, when it is determined that the adjacent pixels b1 and b2 in the same block are similar to the corner pixel B, it can move to step S2203 to continue processing the selected corner outlier.

[0458] In step S2202, it is determined whether the horizontal block boundaries and vertical block boundaries adjacent to the corner outliers are edges of image regions contained in the image. In some cases, step S2203 may not be performed, for example, the decision to omit step S2203 may be based on the characteristics of the image or on the information of the signaling.

[0459] Step S2203 is to determine whether corner pixel B, selected as a corner outlier, is unsuitable for filtering because it belongs to a different image region than its neighboring corner pixels A, C, and D. For example, when the image region to which corner pixel B belongs is different from the image regions to which corner pixels A, C, and D belong, the pixel value of corner pixel B may show a large difference from the pixel values ​​of its neighboring corner pixels A, C, and D. In the case described above, it may not be possible to attribute the difference in pixel value to noise caused by factors such as block unit quantization. Therefore, it is advisable not to perform corner outlier filtering on corner pixel B in this case.

[0460] In step S2203, it is determined whether the horizontal and vertical block boundaries adjacent to the outlier corner pixel B are edges of the image region. If the horizontal and vertical block boundaries adjacent to corner pixel B are determined to be edges of the image region, it can be determined that corner pixel B belongs to a different image region from its neighboring corner pixels A, C, and D.

[0461] In one embodiment of the present invention, the determination of whether an edge exists can be performed using at least one pixel adjacent to the horizontal and vertical block boundaries of the blocks 2102, 2103, and 2104 that are adjacent to the corner outlier (corner pixel B) and a third threshold. The third threshold can be set based on quantization parameters. For example, it can be one of the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104, or it can be the maximum, minimum, most frequent, mode, median, average, weighted average, and / or a value induced by scaling the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104 using a specific constant value. The specific constant value can be a fixed value or a variable value, or it can be obtained based on information contained in the bitstream for signaling. However, the third threshold is not limited to this; different values ​​set according to the characteristics of the image or values ​​signaled via bitstream can also be used. In one embodiment to which the present invention is applied, the aforementioned third threshold can be set to 1 / 6 of the average quantization parameters of the four adjacent blocks, i.e., QP / 6. However, the third threshold is not limited to this; different values ​​set according to the characteristics of the image or values ​​signaled via bitstream can also be used.

[0462] In one embodiment of the present invention, the determination of whether an edge exists can be made by comparing the change in the number of pixels adjacent to the horizontal and vertical block boundaries of the pixels in the block adjacent to the corner outlier with a third threshold. For example, when corner pixel B is selected as the corner outlier, the following formula 12 can be used to determine whether the horizontal and vertical block boundaries adjacent to corner pixel B are edges of the image region.

[0463] [Formula 12]

[0464]

[0465] Determined as an edge

[0466] No m

[0467] Proceed to the next step

[0468] In Formula 12 above, to determine whether the boundary of the horizontal block adjacent to the outlier corner pixel B is an edge, the pixels contained in the block adjacent to corner pixel B, namely pixels c1, C, D, and d1 adjacent to the horizontal block boundary, can be used. As the variation values ​​of pixels c1, C, D, and d1, the difference between the average pixel value of pixels c1, C, D, and d1 and the pixel value of pixel c1, and / or the difference between the average pixel value of pixels c1, C, D, and d1 and the pixel value of pixel d1, can be used. Alternatively, a process of comparing the difference values ​​between two or more pixel values ​​among the pixels adjacent to the horizontal block boundary with a specific standard value can be used. This specific standard value can be determined or signaled based on the characteristics of the image. When the changes in the aforementioned pixels c1, C, D, and d1 are less than the third threshold, i.e., QP / 6, it is determined that the changes in the aforementioned pixels c1, C, D, and d1 are small, thereby further determining that the horizontal block boundaries adjacent to the aforementioned pixels c1, C, D, and d1 are the edges of the image region.

[0469] Similarly, to determine whether the boundary of a vertical block adjacent to a corner outlier (corner pixel B) is an edge, the pixels contained in the block adjacent to corner pixel B, namely pixels a2, A, D, and d2 adjacent to the aforementioned vertical block boundary, can be used. As the variation values ​​of pixels a2, A, D, and d2, the difference between the average pixel value of pixels a2, A, D, and d2 and the pixel value of pixel a2, and / or the difference between the average pixel value of pixels a2, A, D, and d2 and the pixel value of pixel d2, can be used. Alternatively, a process can be used to compare the difference values ​​between two or more pixel values ​​among the pixels adjacent to the aforementioned vertical block boundary with a specific standard value. This specific standard value can be determined or signaled based on the characteristics of the image. When the changes in the aforementioned pixels a2, A, D, and d2 are less than the third threshold, i.e., QP / 6, it is determined that the changes in the aforementioned pixels a2, A, D, and d2 are small, thereby further determining that the vertical block boundaries adjacent to the aforementioned pixels a2, A, D, and d2 are the edges of the image region.

[0470] Although Formula 12 is used in one embodiment of the present invention to determine whether the boundary of a block is an edge of an image region, the method for determining whether an image region is an edge is not limited to this, and other methods can also be applied.

[0471] In step S2203, based on the judgment result of the above formula 12, when it is determined that the horizontal block boundary or vertical block boundary adjacent to the corner outlier, i.e., the corner pixel B, is not an edge of the image, the corner outlier filtering in step S2204 can be performed.

[0472] In step S2203, based on the judgment result of Formula 12 above, when it is determined that both the horizontal block boundary and the vertical block boundary adjacent to the corner outlier (corner pixel B) are edges of the image, the corner outlier filtering operation for corner pixel B can be terminated, or edge judgment can be additionally performed using Formula 13 below. In some cases, the edge judgment using Formula 13 below can be omitted. For example, it can be decided whether to omit the edge judgment using Formula 13 below based on the characteristics of the image or the information of the signaling.

[0473] [Formula 13]

[0474]

[0475] In Formula 13 above, it is determined whether the difference between the pixel value of a corner outlier (corner pixel B) and its adjacent corner pixel A is less than a fourth threshold. This fourth threshold can be set based on quantization parameters. For example, it can be one of the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104, or it can be the maximum, minimum, most frequent, mode, median, average, weighted average, and / or a value induced by scaling the quantization parameters of the four adjacent blocks 2101, 2102, 2103, and 2104 using a specific constant value. This specific constant value can be a fixed value or a variable value, or it can be obtained based on information included in the signaling within the bitstream. However, the fourth threshold is not limited to this; different values ​​set according to the characteristics of the image or values ​​signaled via the bitstream can also be used. In one embodiment of the present invention, the fourth threshold can be set to half of the average quantization parameter of the four adjacent blocks, i.e., QP / 2. However, the fourth threshold is not limited to this, and different values ​​set according to the characteristics of the image or values ​​signaled via the bitstream can also be used. In the embodiment of the present invention described above, the first to fourth thresholds used can all be the same or different, or some can be the same or different.

[0476] In Formula 13 above, it is determined whether the difference between the pixel value of corner outlier point B and its adjacent corner pixel A is less than QP / 2. When the difference between the pixel value of corner pixel B and its adjacent corner pixel A is less than QP / 2, the vertical block boundary adjacent to corner pixel B can be finally determined as the edge of the image region.

[0477] Similarly, in Formula 13 above, it is determined whether the difference between the pixel value of corner outlier point B and its adjacent corner pixel C is less than QP / 2. When the difference between the pixel value of corner pixel B and its adjacent corner pixel C is less than QP / 2, the horizontal block boundary adjacent to corner pixel B can be finally determined as the edge of the image region.

[0478] In step S2203, based on the judgment results of formula 12 and / or formula 13, when it is determined that the horizontal block boundary and the vertical block boundary adjacent to the corner pixel B are both edges of the image, the filtering in step S2204 can be skipped and the process can end directly.

[0479] In step S2203, based on the judgment results of Formula 12 and / or Formula 13, when it is determined that the horizontal or vertical block boundary adjacent to corner pixel B is not an edge of the image, corner outlier filtering of corner pixel B in step S2204 can be performed. As described above, by sequentially executing steps S2201 to S2203, the filtering object, i.e., corner outliers, can be determined. However, steps S2201 to S2203 are not a limitation on the judgment order, and the judgment order can be adaptively changed within the scope of maintaining the essence of the present invention. Furthermore, the filtering object, i.e., corner outliers, can be selectively determined by at least one of steps S2201 to S2203.

[0480] Filtering of corner outliers and their adjacent pixels can proceed in a direction that reduces differences with adjacent pixels, for example, in a direction that reduces differences in pixel values ​​between corner pixels belonging to other blocks. In one embodiment of the invention, filtering can be performed, for example, using the following formula 14.

[0481] [Formula 14]

[0482] B′=((4×B)+A+C+(2×D)+4)<<3

[0483] b1'=(B'+(3×b1)+2)<<2

[0484] b2′=(B'+(3×b2)+2)<<2

[0485] In formula 14 above, A, B, C, D, b1, and b2 represent respectively Figure 21 The pixel values ​​at the positions shown in (c) are B', b1', and b2', respectively representing the pixel values ​​at the positions shown in the diagram. Figure 21The values ​​of the pixels located at positions B, b1, and b2 in (c) are the filtered values. The filtered pixels can include corner outliers and pixels adjacent to them horizontally, vertically, and / or diagonally. The filtering coefficients used in the filtering process can use specific values ​​known to the encoder / decoder, or they can be derived based on image characteristics, and information related to the filtering coefficients can be signaled. Filtering is not performed only on the boundaries of prediction, quantization, and / or transform blocks; for example, it can be performed only on the boundaries of blocks with specific sizes, such as 8×8, 16×16, 32×32, and 64×63 blocks. Information related to the type and / or size of the blocks to which filtering is performed can be determined using information known to the encoder / decoder, based on image characteristics, or by signaling related information.

[0486] The method applicable to embodiments of the present invention comprises one or more steps and has been described in a specific order. However, the present invention is not limited to the specific order described above. For example, the execution order of the steps can be changed. Alternatively, one or more steps can be executed simultaneously. Or, one or more steps can be added at any position.

[0487] Embodiments of the present invention can be implemented as application program instructions executable by various computer components and stored in a computer-readable storage medium. The computer-readable storage medium can include application program instructions, data files, and data structures, individually or in combination. Embodiments of the present invention can be implemented using a hardware device equipped with one or more processors. Each of the processors can operate as a software module.

[0488] The concept of this invention is not limited to the embodiments described above, and all modifications included within the scope of the following claims and their equivalents are included within the scope of this invention.

[0489] Industry availability

[0490] This invention can be used to encode / decode images.

Claims

1. A video signal decoding method executed by a video signal decoding device, comprising the following steps: Decode the block segmentation information of the current block; Based on the block segmentation information, the current block is divided into multiple sub-blocks; and Decode the sub-block. in, In the segmentation step, at least one of a quadtree structure, a binary tree structure, and a ternary tree structure is used to segment the current block. The quadtree structure is used preferentially over the binary tree structure and the ternary tree structure. The sub-block is a coding unit.

2. A video signal encoding method executed by a video signal encoding device, comprising the following steps: Determine whether to split the current block; Based on the determination, the current block is divided into multiple sub-blocks; Generate block segmentation information for the current block; as well as The block segmentation information and the sub-blocks are encoded. In the segmentation step, at least one of a quadtree structure, a binary tree structure, and a ternary tree structure is used to segment the current block. The quadtree structure is used preferentially over the binary tree structure and the ternary tree structure. The sub-block is a coding unit.

3. A method for transmitting a bit stream, comprising the following steps: Generating bitstreams using video signal encoding methods; and Transmit the bit stream, in, The video signal encoding method includes the following steps: Determine whether to split the current block; Based on the determination, the current block is divided into multiple sub-blocks; Generate block segmentation information for the current block; as well as The block segmentation information and the sub-blocks are encoded. In the segmentation step, at least one of a quadtree structure, a binary tree structure, and a ternary tree structure is used to segment the current block. The quadtree structure is used preferentially over the binary tree structure and the ternary tree structure. The sub-block is a coding unit.

Images