Image decoding method, image encoding method, and device

Abstract

To reduce computational complexity, memory requirements, and memory access bandwidth. [Solution] The image decoding method includes the steps of decoding adaptive loop filter information of an encoded tree block, and applying an adaptive loop filter to the encoded tree block based on the adaptive loop filter information, wherein if the encoded tree block is a luminance component block, the adaptive loop filter is applied using a 7x7 diamond-shaped filter, and if the encoded tree block is a chrominance component block, the adaptive loop filter is applied using a 5x5 diamond-shaped filter.

Description

[Technical Field]

[0001] The present invention relates to an image decoding method, an image encoding method, and a device. Specifically, the present invention relates to an image decoding method, an image encoding method, and a device based on intra-loop filtering. [Background technology]

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) and UHD (Ultra High Definition) images, has been increasing in various application fields. As image data becomes higher resolution and higher quality, the relative amount of data increases compared to conventional image data. Therefore, when transmitting image data using conventional wired or wireless broadband lines or storing it using conventional storage media, transmission and storage costs increase. To solve these problems that arise with higher resolution and quality image data, highly efficient image encoding / decoding technologies for images with higher resolution and image quality are required.

[0003] Various image compression techniques exist, including inter-frame prediction techniques that predict pixel values ​​contained in the current picture from previous or subsequent pictures; intra-frame prediction techniques that predict pixel values ​​contained in the current picture using pixel information within the current picture; transformation and quantization techniques for compressing the energy of residual signals; and entropy coding techniques that assign short codes to frequently occurring values ​​and long codes to less frequently occurring values. By utilizing these image compression techniques, image data can be effectively compressed and transmitted or stored.

[0004] Deblocking filters aim to reduce blocking phenomena that occur at block boundaries by applying vertical and horizontal filtering to the block boundaries. However, deblocking filters have the drawback that they cannot minimize the distortion between the original image and the reconstructed image when filtering block boundaries.

[0005] Sample-adaptive offsetting is a method used to reduce ringing by comparing pixel values ​​with adjacent pixels on a pixel-by-pixel basis, and then applying an offset to specific pixels or to pixels whose pixel values ​​belong to a specific range. Sample-adaptive offsetting uses rate-distortion optimization to partially minimize the distortion between the original image and the reconstructed image, but there are limitations in terms of distortion minimization when the difference in distortion between the original image and the reconstructed image is large. [Overview of the project] [Problems that the invention aims to solve]

[0006] The present invention aims to provide an image coding / decoding method and apparatus using loop filtering.

[0007] Furthermore, the present invention aims to provide an in-loop filtering method and apparatus using subsample-based block classification to reduce the computational complexity and memory access bandwidth of the image encoder / decoder.

[0008] Furthermore, the present invention aims to provide an in-loop filtering method and apparatus using multiple filter shapes in order to reduce the computational complexity, memory requirements, and memory access bandwidth of an image encoder / decoder.

[0009] Furthermore, the present invention aims to provide a recording medium that stores a bitstream generated by the image encoding / decoding method or apparatus of the present invention. [Means for solving the problem]

[0010] According to the present invention, an image decoding method can be provided, which includes the steps of: decoding filter information for encoding units; classifying the encoding units into block classification units; and filtering the encoding units classified into block classification units using the filter information.

[0011] The image decoding method according to the present invention further includes the step of assigning a block classification index to an encoding unit classified by the block classification unit, wherein the block classification index can be determined using directional information and activity information.

[0012] In the image decoding method according to the present invention, at least one of the directional information and activity information can be determined based on a gradient value for at least one of the vertical, horizontal, first diagonal, and second diagonal directions.

[0013] In the image decoding method according to the present invention, the gradient value can be obtained using a 1D Laplacian operation at the block classification unit.

[0014] In the image decoding method according to the present invention, the 1D Laplacian operation may be a 1D Laplacian operation in which the execution position of the operation is subsampled.

[0015] In the image decoding method according to the present invention, the gradient value can be determined based on a temporal layer identifier.

[0016] In the image decoding method according to the present invention, the filter information may include at least one of the following: whether filtering is performed, filter coefficient value, number of filters, number of filter taps (filter length) information, filter shape information, filter type information, information on whether a fixed filter is used for a block classification index, and information on the symmetrical shape of the filter.

[0017] In the image decoding method according to the present invention, the filter shape information may include information on at least one of the following shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0018] In the image decoding method according to the present invention, the filter coefficient values ​​may include filter coefficient values ​​that have been geometrically transformed based on coding units classified by the block classification units.

[0019] In the image decoding method according to the present invention, the information regarding the symmetry shape of the filter may include information regarding at least one of the following: point symmetry, transverse symmetry, vertical symmetry, and diagonal symmetry.

[0020] Furthermore, according to the present invention, an image encoding method can be provided which includes the steps of: classifying encoding units into block classification units; filtering the encoding units classified into block classification units using filter information for the encoding units; and encoding the filter information.

[0021] The image coding method according to the present invention further includes the step of assigning a block classification index to coding units classified by the block classification unit, wherein the block classification index can be determined using directional information and activity information.

[0022] In the image coding method according to the present invention, at least one of the directional information and activity information can be determined based on a gradient value for at least one of the vertical, horizontal, first diagonal, and second diagonal directions.

[0023] In the image coding method according to the present invention, the gradient value can be obtained using a 1D Laplacian operation at the block classification unit.

[0024] In the image coding method according to the present invention, the 1D Laplacian operation may be a 1D Laplacian operation in which the execution position of the operation is subsampled.

[0025] In the image coding method according to the present invention, the gradient value can be determined based on a temporal hierarchy identifier.

[0026] In the image coding method according to the present invention, the filter information may include at least one of the following: whether filtering is performed, filter coefficient values, number of filters, number of filter taps (filter length) information, filter shape information, filter type information, information on whether a fixed filter is used for a block classification index, and information on the symmetric shape of the filter.

[0027] In the image coding method according to the present invention, the filter shape information may include information on at least one of the following shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0028] In the image coding method according to the present invention, the filter coefficient values ​​may include filter coefficient values ​​that have been geometrically transformed based on coding units classified by the block classification units.

[0029] Furthermore, the recording medium according to the present invention can store a bitstream generated by the image encoding method according to the present invention. [Effects of the Invention]

[0030] According to the present invention, an image encoding / decoding method and apparatus using loop filtering can be provided.

[0031] Furthermore, according to the present invention, a method and apparatus for intra-loop filtering using subsample-based block classification can be provided to reduce the computational complexity and memory access bandwidth of the image encoder / decoder.

[0032] Furthermore, according to the present invention, a loop-based filtering method and apparatus using a multiplex filter shape can be provided to reduce the computational complexity, memory requirements, and memory access bandwidth of the image encoder / decoder.

[0033] Furthermore, according to the present invention, a recording medium storing a bitstream generated by the image encoding / decoding method or apparatus of the present invention can be provided.

[0034] Furthermore, according to the present invention, the efficiency of image encoding and / or decoding can be improved. [Brief explanation of the drawing]

[0035] [Figure 1] This is a block diagram showing the configuration of one embodiment of an encoding device to which the present invention is applied. [Figure 2] This is a block diagram showing the configuration of one embodiment of a decoding device to which the present invention is applied. [Figure 3] This is a schematic diagram showing the image partitioning structure when encoding and decoding an image. [Figure 4] This is a diagram illustrating an embodiment of the in-screen prediction process. [Figure 5] This is a diagram illustrating an embodiment of the inter-screen prediction process. [Figure 6] This is a diagram illustrating the transformation and quantization process. [Figure 7] This flowchart shows an image decoding method according to one embodiment of the present invention. [Figure 8] This flowchart shows an image encoding method according to one embodiment of the present invention. [Figure 9] This is an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions. [Figure 10] This is an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 11]This is an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 12] This is an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 13] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 14] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 15] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 16] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 17] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 18] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 19] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 20] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 21] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 22] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 23] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 24] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 25] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 26] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 27] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 28] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 29] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 30] This is an example of determining gradient values ​​in the lateral, vertical, first diagonal, and second diagonal directions at a specific sample position according to one embodiment of the present invention. [Figure 31] This is an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions when the temporal hierarchy identifier is the highest level. [Figure 32] This figure shows various computational techniques that can replace the 1D Laplacian operation according to one embodiment of the present invention. [Figure 33] This figure shows a rhombus-shaped filter according to one embodiment of the present invention. [Figure 34] This figure shows a filter with 5x5 filter taps according to one embodiment of the present invention. [Figure 35a] This figure shows various filter shapes based on one embodiment of the present invention. [Figure 35b] This figure shows various filter shapes based on one embodiment of the present invention. [Figure 36]This figure shows a horizontally or vertically symmetrical filter according to one embodiment of the present invention. [Figure 37] This figure shows filters that have undergone geometric transformations to square, octagonal, snowflake, and rhombus-shaped filters according to one embodiment of the present invention. [Figure 38] This figure shows the process of converting a 9x9 rhombus-shaped filter coefficient to a 5x5 square-shaped filter coefficient according to one embodiment of the present invention. [Figure 39] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 40a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 40b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 40c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 40d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 41a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 41b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 41c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 41d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 42a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 42b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 42c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 42d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 43] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 44a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 44b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 44c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 44d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 45a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 45b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 45c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 45d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 46a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 46b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 46c]This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 46d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 47a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 47b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 47c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 47d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 48] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 49a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 49b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 49c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 49d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 50a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 50b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 50c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 50d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 51a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 51b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 51c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 51d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 52] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 53a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 53b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 53c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 53d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 54a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 54b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 54c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 54d]This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 55a] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 55b] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 55c] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Figure 55d] This is another example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample. [Modes for carrying out the invention]

[0036] The present invention can be modified in various ways and may have various embodiments; therefore, specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this should not be understood as limiting the present invention to specific embodiments, but rather as including all modifications, equivalents, or substitutes that fall within the spirit and technical scope of the present invention. Similar reference numerals in the drawings refer to the same or similar functions in various aspects. The shape and size of elements in the drawings may be exaggerated for clearer explanation. Detailed descriptions of the exemplary embodiments described below refer to the accompanying drawings that illustrate specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to carry out the embodiments. It should be understood that the various embodiments are different from one another but do not need to be mutually exclusive. For example, specific shapes, structures, and characteristics described herein can be realized in various embodiments in relation to one embodiment without departing from the spirit and scope of the present invention. It should also be understood that the position or arrangement of individual components within each disclosed embodiment can be modified without departing from the spirit and scope of the embodiment. Therefore, the detailed descriptions set forth below should not be taken as restrictive, and the scope of the exemplary embodiments is limited only to all equivalents of those claims and the appended claims, if appropriately described.

[0037] In this invention, terms such as "first," "second," etc., can be used to describe various components, but these components should not be limited by the above terms. These terms are used solely for the purpose of distinguishing one component from another. For example, as long as it does not fall outside the scope of the rights of this invention, the first component may be named the second component, and similarly, the second component may be named the first component. The term "and / or" includes a combination of multiple related descriptions or any of multiple related descriptions.

[0038] When a component of the present invention is described as being "connected" or "linked" to another component, it should be understood that it may be directly connected or linked to the other component, but there may also be another component interposed between them. Conversely, when a component is described as being "directly connected" or "directly linked" to another component, it should be understood that there is no other component interposed between them.

[0039] The components shown in the embodiments of the present invention are illustrated independently to illustrate distinct characteristic functions, and this does not mean that each component consists of separate hardware or a single software component. That is, for the sake of explanation, each component is listed and included within each component, and at least two of the components from each component may be combined to form a single component, or each component may be divided into multiple components to perform functions. Such integrated and separated embodiments of each component are also included within the scope of the present invention, as long as they do not deviate from the essence of the present invention.

[0040] The terms used in this invention are used solely to describe specific embodiments and do not limit the invention. A singular expression includes plural expressions unless the context clearly indicates otherwise. In this invention, terms such as “includes” or “having” specify the presence of features, figures, steps, operations, components, parts, or combinations thereof described in the specification, and should be understood not to pre-exist the presence or possibility of adding one or more other features, figures, steps, operations, components, parts, or combinations thereof. In other words, in this invention, the description of a particular configuration as “includes” does not exclude configurations other than that configuration, but rather means that additional configurations may be included within the scope of the implementation of this invention or the technical idea of ​​this invention.

[0041] Some components of the present invention may not be essential components that perform an essential function in the present invention, but rather optional components that merely improve performance. The present invention can be realized by including only components that are essential to realizing the essence of the present invention, excluding components used solely for performance improvement, and a structure including only essential components, excluding optional components used solely for performance improvement, is also within the scope of the rights of the present invention.

[0042] Embodiments of the present invention will be described in detail below with reference to the drawings. In describing the embodiments of this specification, if it is determined that a specific description of a related known configuration or function may obscure the gist of this specification, such detailed description will be omitted, the same reference numerals will be used for the same components in the drawings, and redundant descriptions of the same components will be omitted.

[0043] In the following, "image" may refer to a single picture that makes up a video, or it may refer to the video itself. For example, "encoding and / or decoding of an image" may refer to "encoding and / or decoding of a video," or it may refer to "encoding and / or decoding of any of the images that make up a video."

[0044] In the following, the terms "moving images" and "video" may be used interchangeably or interchangeably.

[0045] In the following, the target image may be an image to be encoded and / or an image to be decoded. Furthermore, the target image may be an input image input to an encoding device, or an input image input to a decoding device. Here, the target image can have the same meaning as the current image.

[0046] In the following, the terms "image," "picture," "frame," and "screen" may be used interchangeably or interchangeably.

[0047] In the following, the target block may be the target block to be encoded and / or the target block to be decoded. The target block may also be the current block currently being encoded and / or decoded. For example, the terms "target block" and "current block" may be used interchangeably or interchangeably.

[0048] In the following, the terms "block" and "unit" may be used interchangeably or interchangeably. Alternatively, "block" may refer to a specific unit.

[0049] In the following, the terms "region" and "segment" may be used interchangeably.

[0050] In the following, a particular signal may indicate a particular block. For example, the original signal may indicate the target block. The prediction signal may indicate the predicted block. The residual signal may indicate the residual block.

[0051] In the embodiment, each of the following can have a value: information, data, flags, indices, elements, and attributes. A value of "0" for information, data, flags, indices, elements, and attributes can represent logical false or a first predefined value. In other words, the values ​​"0", false, logical false, and the first predefined value may be used interchangeably. A value of "1" for information, data, flags, indices, elements, and attributes can represent logical true or a second predefined value. In other words, the values ​​"1", true, logical true, and the second predefined value may be used interchangeably.

[0052] When variables such as i or j are used to indicate rows, columns, or indices, the value of i may be a non-negative integer or a non-negative integer of 1 or more. In other words, in some embodiments, rows, columns, and indices may be counted from 0 or from 1.

[0053] Glossary Encoder: This refers to a device that performs encoding. In other words, it can mean an encoding device.

[0054] A decoder is a device that performs decoding. In other words, it can mean a decoding device.

[0055] A block is an M × N array of samples. Here, M and N can represent positive integer values, and a block generally represents a two-dimensional array of samples. A block can also represent a unit. The current block can represent the target block to be encoded during encoding, or the target block to be decoded during decoding. Furthermore, the current block can be at least one of the encoded block, prediction block, residual block, and transformation block.

[0056] Sample: The basic unit that makes up a block. Its value ranges from 0 to 2 depending on the bit depth (Bd). Bd It can be expressed as a value up to -1. In this invention, "sample" can be used interchangeably with "pixel." That is, "sample," "pixel," and "pixel" can have the same meaning as one another.

[0057] A unit can refer to a unit of image coding and decoding. In image coding and decoding, a unit can be a region into which a single image has been divided. Furthermore, when an image is divided into subdivided units for coding or decoding, a unit can refer to one of these subdivided units. That is, a single image can be divided into multiple units. In image coding and decoding, predefined processing can be performed on each unit. A single unit can be further divided into subunits that are smaller than the unit itself. Depending on its function, a unit can refer to a block, macroblock, coding tree unit, coding tree block, coding unit, coding block, prediction unit, prediction block, residual unit, residual block, transform unit, transform block, and so on. Furthermore, a unit can mean that it includes a luminance (Luma) component block, a corresponding chroma component block, and syntactic elements for each block, in order to distinguish it from a block. Units can have various sizes and shapes, and in particular, the shape of a unit can include not only squares but also two-dimensional geometric shapes such as rectangles, trapezoids, triangles, and pentagons. Unit information can also include at least one of the following: the type of unit that points to an encoded unit, a prediction unit, a residual unit, a transform unit, etc.; the size of the unit; the depth of the unit; and the encoding and decoding order of the unit.

[0058] A coding tree unit consists of two chrominance component (Cb,Cr) coding tree blocks associated with one luminance component (Y) coding tree block. It can also mean that the unit includes the aforementioned blocks and the syntactic elements for each block. Each coding tree unit can be divided using one or more partitioning schemes, such as a quad tree, binary tree, or ternary tree, to form subunits such as coding units, prediction units, and transformation units. It can be used as a term to refer to a sample block that becomes a processing unit in the image decoding / coding process, such as the partitioning of an input image. Here, a quad tree can mean a quarternary tree.

[0059] Coding Tree Block: This term can be used to refer to any of the following: Y-coded tree block, Cb-coded tree block, and Cr-coded tree block.

[0060] A neighbor block can refer to a block adjacent to the current block. A neighbor block can refer to a block whose boundary is adjacent to the current block, or a block located within a predetermined distance from the current block. A neighbor block can also refer to a block adjacent to the vertex of the current block. Here, a block adjacent to the vertex of the current block may be a block adjacent vertically to the horizontally adjacent block of the current block, or a block adjacent horizontally to the vertically adjacent block of the current block. A neighbor block can also refer to a restored neighbor block.

[0061] A reconstructed neighbor block can refer to a neighbor block that has already been encoded or decoded spatially / temporally around the current block. In this case, a reconstructed neighbor block can refer to a reconstructed neighbor unit. A reconstructed spatial neighbor block may be a block within the current picture that has already been reconstructed through encoding and / or decoding. A reconstructed temporal neighbor block may be a reconstructed block or its neighbor at a position corresponding to the current block of the current picture within the reference image.

[0062] Unit depth: This can represent the degree to which a unit has been divided. In a tree structure, the top-level node (Root Node) can correspond to the first, undivided unit. The top-level node is sometimes called the root node. The top-level node can also have the minimum depth value. In this case, the top-level node can have a depth of Level 0. A node with a depth of Level 1 can represent a unit created when the first unit is divided once. A node with a depth of Level 2 can represent a unit created when the first unit is divided twice. A node with a depth of Level n can represent a unit created when the first unit is divided n times. A leaf node is the lowest-level node and can be a node that cannot be divided further. The depth of a leaf node can be the maximum level. For example, a predefined value for the maximum level can be 3. The root node has the shallowest depth, and the leaf node has the deepest depth. Also, when a unit is represented in a tree structure, the level at which the unit exists can represent the unit depth.

[0063] A bitstream can refer to a sequence of bits containing encoded image information.

[0064] Parameter Set: This corresponds to the header information within the structure of a bitstream. A parameter set may include at least one of the following: video parameter set, sequence parameter set, picture parameter set, or adaptation parameter set. A parameter set may also include slice header, tile group header, and tile header information. The tile group can mean a group containing various tiles and may be synonymous with slice.

[0065] Parsing: This can mean determining the values ​​of syntax elements by entropy decoding a bitstream, or it can mean the entropy decoding process itself.

[0066] A symbol can represent at least one of the following: a syntactic element of the unit to be encoded / decoded, a coding parameter, or a transform coefficient value. A symbol can also represent the target of entropy coding or the result of entropy decoding.

[0067] Prediction Mode: This may indicate a mode that is encoded / decoded by in-screen prediction, or a mode that is encoded / decoded by cross-screen prediction.

[0068] A prediction unit can refer to the basic unit used when performing predictions, such as inter-screen prediction, intra-screen prediction, inter-screen compensation, intra-screen compensation, and motion compensation. A single prediction unit may be divided into multiple partitions or sub-prediction units of smaller size. Multiple partitions can also be basic units in the execution of prediction or compensation. Partitions generated by the division of a prediction unit can also be prediction units.

[0069] Prediction Unit Partition: This can refer to a configuration in which the prediction unit is divided.

[0070] A Reference Picture List (CP) can refer to a list containing one or more reference images used for inter-screen prediction or motion compensation. Types of CP lists include LC (List Combined), L0 (List 0), L1 (List 1), L2 (List 2), and L3 (List 3). One or more CP lists can be used for inter-screen prediction.

[0071] Inter-Prediction Indicator: This can represent the inter-prediction direction of the current block (unidirectional prediction, bidirectional prediction, etc.). Alternatively, it can represent the number of reference images used when generating prediction blocks for the current block. Alternatively, it can represent the number of prediction blocks used when performing inter-prediction or motion compensation on the current block.

[0072] The prediction list utilization flag indicates whether or not to generate a prediction block using at least one reference image within a specific list of reference images. The prediction list utilization flag can be used to derive the inter-screen prediction indicator, and conversely, the prediction list utilization flag can be used to derive the inter-screen prediction indicator. For example, if the prediction list utilization flag indicates a first value of "0", it indicates that no prediction block will be generated using the reference images within that list, and if it indicates a second value of "1", it indicates that a prediction block can be generated using that list of reference images.

[0073] Reference Picture Index: This can refer to an index that points to a specific reference image in a list of reference images.

[0074] Reference Picture: This can refer to an image that a particular block references for inter-screen prediction or motion compensation. Alternatively, a reference picture may be an image containing a reference block that the current block references for inter-screen prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference image" may be used interchangeably or interchangeably.

[0075] Motion Vector: A two-dimensional vector used for inter-screen prediction or motion compensation. The motion vector can represent the offset between the block being encoded / decoded and the reference block. For example, (mvX, mvY) can represent a motion vector, where mvX represents the horizontal component and mvY represents the vertical component.

[0076] Search Range: The search range can be a two-dimensional region where the search for motion vectors takes place during the inter-screen prediction process. For example, the size of the search range can be M x N, where M and N can be positive integers.

[0077] Motion Vector Candidate: This can refer to a block that is a candidate for prediction when predicting a motion vector, or the motion vector of that block. Motion vector candidates may also be included in the motion vector candidate list.

[0078] A motion vector candidate list can refer to a list constructed using one or more motion vector candidates.

[0079] Motion Vector Candidate Index: This can refer to an indicator showing motion vector candidates within a list of motion vector candidates. It may also be the index of a Motion Vector Predictor.

[0080] Motion Information: This can refer to information that includes not only motion vectors, reference image indices, and inter-screen prediction indicators, but also at least one of the following: prediction list utilization flags, reference image list information, reference images, motion vector candidates, motion vector candidate indices, merge candidates, and margin indices.

[0081] A merge candidate list can refer to a list composed of one or more merge candidates.

[0082] Merge Candidate: This can refer to spatial merge candidates, temporal merge candidates, combinational merge candidates, combinational dual-prediction merge candidates, zero merge candidates, etc. Merge candidates can include motion information such as inter-screen prediction indicators, reference image indices for each list, motion vectors, prediction list utilization flags, and inter-screen prediction indicators.

[0083] Merge Index: This can refer to an indicator that points to a merge candidate within a list of merge candidates. The merge index can also indicate the block that led to the merge candidate, among the blocks restored to be spatially / temporarily adjacent to the current block. Furthermore, the merge index can indicate at least one of the movement information pieces associated with the merge candidate.

[0084] A Transform Unit can refer to the fundamental unit used when performing residual signal coding / decoding, such as transform, inverse transform, quantization, inverse quantization, and transform coefficient coding / decoding. A single transform unit can be divided into multiple sub-transform units of even smaller size. Here, a transform / inverse transform can include at least one of a linear transform / inverse transform and a quadratic transform / inverse transform.

[0085] Scaling can be defined as the process of multiplying a quantized level by a factor. Transformation coefficients can be generated as a result of scaling a quantized level. Scaling can also be called dequantization.

[0086] Quantization parameter: This can refer to the value used in quantization to generate the quantized level using transformation coefficients. Alternatively, it can refer to the value used in inverse quantization to generate the transformation coefficients by scaling the quantized level. The quantization parameter may be a value mapped to the quantization step size.

[0087] Delta Quantization Parameter: This can refer to the difference between the predicted quantization parameter and the quantization parameter of the unit being encoded / decoded.

[0088] Scanning can refer to a method of sorting the order of units, blocks, or matrix coefficients. For example, sorting a two-dimensional array into a one-dimensional array is called scanning. Alternatively, sorting a one-dimensional array into a two-dimensional array can also be called scanning or inverse scanning.

[0089] Transform Coefficient: This can refer to a coefficient value generated after a transformation has been performed by an encoder. It can also refer to a coefficient value generated after at least one of entropy decoding and inverse quantization has been performed by a decoder. The quantization level or quantization transformation coefficient level obtained by applying quantization to the transform coefficient or residual signal may also be included in the meaning of transform coefficient.

[0090] Quantized Level: This can refer to the value generated by quantizing the transformation coefficients or residual signal in the encoder. Alternatively, it can refer to the value that is to be dequantized before dequantization is performed in the decoder. Similarly, the quantized transformation coefficient level, which is the result of transformation and quantization, can also be included in the meaning of quantized level.

[0091] Non-zero transform coefficient: This can refer to a transform coefficient whose value is not zero, a transform coefficient level whose value is not zero, or a quantized level.

[0092] A quantization matrix is ​​a matrix used in the quantization or dequantization process to improve the subjective or objective image quality of an image. A quantization matrix can also be called a scaling list.

[0093] Quantization matrix coefficients can represent each element within the quantization matrix. They can also be called matrix coefficients.

[0094] The default matrix can refer to a predetermined quantization matrix that is predefined in the encoder and decoder.

[0095] A non-default matrix can refer to a quantization matrix that is not predefined in the encoder and decoder, but is signaled by the user.

[0096] Statistical value: A statistical value for at least one variable, coding parameter, constant, etc., that has a specific value that can be calculated, can be at least one of the following: mean, sum, weighted mean, weighted sum, minimum, maximum, mode, median, and interpolated value of that specific value.

[0097] Figure 1 is a block diagram showing the configuration of one embodiment of an encoding device to which the present invention is applied.

[0098] The encoding device 100 may be an encoder, a video encoding device, or an image encoding device. The video may contain one or more images. The encoding device 100 can sequentially encode one or more images.

[0099] Referring to Figure 1, the encoding device 100 may include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transformer 130, a quantization unit 140, an entropy encoding unit 150, an inverse quantization unit 160, an inverse transformer 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

[0100] The encoding device 100 can encode an input image in intra-mode and / or inter-mode. The encoding device 100 can also generate a bitstream containing the encoded information through the encoding of the input image, and can output the generated bitstream. The generated bitstream can be stored on a computer-readable recording medium or streamed via a wired / wireless transmission medium. When intra-mode is used as the prediction mode, switch 115 can be switched to intra, and when inter-mode is used as the prediction mode, switch 115 can be switched to inter. Here, intra-mode can mean in-screen prediction mode, and inter-mode can mean inter-screen prediction mode. The encoding device 100 can generate prediction blocks for the input blocks of the input image. Furthermore, after the prediction blocks are generated, the encoding device 100 can encode the residual blocks using the difference (residual) between the input blocks and the prediction blocks. The input image is sometimes referred to as the current image, which is currently being encoded. The input blocks are sometimes referred to as the current blocks or encoding target blocks, which are currently being encoded.

[0101] When the prediction mode is intra mode, the intra prediction unit 120 can also use samples of already encoded / decoded blocks surrounding the current block as reference samples. The intra prediction unit 120 can perform spatial prediction for the current block using the reference samples and generate prediction samples for the input block through spatial prediction. Here, intra prediction can mean in-screen prediction.

[0102] When the prediction mode is intermode, the motion prediction unit 111 can search the reference image for the region that best matches the input block during the motion prediction process, and derive a motion vector using the searched region. In this case, the search region can be used as the region. The reference image can be stored in the reference picture buffer 190. Here, the reference image can be stored in the reference picture buffer 190 after encoding / decoding has been processed.

[0103] The motion compensation unit 112 can generate predicted blocks for the current block by performing motion compensation using motion vectors. Here, inter-prediction can mean inter-screen prediction or motion compensation.

[0104] The motion prediction unit 111 and motion compensation unit 112 can generate prediction blocks by applying an interpolation filter to a portion of the reference image when the motion vector values ​​do not have integer values. In order to perform inter-screen prediction or motion compensation, the system can determine, based on the encoding unit, which of the following methods the motion prediction and motion compensation method of the prediction unit included in the encoding unit is: Skip Mode, Merge Mode, Advanced Motion Vector Prediction (AMVP) Mode, or Current Picture Reference Mode, and perform inter-screen prediction or motion compensation according to each mode.

[0105] The subtractor 125 can generate a residual block using the difference between the input block and the prediction block. The residual block is also called the residual signal. The residual signal can represent the difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming, quantizing, or transforming the difference between the original signal and the prediction signal. The residual block may be a residual signal on a block-by-block basis.

[0106] The transformation unit 130 can perform a transformation on the remaining blocks to generate a transformation coefficient and output the generated transformation coefficient. Here, the transformation coefficient may be a coefficient value generated by performing a transformation on the remaining blocks. When the transformation skip mode is applied, the transformation unit 130 can also omit the transformation on the remaining blocks.

[0107] A quantized level can be generated by applying quantization to the conversion coefficients or residual signal. In the following embodiments, the quantized level may also be referred to as a conversion coefficient.

[0108] The quantization unit 140 can generate a quantization level by quantizing the conversion coefficients or residual signal based on quantization parameters, and can output the generated quantization level. In this case, the quantization unit 140 can quantize the conversion coefficients using a quantization matrix.

[0109] The entropy coding unit 150 can generate a bitstream and output it by performing entropy coding using a probability distribution on values ​​calculated by the quantization unit 140 or coding parameter values ​​calculated during the coding process. The entropy coding unit 150 can perform entropy coding on information about image samples and information for decoding images. For example, information for decoding images may include syntax elements.

[0110] When entropy coding is applied, symbols with a high probability of occurrence are assigned fewer bits, and symbols with a low probability of occurrence are assigned more bits, thereby reducing the size of the bit sequence for the symbols to be coded. The entropy coding unit 150 can use coding methods such as exponential Golomb, CAVLC (Context-Adaptive Variable Length Coding), and CABAC (Context-Adaptive Binary Arithmetic Coding) for entropy coding. For example, the entropy coding unit 150 can perform entropy coding using a Variable Length Coding (VLC) table. Alternatively, the entropy coding unit 150 can derive a binarization method for the target symbols and a probability model for the target symbols / bins, and then perform arithmetic coding using the derived binarization method, probability model, and context model.

[0111] The entropy coding unit 150 can convert two-dimensional block-form coefficients into one-dimensional vectors via a transform coefficient scanning method in order to encode the transformation coefficient level (quantization level).

[0112] Coding parameters can include not only information encoded by the encoder and signaled to the decoder (such as flags and indices), like syntactic elements, but also information induced during the encoding or decoding process, and can represent information necessary when encoding or decoding an image. For example, unit / block size, unit / block depth, unit / block partitioning information, unit / block shape, unit / block partitioning structure, whether or not it's a quadtree partition, whether or not it's a binary tree partition, direction of binary tree partitioning (horizontal or vertical), shape of binary tree partitioning (symmetric or asymmetric), whether or not it's a ternary tree partition, direction of ternary tree partitioning (horizontal or vertical), shape of ternary tree partitioning (symmetric or asymmetric), whether or not it's a composite tree partition, direction of composite tree partitioning (horizontal or vertical), shape of composite tree partitioning (symmetric or asymmetric), partition tree of composite tree (binary or ternary), prediction mode (in-screen prediction or inter-screen prediction), in-screen brightness prediction mode / direction, in-screen color difference prediction mode / direction, in-screen partitioning information, inter-screen partitioning information, encoded block partitioning flag, prediction block partitioning flag, transformed block partitioning flag, reference sample filtering method, reference sample filter tap, reference sample filter coefficient, prediction Block filtering method, prediction block filter tap, prediction block filter coefficient, prediction block boundary filtering method, prediction block boundary filter tap, prediction block boundary filter coefficient, in-screen prediction mode, inter-screen prediction mode, motion information, motion vector, motion vector difference, reference image index, inter-screen prediction direction, inter-screen prediction indicator, prediction list utilization flag, reference image list, reference image, motion vector prediction index, motion vector prediction candidate, motion vector candidate list, merge mode usage status, merge index, merge candidate, merge candidate list, skip mode usage status, interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, motion vector representation accuracy, transformation type, transformation size, information on the use of linear transformation, information on the use of quadratic transformation, linear transformation index, quadratic transformation index, residual signal information, coded block patternPattern), Coded Block Flag, Quantization Parameter, Residual Quantization Parameter, Quantization Matrix, In-Screen Loop Filter Application (or not), In-Screen Loop Filter Coefficient, In-Screen Loop Filter Tap, In-Screen Loop Filter Shape / Form, Deblocking Filter Application (or not), Deblocking Filter Coefficient, Deblocking Filter Tap, Deblocking Filter Strength, Deblocking Filter Shape / Form, Adaptive Sample Offset Application (or not), Adaptive Sample Offset Value, Adaptive Sample Offset Category, Adaptive Sample Offset Type, Adaptive Loop Filter Application (or not), Adaptive Loop Filter Coefficient, Adaptive Loop Filter Tap, Adaptive Loop Filter Shape / Form, Binarization / Inverse Binarization Method, Context Model Determination Method, Context Model Update Method, Regular Mode Execution (or not), Bypass Mode Execution (or not), Context Bin, Bypass Bin, Important Coefficients, Flag, Last Important Coefficient Flag, Coefficient Group Unit Encoding Flag, Last Important Coefficient Position, Flag for Coefficient Value Greater Than 1, Flag for Coefficient Value Greater Than 2 The encoding parameters may include at least one value or combination of the following: a flag indicating whether the coefficient value is greater than 3, remaining coefficient value information, sign information, restored luminance sample, restored chrominance sample, residual luminance sample, residual chrominance sample, luminance conversion coefficient, chrominance conversion coefficient, luminance quantization level, chrominance quantization level, conversion coefficient level scanning method, size of the decoder side motion vector search area, shape of the decoder side motion vector search area, number of decoder side motion vector searches, CTU size information, minimum block size information, maximum block size information, maximum block depth information, minimum block depth information, image display / output order, slice identification information, slice type, slice division information, tile identification information, tile type, tile division information, tile group identification information, tile group type, tile group division information, picture type, input sample bit depth, restored sample bit depth, residual sample bit depth, conversion coefficient bit depth, quantization level bit depth, information for the luminance signal, and information for the chrominance signal.

[0113] Here, signaling a flag or index can mean, in an encoder, entropy encoding the flag or index and including it in the bitstream, and in a decoder, entropy decoding the flag or index from the bitstream.

[0114] When the encoding device 100 performs encoding using interpretation, the encoded current image can be used as a reference image for other images to be processed later. Therefore, the encoding device 100 can further restore or decode the encoded current image and store the restored or decoded image as a reference image in the reference picture buffer 190.

[0115] The quantization level can be dequantized by the dequantization unit 160 and inversely transformed by the inverse transform unit 170. The dequantized and / or inversely transformed coefficients can be combined with the predicted block via the adder 175. By combining the dequantized and / or inversely transformed coefficients with the predicted block, a reconstructed block can be generated. Here, the dequantized and / or inversely transformed coefficients mean coefficients that have undergone at least one of dequantization and / or inverse transformation, and can mean the reconstructed residual block.

[0116] The reconstructed block can pass through the filter section 180. The filter section 180 can apply at least one of the following to the reconstructed sample, reconstructed block, or reconstructed image: a deblocking filter, a sample adaptive offset (SAO), or an adaptive loop filter (ALF). The filter section 180 is also called an in-loop filter.

[0117] Deblocking filters can remove block distortion arising from the boundaries between blocks. To determine whether or not to apply a deblocking filter, the decision can be made based on the samples contained in several columns or rows within the block. When applying a deblocking filter to a block, different filters can be applied depending on the required deblocking filtering strength.

[0118] Sample-adaptive offsetting can be used to compensate for encoding errors by adding an appropriate offset value to the sample value. Sample-adaptive offsetting can correct the offset from the original image on a sample-by-sample basis for deblocked images. One method is to divide the samples contained in the image into a certain number of regions, determine the regions to be offsetted, and apply the offset to those regions, or to apply the offset while considering the edge information of each sample.

[0119] Adaptive loop filters can perform filtering based on a comparison between the reconstructed image and the original image. After dividing the samples contained in the image into predetermined groups, the filter to be applied to each group can be determined, allowing for differential filtering for each group. Information related to whether or not to apply an adaptive loop filter can be signaled on a per-coding unit (CU) basis, and the shape and filter coefficients of the applied adaptive loop filter may differ depending on the block.

[0120] The restored block or restored image after passing through the filter unit 180 can be saved in the reference picture buffer 190. The restored block after passing through the filter unit 180 may be part of the reference image. In other words, the reference image may be a restored image consisting of the restored block after passing through the filter unit 180. The saved reference image can subsequently be used for inter-screen prediction or motion compensation.

[0121] Figure 2 is a block diagram showing the configuration of one embodiment of a decoding device to which the present invention is applied.

[0122] The decoding device 200 may be a decoder, a video decoding device, or an image decoding device.

[0123] Referring to Figure 2, the decoding device 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

[0124] The decoding device 200 can receive the bitstream output from the encoding device 100. The decoding device 200 can receive the bitstream stored on a computer-readable recording medium or receive the bitstream being streamed via a wired / wireless transmission medium. The decoding device 200 can perform decoding on the bitstream in intra-mode or inter-mode. The decoding device 200 can also generate a restored image or a decoded image through decoding and can output a restored image or a decoded image.

[0125] If the prediction mode used for decoding is intra-mode, the switch can be switched to intra-mode. If the prediction mode used for decoding is inter-mode, the switch can be switched to inter-mode.

[0126] The decoding device 200 can decode the input bitstream, obtain the reconstructed residual block, and generate a predicted block. Once the reconstructed residual block and the predicted block are obtained, the decoding device 200 can generate the reconstructed block to be decoded by adding the reconstructed residual block and the predicted block. The block to be decoded is sometimes referred to as the current block.

[0127] The entropy decoding unit 210 can generate symbols by performing entropy decoding based on a probability distribution for the bitstream. The generated symbols may include symbols in quantization level form. Here, the entropy decoding method may be the inverse process of the entropy coding method described above.

[0128] The entropy decoding unit 210 can convert one-dimensional vector form coefficients into two-dimensional block form coefficients by scanning the transformation coefficients in order to decode the transformation coefficient level (quantization level).

[0129] The quantization level can be inversely quantized by the inverse quantization unit 220 and inversely transformed by the inverse transformation unit 230. The quantization level is the result of inverse quantization and / or inverse transformation, and can be generated as a restored residual block. At this time, the inverse quantization unit 220 can apply a quantization matrix to the quantization level.

[0130] When intra-mode is used, the intra-prediction unit 240 can generate predicted blocks by performing spatial predictions on the current block using sample values ​​of already decoded blocks around the block to be decoded.

[0131] When intermode is used, the motion compensation unit 250 can generate predicted blocks by performing motion compensation on the current block using the motion vector and the reference image stored in the reference picture buffer 270. The motion compensation unit 250 can generate predicted blocks by applying an interpolation filter to a portion of the reference image if the value of the motion vector does not have an integer value. In order to perform motion compensation, the motion compensation method of the prediction unit included in the coding unit can be determined based on the coding unit, and motion compensation can be performed according to each mode.

[0132] The adder 225 can generate a restored block by adding the restored residual block and the predicted block. The filter unit 260 can apply at least one of the following to the restored block or restored image: a deblocking filter, a sample-adaptive offset, and an adaptive loop filter. The filter unit 260 can output a restored image. The restored block or restored image is stored in the reference picture buffer 270 and can be used for inter-prediction. The restored block that has passed through the filter unit 260 may be part of the reference image. In other words, the reference image may be a restored image consisting of the restored block that has passed through the filter unit 260. The stored reference image can subsequently be used for inter-frame prediction or motion compensation.

[0133] Figure 3 is a schematic diagram showing the image partitioning structure when an image is encoded and decoded. Figure 3 schematically shows an embodiment in which one unit is divided into multiple subunits.

[0134] To efficiently divide images, coding units (CUs) can be used in encoding and decoding. Coding units can be used as the basic unit of image encoding / decoding. Furthermore, coding units can be used in units that distinguish between in-frame prediction modes and inter-frame prediction modes during image encoding / decoding. Coding units can be the basic unit used for the processes of prediction, transformation, quantization, inverse transformation, inverse quantization, or encoding / decoding of transformation coefficients.

[0135] Referring to Figure 3, Image 300 is sequentially divided into Largest Coding Unit (LCU) units, and the division structure is determined at the LCU level. Here, LCU can be used interchangeably with Coding Tree Unit (CTU). Unit division can mean the division of the block corresponding to the unit. Block division information may include information about the depth of the unit. Depth information can indicate the number and / or extent to which the unit is divided. A single unit can be hierarchically divided into multiple subunits based on a tree structure, with depth information. In other words, a unit and the subunits generated by the division of that unit can correspond to a node and its child nodes, respectively. Each divided subunit can have depth information. Depth information is information indicating the size of the CU and can be stored for each CU. Since unit depth indicates the number and / or extent to which the unit is divided, subunit division information can also include information about the size of the subunit.

[0136] The partition structure can refer to the distribution of coding units (CUs) within the CTU310. Such a distribution can be determined by whether or not a single CU is divided into multiple CUs (2 or more positive integers, including 2, 4, 8, 16, etc.). The width and height of the CUs generated by the partitioning can be half the width and half the height of the original CU, respectively, or they can be smaller than the width and height of the original CU, depending on the number of divisions. A CU can be recursively partitioned into multiple CUs. Recursive partitioning can reduce the size of at least one of the width and height of the partitioned CUs compared to at least one of the width and height of the original CU. The partitioning of CUs can be performed recursively up to a predetermined depth or a predetermined size. For example, the depth of the CTU may be 0, and the depth of the Smallest Coding Unit (SCU) may be a predetermined maximum depth. Here, CTU is the coding unit with the largest coding unit size, as described above, and SCU may be the coding unit with the smallest coding unit size. The division starts from CTU310, and each time the width and / or height of the CU decreases due to the division, the depth of the CU increases by 1. For example, at each depth, an undivided CU can have a size of 2N × 2N. Also, in the case of a divided CU, a 2N × 2N CU can be divided into four CUs, each having a size of N × N. The size of N can be halved each time the depth increases by 1.

[0137] Furthermore, information regarding whether or not a CU is partitioned can be represented through the partitioning information of the CU. The partitioning information can be 1 bit of information. All CUs except SCUs can contain partitioning information. For example, if the value of the partitioning information is the first value, the CU does not need to be partitioned, and if the value of the partitioning information is the second value, the CU may be partitioned.

[0138] Referring to Figure 3, a CTU with a depth of 0 can be a 64x64 block. 0 can be the minimum depth. An SCU with a depth of 3 can be an 8x8 block. 3 can be the maximum depth. CUs of 32x32 and 16x16 blocks can be represented with depths of 1 and 2, respectively.

[0139] For example, if one coding unit is divided into four coding units, the width and height of the four divided coding units can be half the size of the original coding unit. As an example, if a 32x32 coding unit is divided into four coding units, each of the four divided coding units can be 16x16 in size. When one coding unit is divided into four coding units, the coding unit can be said to have been divided in a quad-tree manner (quad-tree partitioning).

[0140] For example, when one coding unit is divided into two coding units, the width or height of the two divided coding units can be half the size of the original coding unit. For example, when a 32x32 coding unit is vertically divided into two coding units, the two divided coding units can each have a size of 16x32. For example, when an 8x32 coding unit is horizontally divided into two coding units, the two divided coding units can each have a size of 8x16. When one coding unit is divided into two coding units, the coding unit can be said to have been divided in a binary-tree manner (binary-tree partition).

[0141] For example, if one coding unit is to be divided into three coding units, this can be achieved by dividing the width or height of the coding unit in a 1:2:1 ratio before the division. For instance, if a 16x32 coding unit is horizontally divided into three coding units, the three divided coding units can have sizes of 16x8, 16x16, and 16x8 from top to bottom. For instance, if a 32x32 coding unit is vertically divided into three coding units, the three divided coding units can have sizes of 8x32, 16x32, and 8x32 from left to right. When one coding unit is divided into three coding units, the coding unit can be said to have been divided in a ternary-tree manner (ternary-tree partition).

[0142] Figure 3 shows CTU320, an example of a CTU to which quadtree, binary tree, and ternary tree partitioning are all applied.

[0143] As mentioned above, at least one of quadtree partitioning, binary tree partitioning, and ternary tree partitioning can be applied to partition a CTU. Each partitioning method can be applied according to a predetermined priority. For example, quadtree partitioning can be applied preferentially to a CTU. A coded unit that cannot be further partitioned using quadtree partitioning can become a leaf node of a quadtree. A coded unit that is a leaf node of a quadtree can become the root node of a binary tree and / or ternary tree. In other words, a coded unit that is a leaf node of a quadtree can be partitioned using binary tree partitioning, ternary tree partitioning, or cannot be partitioned any further. In this case, by preventing further quadtree partitioning of coded units generated by binary tree partitioning or ternary tree partitioning of a coded unit that is a leaf node of a quadtree, block partitioning and / or signaling of partition information can be effectively performed.

[0144] The partitioning of coding units corresponding to each node in a quadtree can be signaled using quad partitioning information. Quad partitioning information with a first value (e.g., "1") can indicate that the coding unit in question will be partitioned in a quadtree. Quad partitioning information with a second value (e.g., "0") can indicate that the coding unit will not be partitioned in a quadtree. Quad partitioning information can be a flag with a predetermined length (e.g., 1 bit).

[0145] There may be no priority order between binary tree splitting and ternary tree splitting. In other words, a coding unit corresponding to a leaf node of a quadtree can be split into a binary tree or a ternary tree. Furthermore, coding units generated by binary tree splitting or ternary tree splitting can either be split into a binary tree or a ternary tree again, or they cannot be split further.

[0146] A partition where there is no priority between binary tree partitioning and ternary tree partitioning is sometimes called a multi-type tree partition. That is, an encoding unit corresponding to a leaf node of a quadtree can become the root node of a multi-type tree. The partitioning of an encoding unit corresponding to each node of a multi-type tree can be signaled using at least one of the following: information on whether or not a multi-type tree is being partitioned, information on the partitioning direction, and information on the partitioned tree. For the partitioning of an encoding unit corresponding to each node of the multi-type tree, the information on whether or not a partition is being partitioned, information on the partitioning direction, and information on the partitioned tree may be signaled sequentially.

[0147] Information indicating whether or not a composite tree is partitioned, having a first value (e.g., "1"), can indicate that the encoding unit will be partitioned into a composite tree. Information indicating whether or not a composite tree is partitioned, having a second value (e.g., "0"), can indicate that the encoding unit will not be partitioned into a composite tree.

[0148] When a coding unit corresponding to each node of a complex tree is subjected to complex tree partitioning, the coding unit may further include partitioning direction information. The partitioning direction information can indicate the direction of the complex tree partitioning. Partitioning direction information having a first value (e.g., "1") can indicate that the coding unit is partitioned vertically. Partitioning direction information having a second value (e.g., "0") can indicate that the coding unit is partitioned horizontally.

[0149] If the coding unit corresponding to each node of a composite tree is subjected to a composite tree partition, the coding unit may further include partition tree information. The partition tree information may indicate the tree used for the composite tree partition. Partition tree information having a first value (e.g., "1") may indicate that the coding unit is subjected to a binary tree partition. Partition tree information having a second value (e.g., "0") may indicate that the coding unit is subjected to a ternary tree partition.

[0150] The information regarding whether or not a partition is being created, the partition tree information, and the partition direction information may each be flags having a predetermined length (e.g., 1 bit).

[0151] At least one of the following can be entropy encoded / decoded: quad partitioning information, information on whether or not a complex tree is partitioned, partitioning direction information, and partition tree information. For the entropy encoding / decoding of the said information, information from surrounding encoding units adjacent to the current encoding unit can be used. For example, the partitioning configuration (whether or not it is partitioned, partition tree, and / or partitioning direction) of the left encoding unit and / or the upper encoding unit has a high probability of being similar to the partitioning configuration of the current encoding unit. Therefore, contextual information for entropy encoding / decoding of the current encoding unit's information can be derived based on the information of the surrounding encoding units. In this case, the information of the surrounding encoding units may include at least one of the following: quad partitioning information of the encoding unit, information on whether or not a complex tree is partitioned, partitioning direction information, and partition tree information.

[0152] As another embodiment, among the binary tree division and the ternary tree division, the binary tree division can be preferentially executed. That is, the binary tree division can be applied first, and the coding unit corresponding to the leaf node of the binary tree can also be set as the root node of the ternary tree. In this case, the quad-tree division and the binary tree division may not be performed on the coding unit corresponding to the node of the ternary tree.

[0153] The coding unit that cannot be further divided by the quad-tree division, the binary tree division, and / or the ternary tree division can be a unit of coding, prediction, and / or transformation. That is, the coding unit may not be further divided for prediction and / or transformation. Therefore, the division structure, division information, etc. for dividing the coding unit into a prediction unit and / or a transformation unit may not exist in the bitstream.

[0154] However, when the size of the coding unit that is the unit of division is larger than the size of the maximum transformation block, the corresponding coding unit can be recursively divided until it becomes the same size as or smaller than the size of the maximum transformation block. For example, when the size of the coding unit is 64x64 and the size of the maximum transformation block is 32x32, the coding unit can be divided into four 32x32 blocks for transformation. For example, when the size of the coding unit is 32x64 and the size of the maximum transformation block is 32x32, the coding unit can be divided into two 32x32 blocks for transformation. In this case, whether the coding unit is divided for transformation is not signaled separately and can be determined by comparing the horizontal or vertical of the coding unit with the horizontal or vertical of the maximum transformation block. For example, when the horizontal of the coding unit is larger than the horizontal of the maximum transformation block, the coding unit can be bisected vertically. Also, when the vertical of the coding unit is larger than the vertical of the maximum transformation block, the coding unit can be bisected horizontally.

[0155] Information regarding the maximum and / or minimum size of the symbolization unit and information regarding the maximum and / or minimum size of the conversion block can be signaled or determined at a higher level of the symbolization unit. The higher level may be, for example, the sequence level, picture level, slice level, tile group level, tile level, etc. For example, the minimum size of the symbolization unit may be determined to be 4x4. For example, the maximum size of the conversion block may be determined to be 64x64. For example, the minimum size of the conversion block may be determined to be 4x4.

[0156] Information regarding the minimum size of the symbolization unit (the minimum size of the quadtree) corresponding to the leaf node of the quadtree and / or information regarding the maximum depth (the maximum depth of the composite tree) from the root node to the leaf node of the composite tree can be signaled or determined at a higher level of the symbolization unit. The higher level may be, for example, the sequence level, picture level, slice level, tile group level, tile level, etc. The information regarding the minimum size of the quadtree and / or the information regarding the maximum depth of the composite tree can be signaled or determined for each of the in-picture slice and the inter-picture slice.

[0157] Difference information regarding the size of the CTU and the maximum size of the transformation block can be signaled or determined at a higher level of the coding unit. This higher level may be, for example, the sequence level, picture level, slice level, tile group level, or tile level. Information regarding the maximum size of the coding unit corresponding to each node of a binary tree (maximum size of the binary tree) can be determined based on the size of the coding tree unit and the difference information. The maximum size of the coding unit corresponding to each node of a ternary tree (maximum size of the ternary tree) can have different values ​​depending on the type of slice. For example, in the case of an intra-screen slice, the maximum size of the ternary tree may be 32x32. Also, for example, in the case of an inter-screen slice, the maximum size of the ternary tree may be 128x128. For example, the minimum size of the coding unit corresponding to each node of a binary tree (minimum size of the binary tree) and / or the minimum size of the coding unit corresponding to each node of a ternary tree (minimum size of the ternary tree) can be set as the minimum size of the coding block.

[0158] As another example, the maximum size of a binary tree and / or a ternary tree can be signaled or determined at the slice level. Similarly, the minimum size of a binary tree and / or a ternary tree can be signaled or determined at the slice level.

[0159] Based on the various block size and depth information mentioned above, quad partitioning information, information on whether or not it is a complex tree partition, partition tree information, and / or partitioning direction information may or may not be present in the bitstream.

[0160] For example, if the size of the coding unit is not greater than the minimum size of the quadtree, the coding unit does not contain quad partitioning information, and this quad partitioning information can be inferred into the second value.

[0161] For example, if the size (width and height) of a coding unit corresponding to a node in a composite tree is larger than the maximum size (width and height) of a binary tree and / or a ternary tree, the coding unit may not undergo binary and / or ternary tree splitting. As a result, information about whether or not the composite tree is split is not signaled and can be inferred into a second value.

[0162] Alternatively, if the size (width and height) of the coding unit corresponding to a node in a composite tree is the same as the minimum size (width and height) of a binary tree, or if the size (width and height) of the coding unit is twice the minimum size (width and height) of a ternary tree, the coding unit may not be split into a binary tree and / or a ternary tree. As a result, information on whether or not the composite tree is split is not signaled and can be inferred into a second value. This is because, when the coding unit is split into a binary tree and / or a ternary tree, a coding unit smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree is generated.

[0163] Alternatively, if the depth of the encoding unit corresponding to a node in the composite tree is the same as the maximum depth of the composite tree, the encoding unit may not undergo binary and / or ternary tree splitting. As a result, information on whether or not the composite tree is split is not signaled and can be inferred into a second value.

[0164] Alternatively, information regarding whether or not the composite tree is being split can be signaled only if at least one of the following is possible for the coding unit corresponding to a node of the composite tree: vertical binary tree splitting, horizontal binary tree splitting, vertical ternary tree splitting, and horizontal ternary tree splitting. Otherwise, the coding unit may not be subjected to binary tree splitting and / or ternary tree splitting. As a result, information regarding whether or not the composite tree is being split is not signaled and can be inferred to a second value.

[0165] Alternatively, the splitting direction information can be signaled only if both vertical and horizontal binary tree splitting are possible for the encoding unit corresponding to a node in the composite tree, or if both vertical and horizontal ternary tree splitting are possible. Otherwise, the splitting direction information is not signaled and can be inferred to a value indicating the possible splitting direction.

[0166] Alternatively, the partitioned tree information can be signaled only if both vertical binary tree partitioning and vertical ternary tree partitioning are possible for the encoding unit corresponding to a node of the composite tree, or if both horizontal binary tree partitioning and horizontal ternary tree partitioning are possible. Otherwise, the partitioned tree information is not signaled and can be inferred to a value indicating a tree that can be partitioned.

[0167] Figure 4 is a diagram illustrating an embodiment of the in-screen prediction process.

[0168] The arrows from the center to the outer edge in Figure 4 can indicate the prediction direction for the on-screen prediction mode.

[0169] In-screen encoding and / or decoding may be performed using reference samples from surrounding blocks of the current block. These surrounding blocks may be the reconstructed surrounding blocks. For example, in-screen encoding and / or decoding may be performed using the reference sample values ​​or encoding parameters contained in the reconstructed surrounding blocks.

[0170] A prediction block can mean a block generated as a result of performing an on-screen prediction. A prediction block can correspond to at least one of CU, PU, ​​and TU. The unit of a prediction block can be at least one of the sizes of CU, PU, ​​and TU. A prediction block may be a square-shaped block with sizes such as 2x2, 4x4, 16x16, 32x32, or 64x64, or a rectangular-shaped block with sizes such as 2x8, 4x8, 2x16, 4x16, and 8x16.

[0171] In-screen prediction may be performed depending on the in-screen prediction mode for the current block. The number of in-screen prediction modes that a current block can have is a predetermined fixed value, which may be a value determined differently depending on the attributes of the prediction block. For example, the attributes of the prediction block may include the size and shape of the prediction block.

[0172] The number of in-screen prediction modes can be fixed at N, regardless of the block size. Alternatively, the number of in-screen prediction modes could be 3, 5, 9, 17, 34, 35, 36, 65, or 67, for example. Alternatively, the number of in-screen prediction modes could vary depending on the block size and / or the type of color component. For example, the number of in-screen prediction modes could differ depending on whether the color component is a lumen signal or a chroma signal. For example, the number of in-screen prediction modes could increase as the block size increases. Alternatively, the number of in-screen prediction modes in a lumen component block could be greater than the number of in-screen prediction modes in a chroma component block.

[0173] The on-screen prediction mode can be a non-directional mode or a directional mode. The non-directional mode can be a DC mode or a Planar mode. The directional mode (angular mode) can be a prediction mode with a specific direction or angle. The on-screen prediction mode can be represented by at least one of the following: mode number, mode value, mode digit, mode angle, and mode direction. The number of on-screen prediction modes can be one to M, including the non-directional and directional modes.

[0174] A step may be taken to check whether the samples contained in the restored surrounding blocks, which are used to predict the current block on screen, are available as reference samples for the current block. If there are samples that are not available as reference samples for the current block, the sample values ​​of the samples that are not available as reference samples are replaced with a copy and / or interpolated value of at least one sample value from the restored surrounding blocks, and then the sample values ​​of the samples that are not available as reference samples are made available as reference samples for the current block.

[0175] A filter can be applied to at least one of the reference sample or prediction sample based on at least one of the in-screen prediction mode and the current block size during in-screen prediction.

[0176] In planner mode, when generating a prediction block for the current block, the sample value of the predicted sample can be generated using a weighted sum of the upper and left reference samples of the current sample, and the upper right and lower left reference samples of the current block, based on the position of the predicted sample within the prediction block. In DC mode, the average value of the upper and left reference samples of the current block can be used when generating a prediction block for the current block. In directional mode, the prediction block can be generated using the upper, left, upper right, and / or lower left reference samples of the current block. Interpolation in real units can also be performed to generate the predicted sample value.

[0177] The in-picture prediction mode of the current block can be entropy-encoded / decoded by predicting from the in-picture prediction modes of the blocks existing around the current block. If the in-picture prediction modes of the current block and the surrounding blocks are the same, information indicating that the in-picture prediction modes of the current block and the surrounding blocks are the same can be signaled using predetermined flag information. Also, among the in-picture prediction modes of a plurality of surrounding blocks, indicator information for the in-picture prediction mode that is the same as the in-picture prediction mode of the current block can be signaled. When the in-picture prediction modes of the current block and the surrounding blocks are different from each other, the in-picture prediction mode information of the current block can be entropy-encoded / decoded by performing entropy-encoding / decoding based on the in-picture prediction mode of the surrounding blocks.

[0178] FIG. 5 is a diagram for explaining an embodiment of an inter-picture prediction process.

[0179] The quadrilaterals shown in FIG. 5 can represent images. Also, the arrows in FIG. 4 can indicate the prediction direction. Each image can be classified into an I picture (Intra Picture), a P picture (Predictive Picture), a B picture (Bi-predictive Picture), etc. according to the encoding type.

[0180] An I picture can be encoded / decoded via in-picture prediction without inter-picture prediction. A P picture can be encoded / decoded via inter-picture prediction that uses only a reference image existing in one direction (e.g., the forward direction or the reverse direction). A B picture can be encoded / decoded via inter-picture prediction that uses reference images existing in both directions (e.g., the forward direction and the reverse direction). Also, in the case of a B picture, it can be encoded / decoded via inter-picture prediction that uses reference images existing in both directions or inter-picture prediction that uses a reference image existing in one of the forward direction and the reverse direction. Here, both directions can be the forward direction and the reverse direction. Here, when inter-picture prediction is used, the encoder can perform inter-picture prediction or motion compensation, and the decoder can perform corresponding motion compensation.

[0181] The following describes in detail the inter-screen prediction method according to the embodiment.

[0182] Inter-screen prediction or motion compensation can be performed using reference images and motion information.

[0183] Motion information for the current block can be derived during inter-screen prediction by the encoding device 100 and the decoding device 200, respectively. The motion information can be derived using the motion information of the recovered surrounding blocks, the motion information of the collocated block (col block), and / or blocks adjacent to the collocated block. The collocated block may be a block that corresponds to the spatial position of the current block within an already recovered collocated picture (col picture). Here, the collocated picture may be one of the at least one reference images included in the reference image list.

[0184] The method for deriving motion information may vary depending on the prediction mode of the current block. For example, prediction modes applied for inter-screen prediction may include AMVP mode, merge mode, skip mode, and current picture reference mode. Here, merge mode is sometimes referred to as motion merge mode.

[0185] For example, when AMVP is applied as the prediction mode, at least one of the recovered motion vectors of surrounding blocks, the motion vectors of collated blocks, the motion vectors of blocks adjacent to the collated block, and the (0,0) motion vector can be determined as motion vector candidates to generate a motion vector candidate list. Motion vector candidates can be induced using the generated motion vector candidate list. Based on the induced motion vector candidates, the motion information of the current block can be determined. Here, the motion vector of the collated block or the motion vector of a block adjacent to the collated block may be called a temporal motion vector candidate, and the recovered motion vector of the surrounding block may be called a spatial motion vector candidate.

[0186] The encoding device 100 can calculate the motion vector difference (MVD) between the current block's motion vector and a motion vector candidate, and can entropically encode the MVD. The encoding device 100 can also entropically encode the motion vector candidate index to generate a bitstream. The motion vector candidate index can indicate the optimal motion vector candidate selected from among the motion vector candidates included in the motion vector candidate list. The decoding device 200 entropically decodes the motion vector candidate index from the bitstream and uses the entropically decoded motion vector candidate index to select a motion vector candidate for the block to be decoded from among the motion vector candidates included in the motion vector candidate list. Furthermore, the decoding device 200 can derive the motion vector of the block to be decoded using the sum of the entropically decoded MVD and the motion vector candidate.

[0187] The bitstream may include a reference image index that indicates a reference image. The reference image index can be entropy encoded and signaled from the encoding device 100 to the decoding device 200 via the bitstream. The decoding device 200 can generate a predicted block for the block to be decoded based on the induced motion vector and the reference image index information.

[0188] Another example of a motion information derivation method is the merge mode. The merge mode can mean the merging of motion for multiple blocks. The merge mode can mean a mode in which the motion information of the current block is derived from the motion information of surrounding blocks. When the merge mode is applied, a merge candidate list can be generated using the recovered motion information of surrounding blocks and / or the motion information of collated blocks. The motion information may include at least one of the following: 1) motion vectors, 2) reference image indices, and 3) inter-screen prediction indicators. The prediction indicators can be unidirectional (L0 prediction, L1 prediction) or bidirectional.

[0189] The merge candidate list can represent a list in which motion information is stored. The motion information stored in the merge candidate list may be at least one of the following: motion information of surrounding blocks adjacent to the current block (spatial merge candidate), motion information of blocks corresponding to the current block in the reference image (collocated) (temporal merge candidate), new motion information generated by a combination of motion information already present in the merge candidate list, and zero merge candidates.

[0190] The encoding device 100 can generate a bitstream by entropy encoding at least one of the merge flag and the merge index, and then signal it to the decoding device 200. The merge flag is information indicating whether or not to perform a merge mode on a block-by-block basis, and the merge index may be information about which of the surrounding blocks adjacent to the current block to merge with. For example, the surrounding blocks of the current block may include at least one of the blocks adjacent to the left of the current block, the blocks adjacent above it, and the blocks adjacent in time.

[0191] The skip mode may be a mode in which the motion information of surrounding blocks is applied directly to the current block. When the skip mode is used, the encoding device 100 can entropically encode information about which block's motion information to use as the current block's motion information and signal it to the decoding device 200 via a bitstream. In this case, the encoding device 100 does not need to signal syntactic elements relating to at least one of the motion vector difference information, encoded block flags, and conversion coefficient levels (quantization levels) to the decoding device 200.

[0192] The current picture reference mode can mean a prediction mode that uses an already restored region within the current picture to which the current block belongs. In this case, a vector can be defined to identify the already restored region. Whether or not the current block is encoded in current picture reference mode can be encoded using the reference image index of the current block. A flag or index indicating whether or not the current block is a block encoded in current picture reference mode may be signaled, or may be inferred using the reference image index of the current block. If the current block is encoded in current picture reference mode, the current picture can be added to the reference image list for the current block at a fixed position or at an arbitrary position. The fixed position may be, for example, the position where the reference image index is 0 or the last position. If the current picture is added at an arbitrary position in the reference image list, a separate reference image index indicating the arbitrary position may be signaled.

[0193] Figure 6 is a diagram illustrating the transformation and quantization process.

[0194] As shown in Figure 6, a quantized level can be generated by performing a conversion and / or quantization process on the residual signal. The residual signal can be generated from the difference between the original block and the prediction block (in-screen prediction block or inter-screen prediction block). Here, the prediction block may be a block generated by in-screen prediction or inter-screen prediction. Here, the conversion may include at least one of a linear conversion and a quadratic conversion. By performing a linear conversion on the residual signal, conversion coefficients can be generated, and by performing a quadratic conversion on the conversion coefficients, quadratic conversion coefficients can be generated.

[0195] A primary transform can be performed using at least one of a predetermined set of transform methods. For example, these transform methods may include DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT (Karhunen-Loeve Transform) based transforms. A secondary transform can be performed on the transform coefficients generated after the primary transform. The transform method applied during the primary and / or secondary transform can be determined based on at least one of the coding parameters of the current block and / or surrounding blocks. Alternatively, transform information indicating the transform method may be signaled.

[0196] Quantization levels can be generated by quantizing the result or residual signal after a linear and / or quadratic transformation. The quantization levels can be scanned by at least one of the following: an up-right diagonal scan, a vertical scan, or a horizontal scan, based on at least one of the in-screen prediction mode or the size / shape of the block. For example, scanning the block coefficients using an up-right diagonal scan can convert them to a one-dimensional vector format. Depending on the size of the transformed block and / or the in-screen prediction mode, a vertical scan scanning two-dimensional block shape coefficients in the column direction or a horizontal scan scanning two-dimensional block shape coefficients in the row direction may be used instead of an up-right diagonal scan. The scanned quantization levels can be entropy-encoded and included in the bitstream.

[0197] The decoder can generate quantization levels by entropy decoding the bitstream. These quantization levels can then be inverse scanned and aligned into a two-dimensional block shape. At this time, at least one of the following methods can be used for inverse scanning: upper right diagonal scan, vertical scan, and horizontal scan.

[0198] It is possible to perform inverse quantization at the quantization level, perform a quadratic inverse transform depending on whether a quadratic inverse transform is performed, and then perform a linear inverse transform on the result of the quadratic inverse transform depending on whether a linear inverse transform is performed, thereby generating the restored residual signal.

[0199] The following describes an intra-loop filtering method using subsample-based block classification according to one embodiment of the present invention, with reference to Figures 7 to 55d.

[0200] In the present invention, in-loop filtering may include at least one of deblocking filtering, sample adaptive offset, bilateral filtering, and adaptive in-loop filtering.

[0201] By applying at least one of the deblocking filtering and sample-adaptive offsetting to the reconstructed image, which is generated by combining the predicted blocks within / between screens and the reconstructed residual blocks, blocking artifacts and ringing artifacts present in the reconstructed image can be effectively reduced. However, while the deblocking filter aims to reduce blocking artifacts occurring at the boundaries between blocks by performing vertical and horizontal filtering on the block boundaries, it has the drawback that it cannot minimize the distortion between the original image and the reconstructed image when filtering block boundaries. Furthermore, sample-adaptive offsetting is a method that reduces ringing artifacts by comparing the pixel values ​​of adjacent pixels on a pixel-by-pixel basis and then adding an offset to a specific pixel or to pixels whose pixel values ​​belong to a specific range. Sample-adaptive offsetting partially minimizes the distortion between the original image and the reconstructed image using rate-distortion optimization, but there are limitations in terms of distortion minimization when the difference in distortion between the original image and the reconstructed image is large.

[0202] The aforementioned bidirectional filtering can mean that filtering is performed by determining the filter coefficients based on the distance between the central sample and other samples within the filtering region, and the difference in values ​​between the samples.

[0203] The adaptive intra-loop filtering described above can be interpreted as filtering that minimizes distortion between the original image and the reconstructed image by applying a filter to the reconstructed image that minimizes the distortion between the original image and the reconstructed image.

[0204] In this invention, unless otherwise specified, intra-loop filtering may mean adaptive intra-loop filtering.

[0205] In the present invention, filtering can mean applying a filter to at least one unit from among a sample, block, CU, PU, ​​TU, CTU, slice, tile, tile group, image (picture), and sequence. The filtering may include at least one of block classification, filtering execution, and filter information encoding / decoding.

[0206] In the present invention, CU (Coding Unit), PU (Prediction Unit), TU (Transform Unit), and CTU (Coding Tree Unit) may have the same meaning as CB (Coding Block), PB (Prediction Block), TB (Transform Block), and CTB (Coding Tree Block), respectively.

[0207] In the present invention, a block can mean at least one of the blocks or unit units used in the encoding / decoding process, such as CU, PU, ​​TU, CB, PB, TB, etc.

[0208] The aforementioned intra-loop filtering can be applied to the reconstructed image in the following order: bidirectional filtering, deblocking filtering, sample-adaptive offset, and adaptive intra-loop filtering, to generate a decoded image. However, the decoded image can also be generated by changing the order of the filtering processes included in the intra-loop filtering and applying them to the reconstructed image.

[0209] As an example, loop filtering can be applied to the reconstructed image in the following order: deblocking filtering, sample-adaptive offsetting, and adaptive loop filtering.

[0210] As another example, loop filtering can be applied to the reconstructed image in the following order: bidirectional filtering, adaptive loop filtering, deblocking filtering, and sample-adaptive offsetting.

[0211] As another example, in-loop filtering can be applied to the reconstructed image in the following order: adaptive in-loop filtering, deblocking filtering, and sample-adaptive offsetting.

[0212] As another example, intra-loop filtering can be applied to the reconstructed image in the following order: adaptive intra-loop filtering, sample-adaptive offset, and deblocking filtering.

[0213] In the present invention, the decoded image can mean the image resulting from performing loop filtering or post-processing filtering on a reconstructed image composed of reconstructed blocks generated by adding reconstructed residual blocks to in-screen prediction blocks or inter-screen prediction blocks. Furthermore, in the present invention, the meanings of decoded sample / block / CTU / image and reconstructed sample / block / CTU / image are not different from each other and can mean the same thing.

[0214] The adaptive intra-loop filtering described above can be applied to the reconstructed image to generate the decoded image. Alternatively, the adaptive intra-loop filtering can be applied to a decoded image on which at least one of deblocking filtering, sample-adaptive offsetting, and bidirectional filtering has been performed. Furthermore, the adaptive intra-loop filtering can be applied to a decoded image on which adaptive intra-loop filtering has been performed. In this case, the adaptive intra-loop filtering can be repeated N times on the reconstructed or decoded image, where N is a positive integer.

[0215] In-loop filtering can be performed on a decoded image that has been filtered by at least one of the aforementioned in-loop filtering methods. For example, if at least one other filtering method is performed on a decoded image that has been filtered by at least one of the aforementioned in-loop filtering methods, the parameters used in at least one of the aforementioned in-loop filtering methods are changed, and the modified in-loop filtering method can be applied to the decoded image. In this case, the parameters may include at least one of the following: coding parameters, filter coefficients, number of filter taps (filter length), filter shape, filter type, number of filtering executions, filter strength, and threshold.

[0216] The aforementioned filter coefficients can refer to the coefficients that constitute the filter, and can refer to the coefficient values ​​that correspond to specific mask positions in the mask shape, which are applied to the restored sample.

[0217] The number of filter taps represents the filter length, and if the filter has a symmetric characteristic with respect to one specific direction, the actual number of filter coefficients subject to encoding / decoding can be reduced by half. Furthermore, the number of filter taps can represent either the width of the filter in the horizontal direction or the height of the filter in the vertical direction. In a two-dimensional filter configuration, the number of filter taps can also represent both the width and height of the filter in the horizontal direction. Additionally, the filter can have symmetric characteristics with respect to two or more specific directions.

[0218] The aforementioned filter shape, when the filter has a mask shape, can refer to geometric shapes that can be represented in two dimensions, such as a diamond / rhombus, rectangle (non-square), square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, dodecagon, or combinations thereof. Alternatively, the filter shape may be the shape obtained by projecting a three-dimensional filter two-dimensionally.

[0219] The aforementioned filter types can refer to filters such as Wiener filters, low-pass filters, high-pass filters, linear filters, non-linear filters, and bilateral filters.

[0220] In this invention, while the Wiener filter will be the main focus of the description among the various types of filters mentioned above, the invention is not limited to this, and embodiments or combinations of embodiments of this invention can be applied to the various types of filters mentioned above.

[0221] A Wiener filter can be used as a filter type for adaptive loop filtering. The Wiener filter is an optimal linear filter, and its purpose is to improve encoding efficiency by effectively removing noise, blurring, and distortion in the image. In this case, the Wiener filter can be designed to minimize the distortion between the original image and the reconstructed / decoded image.

[0222] At least one of the filtering methods described above may be performed within the encoding or decoding process. In this case, the encoding or decoding process can mean that the encoding or decoding process is performed in units of at least one of slices, tiles, tile groups, pictures, sequences, CTUs, blocks, CUs, PUs, and TUs. At least one of the filtering methods described above may be performed within the encoding or decoding process in units such as slices, tiles, tile groups, and pictures. For example, a Wiener filter may be performed within the encoding or decoding process in the form of adaptive loop filtering. In other words, "within the loop" in adaptive loop filtering means that filtering is performed within the encoding or decoding process. When adaptive loop filtering is performed, the decoded image that has undergone adaptive loop filtering can be used as a reference image for the image to be encoded / decoded later. In this case, the image to be encoded / decoded later can refer to the decoded image that has undergone adaptive loop filtering to perform inter-screen prediction or motion compensation, thereby improving not only the encoding efficiency of the decoded image that has undergone adaptive loop filtering, but also the encoding efficiency of the image to be encoded / decoded later. Furthermore, at least one of the filtering methods may be performed within a CTU-unit or block-unit encoding or decoding process. For example, a Wiener filter may be performed within a CTU-unit or block-unit encoding or decoding process in the form of adaptive loop filtering. That is, "within the loop" in adaptive loop filtering means that filtering is performed within a CTU-unit or block-unit encoding or decoding process. When adaptive loop filtering is performed on a CTU-unit or block-unit basis, the decoded CTU or decoded block that has undergone adaptive loop filtering can also be used as a reference CTU or reference block for a subsequently encoded / decoded CTU or block. In this case, the subsequently encoded / decoded CTU or block can refer to the decoded CTU or decoded block that has undergone adaptive loop filtering to perform on-screen prediction, etc., thus improving not only the encoding efficiency of the decoded CTU or decoded block that has undergone adaptive loop filtering, but also the encoding efficiency of the subsequently encoded / decoded CTU or block.

[0223] Furthermore, at least one of the filtering methods may be performed after the decoding process in the form of post-processing filtering. For example, a Wiener filter can be applied after the decoding process in the form of post-processing filtering. When a Wiener filter is applied after the decoding process, it can be applied to the restored / decoded image before image output (display). When post-processing filtering is performed, the decoded image that has undergone post-processing filtering cannot be used as a reference image for images that will be subsequently encoded / decoded.

[0224] Adaptive intra-loop filtering can utilize block-based filter adaptation. Here, block-based filter adaptation can be defined as the adaptive selection of filters to be used for each block from a large number of filters. The aforementioned block-based filter adaptation can also be defined as block classification.

[0225] Figure 7 is a flowchart showing an image decoding method according to one embodiment of the present invention.

[0226] Referring to Figure 7, the decoder can decode the filter information for the encoding unit (S701).

[0227] The aforementioned filter information is not limited to encoding units, but can also refer to filter information for slices, tiles, tile groups, pictures, sequences, CTUs, blocks, CUs, PUs, TUs, etc.

[0228] On the other hand, the filter information may include at least one of the following: whether filtering is performed, filter coefficient value, number of filters, number of filter taps (filter length) information, filter shape information, filter type information, information on whether a fixed filter is used for the block classification index, and information on the symmetric shape of the filter.

[0229] On the other hand, the filter shape information may include information on at least one of the following shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0230] On the other hand, the filter coefficient values ​​may include filter coefficient values ​​that have been geometrically transformed based on blocks classified by block classification units.

[0231] On the other hand, information regarding the symmetry shape of the filter may include information about at least one of the following: point symmetry, transverse symmetry, vertical symmetry, and diagonal symmetry.

[0232] Furthermore, the decoder can classify the coding units into block classification units (S702). The decoder can also assign a block classification index to the coding units classified into block classification units.

[0233] The aforementioned block classification is not limited to being performed at the coding unit level, but can also be performed at the slice, tile, tile group, picture, sequence, CTU, block, CU, PU, ​​TU, and other units.

[0234] On the other hand, the aforementioned block classification index can be determined using directional information and activity information.

[0235] On the other hand, at least one of the directional information and activity information can be determined based on a gradient value in at least one of the vertical, horizontal, first diagonal, and second diagonal directions.

[0236] On the other hand, the gradient value can be obtained using a 1D Laplacian operation at the block classification unit level.

[0237] On the other hand, the 1D Laplacian operation may be a 1D Laplacian operation in which the execution position of the operation is subsampled.

[0238] On the other hand, the gradient value can be determined based on a temporal layer identifier.

[0239] Furthermore, the decoder can use the filter information to filter the coding units classified by the block classification units (S703).

[0240] The target of the filtering is not limited to coding units, but may include slices, tiles, tile groups, pictures, sequences, CTUs, blocks, CUs, PUs, TUs, etc.

[0241] Figure 8 is a flowchart showing an image encoding method according to one embodiment of the present invention.

[0242] Referring to Figure 8, the encoder can classify the coding units into block classification units (S801). The encoder can also assign a block classification index to the coding units classified into the block classification units.

[0243] The aforementioned block classification is not limited to being performed at the coding unit level, but can also be performed at the slice, tile, tile group, picture, sequence, CTU, block, CU, PU, ​​TU, and other units.

[0244] On the other hand, the aforementioned block classification index can be determined using directional information and activity information.

[0245] On the other hand, at least one of the directional information and activity information can be determined based on a gradient value in at least one of the vertical, horizontal, first diagonal, and second diagonal directions.

[0246] On the other hand, the gradient value can be obtained using a 1D Laplacian operation at the block classification unit level.

[0247] On the other hand, the 1D Laplacian operation may be a 1D Laplacian operation in which the execution position of the operation is subsampled.

[0248] On the other hand, the gradient value can be determined based on the temporal hierarchy identifier.

[0249] Furthermore, the encoder can filter the encoding units classified by the block classification unit using the filter information for the encoding unit (S802).

[0250] The target of the filtering is not limited to coding units, but may include slices, tiles, tile groups, pictures, sequences, CTUs, blocks, CUs, PUs, TUs, etc.

[0251] On the other hand, the filter information may include at least one of the following: whether or not a filter is executed, filter coefficient value, number of filters, number of filter taps (filter length) information, filter shape information, filter type information, information on whether or not a fixed filter is used for the block classification index, and information on the symmetric shape of the filter.

[0252] On the other hand, the filter shape information may include information on at least one of the following shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0253] On the other hand, the filter coefficient values ​​may include filter coefficient values ​​that have been geometrically transformed based on blocks classified by block classification units.

[0254] Furthermore, the encoder can encode the filter information (S803).

[0255] The aforementioned filter information can refer to filter information for slices, tiles, tile groups, pictures, sequences, CTUs, blocks, CUs, PUs, TUs, etc.

[0256] In an encoder, adaptive intra-loop filtering can consist of a block classification step, a filtering execution step, and a filtering information encoding step.

[0257] More specifically, the encoder can consist of a block classification step, a filter coefficient induction step, a filtering execution decision step, a filter shape determination step, a filtering execution step, and a filter information encoding step. The filter coefficient induction step, the filtering execution decision step, and the filter shape determination step are outside the scope of the present invention and will not be specifically mentioned in this invention, but they can be briefly described as follows. Therefore, the encoder may include a block classification step, a filtering execution step, and a filter information encoding step.

[0258] In the filter coefficient derivation step, Wiener filter coefficients can be derived from the viewpoint of minimizing the distortion between the original image and the filtered image. Wiener filter coefficients can be derived for each block classification index. Furthermore, Wiener filter coefficients can be derived according to at least one of the filter taps and filter shape. When deriving Wiener filter coefficients, an autocorrelation function for the reconstructed samples, a cross-correlation function for the original and reconstructed samples, an autocorrelation matrix, and a cross-correlation matrix can be derived. The Wiener-Hopf equation can be derived using the derived autocorrelation matrix and cross-correlation matrix to calculate the filter coefficients. At this time, the filter coefficients can be calculated using the Gaussian elimination method or the Cholesky decomposition method on the Wiener-Hopf equation.

[0259] In the filtering execution decision step, it is possible to decide, from the perspective of rate-distortion optimization, whether to apply adaptive intra-loop filtering at the slice / picture / tile / tile group level, at the block level, or not to apply adaptive intra-loop filtering at all. Here, the rate can include the filter information to be encoded. The distortion can be a value for the difference between the original image and the reconstructed image, or the difference between the original image and the filtered reconstructed image. MSE (Mean of Squared Error), SSE (Sum of Squared Error), SAD (Sum of Absolute Difference), etc., can be used. At this time, in the filtering execution decision step, it is also possible to decide whether to perform filtering not only on the luminance component but also on the chrominance component.

[0260] In the filter shape determination step, when applying adaptive in-loop filtering, it is possible to determine what type of filter shape to use, what number of taps to use, etc., from the perspective of rate-distortion optimization.

[0261] On the other hand, the decoder may include a filter information decoding step, a block classification step, and a filtering execution step.

[0262] To avoid redundant explanations in this invention, the filter information encoding step and the filter information decoding step will be referred to together as the filter information encoding / decoding step later.

[0263] The block classification step will be described below.

[0264] A block classification index can be assigned to each N×M block unit (or block classification unit) within the reconstructed image, allowing each block to be classified up to L times. Here, the block classification index can be assigned not only to the reconstructed / decoded image, but also to at least one of the following: reconstructed / decoded slices, reconstructed / decoded tile groups, reconstructed / decoded tiles, reconstructed / decoded CTUs, or reconstructed / decoded blocks.

[0265] Here, N, M, and L can be positive integers. For example, N and M can be 2, 4, 8, 16, and 32, respectively, and L can be 4, 8, 16, 20, 25, and 32. If N and M are both 1, then sample-based classification can be performed instead of block-based classification. Also, if N and M have different positive integer values, then the N × M block units can have a non-square shape. Furthermore, N and M can have the same positive integer value.

[0266] For example, a total of 25 different block classification indices can be assigned to a reconstructed image in a 2x2 block unit. For example, a total of 25 different block classification indices can be assigned to a reconstructed image in a 4x4 block unit.

[0267] The aforementioned block classification index can have a value from 0 to L-1, or a value from 1 to L.

[0268] The aforementioned block classification index C is obtained by quantizing the directionality D value and the activity A value. q At least one of the values ​​can be used as the base, and it can be expressed by formula 1. [Formula 1]

number

[0269] In Equation 1, the constant value 5 is an example, and a J value can be used. At this time, J can be a positive integer having a value smaller than L.

[0270] According to an embodiment of the present invention, in the case of 2×2 block classification, the sum g of the respective gradient values in the vertical direction, the horizontal direction, the first diagonal (135-degree diagonal) direction, and the second diagonal (45-degree diagonal) direction using a 1D Laplacian operation v , g h , g d1 , g d2 can be represented by Equations 2 to 5. The D value and the A value can be derived using the sum of the gradient values. The sum of the gradient values is an embodiment, and a statistical value of the gradient values can be used in place of the sum of the gradient values. [Equation 2] [Number] [Equation 3] [Number] [Equation 4] [Number] [Equation 5] [Number]

[0271] In Equations 2 to 5, each of i and j means the horizontal position and the vertical position of the upper left side in the 2×2 block, and R(i, j) can mean the restored sample value at the (i, j) position.

[0272] Also, in Equations 2 to 5, each of k and l is the result V of the 1D Laplacian operation of the sample unit according to the direction k、l , H k、l , D1 k、l , D2 k、lThis can be interpreted as the horizontal and vertical ranges for which the sum is calculated. The result of the sample-unit 1D Laplacian operation in each of the aforementioned directions can be interpreted as the sample-unit gradient value in each direction. That is, the result of the 1D Laplacian operation can be interpreted as the gradient value. The 1D Laplacian operation can be performed in the vertical, horizontal, first diagonal, and second diagonal directions. The gradient value for the corresponding direction can be shown. Furthermore, the results of the 1D Laplacian operation performed in the vertical, horizontal, first diagonal, and second diagonal directions can be interpreted as V k、l H k、l D1 k、l , D2 k、l It can mean that.

[0273] For example, k and l can have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0274] As another example, k and l can have different ranges; that is, the horizontal and vertical lengths of the range over which the sum of the 1D Laplacian operation is calculated may be different.

[0275] As another example, since k has a range from i-2 to i+3 and l has a range from j-2 to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 6x6 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be larger than the size of the block classification unit.

[0276] As another example, since k has a range from i-1 to i+2 and l has a range from j-1 to j+2, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be larger than the size of the block classification unit.

[0277] As another example, since k ranges from i to i+1 and l ranges from j to j+1, the range over which the sum of the 1D Laplacian operation is calculated may be 2x2 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0278] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0279] As another example, the shape of the block classification unit may be at least one two-dimensional geometric shape from among rhombuses, rectangles, squares, trapezoids, diagonals, snowflakes, sharps, clover shapes, crosses, triangles, pentagons, hexagons, octagons, decagons, and dodecagons.

[0280] As another example, the range over which the sum of the 1D Laplacian operation is calculated can have a size of S × T, where S and T can be positive integers, including 0.

[0281] On the other hand, D1, which represents the first diagonal, and D2, which represents the second diagonal, may also mean D0, which represents the first diagonal, and D1, which represents the second diagonal.

[0282] According to one embodiment of the present invention, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using a 1D Laplacian operation. v , g h , g d1 , g d2 This can be expressed by equations 6 to 9. The sum of the gradient values ​​can be used to derive the D value and the A value. The sum of the gradient values ​​is one embodiment. Statistical values ​​of the gradient values ​​can be used as a substitute for the sum of the gradient values. [Formula 6]

number

number

number

number

[0283] In equations 6 through 9, i and j represent the horizontal and vertical positions in the upper left corner of a 4x4 block, respectively, and R(i,j) can be interpreted as the reconstructed sample value at position (i,j).

[0284] Furthermore, in equations 6 through 9, k and l are the results of a 1D Laplacian operation with sample units based on direction, V k、l H k、l D1 k、l , D2 k、l This can be interpreted as the horizontal and vertical ranges for which the sum is calculated. The results of the sample-unit 1D Laplacian operation in each of the aforementioned directions can be interpreted as the sample-unit gradient value in each direction. That is, the results of the 1D Laplacian operation can be interpreted as the gradient value. The 1D Laplacian operation is performed for the vertical, horizontal, first diagonal, and second diagonal directions, and the gradient value for the corresponding direction can be shown. Furthermore, the results of the 1D Laplacian operation performed for the vertical, horizontal, first diagonal, and second diagonal directions are each V k、l H k、l D1 k、l , D2 k、l It can mean that.

[0285] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0286] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be different.

[0287] As another example, since k has a range from i-2 to i+5 and l has a range from j-2 to j+5, the range over which the sum of the 1D Laplacian operation is calculated may be 8x8 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be larger than the size of the block classification unit.

[0288] As another example, since k has a range from i to i+3 and l has a range from j to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0289] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0290] As another example, the range over which the sum of the 1D Laplacian operation is calculated may have a size of S × T, where S and T can be positive integers including 0.

[0291] As another example, the shape of the block classification unit may be at least one two-dimensional geometric shape from among rhombuses, rectangles, squares, trapezoids, diagonals, snowflakes, sharps, clover shapes, crosses, triangles, pentagons, hexagons, octagons, decagons, and dodecagons.

[0292] Figure 9 shows an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions.

[0293] As shown in the example in Figure 9, this is an example of determining gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions in the case of 4x4 block classification.

[0294] The sum of the gradient values ​​in each angular direction, g v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions on a sample basis. That is, 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions can be performed at the positions of V, H, D1, and D2, respectively. In Figure 9, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate the positions of each restored sample, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0295] According to one embodiment of the present invention, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using a 1D Laplacian operation. v , g h , g d1 , g d2 This can be expressed by formulas 10 to 13. The gradient value can be expressed based on subsamples (or subsampling) to reduce the computational complexity of block classification. The D value and A value can be derived using the sum of the gradient values. The sum of the gradient values ​​is one embodiment. Statistical values ​​of the gradient values ​​can be used as a substitute for the sum of the gradient values. [Formula 10]

number

number

number

number

[0296] In equations 10 through 13, i and j represent the horizontal and vertical positions in the upper left corner of a 4x4 block, respectively, and R(i,j) can be interpreted as the reconstructed sample value at position (i,j).

[0297] Furthermore, in equations 10 to 13, k and l are the results of a 1D Laplacian operation with sample units based on direction, V k、l H k、l D1 k、l , D2 k、l This can be interpreted as the horizontal and vertical ranges for which the sum is calculated. The results of the sample-unit 1D Laplacian calculation in each of the aforementioned directions can be interpreted as the sample-unit gradient values ​​in each direction. In other words, the results of the 1D Laplacian calculation can be interpreted as gradient values. The 1D Laplacian calculation is performed for the vertical, horizontal, first diagonal, and second diagonal directions, and the gradient values ​​for each of these directions can be expressed. Furthermore, the results of the 1D Laplacian calculation performed for the vertical, horizontal, first diagonal, and second diagonal directions are each V k、l H k、l D1 k、l , D2 k、l It can mean that.

[0298] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0299] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be different.

[0300] As another example, since k has a range from i-2 to i+5 and l has a range from j-2 to j+5, the range over which the sum of the 1D Laplacian operation is calculated may be 8x8 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be larger than the size of the block classification unit.

[0301] As another example, since k has a range from i to i+3 and l has a range from j to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0302] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0303] As another example, the range over which the sum of the 1D Laplacian operation is calculated may have a size of S × T, where S and T can be positive integers including 0. As another example, the shape of the block classification unit may be at least one two-dimensional geometric shape from among rhombuses, rectangles, squares, trapezoids, diagonals, snowflakes, sharps, clover shapes, crosses, triangles, pentagons, hexagons, octagons, decagons, and dodecagons.

[0304] According to one embodiment of the present invention, a method for calculating a gradient value based on the subsamples can calculate the gradient value by performing a 1D Laplacian operation on samples located in the direction in which the sum of the 1D Laplacian operation is calculated, within the range in which the sum of the 1D Laplacian operation is calculated. Here, a statistical value can be calculated on the result of the 1D Laplacian operation performed on at least one of the samples within the range in which the sum of the 1D Laplacian operation is calculated, thereby calculating a statistical value of the gradient value. In this case, the statistical value may be a sum, a weighted sum, or a mean.

[0305] As an example, when calculating the horizontal gradient value, the 1D Laplacian operation is performed at all sample positions within a row within the range where the sum of the 1D Laplacian operation is calculated. However, it is possible to skip P rows from each row within the range where the sum of the 1D Laplacian operation is calculated and calculate the horizontal gradient value, where P is a positive integer.

[0306] As another example, when calculating the vertical gradient value, the 1D Laplacian operation is performed at all sample positions within a column within the range where the sum of the 1D Laplacian operation is calculated, but the vertical gradient value can be calculated by skipping P columns in each column within the range where the sum of the 1D Laplacian operation is calculated, where P is a positive integer.

[0307] As another example, when calculating the gradient value in the first diagonal direction, the gradient value in the first diagonal direction can be calculated by performing a 1D Laplacian operation on the positions of samples that are skipped in at least one of the columns P and rows Q for at least one of the horizontal and vertical directions, within the range in which the sum of the 1D Laplacian operation is calculated. Here, P and Q are positive integers including 0.

[0308] As another example, when calculating the gradient value in the second diagonal direction, the gradient value in the second diagonal direction can be calculated by performing a 1D Laplacian operation on the positions of samples that are skipped in at least one of the columns P and rows Q for at least one of the horizontal and vertical directions within the range in which the sum of the 1D Laplacian operation is calculated. Here, P and Q are positive integers including 0.

[0309] According to one embodiment of the present invention, a method for calculating the gradient value based on the subsamples can calculate the gradient value by performing a 1D Laplacian operation on at least one sample located within the range in which the sum of the 1D Laplacian operation is calculated. Here, a statistical value can be calculated for the result of the 1D Laplacian operation performed on at least one of the samples within the range in which the sum of the 1D Laplacian operation is calculated, thereby calculating a statistical value for the gradient value. In this case, the statistical value may be the sum, a weighted sum, or the mean.

[0310] As an example, when calculating the gradient value, the 1D Laplacian operation is performed at all sample positions within a row within the range where the sum of the 1D Laplacian operation is calculated. However, the gradient value can be calculated by skipping P rows at a time within the range where the sum of the 1D Laplacian operation is calculated, where P is a positive integer.

[0311] As another example, when calculating the gradient value, the 1D Laplacian operation is performed at all sample positions within the range where the sum of the 1D Laplacian operation is calculated, but the gradient value can be calculated by skipping P columns at a time within the range where the sum of the 1D Laplacian operation is calculated, where P is a positive integer.

[0312] As another example, when calculating the gradient value, the gradient value can be calculated by performing a 1D Laplacian operation on the positions of samples that are skipped in at least one of the columns P and rows Q, in at least one of the horizontal and vertical directions, within the range in which the sum of the 1D Laplacian operation is calculated. Here, P and Q are positive integers including 0.

[0313] As another example, when calculating the gradient value, the gradient value can be calculated by performing a 1D Laplacian operation on the positions of samples that are skipped in columns P and rows Q in both the horizontal and vertical directions, within the range where the sum of the 1D Laplacian operation is calculated. Here, P and Q are positive integers including 0.

[0314] On the other hand, the gradient may include at least one of a lateral gradient, a vertical gradient, a first diagonal gradient, and a second diagonal gradient.

[0315] Figures 10 to 12 show an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample.

[0316] As shown in the example in Figure 10, in the case of 2x2 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated based on the secondary sample. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. That is, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 10, the block classification index C can be assigned to 2x2 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0317] In the drawings of the present invention, the absence of V, H, D1, and D2 in the sample locations can be interpreted as meaning that no 1D Laplacian operation is performed in each direction. In other words, the 1D Laplacian operation is performed in each direction only at the sample locations where V, H, D1, and D2 are displayed. If no 1D Laplacian operation is performed, the result of the 1D Laplacian operation for that sample can be determined to a specific value H, where H can be at least one of a negative integer, 0, and a positive integer.

[0318] As shown in the example in Figure 11, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated based on the secondary sample. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 11, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0319] As shown in the example in Figure 12, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated based on the secondary sample. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 12, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0320] According to one embodiment of the present invention, a gradient value can be calculated by performing a 1D Laplacian operation on a sample located at a specific position within an N × M block unit based on a secondary sample. In this case, the specific position can be at least one of an absolute position or a relative position within the block. Here, a statistical value can be calculated for the gradient value by calculating a statistical value for the result of the 1D Laplacian operation performed on at least one of the samples within the range for which the sum of the 1D Laplacian operation is calculated. In this case, the statistical value can be the sum, a weighted sum, or the mean.

[0321] For example, if it refers to an absolute position, it may mean the upper left position within an N×M block.

[0322] As another example, when referring to an absolute position, it may mean the lower right position within an N×M block.

[0323] As another example, when referring to a relative position, it may mean the central position within an N×M block.

[0324] According to one embodiment of the present invention, the gradient value can be calculated by performing a 1D Laplacian operation on R samples within an N × M block unit based on a subsample. In this case, R is a positive integer including 0. Also, R may be equal to or less than the product of N and M. Here, a statistical value of the gradient value can be calculated by calculating a statistical value for the result of the 1D Laplacian operation performed on at least one of the samples within the range in which the sum of the 1D Laplacian operation is calculated. In this case, the statistical value may be the sum, a weighted sum, or the mean.

[0325] For example, if R is 1, the 1D Laplacian operation may be performed on only one sample within the N×M block.

[0326] As another example, if R is 2, the 1D Laplacian operation may be performed on only two samples within the N×M block.

[0327] As another example, if R is 4, the 1D Laplacian operation may be performed on only 4 samples within the N×M block.

[0328] According to one embodiment of the present invention, a 1D Laplacian operation can be performed on R samples within N × M, which is the range in which the sum of the 1D Laplacian operation is calculated based on the subsamples, to calculate the gradient value. In this case, R is a positive integer including 0. Also, R may be equal to or less than the product of N and M. Here, a statistical value can be calculated on the result of the 1D Laplacian operation performed on at least one of the samples within the range in which the sum of the 1D Laplacian operation is calculated, to obtain a statistical value of the gradient value. In this case, the statistical value may be the sum, the weighted sum, or the mean.

[0329] For example, if R is 1, the 1D Laplacian operation may be performed on only one sample within the N × M range in which the sum of the 1D Laplacian operation is calculated.

[0330] As another example, if R is 2, the 1D Laplacian operation may be performed on only two samples within the range N × M, which is the range in which the sum of the 1D Laplacian operation is calculated.

[0331] As another example, if R is 4, the 1D Laplacian operation may be performed only on 4 samples within the range N × M, which is the range in which the sum of the 1D Laplacian operation is calculated.

[0332] Figures 13 to 18 show other examples of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample.

[0333] As shown in the example in Figure 13, in the case of 4x4 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 13, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0334] As shown in the example in Figure 14, in the case of 4x4 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 14, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be smaller than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0335] As shown in the example in Figure 15, in the case of 4x4 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. That is, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 15, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be smaller than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0336] As shown in the example in Figure 16, in the case of 4x4 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 16, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be smaller than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0337] As shown in the example in Figure 17, in the case of 4x4 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculation in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, the 1D Laplacian calculation can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculation is performed can be subsampled. In Figure 17, the block classification index C can be assigned in shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculation is calculated may be smaller than the size of the block classification unit. Here, since the range over which the sum of the 1D Laplacian calculation is calculated is 1x1, the gradient value can be calculated without performing the sum of the 1D Laplacian calculation. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculation is calculated.

[0338] As shown in the example in Figure 18, in the case of 2x2 block classification, based on the secondary sample, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated using the sample located at a specific position within the NxM block unit. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculation in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, the 1D Laplacian calculation can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculation is performed can be subsampled. In Figure 18, the block classification index C can be assigned in 2x2 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculation is calculated may be smaller than the size of the block classification unit. Here, since the range over which the sum of the 1D Laplacian calculation is calculated is 1x1, the gradient value can be calculated without performing the sum of the 1D Laplacian calculation. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculation is calculated.

[0339] Figures 19 to 30 show an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions at a specific sample position. The specific sample position is a sample position subsampled within a block classification unit, and may be a sample position subsampled within the range in which the sum of the 1D Laplacian operation is calculated. Furthermore, the specific sample position may be the same for each block. Conversely, the specific sample position may differ for each block. Furthermore, the specific sample position may be the same regardless of the direction of the 1D Laplacian operation to be calculated. Conversely, the specific sample position may differ depending on the direction of the 1D Laplacian operation to be calculated.

[0340] As shown in the example in Figure 19, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 19, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0341] As shown in the example in Figure 19, the specific sample positions where the 1D Laplacian calculation is performed may be the same regardless of the direction of the 1D Laplacian calculation. Also, as shown in the example in Figure 19, the pattern of sample positions where the 1D Laplacian calculation is performed can be a checkerboard pattern or a quincunx pattern. Furthermore, the sample positions where the 1D Laplacian calculation is performed can refer to positions within the range or block classification unit or block unit where all values ​​are even in the horizontal (x-axis direction) and vertical (y-axis direction), or where all values ​​are odd.

[0342] As shown in the example in Figure 20, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of them can be calculated. Here, V, H, D1, and D2 represent the results of the respective 1D Laplacian operations in the vertical, horizontal, first diagonal, and second diagonal directions in sample units. That is, 1D Laplacian operations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions at the positions of V, H, D1, and D2, respectively. Also, the positions where the 1D Laplacian operations are performed can be subsampled. In FIG. 20, the block classification index C can be assigned in 4×4 block units with shading. At this time, the range in which the sum of the 1D Laplacian operations is calculated may be larger than the size of the block classification unit. Here, the squares included in the thin solid lines indicate each restored sample position, and the thick solid line indicates the range in which the sum of the 1D Laplacian operations is calculated.

[0343] As in the example of FIG. 20, regardless of the direction of the 1D Laplacian operation, the specific sample positions where the 1D Laplacian operation is performed can be the same. Also, as in the example of FIG. 20, the pattern of the sample positions where the 1D Laplacian operation is performed can be a checkerboard pattern or a quincunx pattern. Also, the sample positions where the 1D Laplacian operation is performed can mean positions having even values and odd values, respectively, in the horizontal (x-axis direction) and vertical (y-axis direction) in the range in which the sum of the 1D Laplacian operations is calculated, or in the block classification unit or block unit, or vice versa.

[0344] As in the example of FIG. 21, in the case of 4×4 block classification, the sum g of the gradient values at specific sample positions v , g h , g d1 , g d2At least one of them can be calculated. Here, V, H, D1, and D2 represent the results of the respective 1D Laplacian operations in the vertical, horizontal, first diagonal, and second diagonal directions in sample units. That is, 1D Laplacian operations in the vertical, horizontal, first diagonal, and second diagonal directions can be performed at the positions of V, H, D1, and D2. Also, the positions where the 1D Laplacian operations are performed can be subsampled. In FIG. 21, the block classification index C can be assigned in 4×4 block units with shading. At this time, the range in which the sum of the 1D Laplacian operations is calculated may be larger than the size of the block classification unit. Here, the rectangles included in the thin solid lines indicate each restored sample position, and the thick solid line indicates the range in which the sum of the 1D Laplacian operations is calculated.

[0345] As in the example of FIG. 22, in the case of 4×4 block classification, the sum g of the gradient values v of g h of g d1 of g d2 At least one of them can be calculated. Here, V, H, D1, and D2 represent the results of the respective 1D Laplacian operations in the vertical, horizontal, first diagonal, and second diagonal directions in sample units. That is, 1D Laplacian operations in the vertical, horizontal, first diagonal, and second diagonal directions can be performed at the positions of V, H, D1, and D2. Also, the positions where the 1D Laplacian operations are performed can be subsampled. In FIG. 22, the block classification index C can be assigned in 4×4 block units with shading. At this time, the range in which the sum of the 1D Laplacian operations is calculated may be larger than the size of the block classification unit. Here, the rectangles included in the thin solid lines indicate each restored sample position, and the thick solid line indicates the range in which the sum of the 1D Laplacian operations is calculated.

[0346] As in the example of FIG. 23, in the case of 4×4 block classification, the sum g of the gradient values v of g h of g d1 of g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 23, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0347] As shown in the example in Figure 23, the specific sample positions where the 1D Laplacian calculation is performed can be the same regardless of the direction of the 1D Laplacian calculation. Also, as shown in the example in Figure 23, the pattern of sample positions where the 1D Laplacian calculation is performed can be a checkerboard pattern or a quincunx pattern. Furthermore, the sample positions where the 1D Laplacian calculation is performed can refer to positions within the range or block classification unit or block unit where all values ​​are even in the horizontal (x-axis direction) and vertical (y-axis direction), or where all values ​​are odd.

[0348] As shown in the example in Figure 24, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 24, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate the positions of each restored sample, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0349] As shown in the example in Figure 24, the specific sample positions where the 1D Laplacian calculation is performed may be the same regardless of the direction of the 1D Laplacian calculation. Also, as shown in the example in Figure 24, the pattern of sample positions where the 1D Laplacian calculation is performed can be a checkerboard pattern or a quincunx pattern. Furthermore, the sample positions where the 1D Laplacian calculation is performed can refer to positions that have even and odd values ​​in the horizontal (x-axis direction) and vertical (y-axis direction), respectively, or positions that have odd and even values ​​in the range or block classification unit or block unit where the sum of the 1D Laplacian calculation is calculated.

[0350] As shown in the example in Figure 25, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 25, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0351] As shown in the example in Figure 26, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculation in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample unit basis. That is, a 1D Laplacian calculation can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. In Figure 26, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated. The aforementioned specific sample position can mean all sample positions within the block classification unit.

[0352] As shown in the example in Figure 27, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 27, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be the same as the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0353] As shown in the example in Figure 28, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions on a sample-by-sample basis. That is, 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions can be performed at the positions of V, H, D1, and D2, respectively. In Figure 28, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated. The aforementioned specific sample position can mean all sample positions within the range over which the sum of the 1D Laplacian calculations is calculated.

[0354] As shown in the example in Figure 29, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculation in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample-by-sample basis. That is, a 1D Laplacian calculation can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. In Figure 29, the block classification index C can be assigned to shaded 4x4 block units. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated. The aforementioned specific sample position can mean all sample positions within the range over which the sum of the 1D Laplacian calculations is calculated.

[0355] As shown in the example in Figure 30, in the case of 4x4 block classification, the sum of gradient values ​​g at a specific sample position. v , g h , g d1 , g d2 At least one of these can be calculated. Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 30, the block classification index C can be assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate each restored sample position, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0356] On the other hand, according to one embodiment of the present invention, it is possible to determine whether or not at least one of the methods for calculating the gradient value is performed according to the temporal layer identifier.

[0357] For example, in the case of a 2x2 block classification, formulas 2 through 5 can be expressed as a single formula, as in formula 14. [Formula 14]

number

[0358] In equation 14, dir includes the horizontal direction, vertical direction, first diagonal direction, and second diagonal direction, therefore, g dir g is the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions. v , g h , g d1 , g d2 This can mean that. Also, i and j represent the horizontal and vertical positions within a 2x2 block, and G dir V is the result of a 1D Laplacian operation in the vertical, horizontal, first diagonal, and second diagonal directions. k、l H k、l D1 k、l , D2 k、l It can mean that.

[0359] In this case, if the temporal hierarchy identifier of the current image (or reconstructed image) is the highest level, then in the case of 2x2 block classification of the current image (or reconstructed image), equation 14 can be expressed as equation 15. [Formula 15]

number

[0360] In formula 15, G dir (i0,j0) represents the gradient value at the top-left position within the 2x2 block, considering the vertical, horizontal, first diagonal, and second diagonal directions.

[0361] Figure 31 shows an example of determining gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions when the temporal hierarchy identifier is the highest level.

[0362] Referring to Figure 31, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is g. v , g h , g d1 , g d2 The calculation can be simplified by calculating the gradient only at the top-left sample position (i.e., the sample position where the shading is displayed) in each 2x2 block.

[0363] According to one embodiment of the present invention, a weight (weighting factor) can be applied to the result of a 1D Laplacian operation performed on at least one of the samples within the range in which the sum of the 1D Laplacian operation is calculated, and a weighted sum can be calculated to obtain a statistical value of the gradient. In this case, at least one of the statistical values ​​such as the weighted mean, median, minimum, maximum, and mode can be used instead of the weighted sum.

[0364] The application of weights and the calculation of weighted sums can be determined based on various conditions or coding parameters related to the current block and surrounding blocks.

[0365] For example, the weighted sum can be calculated using at least one of the following units: sample unit, sample group unit, line unit, and block unit. In this case, the weighted sum can be calculated using different weights for at least one of the following units: sample unit, sample group unit, line unit, and block unit.

[0366] As another example, the weight values ​​can be applied differently depending on at least one of the following: the size of the current block, the shape of the current block, and the sample position.

[0367] As another example, the execution of the weighted sum can be determined based on criteria pre-set in the encoder and decoder.

[0368] As another example, the weight can be adaptively determined using at least one of the coding parameters, such as the size of at least one block from the current block and surrounding blocks, the shape of the block, and the in-screen prediction mode.

[0369] As another example, the decision to perform a weighted sum can be adaptively determined using at least one encoding parameter, such as the size of at least one block from the current block and surrounding blocks, the shape of the block, or the in-screen prediction mode.

[0370] As another example, if the range over which the sum of the 1D Laplacian operation is calculated is larger than the size of the block classification unit, then at least one weight value applied to samples within the block classification unit may be greater than at least one weight value applied to samples outside the block classification unit.

[0371] As another example, if the range over which the sum of the 1D Laplacian operation is calculated is the same as the size of the block classification unit, then the weight values ​​applied to all samples within the block classification unit may be the same.

[0372] On the other hand, information regarding whether or not the weights and / or weighted sums are performed can be entropy encoded by the encoder and signaled to the decoder.

[0373] According to one embodiment of the present invention, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is g v , g h , g d1 , g d2When calculating at least one of the values, if there are currently unavailable samples around the sample, padding can be performed, and the gradient value can be calculated using the padded samples. This padding can mean copying an available sample value adjacent to the unavailable sample to the unavailable sample. Alternatively, a sample value or statistical value obtained based on an available sample value adjacent to the unavailable sample can be used. This padding can be repeated as many times as there are columns P and rows R, where P and R can be positive integers.

[0374] Here, an unavailable sample can mean a sample that lies outside the boundaries of a CTU, CTB, slice, tile, tile group, or picture. Alternatively, an unavailable sample can mean a sample that belongs to at least one of the CTUs, CTBs, slices, tiles, tile groups, and pictures that are different from at least one of the CTUs, CTBs, slices, tiles, tile groups, and pictures to which the sample currently belongs.

[0375] According to one embodiment of the present invention, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is g v , g h , g d1 , g d2 When calculating at least one of these values, it is not necessary to use a predetermined sample.

[0376] As an example, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions, g v , g h , g d1 , g d2 When calculating at least one of these values, it is not necessary to use padded samples.

[0377] As another example, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions, gv , g h , g d1 , g d2 When calculating at least one of these, if there are currently unavailable samples around the sample, those unavailable samples do not need to be used in calculating the sum of the gradient values.

[0378] As another example, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions, g v , g h , g d1 , g d2 When calculating at least one of these values, if the samples currently surrounding the sample are located outside the CTU or CTB boundary, the samples in the surrounding area do not need to be used in calculating the sum of the gradient values.

[0379] According to one embodiment of the present invention, when calculating at least one of the 1D Laplacian calculation values, if there are currently unavailable samples around the sample, padding can be performed by copying the available sample values ​​adjacent to the unavailable sample values, and then the 1D Laplacian calculation can be performed using the padded samples.

[0380] According to one embodiment of the present invention, it is not necessary to use a predetermined sample when performing the 1D Laplacian calculation.

[0381] For example, when performing the 1D Laplacian calculation, it is not necessary to use padded samples.

[0382] As another example, when calculating at least one of the 1D Laplacian values, if there are currently unavailable samples around the sample, those unavailable samples do not need to be used in the 1D Laplacian calculation.

[0383] As another example, when calculating at least one of the 1D Laplacian values, if the samples currently surrounding the sample are located outside the CTU or CTB boundary, the samples in the surrounding area do not need to be used in the 1D Laplacian calculation.

[0384] According to one embodiment of the present invention, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is g v , g h , g d1 , g d2 When calculating at least one of the values, or when calculating at least one of the 1D Laplacian calculation values, samples to which at least one of deblocking filtering, adaptive sample offsetting, and adaptive in-loop filtering has been applied can be used.

[0385] According to one embodiment of the present invention, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is g v , g h , g d1 , g d2 When calculating at least one of the values, or when calculating at least one of the 1D Laplacian calculation values, if a sample currently present around the sample is located outside the CTU or CTB boundary, at least one of deblocking filtering, adaptive sample offset, and adaptive in-loop filtering can be applied to the sample.

[0386] Alternatively, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions, g v , g h , g d1 , g d2When calculating at least one of the values, or when calculating at least one of the 1D Laplacian calculation values, if a sample currently present around the sample is located outside the CTU or CTB boundary, it is not necessary to apply at least one of deblocking filtering, adaptive sample offset, and adaptive in-loop filtering to that sample.

[0387] According to one embodiment of the present invention, when the range in which the sum of the 1D Laplacian calculation is calculated includes samples that are unavailable because they are located outside the CTU or CTB boundary, etc., it can be used when calculating the sum of the 1D Laplacian calculation without applying at least one of deblocking filtering, adaptive sample offsetting, and adaptive in-loop filtering to the unavailable samples.

[0388] According to one embodiment of the present invention, when a block classification unit contains samples that are unavailable because they are located outside the CTU or CTB boundary, it can be used to calculate the 1D Laplacian operation without applying at least one of deblocking filtering, adaptive sample offsetting, and adaptive in-loop filtering to the unavailable samples.

[0389] On the other hand, the method of calculating the gradient value based on the subsamples performs the 1D Laplacian operation on the subsample samples within the range in which the sum of the 1D Laplacian operation is calculated, rather than on the entire sample range in which the sum of the 1D Laplacian operation is calculated. This reduces the number of operations required for the block classification step, such as multiplication, shift, addition, and absolute value operations. It also reduces the memory access bandwidth required when using the recovered samples, thereby reducing the complexity of the encoder and decoder. In particular, performing the 1D Laplacian operation on the subsample samples reduces the timing required for the block classification step when implementing the encoder and decoder in hardware, which is advantageous from the standpoint of hardware implementation complexity.

[0390] Furthermore, if the range over which the sum of the 1D Laplacian operation is calculated is the same as or smaller than the size of the block classification unit, the number of addition operations required for the block classification step can be reduced. Additionally, the memory access bandwidth required when using recovered samples can be reduced, thereby decreasing the complexity of the encoder and decoder.

[0391] On the other hand, the method for calculating the gradient value based on the aforementioned subsamples involves varying at least one of the following in order to perform the 1D Laplacian operation: the position of the sample, the number of samples, and the direction of the sample position, depending on the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions to be calculated: the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions. v , g h , g d1 , g d2 At least one of these can be calculated.

[0392] Furthermore, the method for calculating the gradient value based on the aforementioned subsamples is to keep the position of the sample, the number of samples, and the direction of the sample position the same, regardless of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions to be calculated, and sum the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions g v , g h , g d1 , g d2 At least one of these can be calculated.

[0393] Furthermore, using at least one combination of the methods for calculating the gradient values, a 1D Laplacian operation is performed using the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions, and the sum of the respective gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated. v , g h , g d1 , g d2 At least one of these can be calculated.

[0394] According to one embodiment of the present invention, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions g v , g h , g d1 , g d2 At least two of these values ​​can be compared to each other.

[0395] As an example, after calculating the sum of the gradient values, the sum of the vertical gradient values ​​g v and the sum of the horizontal gradient values ​​g h Compared to this, find the maximum value of the sum of the gradient values ​​in the horizontal and vertical directions.

number

number

number

[0396] At this time, the sum of the vertical gradient values ​​g v and the sum of the horizontal gradient values ​​g h To compare them, the magnitudes of the sums of the gradient values ​​can be compared with each other, as shown in the example in formula 17. [Formula 17]

number

[0397] Another example is the sum of the gradient values ​​in the first diagonal direction, g. d1 and the sum of the gradient values ​​in the second diagonal direction g d2 Compared with the maximum value of the sum of the gradient values ​​in the first diagonal direction and the second diagonal direction.

number

number

number

[0398] At this time, the sum of the gradient values ​​in the first diagonal direction is g. d1 and the sum of the gradient values ​​in the second diagonal direction g d2 To compare them, the magnitudes of the sums of the gradient values ​​can be compared with each other, as shown in the example in formula 19. [Formula 19]

number

[0399] According to one embodiment of the present invention, in order to calculate the directional D value, the maximum value and the minimum value can be compared with two threshold values ​​t1 and t2 as follows.

[0400] The directionality D value can be a positive integer, including 0. For example, the directionality D value can be between 0 and 4. Another example is that the directionality D value can be between 0 and 2.

[0401] Furthermore, the directional D value can be determined according to the characteristics of the region. For example, a directional D value of 0 indicates a texture region, a directional D value of 1 indicates strong horizontal / vertical directionality, a directional D value of 2 indicates weak horizontal / vertical directionality, a directional D value of 3 indicates strong first / second diagonal directionality, and a directional D value of 4 indicates weak first / second diagonal directionality. The determination of the directional D value can be performed by the following steps 1 to 4. Step 1:

number

number

number

number

number

[0402] Here, the thresholds t1 and t2 are positive integers. t1 and t2 may have the same value or different values. For example, t1 and t2 may be 2 and 9, respectively. Another example is that t1 and t2 may be 1, respectively. Yet another example is that t1 and t2 may be 1 and 9, respectively.

[0403] The activity level A value can be expressed as shown in example formula 20 when classifying into 2x2 blocks. [Formula 20]

number

[0404] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0405] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be different.

[0406] As another example, since k is in the range i-2 to i+3 and l is in the range j-2 to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 6x6 in size.

[0407] As another example, since k is in the range i-1 to i+2 and l is in the range j-1 to j+2, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size.

[0408] As another example, since k ranges from i to i+1 and l ranges from j to j+1, the range over which the sum of the 1D Laplacian operation is calculated may be 2x2 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0409] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0410] Furthermore, the activity A value can be expressed as shown in the example in formula 21 when classifying into 4x4 blocks. [Formula 21]

number

[0411] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0412] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be different.

[0413] As another example, since k is in the range i-2 to i+5 and l is in the range j-2 to j+5, the range over which the sum of the 1D Laplacian operation is calculated may be 8x8 in size.

[0414] As another example, since k has a range from i to i+3 and l has a range from j to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0415] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0416] Furthermore, the activity A value can be expressed as shown in the example in formula 22 for the 2x2 block classification. Here, at least one of the 1D Laplacian values ​​for the first and second diagonal directions can be used to further calculate the activity A value. [Formula 22]

number

[0417] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0418] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be different.

[0419] As another example, since k is in the range i-2 to i+3 and l is in the range j-2 to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 6x6 in size.

[0420] As another example, since k is in the range i-1 to i+2 and l is in the range j-1 to j+2, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size.

[0421] As another example, since k ranges from i to i+1 and l ranges from j to j+1, the range over which the sum of the 1D Laplacian operation is calculated may be 2x2 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0422] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0423] Furthermore, the activity A value can be expressed as shown in the example in formula 23 for the 4x4 block classification. Here, at least one of the 1D Laplacian values ​​for the first and second diagonal directions can be used to further calculate the activity A value. [Formula 23]

number

[0424] For example, k and l may have the same range. That is, the horizontal and vertical lengths of the range in which the sum of the 1D Laplacian operation is calculated may be the same.

[0425] As another example, k and l may have different ranges; that is, the horizontal and vertical lengths of the range over which the sum of the 1D Laplacian operation is calculated may be different.

[0426] As another example, since k is in the range i-2 to i+5 and l is in the range j-2 to j+5, the range over which the sum of the 1D Laplacian operation is calculated may be 8x8 in size.

[0427] As another example, since k has a range from i to i+3 and l has a range from j to j+3, the range over which the sum of the 1D Laplacian operation is calculated may be 4x4 in size. In this case, the range over which the sum of the 1D Laplacian operation is calculated may be the same size as the block classification unit.

[0428] As another example, the range over which the sum of the results of the 1D Laplacian operation is calculated may be at least one of the following two-dimensional geometric shapes: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon.

[0429] On the other hand, the activity A value is quantized to a quantized activity A value in the range between I and J. qIt can have a value. Here, I and J are positive integers, including 0, and may be 0 and 4, respectively.

[0430] Quantized activity A q The value can be determined using a predetermined method.

[0431] As an example, quantized activity A q The value can be expressed as in the example in equation 24. In this case, the quantized activity A q The value may fall within a range of a specific minimum value X and a specific maximum value Y. [Formula 24]

number

[0432] Activity A quantized using Equation 24 q The value can be calculated by multiplying the activity A value by a specific constant W value and performing a right shift operation by R. In this case, X, Y, W, and R are positive integers including 0. For example, W may be 24 and R may be 13. Another example is that W may be 64 and R may be (3+N bits). For example, N is a positive integer and may be 8 or 10. Yet another example is that W may be 32 and R may be (3+N bits). For example, N is a positive integer and may be 8 or 10.

[0433] As another example, the quantized activity A q The values ​​are obtained using a Look Up Table (LUT) method, and consist of the activity A value and the quantized activity A value. q A mapping relationship with the value can be set. That is, calculations are performed on the activity A value, and the quantized activity A is obtained using the lookup table. q A value can be calculated. In this case, the operation may include at least one of the following: multiplication, division, right shift, left shift, addition, and subtraction.

[0434] On the other hand, in the case of color difference components, the block classification process described above can be omitted, and filtering can be performed on each color difference component using K filters. Here, K is a positive integer including 0, and may also be 1. Furthermore, in the case of color difference components, the block classification process described above can be omitted, and filtering can be performed using a block classification index determined by the luminance component at the corresponding position. Also, in the case of color difference components, the filter information for the color difference components is not signaled, and a fixed form of filter can be used.

[0435] Figure 32 shows various calculation techniques that can be used as alternatives to the 1D Laplacian calculation according to one embodiment of the present invention.

[0436] According to one embodiment of the present invention, at least one of the operations described in Figure 32 can be used instead of the 1D Laplacian operation. Referring to Figure 32, the operation can include at least one of the 2D Laplacian, 2D Sobel, 2D edge extraction, and 2D LoG (Laplacian of Gaussian) operations. Here, the LoG operation can mean that a combination of a Gaussian filter and a Laplacian filter is applied to the reconstructed sample. In addition, at least one of the 1D / 2D edge extraction filters can be used instead of the 1D Laplacian operation. Furthermore, the DoG (Difference of Gaussian) operation can be used. Here, the DoG operation can mean that a combination of Gaussian filters with different internal parameters is applied to the reconstructed sample.

[0437] Furthermore, an N×M size LoG operation can be used to calculate the directional D value or the activity A value. Here, N and M may be positive integers. As an example, at least one of the 5×5 2D LoG operation in Figure 32(i) or the 9×9 2D LoG operation in Figure 32(j) may be used. As another example, a 1D LoG operation may be used instead of a 2D LoG operation.

[0438] According to one embodiment of the present invention, each 2x2 block of luminance can be classified based on directionality and 2D Laplacian activity. For example, the transverse / vertical gradient characteristics can be obtained using a Sobel filter. The directionality D value can be obtained using equations 25 to 26.

[0439] The representative vector can be calculated such that the condition in [Equation 25] is satisfied for the gradient vector within a given window size (e.g., a 6x6 block). The direction and deformation can be identified by Θ. [Formula 25]

number

[0440] The similarity between the representative vector and each gradient vector within the aforementioned window can be calculated using the dot product, as shown in Equation 26. [Formula 26]

number

[0441] The directional D value can then be determined using the S value calculated by formula 26. Step 1:

number

number

number

number

number

[0442] Here, the total number of block classification indices can be 25.

[0443] According to one embodiment of the present invention, the restored sample

number

number

[0444] In equation 27, I is the restored sample.

number

number

number

number

number

[0445] If classification_idx=0, then block classifier based on the aforementioned directionality and activity.

number

[0446] If classification_idx=1, then sample-based feature classifier

number

number

number

number

[0447] Here, B is the bit depth of the sample, the number of class K is set to K=27, and the operator

number

[0448] If classification_idx=2, then a rank-based sample-based feature classifier.

number

number

number

number

number

[0449] classifier

number

number

number

number

number

number

[0450] In equation 30, T1 and T2 are predefined thresholds. That is, the dynamic range of the samples is divided into three bands, and within each band, the rank of the samples in the local periphery is used as an additional criterion. A rank-based sample-based feature classifier can provide K=27 classes.

[0451] If classification_idx=3, then a feature classifier based on rank and regional change.

number

number

number

[0452] In equation 31, T3 or T4 is a predefined threshold. Regional change at each sample location (i,j).

number

number

[0453] Each sample first has a local variable

number

number

[0454] According to one embodiment of the present invention, at the slice level, up to 16 filter sets are currently available for a slice, using three pixel classification methods such as IntensityClassifier, HistogramClassifier, and DirectionalActivityClassifier. At the CTU level, three modes—NewFilterMode, SpatialFilterMode, and SliceFilterMode—can be supported per CTU based on control flags signaled from the slice header.

[0455] Here, the IntensityClassifier may be similar to the band offset of the SAO. The range of sample intensity can be divided into 32 groups, and the group index can be determined based on the intensity of the sample being processed.

[0456] Furthermore, SimilarityClassifier can be used to compare surrounding samples with the sample being filtered in a 5x5 diamond filter shape. The group index of the sample being filtered can initially be initialized to 0. If the difference between the surrounding sample and the sample being filtered is greater than a predetermined threshold, the group index can be incremented by 1. If the difference between the surrounding sample and the sample being filtered is more than twice as large as a predefined threshold, an additional 1 can be added to the group index. In this case, SimilarityClassifier can have up to 25 groups.

[0457] Furthermore, RotBAClassifier can reduce the range over which the sum of 1D Laplacian operations on a single 2x2 block is calculated from 6x6 to 4x4. This classification can have up to 25 groups. While the number of groups can be up to 25 or 32 in various classifiers, the number of filters in the slice filter set can be limited to a maximum of 16. That is, the encoder can merge consecutive groups and keep the number of merged groups at or below 16.

[0458] According to one embodiment of the present invention, when determining the block classification index, the block classification index can be determined based on at least one coding parameter of the current block and surrounding blocks. The block classification index may differ from one another depending on at least one of the coding parameters. In this case, the coding parameter may include at least one of the following: prediction mode (inter-screen prediction or intra-screen prediction), intra-screen prediction mode, inter-screen prediction mode, inter-screen prediction indicator, motion vector, reference image index, quantization parameter, size of the current block, shape of the current block, size of the block classification unit, and coded block flag / pattern.

[0459] As an example, the block classification can be determined based on a quantization parameter. For instance, if the quantization parameter is less than a threshold T, J block classification indices can be used; if the quantization parameter is greater than a threshold R, H block classification indices can be used; and otherwise, G block classification indices can be used. Here, T, R, J, H, and G are positive integers, including 0. Also, J may be the same as or greater than H. Here, the larger the value of the quantization parameter, the fewer block classification indices can be used.

[0460] As another example, the block classification can be determined based on the current block size. For example, if the current block size is less than threshold T, J block classification indices can be used; if the current block size is greater than threshold R, H block classification indices can be used; and otherwise, G block classification indices can be used. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the block size relatively, the fewer block classification indices can be used.

[0461] As another example, the block classification can be determined according to the size of the block classification unit. For example, if the size of the block classification unit is less than the threshold T, J block classification indices can be used; if the size of the block classification unit is greater than the threshold R, H block classification indices can be used; and otherwise, G block classification indices can be used. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the size of the block classification unit, the fewer block classification indices can be used.

[0462] According to one embodiment of the present invention, the sum of at least one of the gradient values ​​among the sum of gradient values ​​of corresponding positions in the previous image, the sum of gradient values ​​of surrounding blocks of the current block, and the sum of gradient values ​​of block classification units around the current block classification unit can be determined as at least one of the sum of gradient values ​​of the current block and the sum of gradient values ​​of the current block classification unit. Here, the corresponding position in the previous image may be a position or surrounding position in the previous image that has a spatial position corresponding to the corresponding reconstructed sample in the current image.

[0463] For example, if the difference between at least one of the sums of the vertical and horizontal gradient values ​​of a current block unit, gv and gh, and at least one of the sums of the horizontal and vertical gradient values ​​of block classification units surrounding the current block classification unit is less than or equal to threshold E, then at least one of the sums of the first and second diagonal gradient values ​​of block classification units surrounding the current block classification unit, gd1 and gd2, can be determined as at least one of the sums of the gradient values ​​of the current block unit. Here, threshold E is a positive integer including 0.

[0464] As another example, if the difference between the sum of the vertical and horizontal gradient values ​​gv and gh for the current block unit and the sum of the horizontal and vertical gradient values ​​of the block classification units surrounding the current block classification unit is less than or equal to a threshold E, then at least one of the sums of gradient values ​​of the block classification units surrounding the current block classification unit can be determined as at least one of the sums of gradient values ​​for the current block unit. Here, the threshold E is a positive integer including 0.

[0465] As another example, if the difference between at least one statistic of the currently reconstructed samples within a block unit and at least one statistic of the reconstructed samples within a block classification unit surrounding the current block classification unit is less than or equal to a threshold E, then at least one of the sums of gradient values ​​of the block classification units surrounding the current block unit can be determined as at least one of the sums of gradient values ​​of the current block unit. Here, the threshold E is a positive integer including 0. The threshold E can be derived from the spatially adjacent blocks and / or temporally adjacent blocks of the current block. Furthermore, the threshold E may be a value predefined in the encoder and decoder.

[0466] According to one embodiment of the present invention, at least one block classification index from among the block classification index of a previously corresponding position in an image, the block classification index of blocks surrounding the current block, and the block classification index of block classification units surrounding the current block classification unit can be determined as at least one of the block classification index of the current block and the block classification index of the current block classification unit.

[0467] For example, if the difference between at least one of the sums of the vertical and horizontal gradient values ​​of the current block unit, gv and gh, and at least one of the sums of the horizontal and vertical gradient values ​​of the block classification units surrounding the current block classification unit is less than or equal to the threshold E, then the block classification index of the block classification units surrounding the current block classification unit can be determined as the block classification index of the current block unit. Here, the threshold E is a positive integer including 0.

[0468] As another example, if the difference between the sum of the vertical and horizontal gradient values ​​gv and gh for the current block unit and the sum of the horizontal and vertical gradient values ​​for the block classification units surrounding the current block classification unit is less than or equal to threshold E, then the block classification index of the block classification units surrounding the current block classification unit can be determined as the block classification index of the current block unit. Here, the threshold E is a positive integer including 0.

[0469] As another example, if the difference between at least one statistic from the currently reconstructed samples within a block unit and at least one statistic from the reconstructed samples within the block classification units surrounding the current block classification unit is less than or equal to a threshold E, then the block classification index of the block classification units surrounding the current block unit can be determined as the block classification index of the current block unit. Here, the threshold E is a positive integer including 0.

[0470] As another example, the block classification index can be determined using at least one combination of the methods for determining the block classification index described above.

[0471] The filtering execution steps will be described below.

[0472] According to one embodiment of the present invention, filtering can be performed on samples or blocks in the reconstructed / decoded image using a filter corresponding to the determined block classification index. When performing the filtering, any of L types of filters can be selected, where L is a positive integer including 0.

[0473] For example, one filter can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image at the reconstructed / decoded sample level.

[0474] As another example, one filter can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image for each block classification unit.

[0475] As another example, one filter can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image at the CU (Unit of Cavity) level.

[0476] As another example, one filter can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image on a block-by-block basis.

[0477] As another example, U filters are selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image at the reconstructed / decoded sample level. In this case, U is a positive integer.

[0478] As another example, U filters can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image for each block classification unit. In this case, U is a positive integer.

[0479] As another example, U filters can be selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image in units of CUs. In this case, U is a positive integer.

[0480] As another example, U filters are selected from L types of filters for each block classification unit, and filtering can be performed on the reconstructed / decoded image on a block-by-block basis. In this case, U is a positive integer.

[0481] On the other hand, the L types of filters mentioned above can be referred to as a filter set.

[0482] According to one embodiment of the present invention, the L-type filters may differ from each other in at least one of the following: filter coefficients, number of filter taps (filter length), filter shape, and filter type.

[0483] As an example, the L-type filters may have at least one of the following characteristics identical to each other: filter coefficient, number of filter taps (filter length), filter shape, and filter type, for at least one unit from among blocks, CUs, PUs, TUs, CTUs, slices, tiles, tile groups, images (pictures), and sequences.

[0484] As another example, the L-type filters may have at least one of the following characteristics differ from one another: filter coefficients, filter tap count (filter length), filter shape, and filter type, for at least one unit from among blocks, CUs, PUs, TUs, CTUs, slices, tiles, tile groups, images (pictures), and sequences.

[0485] Furthermore, the filtering may be performed using the same or different filters on at least one unit from among samples, blocks, CUs, PUs, TUs, CTUs, slices, tiles, tile groups, images, and sequences.

[0486] Furthermore, the filtering may be performed based on filtering execution status information for at least one unit among samples, blocks, CUs, PUs, TUs, CTUs, slices, tiles, tile groups, images (pictures), and sequences. The filtering execution status information may be information signaled from the encoder to the decoder in at least one unit among samples, blocks, CUs, PUs, TUs, CTUs, slices, tiles, tile groups, images (pictures), and sequences.

[0487] According to one embodiment of the present invention, as filters for filtering, N types of filters having a diamond (rhombus) shape and a different number of filter taps can be used. Here, N can be a positive integer. For example, diamonds with 5x5, 7x7, and 9x9 filter taps can be represented as shown in Figure 33.

[0488] Figure 33 shows a rhomboid-shaped filter according to one embodiment of the present invention.

[0489] Referring to Figure 33, the filter index can be entropy encoded / decoded at the picture / tile / tile group / slice / sequence level to signal from the encoder to the decoder which of the three diamond-shaped filters with filter tap counts of 5x5, 7x7, and 9x9 should be used. In other words, the filter index can be entropy encoded / decoded at the sequence parameter set, picture parameter set, slice header, slice data, tile header, tile group header, etc., within the bitstream.

[0490] According to one embodiment of the present invention, if the number of filter taps in the encoder / decoder is fixed to one, filtering can be performed in the encoder / decoder using the filter without entropy encoding / decoding the filter index. Here, the number of filter taps may be a 7x7 diamond shape in the case of the luminance component, and a 5x5 diamond shape in the case of the chrominance component.

[0491] According to one embodiment of the present invention, at least one of the three types of rhombic-shaped filters can be used for sample filtering of at least one of the luminance component and chrominance component.

[0492] For example, for luminance component restoration / decoding samples, at least one of the three types of diamond-shaped filters shown in Figure 33 can be used for restoration / decoding sample filtering.

[0493] As another example, for color difference component restoration / decoding samples, the 5x5 rhombic filter shown in Figure 33 can be used for filtering the restoration / decoding sample.

[0494] As another example, the color difference component restoration / decoding sample may be filtered using a filter selected from the luminance components corresponding to the color difference components.

[0495] On the other hand, the numbers within each filter shape in Figure 33 represent the filter coefficient index, and the filter coefficient index can have a symmetric shape around the filter. In other words, the filter in Figure 33 can be a point symmetric filter.

[0496] On the other hand, as shown in Figure 33(a), a 9x9 diamond-shaped filter can entropy encode / decode a total of 21 filter coefficients, as shown in Figure 33(b), a 7x7 diamond-shaped filter can entropy encode / decode a total of 13 filter coefficients, and as shown in Figure 33(c), a 5x5 diamond-shaped filter can entropy encode / decode a total of 7 filter coefficients. In other words, a maximum of 21 filter coefficients must be entropy encoded / decoded.

[0497] Furthermore, as shown in Figure 33(a), a 9x9 diamond-shaped filter requires a total of 21 multiplications per sample; as shown in Figure 33(b), a 7x7 diamond-shaped filter requires a total of 13 multiplications per sample; and as shown in Figure 33(c), a 5x5 diamond-shaped filter requires a total of 7 multiplications per sample. In other words, filtering must be performed using a maximum of 21 multiplications per sample.

[0498] Furthermore, as shown in Figure 33(a), in the case of a 9x9 rhombus-shaped filter, since it has a 9x9 size, a line buffer of 4 lines, which is half the vertical length of the filter, is required when implementing it in hardware. In other words, a line buffer for a maximum of 4 lines is required.

[0499] According to one embodiment of the present invention, as the filter for filtering, the number of filter taps is the same 5x5, but the filter shape can be any filter that includes at least one of the following: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon. For example, if the number of filter taps is 5x5 and the filter shape is a square, octagon, snowflake, or rhombus, it can be represented as shown in Figure 34.

[0500] Here, the number of filter taps is not limited to 5x5, and at least one filter with HxV filter taps can be used, such as 3x3, 4x4, 5x5, 6x6, 7x7, 8x8, 9x9, 5x3, 7x3, 9x3, 7x5, 9x5, 9x7, 11x7, etc. Here, H and V are positive integers and may have the same value or different values. Also, at least one of H and V is a value predefined in the encoder / decoder and may be a value signaled from the encoder to the decoder. Furthermore, one of the values ​​of H and V can be used to define the other value. Also, the final value of H or V can be used to define the final value of H or V.

[0501] On the other hand, in order to signal from the encoder to the decoder which filter to use among the filters shown in the example in Figure 34, the filter index can be entropy encoded / decoded at the picture / tile / tile group / slice / sequence level. In other words, the filter index can be entropy encoded / decoded at the sequence parameter set, picture parameter set, slice header, slice data, tile header, tile group header, etc., within the bitstream.

[0502] On the other hand, at least one of the square, octagonal, snowflake, and rhombic-shaped filters, as shown in the example in Figure 34, can be used for reconstructing / decoding sample filtering of at least one of the luminance and chrominance components.

[0503] On the other hand, the numbers within each filter shape in Figure 34 represent the filter coefficient index, and the filter coefficient index can have a symmetrical shape around the filter. In other words, the filter in Figure 34 can be a point-symmetric filter shape.

[0504] According to one embodiment of the present invention, when filtering a restored image on a sample-by-sample basis, the encoder can determine which filter shape to use for each image / slice / tile / tile group from the perspective of rate-distortion optimization. Furthermore, filtering can be performed using the determined filter shape. As shown in Figure 34, the degree of improvement in encoding efficiency and the amount of filter information (number of filter coefficients) differ depending on the filter shape, so it is necessary to determine the optimal filter shape for each image / slice / tile / tile group. In other words, the optimal filter shape can be determined to differ from the example filter shapes in Figure 34 depending on the image resolution, image characteristics, bit rate, etc.

[0505] According to one embodiment of the present invention, using a filter like the example in Figure 34 can reduce the computational complexity of the encoder / decoder compared to using a filter like the example in Figure 33.

[0506] As an example, in the case of a 5x5 square-shaped filter as shown in Figure 34(a), a total of 13 filter coefficients can be entropy coded / decoded. In the case of a 5x5 octagon-shaped filter as shown in Figure 34(b), a total of 11 filter coefficients can be coded / decoded. In the case of a 5x5 snowflake-shaped filter as shown in Figure 34(c), a total of 9 filter coefficients can be coded / decoded. In other words, the number of filter coefficients to be entropy coded / decoded can differ depending on the shape of the filter. Here, the maximum number of filter coefficients in a filter like the example in Figure 34 (13) is smaller than the maximum number of filter coefficients in a filter like the example in Figure 33 (21). Therefore, when using a filter like the example in Figure 34, the number of filter coefficients to be entropy coded / decoded decreases, thus reducing the computational complexity of the encoder / decoder.

[0507] As other examples, a 5x5 square-shaped filter as shown in Figure 34(a) requires a total of 13 multiplications per sample, a 5x5 octagon-shaped filter as shown in Figure 34(b) requires a total of 11 multiplications per sample, a 5x5 snowflake-shaped filter as shown in Figure 34(c) requires a total of 9 multiplications per sample, and a 5x5 rhombus-shaped filter as shown in Figure 34(d) requires a total of 7 multiplications per sample. The maximum number of multiplications per sample for a filter within a filter, as in the example in Figure 34 (13), is smaller than the maximum number of multiplications per sample for a filter within a filter, as in the example in Figure 33 (21). Therefore, when using a filter like the one in the example in Figure 34, the number of multiplications per sample is reduced, which can reduce the computational complexity of the encoder / decoder.

[0508] As another example, in the filter shown in Figure 34, each filter shape has a 5x5 size, so a line buffer of 2 lines, which is half the vertical length of the filter, is required for hardware implementation. Here, the number of lines in the line buffer required to implement the filter shown in Figure 34 (2 lines) is smaller than the number of lines in the line buffer required to implement the filter shown in Figure 33 (4 lines). Therefore, when using a filter like the one shown in Figure 34, the size of the line buffer can be reduced, which in turn reduces the implementation complexity of the encoder / decoder, the memory requirements, and the memory access bandwidth.

[0509] According to one embodiment of the present invention, a filter can be used for filtering whose filter shape includes at least one of the following: rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagon. For example, as shown in the example in Figures 35a and / or 35b, it can have the shapes of a square, octagon, snowflake, rhombus, hexagon, rectangle, cross, sharp, clover, diagonal, etc.

[0510] As an example, a filter set can be constructed using at least one of the filters in Figure 35a and / or Figure 35b, where the vertical length of the filter is 5, and then filtering can be performed.

[0511] As another example, a filter set can be constructed using at least one of the filters with a vertical length of 3 as shown in Figure 35a and / or Figure 35b, and then filtering can be performed.

[0512] As another example, a filter set can be constructed using at least one of the filters in Figure 35a and / or Figure 35b, where the vertical lengths of the filters are 5 and 3, and then filtering can be performed.

[0513] On the other hand, in Figures 35a and / or 35b, the filter shape is designed based on a vertical length of 3 or 5, but it is not limited to this, and can be designed and used for filtering when the vertical length of the filter is M. Here, M is a positive integer.

[0514] On the other hand, in order to signal from the encoder to the decoder which filter to use by constructing a filter set of only H filters, as in the example shown in Figure 35a and / or Figure 35b, the filter index can be entropy encoded / decoded on a picture / tile / tile group / slice / sequence basis, where H is a positive integer. In other words, the filter index can be entropy encoded / decoded on a sequence parameter set, picture parameter set, slice header, slice data, tile header, tile group header, etc., within a bitstream.

[0515] On the other hand, at least one of the rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagonal filters can be used for reconstructing / decoding sample filtering of at least one of the luminance component and chrominance component.

[0516] On the other hand, the numbers within each filter shape in Figure 35a and / or Figure 35b indicate the filter coefficient index, and the filter coefficient index has a symmetrical shape around the filter. In other words, the filters in Figure 35a and / or Figure 35b can be considered point-symmetric filter shapes.

[0517] According to one embodiment of the present invention, using a filter as shown in the example in Figure 35a and / or Figure 35b can reduce the computational complexity of the encoder / decoder compared to using a filter as shown in the example in Figure 33.

[0518] For example, by using at least one of the filters shown in Figure 35a and / or Figure 35b, the number of filter coefficients to be entropy coded / decoded can be reduced compared to the number of filter coefficients in a 9x9 rhombus shape as shown in Figure 33, thereby reducing the computational complexity of the encoder / decoder.

[0519] As another example, using at least one of the filters shown in Figure 35a and / or Figure 35b allows for a reduction in the number of multiplications per sample required for filtering with a 9x9 diamond-shaped filter coefficient as shown in Figure 33, thereby reducing the computational complexity of the encoder / decoder.

[0520] As another example, by using at least one of the filters shown in Figure 35a and / or Figure 35b, the number of lines in the line buffer can be reduced to less than the number of lines in the line buffer required for filtering the 9x9 diamond-shaped filter coefficients shown in Figure 33, thereby reducing the implementation complexity of the encoder / decoder, memory requirements, and memory access bandwidth.

[0521] According to one embodiment of the present invention, instead of the point-symmetric filter shape, a filter including at least one of the horizontal / vertical symmetric filter shapes, as shown in the example in Figure 36, can be used for filtering. Alternatively, in addition to point-symmetric and horizontal / vertical symmetric filters, diagonally symmetric filters can also be used. In Figure 36, the numbers within each filter shape may indicate the filter coefficient index.

[0522] As an example, in Figure 36, a filter set can be constructed using at least one filter with a vertical length of 5, and then filtering can be performed.

[0523] As another example, in Figure 36, a filter set can be constructed using at least one filter with a vertical length of 3, and then filtering can be performed.

[0524] As another example, in Figure 36, a filter set can be constructed using at least one of the filters whose vertical lengths are 5 and 3, and then filtering can be performed.

[0525] On the other hand, in Figure 36, the filter shape is designed based on a vertical length of 3 or 5, but it is not limited to this; it can also be designed and used for filtering when the vertical length of the filter is M. Here, M is a positive integer.

[0526] On the other hand, in order to signal from the encoder to the decoder which filter to use by constructing a filter set consisting only of H filters, as in the example in Figure 36, the filter index can be entropy encoded / decoded at the picture / tile / tile group / slice / sequence level. Here, H is a positive integer. In other words, the filter index can be entropy encoded / decoded at the sequence parameter set, picture parameter set, slice header, slice data, tile header, tile group header, etc., within the bitstream.

[0527] On the other hand, at least one of the rhombus, rectangle, square, trapezoid, diagonal, snowflake, sharp, clover, cross, triangle, pentagon, hexagon, octagon, decagon, and dodecagonal filters can be used for reconstructing / decoding sample filtering of at least one of the luminance component and chrominance component.

[0528] According to one embodiment of the present invention, using a filter like the example in Figure 36 can reduce the computational complexity of the encoder / decoder compared to using a filter like the example in Figure 33.

[0529] For example, by using at least one of the filters shown in Figure 36, the number of filter coefficients to be entropy coded / decoded can be reduced compared to the number of filter coefficients in the 9x9 rhombus shape shown in Figure 33, thereby reducing the computational complexity of the encoder / decoder.

[0530] As another example, using at least one of the filters shown in Figure 36 reduces the number of multiplications per sample required for filtering with a 9x9 diamond-shaped filter coefficient as shown in Figure 33, thereby reducing the computational complexity of the encoder / decoder.

[0531] As another example, if at least one of the filters shown in Figure 36 is used, the number of lines in the line buffer can be reduced compared to the number of lines in the line buffer required for filtering the 9x9 diamond-shaped filter coefficients shown in Figure 33, thereby reducing the implementation complexity of the encoder / decoder, the memory requirements, and the memory access bandwidth.

[0532] According to one embodiment of the present invention, before performing filtering by block classification unit, the sum of gradient values ​​calculated by block classification unit (i.e., the sum of gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions) v , g h , g d1 , g d2 A geometric transformation can be performed on the filter coefficients f(k,l) based on at least one of the following. In this case, the geometric transformation on the filter coefficients can mean calculating the geometrically transformed filter by performing at least one of the following on the filter: 90-degree rotation, 180-degree rotation, 270-degree rotation, second diagonal flipping, first diagonal flipping, vertical flipping, horizontal flipping, vertical and horizontal flipping, or zoom in / out.

[0533] On the other hand, performing a geometric transformation on the filter coefficients and then filtering the reconstructed / decoded samples using the geometrically transformed filter coefficients may be equivalent to performing a geometric transformation on at least one of the reconstructed / decoded samples to which the filter is applied, and then filtering the reconstructed / decoded samples using the filter coefficients.

[0534] According to one embodiment of the present invention, geometric transformations can be performed as shown in the examples of equations 33 to 35. [Formula 33]

number

number

number

[0535] Here, equation 33 represents an example of an equation relating to second diagonal flipping, equation 34 to vertical flipping, and equation 35 to 90-degree rotation. In equations 34 and 35, K is the number of filter taps (filter length) in the horizontal and vertical directions, and 0 ≤ k and l ≤ K-1 can represent the coordinates of the filter coefficients. For example, (0,0) can represent the top-left corner, and (K-1,K-1) can represent the bottom-right corner.

[0536] Table 1 also shows an example of the types of geometric transformations applied to the filter coefficients f(k,l) obtained by summing gradient values. [Table 1]

[0537] Figure 37 shows filters that have undergone geometric transformations to square, octagonal, snowflake, and rhombus-shaped filters according to one embodiment of the present invention.

[0538] Referring to Figure 37, at least one geometric transformation from second diagonal flipping, vertical flipping, and 90-degree rotation can be applied to square, octagonal, snowflake, and rhombus-shaped filter coefficients. The filter coefficients obtained by the geometric transformation can then be used for filtering. On the other hand, filtering the reconstructed / decoded samples using the geometrically transformed filter coefficients after applying the geometric transformation to the filter coefficients may be equivalent to applying the geometric transformation to at least one of the reconstructed / decoded samples to which the filter is applied, and then filtering the reconstructed / decoded samples using the filter coefficients.

[0539] According to one embodiment of the present invention, the restored / decoded sample

number

number

number

[0540] In formula 36, ​​L is the number of filter taps (filter length) in the horizontal or vertical direction.

number

[0541] On the other hand, the decoded samples filtered during the execution of the filtering process

number

[0542] On the other hand, the filtered decoded sample can be truncated so that it is represented within N bits, where N is a positive integer. For example, if filtering is performed on the restored / decoded sample, and the resulting filtered decoded sample is truncated to 10 bits, the final decoded sample value can be between 0 and 1023.

[0543] According to one embodiment of the present invention, in the case of color difference components, filtering can be performed based on filtering information of the luminance component.

[0544] For example, chromatic difference component restoration image filtering can only be performed when luminance component restoration image filtering is performed. Here, chromatic difference component restoration image filtering can be performed on at least one of the U(Cr) and V(Cb) components.

[0545] As another example, in the case of color difference components, filtering can be performed using at least one of the following: the filter coefficient of the corresponding luminance component, the number of filter taps, the filter shape, and information on whether filtering is performed or not.

[0546] According to one embodiment of the present invention, when filtering is performed, if there are currently unavailable samples around the sample, padding can be performed, and then filtering can be performed using the padded sample. The padding can mean a method of copying an available sample value adjacent to the unavailable sample to the unavailable sample value. Alternatively, a sample value or statistical value obtained based on an available sample value adjacent to the unavailable sample can be used. The padding can be repeated as many times as there are columns P and rows R, where P and R may be positive integers.

[0547] Here, an unavailable sample can mean a sample that lies outside the boundaries of a CTU, CTB, slice, tile, tile group, or picture. Alternatively, an unavailable sample can mean a sample that belongs to at least one of the CTUs, CTBs, slices, tiles, tile groups, and pictures that is different from at least one of the CTUs, CTBs, slices, tiles, tile groups, and pictures to which the sample currently belongs.

[0548] Furthermore, it is not necessary to use a predetermined sample when performing filtering.

[0549] For example, when performing the filtering described above, it is not necessary to use padded samples.

[0550] As another example, if, during the execution of the filtering, there are currently unavailable samples around the sample, those unavailable samples do not need to be used for filtering.

[0551] As another example, if, during the execution of the filtering, samples currently surrounding the sample are located outside the CTU or CTB boundary, those surrounding samples do not need to be used for filtering.

[0552] Furthermore, when performing the filtering described above, samples to which at least one of deblocking filtering, adaptive sample offsetting, and adaptive intra-loop filtering has been applied can be used.

[0553] Furthermore, when performing the filtering described above, if samples currently surrounding the sample are located outside the CTU or CTB boundary, at least one of deblocking filtering, adaptive sample offsetting, and adaptive in-loop filtering does not need to be applied.

[0554] Furthermore, if the samples used for filtering include samples located outside the CTU or CTB boundary and therefore unavailable, the unavailable samples can be used for filtering without applying at least one of deblocking filtering, adaptive sample offsetting, and adaptive in-loop filtering.

[0555] According to one embodiment of the present invention, when performing the filtering, filtering can be performed on at least one of the samples that exist around at least one boundary of CU, PU, ​​TU, block, block classification unit, CTU, and CTB. In this case, the boundary may include at least one of a vertical boundary, a horizontal boundary, and a diagonal boundary. Furthermore, the samples that exist around the boundary may be at least one of U rows, U columns, and U samples relative to the boundary, where U may be a positive integer.

[0556] According to one embodiment of the present invention, when performing the filtering, filtering can be performed on at least one of the samples located inside a block, excluding samples located around at least one boundary of CU, PU, ​​TU, block, block classification unit, CTU, and CTB. In this case, the boundary may include at least one of a vertical boundary, a horizontal boundary, and a diagonal boundary. Furthermore, the samples located around the boundary may be at least one of U rows, U columns, and U samples relative to the boundary, where U may be a positive integer.

[0557] According to one embodiment of the present invention, when performing the filtering, it can be determined whether or not to perform it based on at least one coding parameter among the current block and surrounding blocks. In this case, the coding parameter may include at least one of the following: prediction mode (whether it is inter-screen prediction or intra-screen prediction), intra-screen prediction mode, inter-screen prediction mode, inter-screen prediction indicator, motion vector, reference image index, quantization parameter, current block size, current block shape, block classification unit size, and coded block flag / pattern.

[0558] Furthermore, when the filtering is performed, at least one of the filter coefficients, filter taps (filter length), filter shape, and filter type can be determined based on at least one encoding parameter from the current block and surrounding blocks. Based on at least one of the encoding parameters, at least one of the filter coefficients, filter taps (filter length), filter shape, and filter type may be different from each other.

[0559] As an example, the number of filters used in the filtering process can be determined based on the quantization parameter. For instance, if the quantization parameter is less than the threshold T, J filters can be used for filtering; if the quantization parameter is greater than the threshold R, H filters can be used for filtering; and otherwise, G filters can be used for filtering. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the value of the quantization parameter, the fewer filters can be used.

[0560] As another example, the number of filters used during the filtering process can be determined based on the current block size. For example, if the current block size is less than threshold T, J filters can be used; if the current block size is greater than threshold R, H filters can be used; and otherwise, G filters can be used. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the block size relatively, the fewer filters can be used.

[0561] As another example, the number of filters used during the filtering process can be determined according to the size of the block classification unit. For example, if the size of the block classification unit is less than the threshold T, J filters can be used; if the size of the block classification unit is greater than the threshold R, H filters can be used; and otherwise, G filters can be used. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the size of the block classification unit, the fewer filters can be used.

[0562] As another example, filtering can be performed using at least one combination of the filtering methods described above.

[0563] The steps for encoding / decoding the filter information will be described below.

[0564] According to one embodiment of the present invention, filter information can be entropy encoded / decoded between the slice header in the bitstream and the first CTU syntax element in the slice data.

[0565] Furthermore, filter information can be entropy encoded / decoded using sequence parameter sets, picture parameter sets, slice headers, slice data, tile headers, tile group headers, CTUs, CTBs, etc., within the bitstream.

[0566] On the other hand, the filter information may include at least one of the following: information on whether luminance component filtering is performed, whether chrominance component filtering is performed, the value of the filter coefficient, the number of filters, the number of filter taps (filter length) information, filter shape information, filter type information, information on whether slice / tile / tile group / picture / CTU / CTB / block / CU unit filtering is performed, information on whether CU unit filtering is performed, CU maximum depth filtering information, information on whether CU unit filtering is performed, information on whether a previously referenced image filter is used, the previously referenced image filter index, information on whether a fixed filter is used for the block classification index, index information for the fixed filter, filter merge information, information on whether different filters are used for the luminance component and the chrominance component, and information on the symmetrical shape of the filter.

[0567] Here, the number of filter taps can be at least one of the following: the horizontal length of the filter, the vertical length of the filter, the first diagonal length of the filter, the second diagonal length of the filter, the horizontal and vertical lengths of the filter, and the number of filter coefficients within the filter.

[0568] On the other hand, the filter information may include up to L luminance filters, where L is a positive integer and may be 25. Also, the filter information may include up to L chrominance filters, where L is a positive integer and may be 1.

[0569] On the other hand, a single filter may contain up to K luminance filter coefficients, where K is a positive integer and may be 13. Furthermore, the filter information may contain up to K chrominance filter coefficients, where K is a positive integer and may be 7.

[0570] For example, information regarding the symmetry of a filter may be information regarding whether the filter shape is point-symmetric, horizontally symmetric, vertically symmetric, or a combination thereof.

[0571] On the other hand, only some of the filter coefficients can be signaled. For example, if the filter has a symmetric shape, only one set of filter coefficients that is symmetric to the information about the filter's symmetric shape can be signaled. Also, for example, the filter coefficients at the center of the filter can be implicitly derived and therefore do not need to be signaled.

[0572] According to one embodiment of the present invention, the filter coefficient values ​​among the filter information can be quantized by an encoder, and the quantized filter coefficient values ​​can be entropy encoded. Similarly, the quantized filter coefficient values ​​can be entropy decoded by a decoder, and the quantized filter coefficient values ​​can be inversely quantized to restore the filter coefficient values. The filter coefficient values ​​can be quantized and inversely quantized within a range of values ​​that can be represented by a fixed M bits. Furthermore, at least one of the filter coefficients can be quantized and inversely quantized with different bits. Conversely, at least one of the filter coefficients can be quantized and inversely quantized with the same bits. The M bits can be determined based on quantization parameters. The M bits can also be a constant value predetermined in the encoder and decoder. Here, M is a positive integer and may be 8 or 10. Furthermore, the M bits may be the same as or less than the number of bits required to represent a sample in the encoder / decoder. For example, if the number of bits required to represent a sample is 10, then M may be 8. The first filter coefficient within the filter is -2 M From 2 M The second filter coefficient can have values ​​down to -1, and the second filter coefficient can range from 0 to 2. M It can have values ​​down to -1. Here, the first filter coefficient can represent the filter coefficients remaining after excluding the central filter coefficient, and the second filter coefficient can represent the central filter coefficient.

[0573] Furthermore, among the filter information, the filter coefficient values ​​are truncated by at least one of the encoder and decoder, and at least one of the minimum and maximum values ​​related to the truncation (clipping) can be entropy coded / decoded. The filter coefficient values ​​can be truncated within the range of the minimum and maximum values. Also, at least one of the minimum and maximum values ​​can be different values ​​for each filter coefficient. Conversely, at least one of the minimum and maximum values ​​can be the same value for each filter coefficient. Also, at least one of the minimum and maximum values ​​can be determined based on the quantization parameter. Also, at least one of the minimum and maximum values ​​can be a constant value predefined in the encoder and decoder.

[0574] According to one embodiment of the present invention, at least one of the filter information can be entropy coded / decoded based on coding parameters of at least one of the current block and surrounding blocks. In this case, the coding parameters may include at least one of the following: prediction mode (whether it is inter-screen prediction or intra-screen prediction), intra-screen prediction mode, inter-screen prediction mode, inter-screen prediction indicator, motion vector, reference image index, quantization parameter, size of the current block, shape of the current block, size of the block classification unit, and coded block flag / pattern.

[0575] For example, among the filter information mentioned above, the number of filters can be determined based on the quantization parameters of the picture / slice / tile group / tile and CTU / CTB / block. Specifically, if the quantization parameter is smaller than the threshold T, J filters can be entropy coded / decoded; if the quantization parameter is larger than the threshold R, H filters can be entropy coded / decoded; and otherwise, G filters can be entropy coded / decoded. Here, T, R, J, H, and G are positive integers including 0. Also, J may be the same as or greater than H. Here, the larger the value of the quantization parameter, the fewer filters can be entropy coded / decoded.

[0576] According to one embodiment of the present invention, whether or not filtering is performed on at least one of the luminance component and the chrominance component can be determined by using filtering execution status information (flag).

[0577] As an example, whether or not filtering is performed on at least one of the luminance component and chrominance component can be determined using CTU / CTB / CU / block-level filtering execution information (flags). For example, if the filtering execution information for a CTB is a first value, filtering may be performed on that CTB, and if it is a second value, filtering may not be performed on that CTB. In this case, the filtering execution information can be entropy encoded / decoded for each CTB. As another example, by further entropy encoding / decoding information regarding the maximum depth or minimum size of a CU (filtering information for the maximum depth of the CU), the filtering execution information for CUs can be entropy encoded / decoded only up to that maximum depth or minimum size.

[0578] For example, a CU-unit flag can only be entropy encoded / decoded up to the depth of the square block partition, depending on the block structure, if both square and non-square block partitioning are possible. Furthermore, the CU-unit flag can be entropy encoded / decoded up to the depth of the non-square block partitioning.

[0579] As another example, information regarding whether or not filtering is performed on at least one of the luminance component and chrominance component can be expressed using a block-level flag. For example, if the block-level flag is a first value, filtering may be performed on that block, and if it is a second value, filtering may not be performed on that block. The size of the block-level unit is N × M, where N and M can be positive integers.

[0580] As another example, information on whether filtering is performed on at least one of the luminance component and chrominance component can be expressed using a flag in the CTU unit. For example, if the flag in the CTU unit is a first value, filtering may be performed on that CTU, and if it is a second value, filtering may not be performed on that CTU. The size of the CTU unit is N × M, where N and M can be positive integers.

[0581] As another example, whether or not filtering is performed on at least one of the luminance and color difference components can be determined based on the picture / slice / tile group / tile type, and information on whether or not filtering is performed on at least one of the luminance and color difference components can be provided using flags at the picture / slice / tile group / tile level.

[0582] According to one embodiment of the present invention, filter coefficients corresponding to different block classifications can be merged in order to reduce the amount of filter coefficients that need to be entropy coded / decoded. In this case, filter merge information regarding whether or not the filter coefficients are merged can be entropy coded / decoded.

[0583] Furthermore, in order to reduce the amount of filter coefficients that need to be entropy coded / decoded, the filter coefficients of a reference image can be used as the filter coefficients of the current image. In this case, the method of using the filter coefficients of a reference image can be called temporal filter coefficient prediction. For example, the said temporal filter coefficient prediction can be used for inter-screen prediction images (B / P images / slices / tile groups / tiles). On the other hand, the filter coefficients of the reference image can be stored in memory. Also, when using the filter coefficients of a reference image in the current image, the entropy coding / decoding of the filter coefficients for the current image can be omitted. In this case, the previously referenced image filter index for which reference image filter coefficients to use can be entropy coded / decoded.

[0584] As an example, when using temporal filter coefficient prediction, a candidate list of filter sets can be constructed. Before decoding a new sequence, the candidate list of filter sets is empty, but each time an image is decoded, the filter coefficients of that image may be added to the candidate list of filter sets. If the number of filters in the candidate list of filter sets reaches the maximum number of filters G, a new filter can replace the oldest filter in terms of the decoding order. In other words, the candidate list of filter sets can be updated in a FIFO (first-in-first-out) manner, where G is a positive integer, and may be 6. To prevent duplication of filters in the candidate list of filter sets, the filter coefficients of images that do not use temporal filter coefficient prediction may be added to the candidate list of filter sets.

[0585] As another example, when using temporal filter coefficient prediction, candidate lists of filter sets for multiple temporal layer indices can be constructed to support temporal scalability. That is, a candidate list of filter sets can be constructed for each temporal layer. For example, the candidate list of filter sets for each temporal layer can include filter sets for previously decoded images that have a temporal layer index equal to or smaller than the image's temporal layer index. Also, after decoding each image, the filter coefficients for that image may be included in a candidate list of filter sets with temporal layer indices equal to or greater than the image's temporal layer index.

[0586] According to one embodiment of the present invention, the filtering can be performed using a fixed filter set.

[0587] While temporal filter coefficient prediction cannot be used for in-screen predicted images (I-images / slices / tile groups / tiles), at least one filter from up to 16 fixed filter sets can be used for filtering, depending on each block classification index. To signal from the encoder to the decoder whether or not to use a fixed filter set, the information on whether or not to use a fixed filter for each block classification index can be entropy encoded / decoded, and if a fixed filter is used, the index information for that fixed filter can also be entropy encoded / decoded. Even if a fixed filter is used for a specific block classification index, the filter coefficients can be entropy encoded / decoded, and the reconstructed image can be filtered using the entropy encoded / decoded filter coefficients and the fixed filter coefficients.

[0588] Furthermore, the fixed filter set can also be used for the inter-screen prediction image (B / P-image / slice / tile group / tile).

[0589] Furthermore, the adaptive intra-loop filtering can be performed with a fixed filter without entropy coding / decoding of the filter coefficients. Here, the fixed filter can mean a set of filters predefined by the encoder and decoder. In this case, without entropy coding / decoding of the filter coefficients, the index information for the fixed filter regarding which filter or which filter set to use from the predefined set of filters by the encoder and decoder can be entropy coded / decoded. At least one of the filter coefficient values, filter tap count (filter length), and filter shape can be filtered by another fixed filter at least one of the block classification unit, block unit, CU unit, CTU unit, slice, tile, tile group, and picture unit.

[0590] On the other hand, at least one filter within the fixed filter set can be converted to a filter with a different number of filter taps and filter shape. For example, as shown in the example in Figure 38, a 9x9 rhombus-shaped filter coefficient can be converted to a 5x5 square-shaped filter coefficient. Specifically, as will be described later, a 9x9 rhombus-shaped filter coefficient can be converted to a 5x5 square-shaped filter coefficient.

[0591] For example, the sum of the filter coefficients corresponding to filter coefficient indices 0, 2, and 6 in a 9x9 rhombus shape may be assigned to filter coefficient index 2 in a 5x5 square shape.

[0592] As another example, the sum of the filter coefficients corresponding to filter coefficient indices 1 and 5 in a 9x9 rhombus shape may be assigned to filter coefficient index 1 in a 5x5 square shape.

[0593] As another example, the sum of the filter coefficients corresponding to filter coefficient indices 3 and 7 in a 9x9 rhombus shape may be assigned to filter coefficient index 3 in a 5x5 square shape.

[0594] As another example, a filter coefficient corresponding to filter coefficient index 4 in a 9x9 rhombus shape may be assigned to filter coefficient index 0 in a 5x5 square shape.

[0595] As another example, a filter coefficient corresponding to filter coefficient index 8 in a 9x9 rhombus shape may be assigned to filter coefficient index 4 in a 5x5 square shape.

[0596] As another example, the sum of the filter coefficients corresponding to filter coefficient indices 9 and 10 in a 9x9 rhombus shape may be assigned to filter coefficient index 5 in a 5x5 square shape.

[0597] As another example, a filter coefficient corresponding to filter coefficient index 11 in a 9x9 rhombus shape may be assigned to filter coefficient index 6 in a 5x5 square shape.

[0598] As another example, the filter coefficient corresponding to index 12 of a 9x9 rhombus-shaped filter coefficient may be assigned to index 7 of a 5x5 square-shaped filter coefficient.

[0599] As another example, a filter coefficient corresponding to filter coefficient index 13 in a 9x9 rhombus shape may be assigned to filter coefficient index 8 in a 5x5 square shape.

[0600] As another example, the sum of the filter coefficients corresponding to filter coefficient indices 14 and 15 in a 9x9 rhombus shape may be assigned to filter coefficient index 9 in a 5x5 square shape.

[0601] As another example, the sum of the filter coefficients corresponding to filter coefficient indices 16, 17, and 18 in a 9x9 rhombus shape may be assigned to filter coefficient index 10 in a 5x5 square shape.

[0602] As another example, a filter coefficient corresponding to filter coefficient index 19 in a 9x9 rhombus shape may be assigned to filter coefficient index 11 in a 5x5 square shape.

[0603] As another example, a filter coefficient corresponding to filter coefficient index 20 in a 9x9 rhombus shape may be assigned to filter coefficient index 12 in a 5x5 square shape.

[0604] Table 2, on the other hand, shows an example of generating filter coefficients by converting 9x9 rhombus-shaped filter coefficients to 5x5 square-shaped filter coefficients. [Table 2]

[0605] In Table 2, the sum of the filter coefficients of at least one of the 9x9 rhombic filters may be the same as the sum of the filter coefficients of at least one of the corresponding 5x5 square filters.

[0606] On the other hand, if a maximum of 16 fixed filter sets are used for 9x9 diamond-shaped filter coefficients, then a maximum of 21 filter coefficients × 25 filters × 16 sets must be stored in memory. If a maximum of 16 fixed filter sets are used for 5x5 square-shaped filter coefficients, then a maximum of 13 filter coefficients × 25 filters × 16 sets must be stored in memory. In this case, the amount of memory required to store the 5x5 square-shaped fixed filter coefficients is less than the amount of memory required to store the 9x9 diamond-shaped fixed filter coefficients, thus reducing the memory requirements and memory access bandwidth required when implementing the encoder / decoder.

[0607] On the other hand, color difference component restoration / decoding samples can be filtered using a filter whose filter tap count and / or filter shape have been transformed from the luminance component at the corresponding position.

[0608] According to one embodiment of the present invention, filter coefficient prediction can be prohibited from a predetermined fixed filter.

[0609] According to one embodiment of the present invention, multiplication operations can be replaced with shift operations. First, the filter coefficients used to filter the luminance and / or chrominance blocks can be divided into two groups. For example, they can be divided into a first group {L0, L1, L2, L3, L4, L5, L7, L8, L9, L10, L14, L15, L16, L17} and a second group containing the remaining coefficients. The first group can be restricted to having coefficient values ​​of {-64, -32, -16, -8, -4, 0, 4, 8, 16, 32, 64}. In this case, the multiplication operation between the filter coefficients included in the first group and the restored / decoded samples can be implemented with a single bit shift operation. Thus, the filter coefficients included in the first group can be mapped to values ​​that have each undergone a bit shift operation before binarization in order to reduce signaling overhead.

[0610] According to one embodiment of the present invention, the block classification and / or decision on whether or not to perform filtering on the color difference component can directly reuse the results of the luminance component at the same position. Furthermore, the filter coefficients for the color difference component can reuse the filter coefficients for the luminance component, for example, a constant 5x5 diamond filter shape can be used.

[0611] As an example, the filter coefficients can be converted from a 9x9 filter shape for the luminance component to a 5x5 filter shape for the chrominance component. In this case, the outermost filter coefficient can be set to 0.

[0612] As another example, in the case of a 5x5 filter shape for the luminance component, the filter coefficients may be the same as those for the chrominance component. In other words, the filter coefficients for the luminance component can be directly applied to the filter coefficients for the chrominance component.

[0613] As another example, to maintain a 5x5 filter shape for filtering color difference components, filter coefficients outside the 5x5 diamond filter shape can be substituted with coefficient values ​​at the boundaries of the 5x5 diamond.

[0614] On the other hand, in-loop filtering can be performed individually on luminance blocks and chrominance blocks. Control flags can be signaled at the picture / slice / tile group / tile CTU / CTB level to indicate whether or not individual adaptive in-loop filtering support is available for chrominance blocks. Flags can also be signaled to indicate a mode that performs adaptive in-loop filtering integrally on luminance blocks and chrominance blocks, or a mode that supports individual adaptive in-loop filtering on luminance blocks and chrominance blocks.

[0615] According to one embodiment of the present invention, when entropy encoding / decoding at least one of the filter information, at least one of the following binarization methods can be used. Method for binarizing truncated rice K-th order Exp_Golomb (K-th order Exp_Golomb) binarization method Restricted K-th order exponential Golomb binarization method Fixed-length binarization method Unary binarization method Truncated Unary Binarization Method

[0616] As an example, the filter coefficient values ​​for luminance filters and chrominance filters can be entropy coded / decoded using different binarization methods for the luminance and chrominance filters.

[0617] As another example, the filter coefficient values ​​for a luminance filter can be entropy coded / decoded using different binarization methods for the coefficients within the luminance filter. Alternatively, the filter coefficient values ​​for a luminance filter can be entropy coded / decoded using the same binarization method for the coefficients within the luminance filter.

[0618] As another example, the filter coefficient values ​​for a color difference filter can be entropy coded / decoded using different binarization methods for the coefficients within the color difference filter. Alternatively, the filter coefficient values ​​for a color difference filter can be entropy coded / decoded using the same binarization method for the coefficients within the color difference filter.

[0619] Furthermore, when entropy encoding / decoding at least one of the filter information, a context model can be determined using at least one of the filter information from the surrounding block, or at least one of the previously encoded / decoded filter information, or the filter information previously encoded / decoded from the image.

[0620] Furthermore, when entropy coding / decoding at least one of the filter information, a context model can be determined using at least one of the filter information components that are different from each other.

[0621] Furthermore, when entropy coding / decoding the filter coefficients, the context model can be determined using at least one of the other filter coefficients within the filter.

[0622] Furthermore, when entropy coding / decoding at least one of the filter information, entropy coding / decoding can be performed using at least one of the filter information from the surrounding block, or at least one of the previously coded / decoded filter information, or a filter information coded / decoded in a previous image as a predicted value for the filter information.

[0623] Furthermore, when entropy coding / decoding at least one of the filter information, at least one of the filter information components that are different from each other can be used as a predicted value for the filter information when entropy coding / decoding is performed.

[0624] Furthermore, when entropy coding / decoding the filter coefficients, at least one of the other filter coefficients in the filter can be used as a predicted value for entropy coding / decoding.

[0625] Furthermore, the filter information can be entropy encoded / decoded using at least one combination of the above-mentioned filter information entropy encoding / decoding methods.

[0626] According to one embodiment of the present invention, the adaptive intra-loop filtering may be performed in units of at least one of the following: blocks, CUs, PUs, TUs, CBs, PBs, TBs, CTUs, CTBs, slices, tiles, tile groups, and picture units. When performed in units of the aforementioned, it can be said that the block classification step, the filtering execution step, and the filter information encoding / decoding step are performed in units of at least one of the following: blocks, CUs, PUs, TUs, CBs, PBs, TBs, CTUs, CTBs, slices, tiles, tile groups, and picture units.

[0627] According to one embodiment of the present invention, the adaptive intra-loop filtering can be performed by determining whether or not to perform at least one of the following: deblocking filtering, sample-adaptive offsetting, and bidirectional filtering.

[0628] As an example, adaptive intra-loop filtering may be performed on samples among the currently reconstructed / decoded samples in the image to which at least one of the following has been applied: deblocking filter, sample-adaptive offset, and bidirectional filtering.

[0629] As another example, adaptive intra-loop filtering may not be performed on samples in the image that have been restored / decoded using at least one of the following methods: deblocking filter, sample-adaptive offset, and bidirectional filtering.

[0630] As another example, for reconstructed / decoded samples in the current image to which at least one of a deblocking filter, sample-adaptive offset, and bidirectional filtering has been applied, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using L filters without performing block classification, where L is a positive integer.

[0631] According to one embodiment of the present invention, the adaptive intra-loop filtering can be determined to be performed or not depending on the slice type or tile group type of the current image.

[0632] As an example, the adaptive intra-loop filtering may be performed only if the current image slice or tile group type is an I-slice or I-tile group.

[0633] As another example, the adaptive intra-loop filtering may be performed when the current image slice or tile group type is at least one of I-slice, B-slice, P-slice, I-tile group, B-tile group, and P-tile group.

[0634] As another example, if the slice or tile group type of the current image is at least one of I-slice, B-slice, P-slice, I-tile group, B-tile group, and P-tile group, adaptive intra-loop filtering of the current image may be performed on the reconstructed / decoded samples in the current image using L filters without performing block classification, where L is a positive integer.

[0635] As another example, if the current image slice or tile group type is at least one of I-slice, B-slice, P-slice, I-tile group, B-tile group, and P-tile group, adaptive intra-loop filtering may be performed using a single filter shape.

[0636] As another example, if the current image slice or tile group type is at least one of I-slice, B-slice, P-slice, I-tile group, B-tile group, and P-tile group, adaptive intra-loop filtering may be performed using one filter tap.

[0637] As another example, if the current image slice or tile group type is at least one of I-slice, B-slice, P-slice, I-tile group, B-tile group, and P-tile group, then at least one of block classification and filtering may be performed in units of N × M block size. In this case, N and M are positive integers, and may be 4.

[0638] According to one embodiment of the present invention, the adaptive intra-loop filtering can be performed or not performed depending on whether the current image is used as a reference image.

[0639] For example, if the current image is used as a reference image for an image that will be subsequently encoded / decoded, the adaptive in-loop filtering may be performed on the current image.

[0640] As another example, if the current image is not used as a reference image for the image to be subsequently encoded / decoded, the adaptive in-loop filtering may not be performed on the current image.

[0641] As another example, if the current image is not used as a reference image for the image to be subsequently encoded / decoded, adaptive intraloop filtering of the current image may be performed using L filters on the reconstructed / decoded samples within the current image without performing block classification, where L is a positive integer.

[0642] As another example, if the current image is not used as a reference image for the image to be subsequently encoded / decoded, adaptive intra-loop filtering may be performed using a single filter shape.

[0643] As another example, if the current image is not used as a reference image for the image to be subsequently encoded / decoded, adaptive intra-loop filtering may be performed using a single filter tap.

[0644] As another example, if the current image is not to be used as a reference image for the image to be subsequently encoded / decoded, at least one of block classification and filtering may be performed in units of N × M block size. In this case, N and M are positive integers, and may be 4.

[0645] According to one embodiment of the present invention, the adaptive intra-loop filtering can be determined to be performed based on a temporal hierarchy identifier.

[0646] For example, if the temporal hierarchy identifier of the current image indicates the lowest hierarchy level 0, the adaptive loop filtering may be performed on the current image.

[0647] As another example, if the temporal hierarchy identifier of the current image indicates the highest hierarchy (e.g., 4), the adaptive in-loop filtering may be performed on the current image.

[0648] As another example, if the temporal hierarchy identifier of the current image indicates the highest hierarchy (e.g., 4), adaptive intraloop filtering of the current image may be performed using L filters on the reconstructed / decoded samples in the current image without performing block classification, where L is a positive integer.

[0649] As another example, if the current image's temporal hierarchy identifier points to the highest hierarchy (e.g., 4), adaptive intra-loop filtering may be performed using a single filter shape.

[0650] As another example, if the current image's temporal hierarchy identifier points to the highest hierarchy (e.g., 4), adaptive intra-loop filtering may be performed using a single filter tap.

[0651] As another example, if the current image's temporal hierarchy identifier indicates the highest hierarchy (e.g., 4), then at least one of block classification and filtering may be performed in units of N × M block size. In this case, N and M are positive integers and may be 4.

[0652] According to one embodiment of the present invention, at least one of the block classification methods can be determined to be executed based on a temporal layer identifier.

[0653] For example, if the temporal hierarchy identifier of the current image indicates the lowest hierarchy level 0, at least one of the block classification methods described above may be performed on the current image.

[0654] As another example, if the temporal hierarchy identifier of the current image indicates the highest hierarchy (e.g., 4), at least one of the block classification methods may be performed on the current image.

[0655] As another example, different block classification methods may be used within the block classification method described above, depending on the value of the time hierarchy identifier.

[0656] As another example, if the temporal hierarchy identifier of the current image indicates the highest hierarchy (e.g., 4), adaptive intraloop filtering of the current image may be performed using L filters on the reconstructed / decoded samples in the current image without performing block classification, where L is a positive integer.

[0657] As another example, if the current image's temporal hierarchy identifier points to the highest hierarchy (e.g., 4), adaptive intra-loop filtering may be performed using a single filter shape.

[0658] As another example, if the current image's temporal hierarchy identifier points to the highest hierarchy (e.g., 4), adaptive intra-loop filtering may be performed using a single filter tap.

[0659] As another example, if the current image's temporal hierarchy identifier indicates the highest hierarchy (e.g., 4), then at least one of block classification and filtering may be performed in units of N × M block size. In this case, N and M are positive integers and may be 4.

[0660] On the other hand, when performing adaptive intra-loop filtering on the current image, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using L filters without performing block classification, where L is a positive integer. In this case, regardless of the temporal hierarchical identifier, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using L filters without performing block classification.

[0661] On the other hand, when performing adaptive intra-loop filtering on the current image, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using L filters, regardless of whether block classification is performed or not. Here, L is a positive integer. In this case, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using L filters, regardless of whether the temporal hierarchical identifier and block classification are performed or not.

[0662] On the other hand, adaptive intra-loop filtering can be performed using a single filter shape. In this case, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using a single filter shape without performing block classification. Alternatively, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using a single filter shape, regardless of whether block classification is performed or not.

[0663] On the other hand, adaptive intra-loop filtering can be performed using a single filter tap. In this case, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using a single filter tap without performing block classification. Alternatively, adaptive intra-loop filtering can be performed on the reconstructed / decoded samples in the current image using a single filter tap, regardless of whether block classification is performed or not.

[0664] On the other hand, the adaptive intra-loop filtering may be performed on a specific unit. For example, the specific unit may be at least one of the following: picture, slice, tile, tile group, CTU, CTB, CU, PU, ​​TU, CB, PB, TB, or M×N size block. Here, M and N are positive integers and can have the same value or different values. Also, at least one of M and N may be a value predefined in the encoder / decoder and may be a value signaled from the encoder to the decoder.

[0665] On the other hand, Figures 39 to 55d show another example of determining the sum of gradient values ​​in the horizontal, vertical, first diagonal, and second diagonal directions based on a secondary sample.

[0666] Referring to Figures 39 to 55d, filtering can be performed in units of 4x4 luminance blocks. In this case, filtering can be performed using different filter coefficients for each 4x4 luminance block. A subsampled Laplacian operation can be performed to classify the 4x4 luminance blocks. The filter coefficients for filtering can be changed for each 4x4 luminance block. Each 4x4 luminance block can be classified into one of up to 25 classes. The classification index corresponding to the filter index of the 4x4 luminance block can be derived based on the orientation and / or quantization activity value of the block. Here, in order to calculate the orientation and / or quantization activity value for each 4x4 luminance block, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and / or second diagonal directions can be calculated by summing the results of 1D Laplacian operations calculated at subsampled positions within an 8x8 block range.

[0667] Specifically, referring to Figure 39, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions based on the secondary sample is g v , g h , g d1 , g d2At least one of the following can be calculated (hereinafter referred to as "the first method"). Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculation in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, the 1D Laplacian calculation can be performed in the horizontal, vertical, first diagonal, and second diagonal directions at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculation is performed can be subsampled. In Figure 39, the block classification index C can be assigned in 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculation is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate the positions of each restored sample, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculation is calculated.

[0668] Here, Figures 40a to 40d are examples showing the encoding / decoding process of block classification by the first method. Figures 41a to 41d are other examples showing the encoding / decoding process of block classification by the first method. Figures 42a to 42d are yet another example showing the encoding / decoding process of block classification by the first method.

[0669] Referring to Figure 43, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated based on the secondary sample. v , g h , g d1 , g d2At least one of these can be calculated (hereinafter referred to as "the second method"). Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample-by-sample basis. In other words, 1D Laplacian calculations can be performed in the horizontal, vertical, first diagonal, and second diagonal directions, respectively, at the positions of V, H, D1, and D2. Furthermore, the positions where the 1D Laplacian calculations are performed can be subsampled. In Figure 43, the block classification index C is assigned to 4x4 block units where shading is displayed. In this case, the range over which the sum of the 1D Laplacian calculations is calculated may be larger than the size of the block classification unit. Here, the rectangles contained within the thin solid lines indicate the positions of each restored sample, and the thick solid lines indicate the range over which the sum of the 1D Laplacian calculations is calculated.

[0670] Specifically, the second method can be interpreted as meaning that a 1D Laplacian operation is performed at the (x,y) position if both the x and y coordinate values ​​are even, or if both the x and y coordinate values ​​are odd. If both the x and y coordinate values ​​are not even, or if both the x and y coordinate values ​​are not odd, the result of the 1D Laplacian operation at the (x,y) position is assigned to 0. In other words, it can be interpreted as meaning that a 1D Laplacian operation is performed on a checkerboard pattern based on the x and y coordinate values.

[0671] On the other hand, referring to Figure 43, the positions where the 1D Laplacian calculation is performed in the horizontal, vertical, first diagonal, and second diagonal directions may be the same. In other words, regardless of whether it is the vertical, horizontal, first diagonal, or second diagonal direction, the 1D Laplacian calculation can be performed for each of the aforementioned directions using a single (unified) subsampled 1D Laplacian calculation position.

[0672] Here, Figures 44a to 44d are examples showing the encoding / decoding process of block classification by the second method. Figures 45a to 45d are other examples showing the encoding / decoding process of block classification by the second method. Figures 46a to 46d are yet another example showing the encoding / decoding process of block classification by the second method. Figures 47a to 47d are yet another example showing the encoding / decoding process of block classification by the second method.

[0673] Referring to Figure 48, in the case of 4x4 block classification, the sum of the gradient values ​​in the vertical, horizontal, first diagonal, and second diagonal directions is calculated based on the secondary sample. v , g h , g d1 , g d2 At least one of these can be calculated (hereinafter referred to as the "third method"). Here, V, H, D1, and D2 represent the results of the 1D Laplacian calculations in the vertical, horizontal, first diagonal, and second diagonal directions, respectively, on a sample basis. In other words, 1D Laplacian calculations can be performed in the horizontal, vertical, first diagonal, and second diagonal directions, respectively, at the posi...

Claims

1. Assign a block class index to the block classification unit within the coding tree block. Based on the block class index, an adaptive loop filter is applied to the samples in the coding tree block. An image decoding method, Whether or not to apply the adaptive loop filter to the coding tree block is determined based on the adaptive loop filter flag decoded from the bitstream. The aforementioned block class index is determined based on directional information and activity information. At least one of the directional information and the activity information is determined based on a gradient value in at least one of the vertical, horizontal, first diagonal, and second diagonal directions. The gradient value is obtained using a one-dimensional Laplacian operation on the block classification unit, The one-dimensional Laplacian operation is performed only on specific samples included in the block classification unit. The aforementioned specific sample includes only samples at even positions where both the horizontal and vertical coordinates are even, and only samples at odd positions where both the horizontal and vertical coordinates are odd. Image decoding method.

2. If there are unavailable peripheral samples among the peripheral samples of the sample in the coding tree block to which the adaptive loop filter is applied, the adaptive loop filter is applied by padding the unavailable peripheral samples. The image decoding method according to claim 1.

3. The filter coefficient values ​​used in the adaptive loop filter are represented by 8 bits. The image decoding method according to claim 1.

4. Assign a block class index to the block classification unit within the coding tree block. Based on the aforementioned block class index, an adaptive loop filter is applied to the samples within the coding tree block. An image encoding method, An adaptive loop filter flag indicating whether to apply the adaptive loop filter to the encoding tree block is encoded into the bitstream. The aforementioned block class index is determined based on directional information and activity information. At least one of the directional information and the activity information is determined based on a gradient value in at least one of the vertical, horizontal, first diagonal, and second diagonal directions. The gradient value is obtained using a one-dimensional Laplacian operation on the block classification unit, The one-dimensional Laplacian operation is performed only on specific samples included in the block classification unit. The aforementioned specific sample includes only samples at even positions where both the horizontal and vertical coordinates are even, and only samples at odd positions where both the horizontal and vertical coordinates are odd. Image encoding method.

5. A device for transmitting a bitstream, A processor configured to generate the aforementioned bitstream, A transmitting unit configured to transmit the bitstream, Equipped with, The generation of the aforementioned bitstream is Assign a block class index to the block classification unit within the coding tree block. This includes applying an adaptive loop filter to the samples in the coding tree block based on the aforementioned block class index, An adaptive loop filter flag indicating whether to apply the adaptive loop filter to the encoding tree block is encoded into the bitstream. The aforementioned block class index is determined based on directional information and activity information. At least one of the directional information and the activity information is determined based on a gradient value in at least one of the vertical, horizontal, first diagonal, and second diagonal directions. The gradient value is obtained using a one-dimensional Laplacian operation on the block classification unit, The one-dimensional Laplacian operation is performed only on specific samples included in the block classification unit. The aforementioned specific sample includes only samples at even positions where both the horizontal and vertical coordinates are even, and only samples at odd positions where both the horizontal and vertical coordinates are odd. device.

Images