Image encoding / decoding method and apparatus using sample filter

Abstract

Disclosed are an image encoding / decoding method and apparatus using sample filtering. An image encoding / decoding method and apparatus are provided. The image encoding method of the disclosure includes the steps of determining whether to apply filtering to a sample to be filtered, determining a filter type based on a result of the determination, and filtering the sample to be filtered by using the determined filter type.

Description

[0001] This application is a divisional application of the patent application with application number 201980018128.1 and title "Image encoding / decoding method and apparatus using sample filter" filed on March 8, 2019. TECHNICAL FIELD

[0002] The present application relates to an image encoding / decoding method and apparatus, and a recording medium storing a bitstream. More particularly, the present application relates to an image encoding / decoding method and apparatus based on sample filtering. BACKGROUND

[0003] Currently, in various applications, there is an increasing demand for high-resolution, high-quality videos, such as high-definition (HD) videos and ultra-high-definition (UHD) videos. Since videos have higher resolution and quality, the amount of video data increases compared to existing video data. Therefore, when the video data is transmitted through a medium such as a wired / wireless broadband line or stored in an existing storage medium, the transmission or storage cost increases. In order to solve this problem under high-resolution, high-quality video data, an efficient video encoding / decoding technique is required.

[0004] There are various video compression techniques, such as an inter prediction technique for predicting pixel values in a current picture from pixel values within a previous picture or a subsequent picture, an intra prediction technique for predicting pixel values in one region of a current picture from another region of the current picture, a transform and quantization technique for compressing the energy of a residual signal, and an entropy encoding technique for assigning a short code to frequently occurring pixel values and a long code to less frequently occurring pixel values. With these video compression techniques, video data can be efficiently compressed, transmitted, and stored. SUMMARY

[0005] TECHNICAL PROBLEM

[0006] An object of the present application is to provide an image encoding / decoding method and apparatus using sample filtering.

[0007] Further, another object of the present application is to provide an image encoding / decoding method and apparatus in which encoding efficiency is improved by determining a filter target sample or region or both the filter target sample and region and performing filtering by determining whether to apply filtering or determining a type or both.

[0008] Further, another object of the present application is to provide a recording medium storing a bitstream generated by the image encoding / decoding method or apparatus of the present application.

[0009] TECHNICAL SOLUTION

[0010] The video encoding method according to the present application can include determining whether to apply filtering to a filtering target sample, determining a filter type based on the determination, and applying filtering to the filtering target sample by using the determined filter type.

[0011] In the video encoding method according to the present application, the filtering target sample can include at least one of a prediction sample, a reference sample, a reconstructed sample, and a residual sample.

[0012] In the video encoding method according to the present application, whether to apply filtering can be determined based on at least one of a coding mode, an intra prediction mode, an inter prediction mode, transform information, filtering information, a size (a horizontal size or a vertical size), a shape, and a reference sample line of the current block, and a coding mode, an intra prediction mode, an inter prediction mode, transform information, filtering information, a size (a horizontal size or a vertical size), a shape, and a reference sample line of a neighboring block.

[0013] In the video encoding method according to the present application, whether to apply filtering can be determined based on whether the current block has a non-square shape.

[0014] In the video encoding method according to the present application, the method can further include determining whether an intra prediction mode of the current block is a predetermined direction mode.

[0015] In the video encoding method according to the present application, the filter type can include at least one of a filter tap, a filter shape, and a filter coefficient.

[0016] In the video encoding method according to the present application, the filter type can be determined when it is determined to apply filtering to the filtering target sample.

[0017] In the video encoding method according to the present application, the filter type can be determined based on a position of the filtering target sample.

[0018] In the video encoding method according to the present application, the filter type can be determined based on at least one of a size and a prediction mode of the current block.

[0019] The video decoding method according to the present application can include determining whether to apply filtering to a filtering target sample, determining a filter type based on the determination, and applying filtering to the filtering target sample by using the determined filter type.

[0020] In the video decoding method according to the present application, the filtering target sample can include at least one of a prediction sample, a reference sample, a reconstructed sample, and a residual sample.

[0021] In the video decoding method according to the present application, whether to apply filtering is determined based on at least one of the coding mode, the intra prediction mode, the inter prediction mode, the transform information, the filter information, the size (horizontal size or vertical size), the shape, and the reference sample line of the current block, and the coding mode, the intra prediction mode, the inter prediction mode, the transform information, the filter information, the size (horizontal size or vertical size), the shape, and the reference sample line of the neighboring block.

[0022] In the video decoding method according to the present application, whether to apply filtering is determined based on whether the current block has a non-square shape.

[0023] In the video decoding method according to the present application, the method can further include determining whether the intra prediction mode of the current block is a predetermined direction mode.

[0024] In the video decoding method according to the present application, the filter type includes at least one of a filter tap, a filter shape, and a filter coefficient.

[0025] In the video decoding method according to the present application, the filter type is determined when it is determined to apply filtering to the filter target sample.

[0026] In the video decoding method according to the present application, the filter type is determined based on the position of the filter target sample.

[0027] In the video decoding method according to the present application, the filter type is determined based on at least one of the size and the prediction mode of the current block.

[0028] In the video decoding method according to the present application, applying filtering to the filter target sample is performed based on the distance between the filter target sample and the reference sample.

[0029] Further, the computer-readable recording medium according to the present application can store a bitstream generated by the video encoding method according to the present application.

[0030] Advantageous Effects

[0031] According to the present application, there are provided an image encoding / decoding method and apparatus using sample filtering

[0032] Further, according to the present application, there are provided an image encoding / decoding method and apparatus in which coding efficiency is improved by determining a filter target sample or region or both the filter target sample and region, and performing filtering by determining whether to apply filtering or determining a type or both.

[0033] Further, according to the present application, there is provided a recording medium storing a bitstream generated by the image encoding / decoding method or apparatus of the present application.

[0034] Further, according to the present application, it is possible to improve the efficiency of image encoding or decoding or both of image encoding and decoding. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a block diagram showing the configuration of an embodiment of an encoding apparatus to which the present application is applied.

[0036] Figure 2 is a block diagram showing the configuration of an embodiment of a decoding apparatus to which the present application is applied.

[0037] Figure 3 is a diagram schematically showing the partition structure of an image when image encoding and decoding are performed.

[0038] Figure 4 is a diagram showing an embodiment of intra prediction.

[0039] Figure 5 is a diagram showing an embodiment of inter prediction.

[0040] Figure 6 is a diagram showing an embodiment of transform and quantization.

[0041] Figure 7 is a diagram showing filtering using variations of a plurality of reference sample lines according to an embodiment of the present application.

[0042] Figure 8 is a diagram showing determination of whether to apply filtering to a reference sample, a prediction sample, and a reconstructed sample and applying a filter thereto according to an embodiment of the present application.

[0043] Figure 9 is a diagram showing a flowchart of an image encoding method according to an embodiment of the present application.

[0044] Figure 10 is a diagram showing a flowchart of an image decoding method according to an embodiment of the present application. DETAILED DESCRIPTION

[0045] Various modifications can be made to the disclosure and there are many embodiments of the disclosure, some of which are now to be provided and described in detail with reference to the accompanying drawings. However, the disclosure is not limited thereto, although the exemplary embodiments can be interpreted to include all modifications, equivalents, or alternatives within the technical concept and technical scope of the disclosure. In various aspects, like reference numerals refer to the same or similar functions. In the drawings, the shapes and sizes of elements can be exaggerated for clarity. In the following detailed description of the disclosure, reference is made to the accompanying drawings, which show by way of illustration specific embodiments in which the disclosure can be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure. It is to be understood that the various embodiments of the disclosure, although different, are not necessarily mutually exclusive. For example, specific features, structures, and characteristics described herein with reference to one embodiment can be implemented in other embodiments without departing from the spirit and scope of the disclosure. In addition, it should be understood that the position or arrangement of various elements within each disclosed embodiment can be modified without departing from the spirit and scope of the disclosure. Therefore, the following detailed description should not be considered to be limiting, and the scope of the disclosure is defined only by the appended claims, along with the full range of equivalents that the claims are entitled to in the appropriate jurisdictions.

[0046] The terms "first", "second", and the like used in the specification can be used to describe various components, but the components should not be construed as being limited to these terms. These terms are used only to distinguish one component from another component. For example, a "first" component can be named as a "second" component without departing from the scope of the disclosure, and a "second" component can also be similarly named as a "first" component. The term "and / or" includes a combination of multiple items or any one of the multiple items.

[0047] It will be understood that, in this specification, when an element is simply referred to as "connected to" or "coupled to" another element without being "directly connected to" or "directly coupled to" another element, the element can be "directly connected to" or "directly coupled to" another element, or connected or coupled to another element with other elements intervening between the element and another element. In contrast, it should be understood that when an element is referred to as "directly coupled" or "directly connected" to another element, there is no intervening element.

[0048] Further, the constituent elements shown in the embodiments of the present application are independently shown to represent different characteristic functions from each other. Thus, this does not mean that each of the constituent elements is configured as a separate hardware or software constituent unit. In other words, each of the constituent elements includes each of the enumerated constituent elements for convenience. Thus, at least two of the constituent elements of each of the constituent elements can be combined to form one constituent element, or one constituent element can be divided into multiple constituent elements to perform each function. Embodiments in which each of the constituent elements is combined and embodiments in which one constituent element is divided are also included in the scope of the present application, if it does not depart from the essence of the present application.

[0049] The terms used in the specification are merely used to describe particular embodiments, and are not intended to limit the present application. Unless otherwise explicitly described in the context, the expression used in the singular includes the expression used in the plural. In the specification, it will be understood that the terms such as "include" and "have" are intended to indicate that there are the features, numbers, steps, actions, elements, parts, or combinations thereof disclosed in the specification, and are not intended to exclude the possibility that one or more other features, numbers, steps, actions, elements, parts, or combinations thereof are present or can be added.

[0050] Further, certain constituent elements can not be indispensable constituent elements to perform the essential functions of the present application, but can be optional constituent elements to improve the performance thereof. The present application can be implemented by including only the indispensable constituent elements to achieve the essence of the present application, without including the constituent elements to improve the performance. A structure in which only the indispensable constituent elements are included, without including the optional constituent elements to improve the performance, is also included in the scope of the present application.

[0051] Hereinafter, embodiments of the present application will be described in detail with reference to the accompanying drawings. In describing the exemplary embodiments of the present application, a well-known function or construction will not be described in detail since it can unnecessarily obscure the understanding of the present application. The same constituent elements in the drawings are designated by the same reference numerals and repeated description of the same elements will be omitted.

[0052] Hereinafter, an image can refer to a picture constituting a video, or can refer to a video itself. For example, "encoding or decoding or both encoding and decoding an image" can refer to "encoding or decoding or both encoding and decoding a moving picture", and can refer to "encoding or decoding or both encoding and decoding one of the pictures of a moving picture".

[0053] Hereinafter, the terms "moving picture" and "video" can be used as the same meaning and can be replaced with each other.

[0054] Hereinafter, a target image can be an encoding target image as an encoding target and / or a decoding target image as a decoding target. Also, the target image can be an input image input to an encoding apparatus and an input image input to a decoding apparatus. Here, the target image can have the same meaning as a current image.

[0055] Hereinafter, the terms "image", "picture", "frame", and "screen" can be used as the same meaning and can be replaced with each other.

[0056] Hereinafter, a target block can be an encoding target block as an encoding target and / or a decoding target block as a decoding target. Also, the target block can be a current block as a target of current encoding and / or decoding. For example, the terms "target block" and "current block" can be used as the same meaning and can be replaced with each other.

[0057] Hereinafter, the terms "block" and "unit" can be used as the same meaning and can be replaced with each other. Or, the "block" can mean a specific unit.

[0058] Hereinafter, the terms "region" and "segment" can be replaced with each other.

[0059] Hereinafter, a specific signal can be a signal representing a specific block. For example, an original signal can be a signal representing a target block. A prediction signal can be a signal representing a prediction block. A residual signal can be a signal representing a residual block.

[0060] In an embodiment, each of a specific information, data, flag, index, element, and attribute, etc. can have a value. A value of the information, data, flag, index, element, and attribute equal to "0" can represent a logical false or a first predefined value. In other words, the value "0", false, logical false, and the first predefined value can be replaced with each other. A value of the information, data, flag, index, element, and attribute equal to "1" can represent a logical true or a second predefined value. In other words, the value "1", true, logical true, and the second predefined value can be replaced with each other.

[0061] When a variable i or j is used to represent a column, a row, or an index, a value of i can be an integer equal to or greater than 0, or an integer equal to or greater than 1. That is, the column, the row, the index, etc. can be counted from 0, or can be counted from 1.

[0062] Term Description

[0063] Encoder: means an apparatus performing encoding. That is, means an encoding apparatus.

[0064] Decoder: means an apparatus performing decoding. That is, means a decoding apparatus.

[0065] Block: is an array of samples of MxN. Here, M and N can represent positive integers, and the block can represent an array of samples in two-dimensional form. The block can refer to a unit. The current block can represent a coding target block that becomes a target at the time of encoding, or a decoding target block that becomes a target at the time of decoding. Also, the current block can be at least one of a coding block, a prediction block, a residual block, and a transform block.

[0066] Sample: is a basic unit constituting a block. According to a bit depth (Bd), the sample can be represented as a value from 0 to 2 Bd -1. In the present invention, the sample can be used as the meaning of a pixel. That is, the sample, pel, pixel can have the same meaning as each other.

[0067] Unit: can refer to a coding and decoding unit. When an image is encoded and decoded, the unit can be an area generated by partitioning a single image. Also, when a single image is partitioned into sub-partition units during encoding or decoding, the unit can represent a sub-partition unit. That is, an image can be partitioned into a plurality of units. When an image is encoded and decoded, a predetermined process for each unit can be performed. A single unit can be partitioned into sub-units having a size smaller than that of the unit. Depending on the function, the unit can represent a block, a macroblock, a coding tree unit, a coding tree block, a coding unit, a coding block, a prediction unit, a prediction block, a residual unit, a residual block, a transform unit, a transform block, etc. Also, in order to distinguish the unit from the block, the unit can include a luma component block, a chroma component block associated with the luma component block, and a syntax element of each color component block. The unit can have various sizes and shapes, and specifically, the shape of the unit can be a two-dimensional geometric figure such as a square, a rectangle, a trapezoid, a triangle, a pentagon, etc. Also, the unit information can include at least one of a unit type indicating a coding unit, a prediction unit, a transform unit, etc., and a unit size, a unit depth, an order of coding and decoding of the unit, etc.

[0068] Coding tree unit: is configured with a single coding tree block of a luma component Y and two coding tree blocks related to chroma components Cb and Cr. Also, the coding tree unit can represent a block and a syntax element of each block. Each coding tree unit can be partitioned by using at least one of a quad-tree partitioning method, a binary-tree partitioning method, and a ternary-tree partitioning method to configure lower-level units such as a coding unit, a prediction unit, a transform unit, etc. The coding tree unit can be used as a term for designating a sample block that becomes a processing unit at the time of encoding / decoding an image as an input image. Here, the quad-tree can represent a quad.

[0069] Coding tree block: can be used as a term for designating any one of a Y coding tree block, a Cb coding tree block, and a Cr coding tree block.

[0070] Neighbor block: can denote a block adjacent to a current block. The block adjacent to the current block can denote a block in contact with a boundary of the current block, or a block located within a predetermined distance from the current block. The neighbor block can denote a block adjacent to a vertex of the current block. Here, the block adjacent to the vertex of the current block can denote a block vertically adjacent to a neighbor block horizontally adjacent to the current block, or a block horizontally adjacent to a neighbor block vertically adjacent to the current block.

[0071] Reconstructed neighbor block: can denote a neighbor block adjacent to a current block and already spatially / temporally encoded or decoded. Here, the reconstructed neighbor block can denote a reconstructed neighbor unit. The reconstructed spatial neighbor block can be a block within a current picture and already reconstructed by encoding or decoding, or both. The reconstructed temporal neighbor block is a block or a neighbor block of the block at a position corresponding to a current block of a current picture within a reference picture.

[0072] Unit depth: can denote a degree of partitioning of a unit. In a tree structure, a highest node (root node) can correspond to a first unit not partitioned. Also, the highest node can have a smallest depth value. In this case, the depth of the highest node can be level 0. A node having a depth of level 1 can denote a unit generated by partitioning the first unit once. A node having a depth of level 2 can denote a unit generated by partitioning the first unit twice. A node having a depth of level n can denote a unit generated by partitioning the first unit n times. A leaf node can be a lowest node and a node that cannot be further partitioned. The depth of the leaf node can be a maximum level. For example, a predefined value of the maximum level can be 3. The depth of the root node can be the lowest, and the depth of the leaf node can be the deepest. Also, when a unit is represented as a tree structure, a level in which the unit exists can denote a unit depth.

[0073] Bitstream: can denote a bitstream including encoded image information.

[0074] Parameter set: corresponds to header information among configurations within a bitstream. At least one of a video parameter set, a sequence parameter set, a picture parameter set, and an adaptation parameter set can be included in the parameter set. Also, the parameter set can include slice header, tile group header, and tile header information. The term "tile group" denotes a group of tiles and has the same meaning as a slice.

[0075] Parsing: can denote determining a value of a syntax element by performing entropy decoding, or can denote entropy decoding itself.

[0076] Symbol: can denote at least one of a syntax element, an encoding parameter, and a transform coefficient value of an encoding / decoding target unit. Also, the symbol can denote an entropy encoding target or an entropy decoding result.

[0077] Prediction mode: can refer to information indicating a mode coded / decoded with intra prediction or a mode coded / decoded with inter prediction.

[0078] Prediction unit: can represent a basic unit when performing prediction such as inter prediction, intra prediction, inter compensation, intra compensation, and motion compensation. A single prediction unit can be partitioned into multiple partitions having smaller sizes, or can be partitioned into multiple lower-level prediction units. The multiple partitions can be basic units when performing prediction or compensation. The partitions generated by dividing the prediction unit can also be prediction units.

[0079] Prediction unit partition: can represent a shape obtained by partitioning a prediction unit.

[0080] Reference picture list can refer to a list including one or more reference pictures used for inter prediction or motion compensation. There are several types of available reference picture lists, including LC (list combination), L0 (list 0), L1 (list 1), L2 (list 2), L3 (list 3).

[0081] Inter prediction indicator can refer to a direction of inter prediction (uni-prediction, bi-prediction, etc.) of a current block. Alternatively, the inter prediction indicator can refer to the number of reference pictures used to generate a prediction block of the current block. Alternatively, the inter prediction indicator can refer to the number of prediction blocks used when inter predicting or motion compensating the current block.

[0082] Prediction list utilization flag indicates whether at least one reference picture in a specific reference picture list is used to generate a prediction block. The prediction list utilization flag can be used to derive the inter prediction indicator, and conversely, the inter prediction indicator can be used to derive the prediction list utilization flag. For example, when the prediction list utilization flag has a first value of zero (0), it indicates that the reference picture in the reference picture list is not used to generate the prediction block. On the other hand, when the prediction list utilization flag has a second value of one (1), it indicates that the reference picture list is used to generate the prediction block.

[0083] Reference picture index can refer to an index indicating a specific reference picture in a reference picture list.

[0084] Reference picture can refer to a picture that is referenced by a specific block for the purpose of inter prediction or motion compensation of the specific block. Alternatively, the reference picture can be a picture including a reference block that is referenced by a current block for the purpose of inter prediction or motion compensation. Hereinafter, the terms "reference picture" and "reference image" have the same meaning and can be interchangeable.

[0085] A motion vector can be a two-dimensional vector used for inter prediction or motion compensation. The motion vector can represent an offset between a coding / decoding target block and a reference block. For example, (mvX, mvY) can represent a motion vector. Here, mvX can represent a horizontal component and mvY can represent a vertical component.

[0086] A search range can be a two-dimensional region searched during inter prediction to retrieve a motion vector. For example, a size of the search range can be MxN. Here, M and N are both integers.

[0087] A motion vector candidate can refer to a prediction candidate block or a motion vector of a prediction candidate block when a motion vector is predicted. In addition, the motion vector candidate can be included in a motion vector candidate list.

[0088] A motion vector candidate list can represent a list consisting of one or more motion vector candidates.

[0089] A motion vector candidate index can represent an indicator indicating a motion vector candidate in a motion vector candidate list. Alternatively, it can be an index of a motion vector predictor.

[0090] Motion information can represent information including at least one of a motion vector, a reference picture index, an inter prediction indicator, a prediction list utilization flag, reference picture list information, a reference picture, a motion vector candidate, a motion vector candidate index, a merge candidate, and a merge index.

[0091] A merge candidate list can represent a list consisting of one or more merge candidates.

[0092] A merge candidate can represent a spatial merge candidate, a temporal merge candidate, a combined merge candidate, a combined bi-predictive merge candidate, or a zero merge candidate. The merge candidate can include motion information such as an inter prediction indicator, a reference picture index per list, a motion vector, a prediction list utilization flag, and an inter prediction indicator.

[0093] A merge index can represent an indicator indicating a merge candidate in a merge candidate list. Alternatively, the merge index can indicate a block in a reconstructed block spatially / temporally neighboring a current block, from which the merge candidate has been derived. Alternatively, the merge index can indicate at least one piece of motion information of the merge candidate.

[0094] A transform unit: can represent a basic unit when encoding / decoding a residual signal such as a transform, an inverse transform, quantization, inverse quantization, transform coefficient encoding / decoding. A single transform unit can be partitioned into a plurality of lower-level transform units having smaller sizes. Here, the transform / inverse transform can include at least one of a first transform / first inverse transform and a second transform / second inverse transform.

[0095] Scaling: can mean a process of multiplying a quantized level by a factor. A transform coefficient can be generated by scaling a quantized level. Scaling can also be referred to as dequantization.

[0096] Quantization parameter: can mean a value used when a transform coefficient is used to generate a quantized level during quantization. The quantization parameter can also mean a value used when a transform coefficient is generated by scaling a quantized level during dequantization. The quantization parameter can be a value mapped on a quantization step.

[0097] Delta quantization parameter: can mean a difference value between a predicted quantization parameter and a quantization parameter of a coding / decoding target unit.

[0098] Scan: can mean a method of ordering coefficients within a unit, a block, or a matrix. For example, changing a two-dimensional matrix of coefficients into a one-dimensional matrix can be referred to as a scan, and changing a one-dimensional matrix of coefficients into a two-dimensional matrix can be referred to as a scan or an inverse scan.

[0099] Transform coefficient: can mean a coefficient value generated after a transform is performed in an encoder. The transform coefficient can mean a coefficient value generated after at least one of entropy decoding and dequantization is performed in a decoder. A quantized level obtained by quantizing a transform coefficient or a residual signal, or a quantized transform coefficient level, can also fall within the meaning of a transform coefficient.

[0100] Quantized level: can mean a value generated in an encoder by quantizing a transform coefficient or a residual signal. Alternatively, the quantized level can mean a value that is a dequantization target that undergoes dequantization in a decoder. Similarly, a quantized transform coefficient level that is a result of a transform and quantization can also fall within the meaning of a quantized level.

[0101] Non-zero transform coefficient: can mean a transform coefficient having a value other than zero, or a transform coefficient level or a quantized level having a value other than zero.

[0102] Quantization matrix: can mean a matrix used in a quantization process or a dequantization process performed for improving subjective image quality or objective image quality. The quantization matrix can also be referred to as a scaling list.

[0103] Quantization matrix coefficient: can mean each element within a quantization matrix. The quantization matrix coefficient can also be referred to as a matrix coefficient.

[0104] Default matrix: can mean a predetermined quantization matrix that is predefined in an encoder or a decoder.

[0105] Non-default matrix: can mean a quantization matrix that is not predefined in an encoder or a decoder but is signaled by a user.

[0106] Statistical value: The statistical value for at least one of a variable having a calculable specific value, an encoding parameter, a constant value, etc., can be one or more of an average value, a summation value, a weighted average value, a weighted summation value, a minimum value, a maximum value, a most frequently occurring value, a median value, an interpolation value of the corresponding specific value.

[0107] Figure 1 FIG. 1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present application.

[0108] The encoding apparatus 100 can be an encoder, a video encoding apparatus, or an image encoding apparatus. The video can include at least one image. The encoding apparatus 100 can sequentially encode the at least one image.

[0109] Referring to Figure 1 , the encoding apparatus 100 can include a motion prediction unit 111, a motion compensation unit 112, an intra prediction unit 120, a switch 115, a subtractor 125, a transform unit 130, a quantization unit 140, an entropy encoding unit 150, a dequantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

[0110] The encoding apparatus 100 can perform encoding of an input image by using an intra mode or an inter mode or both the intra mode and the inter mode. Further, the encoding apparatus 100 can generate a bitstream including encoded information by encoding the input image, and output the generated bitstream. The generated bitstream can be stored in a computer-readable recording medium, or can be streamed through a wired / wireless transmission medium. When the intra mode is used as a prediction mode, the switch 115 can switch to intra. Alternatively, when the inter mode is used as the prediction mode, the switch 115 can switch to the inter mode. Here, the intra mode can denote an intra prediction mode, and the inter mode can denote an inter prediction mode. The encoding apparatus 100 can generate a prediction block for an input block of an input image. Further, the encoding apparatus 100 can encode a residual block using a residual of the input block and the prediction block after the prediction block is generated. The input image can be referred to as a current image that is a current encoding target. The input block can be referred to as a current block that is a current encoding target, or as an encoding target block.

[0111] When the prediction mode is the intra mode, the intra prediction unit 120 can use samples of a block that has been encoded / decoded and is adjacent to the current block as reference samples. The intra prediction unit 120 can perform spatial prediction on the current block by using the reference samples, or generate prediction samples of the input block by performing the spatial prediction. Here, the intra prediction can denote prediction within a frame.

[0112] When the prediction mode is the inter mode, the motion prediction unit 111 can retrieve a region most matching the input block from a reference picture when performing motion prediction, and derive a motion vector by using the retrieved region. In this case, the search region can be used as the region. The reference picture can be stored in the reference picture buffer 190. Here, when encoding / decoding of the reference picture is performed, the reference picture can be stored in the reference picture buffer 190.

[0113] The motion compensation unit 112 can generate a prediction block by performing motion compensation on the current block by using the motion vector. Here, the inter prediction can mean prediction between frames or motion compensation.

[0114] When a value of the motion vector is not an integer, the motion prediction unit 111 and the motion compensation unit 112 can generate a prediction block by applying an interpolation filter to a partial region of a reference picture. In order to perform inter-picture prediction or motion compensation on a coding unit, it can be determined which mode among a skip mode, a merge mode, an advanced motion vector prediction (AMVP) mode, and a current picture reference mode is used for motion prediction and motion compensation of a prediction unit included in the corresponding coding unit. Then, according to the determined mode, the inter-picture prediction or the motion compensation can be differently performed.

[0115] The subtractor 125 can generate a residual block by using a residual of the input block and the prediction block. The residual block can be referred to as a residual signal. The residual signal can mean a difference between an original signal and a prediction signal. Also, the residual signal can be a signal generated by transforming or quantizing or transforming and quantizing a difference between the original signal and the prediction signal. The residual block can be a residual signal of a block unit.

[0116] The transform unit 130 can generate transform coefficients by performing a transform on the residual block, and output the generated transform coefficients. Here, the transform coefficients can be coefficient values generated by performing a transform on the residual block. When a transform skip mode is applied, the transform unit 130 can skip the transform on the residual block.

[0117] A quantized level can be generated by applying quantization to the transform coefficients or to the residual signal. Hereinafter, the quantized level can also be referred to as a transform coefficient in an embodiment.

[0118] The quantization unit 140 can generate a quantized level by quantizing the transform coefficients or the residual signal according to a parameter, and output the generated quantized level. Here, the quantization unit 140 can quantize the transform coefficients by using a quantization matrix.

[0119] The entropy encoding unit 150 can generate a bitstream by performing entropy encoding on the values calculated by the quantization unit 140 or on the encoding parameter values calculated when encoding is performed according to a probability distribution, and output the generated bitstream. The entropy encoding unit 150 can perform entropy encoding on the sample information of the image and information used to decode the image. For example, the information used to decode the image can include syntax elements.

[0120] When entropy encoding is applied, symbols are represented such that a smaller number of bits is allocated to symbols having a high generation probability, and a larger number of bits is allocated to symbols having a low generation probability, and thus, the size of a bitstream of the symbols to be encoded can be reduced. The entropy encoding unit 150 can use an encoding method for entropy encoding such as exponential Golomb, context adaptive variable length coding (CAVLC), context adaptive binary arithmetic coding (CABAC), or the like. For example, the entropy encoding unit 150 can perform entropy encoding by using a variable length coding / code (VLC) table. Also, the entropy encoding unit 150 can derive a binarization method of a target symbol and a probability model of the target symbol / binary bit, and perform arithmetic encoding by using the derived binarization method and context model.

[0121] In order to encode the transform coefficient levels (quantized levels), the entropy encoding unit 150 can change the coefficients in a two-dimensional block form to a one-dimensional vector form by using a transform coefficient scanning method.

[0122] The coding parameters can include information such as syntax elements (flags, indices, etc.) that are coded in the encoder and signaled to the decoder, and information derived when performing encoding or decoding. The coding parameters can represent information needed when encoding or decoding an image. For example, at least one value or combination of the following can be included in the coding parameters: unit / block size, unit / block depth, unit / block partition information, unit / block shape, unit / block partition structure, whether or not partitioning in a quad-tree form is performed, whether or not partitioning in a binary tree form is performed, partitioning direction in a binary tree form (horizontal direction or vertical direction), partitioning form in a binary tree form (symmetric partitioning or asymmetric partitioning), whether or not the current coding unit is partitioned by triple-tree partitioning, triple-tree partitioning direction (horizontal direction or vertical direction), triple-tree partitioning type (symmetric type or asymmetric type), whether or not the current coding unit is partitioned by multi-type tree partitioning, multi-type tree partitioning direction (horizontal direction or vertical direction), multi-type tree partitioning type (symmetric type or asymmetric type), multi-type tree partitioning tree (binary tree or triple tree) structure, prediction mode (intra prediction or inter prediction), luma intra prediction mode / direction, chroma intra prediction mode / direction, intra partition information, inter partition information, coding block partition flag, prediction block partition flag, transform block partition flag, reference sample filtering method, reference sample filter tap, reference sample filter coefficient, prediction block filtering method, prediction block filter tap, prediction block filter coefficient, prediction block boundary filtering method, prediction block boundary filter tap, prediction block boundary filter coefficient, intra prediction mode, inter prediction mode, motion information, motion vector, motion vector difference, reference picture index, inter prediction angle, inter prediction indicator, prediction list utilization flag, reference picture list, reference picture, motion vector predictor index, motion vector predictor candidate, motion vector candidate list, whether or not to use merge mode, merge index, merge candidate, merge candidate list, whether or not to use skip mode, interpolation filter type, interpolation filter tap, interpolation filter coefficient, motion vector size, representation precision of motion vector, transform type, transform size, information whether or not first (primary) transform is used, information whether or not secondary transform is used, first transform index, secondary transform index, information whether or not residual signal exists, coding block pattern, coding block flag (CBF), quantization parameter, quantization parameter residual, quantization matrix, whether or not to apply intra loop filter, intra loop filter coefficient, intra loop filter tap, intra loop filter shape / form, whether or not to apply deblocking filter, deblocking filter coefficient, deblocking filter tap, deblocking filter strength, deblocking filter shape / form, whether or not to apply adaptive sample offset, adaptive sample offset value, adaptive sample offset class, adaptive sample offset type, whether or not to apply adaptive loop filter, adaptive loop filter coefficient, adaptive loop filter tap, adaptive loop filter shape / form,binarization / de-binarization method, context model determination method, context model update method, whether to perform normal mode, whether to perform bypass mode, context bin, bypass bin, significant coefficient flag, last significant coefficient flag, coding flag for a unit of a coefficient group, position of last significant coefficient, flag as to whether a value of a coefficient is greater than 1, flag as to whether a value of a coefficient is greater than 2, flag as to whether a value of a coefficient is greater than 3, information as to remaining coefficient values, sign information, reconstructed luma sample, reconstructed chroma sample, residual luma sample, residual chroma sample, luma transform coefficient, chroma transform coefficient, quantized luma level, quantized chroma level, transform coefficient level scanning method, size of a motion vector search region at a decoder side, shape of a motion vector search region at a decoder side, number of times of motion vector search at a decoder side, information as to CTU size, information as to minimum block size, information as to maximum block size, information as to maximum block depth, information as to minimum block depth, image display / output order, slice identification information, slice type, slice partition information, parallel block identification information, parallel block type, parallel block partition information, parallel block group identification information, parallel block group type, parallel block group partition information, picture type, bit depth of an input sample, bit depth of a reconstructed sample, bit depth of a residual sample, bit depth of a transform coefficient, bit depth of a quantized level, and information as to a luma signal or information as to a chroma signal.

[0123] Here, signaling a flag or an index can mean that the corresponding flag or index is entropy-encoded by an encoder and included in a bitstream, and can mean that the corresponding flag or index is entropy-decoded from the bitstream by a decoder.

[0124] When the encoding apparatus 100 performs encoding through inter prediction, the encoded current picture can be used as a reference picture for another picture which is processed later. Accordingly, the encoding apparatus 100 can reconstruct or decode the encoded current picture, or store the reconstructed or decoded picture in the reference picture buffer 190 as a reference picture.

[0125] The quantized level can be dequantized in the dequantization unit 160, or can be inverse-transformed in the inverse transform unit 170. The coefficient which is dequantized or inverse-transformed or both can be added to the prediction block by the adder 175. By adding the coefficient which is dequantized or inverse-transformed or both to the prediction block, a reconstructed block can be generated. Here, the coefficient which is dequantized or inverse-transformed or both can mean a coefficient for which at least one of dequantization and inverse transformation is performed, and can mean a reconstructed residual block.

[0126] The reconstructed block can pass through a filter unit 180. The filter unit 180 can apply at least one of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed sample, the reconstructed block, or the reconstructed picture. The filter unit 180 can be referred to as an in-loop filter.

[0127] The deblocking filter can remove blocking distortion generated in a boundary between blocks. In order to determine whether to apply the deblocking filter, it can be determined whether to apply the deblocking filter to the current block based on samples included in a number of rows or columns included in the block. When the deblocking filter is applied to the block, another filter can be applied according to a required deblocking filter strength.

[0128] In order to compensate for encoding errors, a suitable offset value can be added to a sample value by using a sample adaptive offset. The sample adaptive offset can correct an offset of a deblocked picture from an original picture in units of samples. A method of applying an offset considering edge information about each sample can be used, or a method of partitioning samples of a picture into a predetermined number of regions, determining a region to which an offset is applied, and applying the offset to the determined region can be used.

[0129] The adaptive loop filter can perform filtering based on a comparison result of a filtered reconstructed picture and an original picture. Samples included in the picture can be partitioned into a predetermined group, a filter to be applied to each group can be determined, and differential filtering can be performed on each group. Information on whether to apply the ALF can be signaled through a coding unit (CU), and a form and coefficients of the ALF to be applied to each block can vary.

[0130] The reconstructed block or the reconstructed picture that has passed through the filter unit 180 can be stored in a reference picture buffer 190. The reconstructed block processed by the filter unit 180 can be a part of a reference picture. That is, the reference picture is a reconstructed picture composed of the reconstructed blocks processed by the filter unit 180. The stored reference picture can be used later in inter prediction or motion compensation.

[0131] Figure 2 FIG. 1 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment and to which the present application is applied.

[0132] The decoding apparatus 200 can be a decoder, a video decoding apparatus, or a picture decoding apparatus.

[0133] Referring to Figure 2 The decoding apparatus 200 can include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an intra prediction unit 240, a motion compensation unit 250, an adder 255, a filter unit 260, and a reference picture buffer 270.

[0134] The decoding apparatus 200 can receive a bitstream output from the encoding apparatus 100. The decoding apparatus 200 can receive a bitstream stored in a computer-readable recording medium, or can receive a bitstream being streamed through a wired / wireless transmission medium. The decoding apparatus 200 can decode the bitstream by using an intra mode or an inter mode. Furthermore, the decoding apparatus 200 can generate a reconstructed image or a decoded image produced by decoding, and output the reconstructed image or the decoded image.

[0135] When the prediction mode used at the time of decoding is the intra mode, the switch can be switched to the intra. Alternatively, when the prediction mode used at the time of decoding is the inter mode, the switch can be switched to the inter mode.

[0136] The decoding apparatus 200 can obtain a reconstructed residual block by decoding an input bitstream, and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 can generate a reconstructed block that is a decoding target by adding the reconstructed residual block to the prediction block. The decoding target block can be referred to as a current block.

[0137] The entropy decoding unit 210 can generate a symbol by entropy-decoding a bitstream according to a probability distribution. The generated symbol can include a quantized level form of a symbol. Here, the entropy-decoding method can be an inverse process of the above-described entropy-encoding method.

[0138] In order to decode a transform coefficient level (quantized level), the entropy decoding unit 210 can change a coefficient in a one-dimensional vector form to a two-dimensional block form by using a transform coefficient scanning method.

[0139] The quantized level can be inverse-quantized in the inverse quantization unit 220, or can be inverse-transformed in the inverse transform unit 230. The quantized level can be a result of inverse-quantization or inverse-transformation, or both inverse-quantization and inverse-transformation, and can be generated as a reconstructed residual block. Here, the inverse quantization unit 220 can apply a quantization matrix to the quantized level.

[0140] When the intra mode is used, the intra prediction unit 240 can generate a prediction block by performing spatial prediction on the current block, in which the spatial prediction uses sample values of blocks neighboring the decoding target block and already decoded.

[0141] When the inter mode is used, the motion compensation unit 250 can generate a prediction block by performing motion compensation on the current block, in which the motion compensation uses a motion vector and a reference image stored in the reference picture buffer 270.

[0142] The adder 225 can generate a reconstructed block by adding the reconstructed residual block to the prediction block. The filter unit 260 can apply at least one of a deblocking filter, a sample adaptive offset, and an adaptive loop filter to the reconstructed block or the reconstructed picture. The filter unit 260 can output the reconstructed picture. The reconstructed block or the reconstructed picture can be stored in the reference picture buffer 270 and used when performing inter prediction. The reconstructed block processed by the filter unit 260 can be a part of a reference picture. That is, the reference picture is a reconstructed picture composed of the reconstructed blocks processed by the filter unit 260. The stored reference picture can be used later in inter prediction or motion compensation.

[0143] Figure 3 FIG. 1 is a diagram schematically illustrating a partition structure of an image when encoding and decoding the image. Figure 3 FIG. 2 schematically illustrates an example of partitioning a single unit into a plurality of lower-level units.

[0144] To effectively partition an image, a coding unit (CU) can be used when encoding and decoding. The coding unit can be used as a basic unit when encoding / decoding an image. Also, the coding unit can be used as a unit for distinguishing an intra prediction mode from an inter prediction mode when encoding / decoding an image. The coding unit can be a basic unit for prediction, transform, quantization, inverse transform, dequantization, or encoding / decoding processing of transform coefficients.

[0145] Referring to Figure 3 , the image 300 is sequentially partitioned in a maximum coding unit (LCU) and the LCU unit is determined as a partition structure. Here, the LCU can be used in the same meaning as a coding tree unit (CTU). The unit partitioning can denote partitioning of a block associated with the unit. In the block partitioning information, information of a unit depth can be included. The depth information can denote either or both of a number or degree of partitioning of a unit or a number and degree of partitioning of a unit. A single unit can be partitioned into a plurality of lower-level units hierarchically associated with the depth information based on a tree structure. In other words, a unit and lower-level units generated by partitioning the unit can correspond to a node and child nodes of the node, respectively. Each of the partitioned lower-level units can have the depth information. The depth information can be information denoting a size of a CU and can be stored in each CU. The unit depth denotes a number and / or degree related to partitioning of a unit. Accordingly, the partitioning information of the lower-level units can include information on sizes of the lower-level units.

[0146] The partition structure can represent a distribution of coding units (CUs) within the LCU 310. The distribution can be determined according to whether a single CU is partitioned into a plurality of (a positive integer equal to or greater than 2, including 2, 4, 8, 16, etc.) CUs. The horizontal and vertical sizes of the CUs generated by the partitioning can be half of the horizontal and vertical sizes of the CU before the partitioning, respectively, or can have sizes smaller than the horizontal and vertical sizes before the partitioning according to the number of times of partitioning. The CU can be recursively partitioned into a plurality of CUs. At least one of the height and width of the CU after the partitioning can be reduced compared to at least one of the height and width of the CU before the partitioning by the recursive partitioning. The partitioning of the CU can be recursively performed until a predefined depth or a predefined size. For example, the depth of the LCU can be 0, and the depth of a smallest coding unit (SCU) can be a predefined maximum depth. Here, as described above, the LCU can be a coding unit having a maximum coding unit size, and the SCU can be a coding unit having a minimum coding unit size. The partitioning starts from the LCU 310, and the CU depth is increased by 1 when the horizontal size or the vertical size, or both, of the CU is reduced by the partitioning. For example, the size of the CU that is not partitioned can be 2Nx2N for each depth. Also, in the case of the CU that is partitioned, the CU having a size of 2Nx2N can be partitioned into four CUs having a size of NxN. As the depth is increased by 1, the size of N can be halved.

[0147] Also, information on whether the CU is partitioned can be represented by using partitioning information of the CU. The partitioning information can be 1-bit information. All CUs except for the SCU can include the partitioning information. For example, when the value of the partitioning information is a first value, the CU can not be partitioned, and when the value of the partitioning information is a second value, the CU can be partitioned.

[0148] Referring to Figure 3 The LCU having a depth of 0 can be a 64x64 block. 0 can be a minimum depth. The SCU having a depth of 3 can be an 8x8 block. 3 can be a maximum depth. The CUs of the 32x32 block and the 16x16 block can be represented as depths 1 and 2, respectively.

[0149] For example, when a single coding unit is partitioned into four coding units, the horizontal and vertical sizes of the partitioned four coding units can be half the size of the horizontal and vertical sizes of the CU before being partitioned. In one embodiment, when a coding unit having a size of 32x32 is partitioned into four coding units, each of the partitioned four coding units can have a size of 16x16. When a single coding unit is partitioned into four coding units, the coding unit can be said to be partitioned in a quad-tree form.

[0150] For example, when one coding unit is partitioned into two sub-coding units, each of the two sub-coding units can have a horizontal size or a vertical size (width or height) that is half of a horizontal size or a vertical size of the original coding unit. For example, when a coding unit having a size of 32x32 is vertically partitioned into two sub-coding units, each of the two sub-coding units can have a size of 16x32. For example, when a coding unit having a size of 8x32 is horizontally partitioned into two sub-coding units, each of the two sub-coding units can have a size of 8x16. When one coding unit is partitioned into two sub-coding units, the coding unit can be referred to as being bipartitioned or partitioned by a binary tree partitioning structure.

[0151] For example, when one coding unit is partitioned into three sub-coding units, a horizontal size or a vertical size of the coding unit can be partitioned in a ratio of 1:2:1, thereby resulting in three sub-coding units having a ratio of 1:2:1 in horizontal size or vertical size. For example, when a coding unit having a size of 16x32 is horizontally partitioned into three sub-coding units, the three sub-coding units can have sizes of 16x8, 16x16, and 16x8, respectively, in order from the topmost sub-coding unit to the bottommost sub-coding unit. For example, when a coding unit having a size of 32x32 is vertically partitioned into three sub-coding units, the three sub-coding units can have sizes of 8x32, 16x32, and 8x32, respectively, in order from the leftmost sub-coding unit to the rightmost sub-coding unit. When one coding unit is partitioned into three sub-coding units, the coding unit can be referred to as being tripartitioned or partitioned according to a ternary tree partitioning structure.

[0152] In Figure 3 The coding tree unit (CTU) 320 is an example of a CTU to which all of the quadtree partitioning structure, the binary tree partitioning structure, and the ternary tree partitioning structure are applied.

[0153] As described above, in order to partition a CTU, at least one of the quadtree partitioning structure, the binary tree partitioning structure, and the ternary tree partitioning structure can be applied. The various tree partitioning structures can be applied to the CTU sequentially according to a predetermined priority order. For example, the quadtree partitioning structure can be applied to the CTU first. A coding unit that cannot be partitioned any more using the quadtree partitioning structure can correspond to a leaf node of the quadtree. The coding unit corresponding to the leaf node of the quadtree can be used as a root node of the binary tree and / or the ternary tree partitioning structure. That is, the coding unit corresponding to the leaf node of the quadtree can be further partitioned according to the binary tree partitioning structure or the ternary tree partitioning structure, or can not be further partitioned. Accordingly, by preventing coding blocks resulting from binary tree partitioning or ternary tree partitioning of a coding unit corresponding to a leaf node of the quadtree from undergoing further quadtree partitioning, the block partitioning operation and / or the operation of signaling partitioning information can be efficiently performed.

[0154] The fact that a coding unit corresponding to a node of a quadtree is partitioned can be signaled using quad-partition information. The quad-partition information having a first value (e.g., "1") can indicate that the current coding unit is partitioned according to a quadtree partition structure. The quad-partition information having a second value (e.g., "0") can indicate that the current coding unit is not partitioned according to a quadtree partition structure. The quad-partition information can be a flag having a predetermined length (e.g., one bit).

[0155] There can be no priority between binary tree partitioning and ternary tree partitioning. That is, a coding unit corresponding to a leaf node of a quadtree can be further subjected to any of binary tree partitioning and ternary tree partitioning. Further, a coding unit resulting from binary tree partitioning or ternary tree partitioning can be subjected to further binary tree partitioning or further ternary tree partitioning, or can not be further partitioned.

[0156] A tree structure in which there is no priority between binary tree partitioning and ternary tree partitioning is referred to as a multi-type tree structure. A coding unit corresponding to a leaf node of a quadtree can serve as a root node of a multi-type tree. Whether a coding unit corresponding to a node of a multi-type tree is partitioned can be signaled using at least one of multi-type tree partitioning indication information, partition direction information, and partition tree information. In order to partition a coding unit corresponding to a node of a multi-type tree, the multi-type tree partitioning indication information, the partition direction information, and the partition tree information can be sequentially signaled.

[0157] The multi-type tree partitioning indication information having a first value (e.g., "1") can indicate that the current coding unit is to be subjected to multi-type tree partitioning. The multi-type tree partitioning indication information having a second value (e.g., "0") can indicate that the current coding unit is not to be subjected to multi-type tree partitioning.

[0158] When a coding unit corresponding to a node of a multi-type tree is further partitioned according to a multi-type tree partition structure, the coding unit can include partition direction information. The partition direction information can indicate in which direction the current coding unit is to be partitioned for multi-type tree partitioning. The partition direction information having a first value (e.g., "1") can indicate that the current coding unit is to be vertically partitioned. The partition direction information having a second value (e.g., "0") can indicate that the current coding unit is to be horizontally partitioned.

[0159] When a coding unit corresponding to a node of a multi-type tree is further partitioned according to a multi-type tree partition structure, the current coding unit can include partition tree information. The partition tree information can indicate a tree partition structure to be used for partitioning a node of a multi-type tree. The partition tree information having a first value (e.g., "1") can indicate that the current coding unit is to be partitioned according to a binary tree partition structure. The partition tree information having a second value (e.g., "0") can indicate that the current coding unit is to be partitioned according to a ternary tree partition structure.

[0160] The partition indication information, the partition tree information, and the partition direction information can each be a flag having a predetermined length (e.g., one bit).

[0161] At least any one of the quad-tree partition indication information, the multi-type tree partition indication information, the partition direction information, and the partition tree information can be entropy coded / entropy decoded. In order to entropy code / entropy decode those types of information, information about neighboring coding units adjacent to the current coding unit can be used. For example, it is highly likely that the partition type (partitioned or not partitioned, partition tree, and / or partition direction) of a left neighboring coding unit and / or an above neighboring coding unit of the current coding unit is similar to the partition type of the current coding unit. Thus, context information for entropy coding / entropy decoding information about the current coding unit can be derived from information about the neighboring coding units. The information about the neighboring coding units can include at least any one of the quad-tree partition information, the multi-type tree partition indication information, the partition direction information, and the partition tree information.

[0162] As another example, in the binary tree partition and the ternary tree partition, the binary tree partition can be preferentially performed. That is, the current coding unit can first undergo the binary tree partition, and then the coding unit corresponding to a leaf node of the binary tree can be set as a root node for the ternary tree partition. In this case, for the coding unit corresponding to a node of the ternary tree, neither the quad-tree partition nor the binary tree partition can be performed.

[0163] A coding unit that cannot be partitioned according to the quad-tree partition structure, the binary tree partition structure, and / or the ternary tree partition structure becomes a basic unit for encoding, prediction, and / or transform. That is, the coding unit cannot be further partitioned for prediction and / or transform. Thus, there can be no partition structure information and partition information for partitioning the coding unit into prediction units and / or transform units in the bitstream.

[0164] However, when the size of a coding unit (i.e., a basic unit for partitioning) is larger than the size of the maximum transform block, the coding unit can be recursively partitioned until the size of the coding unit is reduced to be equal to or smaller than the size of the maximum transform block. For example, when the size of the coding unit is 64x64 and when the size of the maximum transform block is 32x32, the coding unit can be partitioned into four 32x32 blocks for transform. For example, when the size of the coding unit is 32x64 and the size of the maximum transform block is 32x32, the coding unit can be partitioned into two 32x32 blocks for transform. In this case, the partitioning of the coding unit for transform is not signaled separately and can be determined by a comparison between the horizontal size or the vertical size of the coding unit and the horizontal size or the vertical size of the maximum transform block. For example, when the horizontal size (width) of the coding unit is larger than the horizontal size (width) of the maximum transform block, the coding unit can be vertically bisected. For example, when the vertical size (length) of the coding unit is larger than the vertical size (length) of the maximum transform block, the coding unit can be horizontally bisected.

[0165] Information of the maximum and / or minimum size of the coding unit and information of the maximum and / or minimum size of the transform block can be signaled or determined at a higher level of the coding unit. The higher level can be, for example, a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. For example, the minimum size of the coding unit can be determined to be 4x4. For example, the maximum size of the transform block can be determined to be 64x64. For example, the minimum size of the transform block can be determined to be 4x4.

[0166] Information of the minimum size of the coding unit corresponding to a leaf node of a quadtree (quadtree minimum size) and / or information of the maximum depth of a multi-type tree from a root node to a leaf node (maximum tree depth of the multi-type tree) can be signaled or determined at a higher level of the coding unit. For example, the higher level can be a sequence level, a picture level, a slice level, a tile group level, a tile level, etc. The information of the minimum size of the quadtree and / or the information of the maximum depth of the multi-type tree can be signaled or determined for each of an intra slice and an inter slice.

[0167] The difference information between the size of CTU and the maximum size of transform block can be signaled or determined at a higher level of coding unit. For example, the higher level can be sequence level, picture level, slice level, parallel block group level, parallel block level, etc. The information of the maximum size of coding unit corresponding to each node of binary tree (hereinafter referred to as the maximum size of binary tree) can be determined based on the size of coding tree unit and the difference information. The maximum size of coding unit corresponding to each node of ternary tree (hereinafter referred to as the maximum size of ternary tree) can vary depending on the type of slice. For example, for an intra slice, the maximum size of ternary tree can be 32x32. For example, for an inter slice, the maximum size of ternary tree can be 128x128. For example, the minimum size of coding unit corresponding to each node of binary tree (hereinafter referred to as the minimum size of binary tree) and / or the minimum size of coding unit corresponding to each node of ternary tree (hereinafter referred to as the minimum size of ternary tree) can be set to the minimum size of coding block.

[0168] As another example, the maximum size of binary tree and / or the maximum size of ternary tree can be signaled or determined at slice level. Alternatively, the minimum size of binary tree and / or the minimum size of ternary tree can be signaled or determined at slice level.

[0169] According to the size information and depth information of the various blocks described above, the quad partition information, the multi-type tree partition indication information, the partition tree information, and / or the partition direction information can or can not be included in the bitstream.

[0170] For example, when the size of coding unit is not greater than the minimum size of quad tree, the coding unit does not contain the quad partition information. Therefore, the quad partition information can be derived from the second value.

[0171] For example, when the size (horizontal size and vertical size) of coding unit corresponding to a node of multi-type tree is greater than the maximum size (horizontal size and vertical size) of binary tree and / or the maximum size (horizontal size and vertical size) of ternary tree, the coding unit can not be partitioned into two or three. Therefore, the multi-type tree partition indication information can not be signaled, but can be derived from the second value.

[0172] Optionally, the coding unit corresponding to a node of the multi-type tree can not be further bi-partitioned or tri-partitioned when the size (horizontal size and vertical size) of the coding unit is the same as the maximum size (horizontal size and vertical size) of the binary tree and / or twice as large as the maximum size (horizontal size and vertical size) of the ternary tree. Thus, the multi-type tree partitioning indication information can not be signaled, but can be derived from the second value. This is because, when the coding unit is partitioned according to the binary tree partitioning structure and / or the ternary tree partitioning structure, a coding unit smaller than the minimum size of the binary tree and / or the minimum size of the ternary tree is generated.

[0173] Optionally, the coding unit corresponding to a node of the multi-type tree can not be further bi-partitioned and / or tri-partitioned when the depth of the coding unit is equal to the maximum depth of the multi-type tree. Thus, the multi-type tree partitioning indication information can not be signaled, but can be derived from the second value.

[0174] Optionally, the multi-type tree partitioning indication information can only be signaled when at least one of the vertical binary tree partitioning, the horizontal binary tree partitioning, the vertical ternary tree partitioning and the horizontal ternary tree partitioning is feasible for the coding unit corresponding to a node of the multi-type tree. Otherwise, the coding unit can not be bi-partitioned and / or tri-partitioned. Thus, the multi-type tree partitioning indication information can not be signaled, but can be derived from the second value.

[0175] Optionally, the partitioning direction information can only be signaled when both the vertical binary tree partitioning and the horizontal binary tree partitioning or both the vertical ternary tree partitioning and the horizontal ternary tree partitioning are feasible for the coding unit corresponding to a node of the multi-type tree. Otherwise, the partitioning direction information can not be signaled, but can be derived from a value indicating the possible partitioning directions.

[0176] Optionally, the partitioning tree information can only be signaled when both the vertical binary tree partitioning and the vertical ternary tree partitioning or both the horizontal binary tree partitioning and the horizontal ternary tree partitioning are feasible for the coding tree corresponding to a node of the multi-type tree. Otherwise, the partitioning tree information can not be signaled, but can be derived from a value indicating the possible partitioning tree structures.

[0177] Figure 4 Fig. 1 is a diagram illustrating an intra prediction process.

[0178] Figure 4 The arrows from the center to the outside can represent the prediction directions of the intra prediction modes.

[0179] Intra coding and / or decoding can be performed by using reference samples of neighboring blocks of a current block. The neighboring blocks can be reconstructed neighboring blocks. For example, the intra coding and / or decoding can be performed by using coding parameters or values of the reference samples included in the reconstructed neighboring blocks.

[0180] A prediction block can represent a block generated by performing intra prediction. The prediction block can correspond to at least one of a CU, a PU, and a TU. A unit of the prediction block can have a size of one of the CU, the PU, and the TU. The prediction block can be a square block having a size of 2x2, 4x4, 16x16, 32x32, or 64x64, etc., or can be a rectangular block having a size of 2x8, 4x8, 2x16, 4x16, and 8x16, etc.

[0181] Intra prediction can be performed according to an intra prediction mode for a current block. The number of the intra prediction modes that the current block can have can be a fixed value, and can be a value determined differently according to properties of the prediction block. For example, the properties of the prediction block can include a size of the prediction block and a shape of the prediction block, etc.

[0182] The number of the intra prediction modes can be fixed to N regardless of a block size. Alternatively, the number of the intra prediction modes can be 3, 5, 9, 17, 34, 35, 36, 65, or 67, etc. Alternatively, the number of the intra prediction modes can vary according to a block size or a color component type or both the block size and the color component type. For example, the number of the intra prediction modes can vary according to whether a color component is a luma signal or a chroma signal. For example, as the block size becomes larger, the number of the intra prediction modes can increase. Alternatively, the number of the intra prediction modes for a luma component block can be greater than the number of the intra prediction modes for a chroma component block.

[0183] The intra prediction mode can be a non-angular mode or an angular mode. The non-angular mode can be a DC mode or a planar mode, and the angular mode can be a prediction mode having a specific direction or angle. The intra prediction mode can be represented by at least one of a mode number, a mode value, a mode number, a mode angle, and a mode direction. The number of the intra prediction modes can be M which is greater than 1, including the non-angular mode and the angular mode.

[0184] In order to perform intra prediction on a current block, a step of determining whether or not a sample included in a reconstructed neighboring block can be used as a reference sample of the current block can be performed. When there is a sample that cannot be used as the reference sample of the current block, a value obtained by copying or performing interpolation on at least one of the sample values of the samples included in the reconstructed neighboring block, or both copying and interpolation, can be used to replace an unavailable sample value of the sample, and thus the replaced sample value is used as the reference sample of the current block.

[0185] When intra prediction is performed, a filter can be applied to at least one of reference samples and prediction samples based on an intra prediction mode and a current block size.

[0186] In the case of the planar mode, when a prediction block of the current block is generated, depending on a position of a prediction target sample within the prediction block, a sample value of the prediction target sample can be generated by using a weighted sum of an upper reference sample and a left reference sample of the current sample and an upper right reference sample and a lower left reference sample of the current block. Also, in the case of the DC mode, when the prediction block of the current block is generated, an average of the upper reference sample and the left reference sample of the current block can be used. Also, in the case of the angular mode, the prediction block can be generated by using the upper reference sample, the left reference sample, the upper right reference sample, and / or the lower left reference sample of the current block. To generate the prediction sample value, interpolation of real units can be performed.

[0187] The intra prediction mode of the current block can be entropy encoded / decoded by predicting the intra prediction mode of a block that exists adjacent to the current block. When the intra prediction mode of the current block is the same as that of the neighboring block, information that the intra prediction mode of the current block is the same as that of the neighboring block can be signaled by using predetermined flag information. Also, indicator information of the intra prediction mode that is the same as the intra prediction mode of the current block among the intra prediction modes of the plurality of neighboring blocks can be signaled. When the intra prediction mode of the current block is different from that of the neighboring block, the intra prediction mode information of the current block can be entropy encoded / decoded by performing entropy encoding / decoding based on the intra prediction mode of the neighboring block.

[0188] Figure 5 is a diagram illustrating an embodiment of an inter prediction process.

[0189] In Figure 5 , a rectangle can represent a picture. In Figure 5 , an arrow indicates a prediction direction. Depending on the encoding type of a picture, the picture can be classified as an intra picture (I picture), a predicted picture (P picture), and a bi-predicted picture (B picture).

[0190] An I picture can be encoded by intra prediction without inter prediction. A P picture can be encoded by inter prediction by using a reference picture that exists in one direction (i.e., forward or backward) with respect to a current block. A B picture can be encoded by inter prediction by using reference pictures that exist in two directions (i.e., forward and backward) with respect to a current block. When inter prediction is used, an encoder can perform inter prediction or motion compensation, and a decoder can perform corresponding motion compensation.

[0191] Hereinafter, an embodiment of inter prediction will be described in detail.

[0192] The inter-picture prediction or motion compensation can be performed using the reference picture and the motion information.

[0193] The motion information of the current block can be derived during inter-picture prediction by each of the encoding apparatus 100 and the decoding apparatus 200. The motion information of the current block can be derived by using the motion information of the reconstructed neighboring block, the motion information of a collocated block (also referred to as a col block or a collocated block), and / or the motion information of a block neighboring the collocated block. The collocated block can denote a block spatially collocated within a previously reconstructed collocated picture (also referred to as a col picture or a collocated picture) with the current block. The collocated picture can be one of one or more reference pictures included in a reference picture list.

[0194] The method of deriving the motion information of the current block can vary according to a prediction mode of the current block. For example, as the prediction mode for inter-picture prediction, there can be an AMVP mode, a merge mode, a skip mode, a current picture reference mode, etc. The merge mode can be referred to as a motion merge mode.

[0195] For example, when the AMVP is used as the prediction mode, at least one of the motion vector of the reconstructed neighboring block, the motion vector of the collocated block, the motion vector of the block neighboring the collocated block, and a (0, 0) motion vector can be determined as a motion vector candidate for the current block, and a motion vector candidate list is generated by using the motion vector candidate. The motion vector candidate of the current block can be derived by using the generated motion vector candidate list. The motion information of the current block can be determined based on the derived motion vector candidate. The motion vector of the collocated block or the motion vector of the block neighboring the collocated block can be referred to as a temporal motion vector candidate, and the motion vector of the reconstructed neighboring block can be referred to as a spatial motion vector candidate.

[0196] The encoding apparatus 100 can calculate a motion vector difference (MVD) between the motion vector of the current block and the motion vector candidate, and can perform entropy encoding on the motion vector difference (MVD). In addition, the encoding apparatus 100 can perform entropy encoding on a motion vector candidate index and generate a bitstream. The motion vector candidate index can indicate a best motion vector candidate among the motion vector candidates included in the motion vector candidate list. The decoding apparatus can perform entropy decoding on the motion vector candidate index included in the bitstream, and can select a motion vector candidate of a decoding target block from the motion vector candidates included in the motion vector candidate list by using the entropy-decoded motion vector candidate index. In addition, the decoding apparatus 200 can add the entropy-decoded MVD to the motion vector candidate extracted through the entropy decoding, thereby deriving the motion vector of the decoding target block.

[0197] The bitstream can include a reference picture index indicating a reference picture. The reference picture index can be entropy encoded by the encoding apparatus 100 and then signaled to the decoding apparatus 200 as the bitstream. The decoding apparatus 200 can generate a prediction block of a decoded target block based on the derived motion vector and the reference picture index information.

[0198] Another example of a method of deriving motion information of a current block can be a merge mode. The merge mode can denote a method of merging motion of a plurality of blocks. The merge mode can denote a mode of deriving motion information of a current block from motion information of neighboring blocks. When the merge mode is applied, a merge candidate list can be generated using motion information of reconstructed neighboring blocks and / or motion information of collocated blocks. The motion information can include at least one of a motion vector, a reference picture index, and an inter prediction indicator. The prediction indicator can indicate a uni-prediction (L0 prediction or L1 prediction) or bi-prediction (L0 prediction and L1 prediction).

[0199] The merge candidate list can be a list of stored motion information. The motion information included in the merge candidate list can be at least one of a zero merge candidate and new motion information, wherein the new motion information is motion information of one neighboring block adjacent to the current block (spatial merge candidate), motion information of a collocated block of the current block included in a reference picture (temporal merge candidate), and a combination of motion information existing in the merge candidate list.

[0200] The encoding apparatus 100 can generate a bitstream by performing entropy encoding on at least one of a merge flag and a merge index, and can signal the bitstream to the decoding apparatus 200. The merge flag can be information indicating whether the merge mode is performed for each block, and the merge index can be information indicating which of the neighboring blocks of the current block is a merge target block. For example, the neighboring blocks of the current block can include a left neighboring block located at the left side of the current block, an above neighboring block arranged above the current block, and a temporal neighboring block adjacent in time to the current block.

[0201] The skip mode can be a mode of applying motion information of a neighboring block as it is to a current block. When the skip mode is applied, the encoding apparatus 100 can perform entropy encoding on information of a fact that which block's motion information is to be used as motion information of the current block to generate a bitstream, and can signal the bitstream to the decoding apparatus 200. The encoding apparatus 100 can not signal syntax elements regarding at least any one of motion vector difference information, a coded block flag, and transform coefficient levels to the decoding apparatus 200.

[0202] A current picture reference mode can represent a prediction mode in which a previously reconstructed region within a current picture to which the current block belongs is used for prediction. Here, a vector can be used to specify the previously reconstructed region. Information indicating whether the current block is to be encoded in the current picture reference mode can be encoded by using a reference picture index of the current block. A flag or index indicating whether the current block is a block encoded in the current picture reference mode can be signaled, and the flag or index can be derived based on the reference picture index of the current block. In a case that the current block is encoded in the current picture reference mode, the current picture can be added to a reference picture list for the current block so as to be located at a fixed position or an arbitrary position in the reference picture list. The fixed position can be, for example, a position indicated by a reference picture index 0, or a last position in the list. When the current picture is added to the reference picture list so as to be located at the arbitrary position, a reference picture index indicating the arbitrary position can be signaled.

[0203] Figure 6 is a diagram illustrating a transform and quantization process.

[0204] As shown in Figure 6 A transform process and / or a quantization process are performed on a residual signal to generate a quantized level signal. The residual signal is a difference between an original block and a prediction block (i.e., an intra-predicted block or an inter-predicted block). The prediction block is a block generated by intra-prediction or inter-prediction. The transform can be a primary transform, a secondary transform, or both the primary transform and the secondary transform. The primary transform of the residual signal generates transform coefficients, and the secondary transform of the transform coefficients generates secondary transform coefficients.

[0205] At least one scheme selected from various pre-defined transform schemes is used to perform the primary transform. For example, examples of the pre-defined transform schemes include a discrete cosine transform (DCT), a discrete sine transform (DST), and a Karhunen-Loève transform (KLT). The transform coefficients generated by the primary transform can undergo the secondary transform. The transform scheme used for the primary transform and / or the secondary transform can be determined according to an encoding parameter of the current block and / or a neighboring block of the current block. Alternatively, the transform scheme can be determined through signaling of transform information.

[0206] Since the residual signal is quantized by the first transform and the second transform, a quantized level signal (quantized coefficient) is generated. Depending on the intra prediction mode or the block size / shape of the block, the quantized level signal can be scanned according to at least one of diagonal up-right scan, vertical scan, and horizontal scan. For example, when the coefficients are scanned according to the diagonal up-right scan, the coefficients in the form of a block become in the form of a one-dimensional vector. In addition to the diagonal up-right scan, horizontal scan or vertical scan can be used to scan the coefficients in the form of a two-dimensional block horizontally or vertically depending on the intra prediction mode and / or size of the transform block. The scanned quantized level coefficients can be entropy coded to be inserted into a bitstream.

[0207] The decoder entropy decodes the bitstream to obtain the quantized level coefficients. The quantized level coefficients can be arranged in the form of a two-dimensional block by inverse scanning. For the inverse scanning, at least one of diagonal up-right scan, vertical scan, and horizontal scan can be used.

[0208] Then, the quantized level coefficients can be dequantized, then inversely transformed by the second inverse transform as necessary, and finally inversely transformed by the first inverse transform as necessary, to generate a reconstructed residual signal.

[0209] Hereinafter, a reference will be made to Figures 7 to 10 An image encoding / decoding method using a sample filter according to an embodiment of the present application will be described.

[0210] In the encoder / decoder, the sample filter can include determining a filter target sample, determining whether to apply a filter, determining a filter type, and applying the filter.

[0211] Hereinafter, determining a filter target sample will be described.

[0212] In determining the filter target sample, a filter application target sample can be determined. For example, coding efficiency can be improved by applying a filter to at least one sample. The at least one sample can be at least one of a reference sample, a prediction sample, a reconstructed sample, and a residual sample. Here, the filter can mean performing a weighted sum using the filter target sample and at least one neighboring sample. Alternatively, the filter can be in the form of reflecting an offset value to the filter target sample.

[0213] According to an embodiment of the present application, the filter target sample can be determined as at least one of the following.

[0214] In one example, when intra prediction or inter prediction is performed, the filter target sample can be a prediction sample.

[0215] In another example, the filter target sample can be a reference sample neighboring a current block for prediction.

[0216] In another example, the filter target samples can be reference samples included in a reference picture used for prediction.

[0217] In another example, the filter target samples can be reconstructed samples within the current block or at least one neighboring block.

[0218] In another example, the filter target samples can be samples obtained by applying at least one filter to the reconstructed samples.

[0219] In another example, the filter target samples can be residual samples generated by applying at least one of entropy decoding, inverse quantization, and inverse transform to a bitstream.

[0220] According to an embodiment of the present invention, the filter target samples can be all or partial samples within the current block, and the block can be at least one of a CTU, a CU, a PU, and a TU.

[0221] According to an embodiment of the present invention, the filter target samples can be samples of a specific sub-block within the current block. Here, the specific sub-block can be a pre-defined sub-block.

[0222] According to an embodiment of the present invention, the filter target samples can be intra-predicted samples, and the target samples can be determined based on at least one of an intra-prediction mode of the current block, a size / shape of the current block, and a reference sample line.

[0223] In one example, when the intra-prediction mode is a non-directional mode such as a DC mode or a planar mode, a predetermined prediction sample line adjacent to the reference samples can be determined as the target samples.

[0224] In another example, the target samples can be determined as the prediction sample line according to a direction of the intra-prediction mode. When the intra-prediction mode is a vertical directional mode, a predetermined prediction sample line adjacent to the left reference samples can be determined as the target samples. Also, when the intra-prediction mode is a horizontal directional mode, a predetermined prediction sample line adjacent to the upper reference samples can be determined as the target samples.

[0225] In another example, when the size of the current block is 4x4, a predetermined prediction sample line can be determined as the target samples, and when the size of the block becomes larger, more prediction sample lines can be determined as the target samples.

[0226] In another example, when the current block has a horizontally long shape, a predetermined prediction sample line adjacent to the upper reference samples can be determined as the target samples. Also, when the current block has a vertically long shape, a predetermined prediction sample line adjacent to the left reference samples can be determined as the target samples.

[0227] In another example, when the reference sample line for prediction is a first line adjacent to the current block, the predetermined prediction sample line can be determined as the target sample. In addition, when the reference sample line is a second line or more, the prediction sample can not be determined as the filter target sample.

[0228] Hereinafter, it will be described whether to apply filtering.

[0229] In determining whether to apply filtering, it can be determined whether to apply filtering to the filter target sample. It can be determined whether to apply filtering based on at least one of the coding mode, the intra prediction mode, the inter prediction mode, the transform information, the filter information, the size (horizontal size or vertical size), the shape, and the reference sample line of the current block and the coding mode, the intra prediction mode, the inter prediction mode, the transform information, the filter information, the size (horizontal size or vertical size), the shape, and the reference sample line of the neighboring block. Alternatively, it can be determined whether to apply filtering based on at least one coding parameter. Alternatively, an indicator indicating whether to apply filtering can be signaled through at least one of SPS, PPS, and slice header.

[0230] According to an embodiment of the present application, it can be determined whether to apply filtering based on the coding mode of the current block. For example, when the coding mode is an intra mode, it can be determined to apply filtering to the current block, and when the coding mode is an inter mode, it can be determined not to apply filtering to the current block.

[0231] According to an embodiment of the present application, it can be determined whether to apply filtering based on the intra prediction mode of the current block. For example, when the intra prediction mode of the current block corresponds to a predetermined mode, it can be determined that filtering will be applied. As an example, the predetermined mode can be at least one of a planar mode, a DC mode, a horizontal mode, a vertical mode, a 45 / 135 / 225 angle mode, and an arbitrary angle mode. Figure 4

[0232] In one example, it can be determined whether to apply filtering according to whether the intra prediction mode or the mode number is even or odd. For example, when the intra prediction mode has an odd-numbered value, filtering can be applied, and when the intra prediction mode has an even-numbered value, filtering can not be applied. When determining whether the intra prediction mode is an odd-numbered or even-numbered value, a modulo operation can be used.

[0233] ​In another example, it can be determined whether to apply the filtering according to whether the intra prediction mode corresponds to a predetermined range. For example, when the intra prediction mode is greater than a predetermined mode, it can be determined to apply the filtering. In contrast, when the intra prediction mode is less than the predetermined mode, it can be determined to apply the filtering. Also, for example, when the intra prediction mode is equal to or greater than the predetermined mode, the filtering can be applied. Here, the predetermined mode can be mode 58. Also, for example, when the intra prediction mode is equal to or less than the predetermined mode, it can be determined to apply the filtering. Here, the predetermined mode can be mode 10.

[0234] Also, when performing the intra prediction or the inter prediction, the filtering target sample can be a prediction sample.

[0235] In another example, the filtering can be applied to a mode corresponding to a predetermined range based on the intra prediction mode. For example, the filtering can be applied to a mode increased by N or decreased by N from the intra prediction mode, and N can be an integer equal to or greater than 1. For example, when the intra prediction mode is a vertical mode and N is 2, the filtering can be applied to a mode increased by 2 or decreased by 2 from the vertical mode.

[0236] In another example, it can be determined whether to apply the filtering according to whether the intra prediction mode is a non-directional mode (a planar mode, a DC mode) or a directional mode (a mode other than the planar mode and the DC mode). For example, when the intra prediction mode is the directional mode, the filtering can be applied. Otherwise, when the intra prediction mode is the non-directional mode, the filtering can not be applied.

[0237] According to an embodiment of the present application, it can be determined whether to apply the filtering based on an inter prediction mode of a current block. For example, when the inter prediction mode of the current block corresponds to a predetermined mode, it can be determined to apply the filtering.

[0238] In one example, when the inter prediction mode is an inter prediction mode other than a skip mode, the filtering can be applied.

[0239] In another example, when the inter prediction mode corresponds to a merge mode, the filtering can be applied.

[0240] In another example, when the inter prediction mode is a mode in which prediction or motion compensation is performed using at least two blocks, the filtering can be applied. For example, when the prediction is performed by combining a single merge mode and a single intra prediction mode, the filtering can be applied. Here, the filtering target sample can be a reference sample for performing the intra prediction, or a prediction sample of the prediction.

[0241] Alternatively, in contrast, in order to reduce complexity, when the prediction is performed by combining a single merge mode and a single intra prediction mode, the filtering can not be applied. For example, when the merge mode is combined with the intra prediction mode, the filtering can not be applied to the reference sample for the intra prediction or the prediction sample.

[0242] According to embodiments of the present invention, it can be determined whether to apply filtering based on a size of the current block (a horizontal size or a vertical size of the current block).

[0243] In one example, filtering can be applied when the horizontal size or the vertical size corresponds to a predetermined range. For example, filtering can be applied when the size of the current block is equal to or greater than 8x8 and equal to or less than 32x32. Alternatively, for example, filtering can be applied when the size of the block is equal to or less than 32x32. Here, information corresponding to the predetermined range can be signaled or predefined.

[0244] In another example, filtering can be applied when a statistical value of the horizontal size and the vertical size corresponds to a predetermined range. For example, filtering can be applied when a sum of the horizontal size and the vertical size is equal to or greater than 16 and equal to or less than 64. Here, the statistical value can be any one of a sum, a product, a weighted sum, or an average.

[0245] In another example, filtering can be applied when the size of the current block is less than 64x64. The size of the current block can be represented as an average logarithmic value of a horizontal length and a vertical length, for example, (Log2(horizontal length) + Log2(vertical length)) » 1. When the size is 64x64, the average logarithmic value can be 6. In other words, filtering can be applied when the average logarithmic value of the horizontal length and the vertical length is less than 6.

[0246] In another example, filtering can be applied when a product of a horizontal size and a vertical size of the current block is greater than a predetermined threshold. The predetermined threshold can be, for example, 32.

[0247] According to embodiments of the present invention, it can be determined whether to apply filtering based on a shape of the current block.

[0248] In one example, filtering can not be applied when the shape of the current block is a square, and filtering can be applied when the shape is a non-square.

[0249] In another example, it can be determined whether to apply filtering based on an intra prediction mode of the current block and the shape of the current block. For example, filtering can be applied when the intra prediction mode of the current block is a predetermined mode and the shape of the current block is a non-square.

[0250] The predetermined mode can be, for example, a predetermined directional mode. For example, the predetermined mode can be a diagonal directional prediction mode (e.g., a left-bottom diagonal directional mode or a right-top diagonal directional mode). Alternatively, the predetermined mode can be a mode obtained by adding a predetermined constant value to the diagonal directional prediction mode. The predetermined constant value can be an integer.

[0251] For example, the filtering can be applied when the intra prediction mode of the current block is 2 and the vertical size of the current block is equal to or greater than the horizontal size of the current block.

[0252] Also, the filtering can be applied when the intra prediction mode of the current block is 66 and the horizontal size of the current block is equal to or greater than the vertical size of the current block.

[0253] The non-square shape can be determined by comparing the horizontal size and the vertical size of the block. Also, the non-square shape can be determined based on the shape of a neighboring block.

[0254] Also, the filtering target sample can be a reference sample for prediction adjacent to the current block, or can be a reference sample included in a reference picture for prediction.

[0255] According to an embodiment of the present application, it can be determined whether to apply filtering based on transform information of the current block. The transform information can be at least one of a coded block flag (CBF), an explicit multi-core transform (EMT) flag, a non-separable secondary transform (NSST) index, a transform skip flag, and a multi-transform selection (MTS).

[0256] In one example, when the CBF is 1, it can be determined that the filtering will be applied, and when the CBF is 0, it can be determined that the filtering will not be applied.

[0257] In another example, when the NSST index is 0, it can be determined that the filtering will not be applied, and when the NSST index is not 0, it can be determined that the filtering will be applied.

[0258] In another example, when the transform skip flag is 1, it can be determined that the filtering will not be applied, and when the transform skip flag is 0, it can be determined that the filtering will be applied.

[0259] In another example, when the MTS is applied to the current block, the filtering can not be applied. In other words, when the cu_mts_flag is 1, it can be determined that the filtering will not be applied, and when the cu_mts_flag is 0, it can be determined that the filtering will be applied.

[0260] In another example, the filtering can not be applied in a specific mode of the MTS. For example, when the mts_idx is 1 to 3, the filtering can not be applied.

[0261] According to an embodiment of the present application, it can be determined whether to apply a second filtering based on first filtering information of the current block. The first filtering or the second filtering can be at least one of a reference sample filter, a prediction sample filter, a residual sample filter, a reconstructed sample filter (e.g., a bilateral filter), a deblocking filter, SAO, and ALF. The filtering information can be at least one of whether to apply the filtering and a filter type.

[0262] In one example, the second filter can not be applied when the first filter is not applied to the current block. Conversely, the second filter can be applied when the first filter is not applied.

[0263] In another example, the second filter can not be applied to reference samples of a reconstructed block including the reference samples of the current block when the first filter is applied to the reconstructed block.

[0264] According to embodiments of the present invention, it can be determined whether to apply a filter based on whether the current block has a constrained intra prediction (CIP) mode.

[0265] For example, it can be determined not to apply at least one of a bilateral filter, a deblocking filter, SAO, and ALF to samples within the current block when the current block is coded / decoded in a CIP mode and a neighboring block adjacent to the current block has an inter mode. Thus, an error occurring in an inter coded / decoded block can not be propagated.

[0266] According to embodiments of the present invention, an indicator indicating whether to apply a filter can be signaled through at least one of an SPS, a PPS, and a slice header. Further, when a block is divided in a sub-block unit, it can be determined whether to apply a filter to a sub-block according to an indicator indicating whether to apply a filter to a super block of the sub-block.

[0267] According to embodiments of the present invention, it can be determined whether to apply a filter to a prediction sample based on a reference sample line used for intra prediction of the current block. For example, when the reference sample line used for prediction corresponds to a first reference sample line (e.g., mrl_idx=0), a filter to the prediction sample can be applied based on an intra prediction mode or a size / shape of the block or both. Further, when the reference sample line used for prediction corresponds to at least one of a second to fourth reference sample line (e.g., mrl_idx≠0), the filter to the prediction sample can not be applied. The mrl_idx can represent a reference sample line index adjacent to the current block, and a value of the mrl_idx can increase as it becomes farther from the current block. In other words, when mrl_idx=0, it can represent a reference sample line closest to the current block among the reference sample lines used for intra prediction of the current block.

[0268] Further, for example, when the reference sample line used for performing prediction corresponds to the first reference sample line (e.g., mrl_idx=0) and an intra prediction mode of the current block has a mode corresponding to one of a planar mode, a DC mode, a horizontal mode, a vertical mode, and a predetermined range, a filter to the prediction sample can be applied. For example, the predetermined range can represent a case where the intra prediction mode of the current block is equal to or greater than a predetermined mode (e.g., mode 58), or a case where the intra prediction mode of the current block is equal to or less than a predetermined mode (e.g., mode 10).

[0269] In addition, when intra prediction or inter prediction is performed, the filtering target sample can be a prediction sample.

[0270] Hereinafter, determining and applying a filter type will be described.

[0271] In determining and applying the filter type, when it is determined that filtering is applied to the filtering target sample, at least one filter type (or filtering type) can be determined and applied. The filter type can be at least one of a filter tap, a filter shape, and a filter coefficient. When the filter is applied, a plurality of sample lines of a neighboring block can be used.

[0272] According to an embodiment of the present application, the filter type can be at least one of the following.

[0273] In one example, the filter type can be an N-tap filter using the filtering target sample and at least one neighboring sample. Here, N can be an integer equal to or greater than 2.

[0274] In another example, the filter type can be a form of adding an offset value to the filtering target sample. Alternatively, the filter type can be a form of performing a weighted sum on the filtering target sample and a predetermined sample.

[0275] In another example, the filter shape can be at least one of a horizontal shape, a vertical shape, a diagonal line shape, and a cross shape.

[0276] In another example, the filter coefficient can be a positive integer or a negative integer, and can have a value that varies according to the number and shape of the filter tap.

[0277] According to an embodiment of the present application, when a reference sample for intra prediction is generated, filtering can be applied to at least one reference sample.

[0278] Filtering can be applied to a plurality of reference sample lines. Here, the filter type applied can be at least one filter type.

[0279] In one example, the same filter can be applied to each of the plurality of reference sample lines.

[0280] In another example, a first filter can be applied to a first reference sample line, and a second filter can be applied to a second reference sample line. For example, the first filter can be a 5-tap filter, and the second filter can be a 3-tap filter.

[0281] In another example, a first filter can be applied to a first reference sample line, and a second filter can not be applied to a second reference sample line.

[0282] In another example, a first filter can be applied to the first reference sample line, and then a second filter can be applied to the first reference sample line.

[0283] The first filter and the second filter can be the same or different.

[0284] In addition, filtering of the reference sample can not always be applied. Thus, the generation of the reference sample can become simple, and the decoding and the decoder can become simple.

[0285] Filtering can be applied to the reference sample based on a combination prediction information (e.g., MH_intra_flag) of inter prediction and intra prediction.

[0286] In one example, when the combination prediction is performed (e.g., MH_intra_flag = 1), filtering can not be applied to the reference sample.

[0287] In another example, when the combination prediction is not performed (e.g., MH_intra_flag = 0), filtering can not be applied to the reference sample.

[0288] According to an embodiment of the present application, when inter prediction is performed, filtering can be applied to a reference picture sample.

[0289] In one example, when motion prediction or compensation is performed for a current block, filtering can be applied to a sample or a block of a reference picture corresponding to motion information.

[0290] In another example, when a current block is divided in a sub-block unit and motion compensation is performed in the sub-block unit, filtering can be applied to a reference block sample of a picture referred by the sub-block. Here, filtering can be applied to a sample adjacent to a boundary of each reference block.

[0291] In addition, when prediction is performed based on at least one intra prediction mode, filtering can be applied to a prediction sample. The filtering application target sample can be all samples within a current block or partial samples adjacent to a reference sample.

[0292] According to an embodiment of the present application, when DC prediction is performed, filtering can be applied to all prediction samples within a current block or to a predetermined prediction sample line adjacent to a boundary of a block adjacent to an upper reference sample and a left reference sample. The predetermined prediction sample line can be determined to vary according to a size of the current block. For example, when the size of the current block becomes larger, more lines can be selected.

[0293] Figure 7 is a diagram illustrating filtering using variations of a plurality of reference sample lines according to an embodiment of the present application.

[0294] Referring toFigure 7 In one example, a value corresponding to a predetermined ratio of a difference between the first reference sample line and the second reference sample line can be applied to the prediction sample. For example, it can be expressed as Equation 1.

[0295] [Equation 1]

[0296] pf[x,0] = clip(p[x,0] + ((r[x,-1] - r[x,-2]) » N))

[0297] In Equation 1, pf[x,0] can denote a sample value obtained by applying filtering, p[x,0] can denote a prediction sample value, r[x,-1] can denote a first reference sample value, r[x,-2] can denote a second reference sample value, N is an integer equal to more than 1, and clip can denote clipping considering a bit depth. Here, N can increase or decrease as x or y of the prediction sample increases.

[0298] In another example, an N-tap filter can be applied by using a plurality of reference sample lines. For example, a 3-tap filter can be applied in accordance with Equation 2 or Equation 3.

[0299] [Equation 2]

[0300] pf[x,0] = (r[x,-2] + 2*r[x,-1] + 5*p[x,0] + 4) » 3

[0301] [Equation 3]

[0302] pf[x,0] = (r[x,-2] + r[x,-1] + 2*p[x,0] + 2) » 2

[0303] In another example, an N-tap filter can be applied by using at least one reference sample in the first reference sample line. For example, a 4-tap filter can be applied in accordance with Equation 4.

[0304] [Equation 4]

[0305] pf[x,y] = (A*r[-1,-1] + B*r[x,-1] + C*r[-1,y] + (64-A-B-C)*p[x,y] + 32) » 6

[0306] In Equation 4, A, B, and C can be filter coefficients applied to each reference sample.

[0307] At least one filter coefficient can vary according to at least one of a position of a prediction sample and an intra prediction mode of a current block.

[0308] For example, when the position of the prediction sample becomes far from the left reference sample or the above reference sample, the filter coefficient value can become small.

[0309] Also, the at least one filter coefficient can become 0 according to the position of the prediction sample. In other words, the number of filter taps can vary based on the position of the prediction sample.

[0310] Also, when the position of the prediction sample becomes far from the left reference sample predetermined position, C can become 0. Thus, here, the multiple filter taps can become 3 taps or 2 taps. Here, the position of the prediction sample can denote the x position of the prediction sample.

[0311] Also, when the position of the prediction sample becomes far from the above reference sample predetermined position, B can become 0. Thus, here, the multiple filter taps can be 3 taps or 2 taps. Here, the position of the prediction sample can denote the y position of the prediction sample.

[0312] Also, when the position of the prediction sample becomes far from the left reference sample and the above reference sample both predetermined positions, filtering can not be applied. Thus, A, B, and C can all become 0. Here, the position of the prediction sample can denote the x position or the y position or both the x position and the y position of the prediction sample.

[0313] Also, the at least one filter coefficient can vary based on whether the intra prediction mode of the current block is a non-directional mode, a vertical directional mode, a horizontal directional mode, a diagonal directional mode, or another directional mode.

[0314] The changing of the filter coefficient or the number of taps or both the filter coefficient and the number of taps according to the position of the prediction sample can be applied identically to filtering based on at least one intra prediction mode, which will be described later.

[0315] According to embodiments of the present invention, when performing planar prediction, filtering can be applied to all prediction samples or to partial prediction samples within the current block.

[0316] In one example, when performing planar prediction, the planar prediction can be performed by using a reference sample to which filtering is not applied. Here, when the prediction sample is generated, filtering can be applied to the prediction sample by using the reference sample.

[0317] In another example, the reference sample used for planar prediction and the reference sample used when the first filter is applied to the prediction sample can be different from each other. For example, the reference sample used for planar prediction can be a sample to which the second filter is applied, and the reference sample used when the first filter is applied to the prediction sample can be a sample to which the second filter is not applied.

[0318] In addition, filtering on the prediction samples can be applied based on a statistical value of the reference samples. For example, when the statistical value of the reference samples satisfies a predetermined threshold, filtering can be applied to the prediction samples, otherwise, filtering can not be applied.

[0319] According to embodiments of the present application, when performing horizontal prediction or vertical prediction, filtering can be applied when generating the prediction samples.

[0320] In one example, when performing vertical prediction, filtering can be performed by using a variation of a plurality of upper reference sample lines. For example, it can be represented as Equation 5, and N can be a fixed value equal to or greater than 1. Alternatively, N can be y+1 or (H-y), and H can represent a vertical dimension of the block.

[0321] [Equation 5]

[0322] pf[x,y] = clip(p[x,y] + ((r[x,-1] - r[x,-2]) » N))

[0323] In another example, when performing vertical prediction, filtering can be performed by using a variation of an upper reference sample and a left reference sample. For example, it can be represented as Equation 6.

[0324] [Equation 6]

[0325] pf[x,y] = clip(p[x,y] + ((r[x,-1] - r[-1,-1]) » N) + ((r[-1,y] - r[-1,-1]) » M))

[0326] In Equation 6, N or M can be a fixed value equal to or greater than 1. Alternatively, N can increase or decrease as x increases. M can increase or decrease as y increases.

[0327] In another example, when performing vertical prediction, filtering can be performed by using a left existing reference sample of a filtering target sample. For example, it can be represented as Equation 7 or Equation 8.

[0328] [Equation 7]

[0329] pf[x,y] = clip(p[x,y] + ((r[-1,y] - r[-1,-1]) » N))

[0330] [Equation 8]

[0331] pf[x,y] = ((64-A)*p[x,y] + A*(r[-1,y] - r[-1,-1]) + 32) » 6

[0332] In Equation 7 or Equation 8, filtering of p[x,y] can be applied by using a difference between r[-1,1] and r[-1,y]. Here, N can be (x+1). Also, A can be a filter coefficient based on an x value, and since a filter target prediction sample becomes a predetermined distance away from a left reference sample and thus the filter coefficient becomes 0, an effect of not applying filtering can be obtained.

[0333] In another example, filtering can be applied to a reference sample existing to the left of a filter target sample, and then filtering of the target sample can be applied. For example, this can be expressed as Equation 9.

[0334] [Equation 9]

[0335] pf[x,y] = clip(p[x,y] + (((r[-1,y-1] + 2*r[-1,y] + r[-1,y+1] + 2) » 2) - r[-1,-1]) » N)

[0336] The above-described embodiment of vertical prediction can be identically applied to horizontal prediction.

[0337] According to an embodiment of the present application, when performing directional prediction having a predetermined angle, filtering can be applied to all or part of prediction samples within a current block.

[0338] In one example, when performing prediction by using an above reference sample, a plurality of left reference sample lines can be used. For example, this can be expressed as Equation 10.

[0339] [Equation 10]

[0340] pf[0,y] = (r[-2,y+2] + 2*r[-1,y+1] + 5*p[0,y] + 4) » 3

[0341] The number of left reference sample lines can vary according to the position of a prediction sample. For example, when a prediction sample is located at a first left line, two reference sample lines can be used, and when a prediction sample is located at a second left line, one reference sample line can be used.

[0342] In another example, filtering can be performed by using at least one reference sample existing on a directional line. For example, an above reference sample r[x',-1] and a left reference sample r[-1,y'] existing at a directional line can be used. x' and y' values can represent the positions of reference samples based on an angle according to each directional mode. Also, r[x',y] (y<0) can be used instead of r[x',-1], and r[x,y'] (x<0) can be used instead of r[-1,y'].

[0343] The above reference sample or the left reference sample or both the above reference sample and the left reference sample can be derived by applying an interpolation filter. Further, when a direction prediction mode of the current block satisfies a predetermined condition, the above reference sample or the left reference sample or both the above reference sample and the left reference sample can be determined by using samples at integer positions adjacent to a direction line of the direction prediction mode. The predetermined condition can be a direction of the prediction mode or a number of the prediction mode.

[0344] When the direction prediction mode is a mode corresponding to 45 degrees to 90 degrees (e.g., mode 63), the prediction sample p[x,y] can be generated by interpolation of the above reference sample r[x',-1] and at least one adjacent reference sample.

[0345] When the filtering is performed on the prediction sample, it can be expressed as Equation 11.

[0346] [Equation 11]

[0347] pf[x,y] = (A*r[-1,y'] + (64-A)*p[x,y] + 32) >> 6,

[0348] In Equation 11, the left reference sample r[-1,y'] can be used, and the left reference sample can be derived by applying an interpolation filter. For example, a bilinear interpolation filter can be applied as Equation 12.

[0349] [Equation 12]

[0350] r[-1,y'] = ((64-Frac)*r[-1,y'] + Frac*r[-1,y'+1] + 32) >> 6

[0351] Here, Frac can be a value of 1 / 64 pixel unit between two samples. For example, Frac can be a distance value in 1 / 64 pixel units between two samples.

[0352] Alternatively, the complexity of the left reference sample can be reduced by using a sample at an integer position close to a direction line as Equation 13. The sample at the integer position can be a sample at an integer position closest to a direction line of the direction prediction mode among samples of a vertical direction line in which the left reference sample is located.

[0353] [Equation 13]

[0354] r[-1,y'] = r[-1,y' + (Frac >> 5)]

[0355] For example, when the direction prediction mode is equal to or greater than mode 58 and less than mode 66, the left reference sample can use a sample at an integer position close to a direction line of the direction prediction mode.

[0356] Also, when the horizontal direction mode (e.g., the directional prediction mode) is a mode corresponding to 3 to 17, the prediction can be performed by using at least one left side reference sample. Also, the filtering of the predicted sample can be performed by using an above reference sample, and the above reference sample can be a sample at an integer position close to the directional line. For example, the above reference sample can be derived in Equation 14. The sample at the integer position can be a sample at an integer position closest to the directional line of the directional prediction mode among the samples of the horizontal direction line where the above reference sample is located.

[0357] [Equation 14]

[0358] r[x', -1] = r[x' + (Frac » 5), -1]

[0359] Also, for example, when the directional prediction mode is equal to or less than mode 10 and greater than mode 2, the above reference sample can use a sample at an integer position close to the directional line of the directional prediction mode.

[0360] According to embodiments of the present invention, the filtering can be applied to a sample predicted using a first intra prediction mode by using a value predicted using a second intra prediction mode.

[0361] In one example, the first intra prediction mode can be a DC mode, and the second intra prediction mode can be a predetermined directional mode. Here, the filtering can be applied by performing a weighted sum on a sample predicted using the DC mode and a sample predicted using the predetermined directional mode.

[0362] In another example, the filtering can be applied by performing a weighted sum on a first intra prediction sample and a second intra prediction sample.

[0363] According to embodiments of the present invention, the reference sample for the intra prediction and the reference sample for the filtering can be different from each other.

[0364] In one example, the reference sample for the prediction can be a reference sample to which the filtering is applied when the reference sample is generated, and the reference sample for the filtering of the predicted sample can be a reference sample to which the filtering is not applied when the reference sample is generated.

[0365] In another example, the reference sample for the prediction can be a plurality of reference sample lines, and the reference sample for the filtering can be a single reference sample line.

[0366] In another example, the reference sample for the prediction can be a first reference sample line adjacent to the current block, and the reference sample for the filtering can be a second reference sample line adjacent to the current block.

[0367] According to embodiments of the present application, when applying filtering to the prediction samples, at least one of filter coefficients, filter taps, filter shapes, and weights can be variably applied according to at least one of a line of prediction samples within the current block, a prediction mode of the current block, and a size of the current block.

[0368] In one example, a 2-tap filter can be applied to a first line of prediction samples adjacent to a boundary of reference samples, and a 3-tap filter can be applied to a second line of prediction samples. Optionally, the number of filter taps can decrease as the line of prediction samples becomes further away from the reference samples.

[0369] In another example, filter coefficient values associated with the second line of prediction samples can be smaller than filter coefficient values associated with the first line of prediction samples. In other words, the filter coefficient values associated with each line can become smaller as the line of prediction samples becomes further away from the reference samples. In other words, the filter coefficient values applied to the left-side reference samples can become smaller as the line of prediction samples becomes further away from the left-side reference samples. Similarly, the filter coefficient values applied to the above reference samples can become smaller as the line of prediction samples becomes further away from the reference samples.

[0370] Here, the filter coefficient values associated with the line of samples can be filter coefficient values applied to the reference samples or the line of prediction samples.

[0371] The sum of the filter coefficient values can be a predefined value. For example, the sum of the filter coefficient values can be 8. Further, the filter coefficient values can vary according to a distance between the prediction samples and the reference samples. For example, the filter coefficient values applied to the prediction samples or the reference samples can become smaller as the prediction samples become further away from the reference samples. Optionally, conversely, the filter coefficient values applied to the prediction samples or the reference samples can become larger as the prediction samples become further away from the reference samples. Further, the filter coefficient values can vary according to a horizontal size or a vertical size of the current block, or both the horizontal size and the vertical size.

[0372] Further, the filtering can be applied by performing a weighted sum on the intra-prediction samples of the current block and the inter-prediction samples of the current block.

[0373] The weights can represent the filter coefficient values.

[0374] In one example, the weights applied to the intra-prediction samples and the weights applied to the inter-prediction samples can be different when an intra-prediction mode of the current block satisfies a predetermined condition.

[0375] In another example, a fixed value can be applied to the weights applied to the intra-prediction samples and the inter-prediction samples when an intra-prediction mode of the current block is a non-directional mode (a DC mode or a planar mode). For example, the fixed value can be 4.

[0376] In another example, when the intra prediction mode of the current block is a directional mode, the weight can be variably applied to the intra prediction samples and the inter prediction samples.

[0377] In another example, when the intra prediction mode of the current block has a vertical direction (prediction mode 50), the weight can be variably applied to the intra prediction samples and the inter prediction samples according to the y position of the prediction samples. Here, the weight applied to the intra prediction samples can become smaller as the prediction samples become further away from the reference samples. Also, the weight applied to the inter prediction samples can become larger as the prediction samples become further away from the reference samples. Further, the weight can vary according to the vertical size of the current block.

[0378] In another example, when the intra prediction mode of the current block has a horizontal direction (prediction mode 18), the weight applied to the intra prediction samples and the inter prediction samples can be variably applied according to the x position of the prediction samples. Here, the weight applied to the intra prediction samples can become smaller as the prediction samples become further away from the reference samples. Also, the weight applied to the inter prediction samples can become larger as the prediction samples become further away from the reference samples. Further, the weight can vary according to the horizontal size of the current block.

[0379] In another example, when the intra prediction mode of the current block is not a non-directional mode, a vertical direction mode, or a horizontal direction mode, a fixed value can be applied to the weight applied to the intra prediction samples and the inter prediction samples. For example, the fixed value can be 4.

[0380] According to embodiments of the present invention, when prediction is performed based on at least one intra prediction mode, a filter can be applied to the prediction samples. The filter application target samples can be all samples within the current block or partial samples adjacent to the reference samples.

[0381] According to embodiments of the present invention, a filter can be applied to the samples predicted based on at least one inter prediction mode.

[0382] In one example, when the inter prediction mode of the current block is not a skip mode, a filter can be applied to the inter prediction samples of the current block.

[0383] In another example, when the inter prediction mode of the current block is a merge mode, a filter can be applied to the inter prediction samples of the current block.

[0384] The filter can have the form of a weighted sum of at least one prediction sample. For example, the filter can be performed by performing a weighted sum of a prediction value of a first inter mode and a prediction value of a second inter mode. Alternatively, the filter can be performed by performing a weighted sum of a prediction value of an inter mode and a prediction value of an intra mode. The prediction value of the inter mode can be a value predicted by using the merge mode.

[0385] In addition, the filtering can be applied to all or part of the prediction samples of the current block. For example, the filtering can be applied to predetermined samples at a boundary of the current block.

[0386] In another example, when the inter prediction is performed by dividing the current block into predetermined sub-blocks, the filtering can be applied to all of the sub-blocks or to part of the samples such as a boundary. For example, the current block can be divided into two sub-blocks, and the filtering can be applied to the prediction samples located at a boundary of the first sub-block and the second sub-block. The filtering can have a form of a weighted sum of the prediction values of the first sub-block and the second sub-block. In addition, for example, the filtering can not be applied to the prediction samples located at the boundary of the first sub-block and the second sub-block.

[0387] According to an embodiment of the present application, the filtering can be applied to the residual samples of the current block. The residual samples can be samples corresponding to at least one of before the transform and the quantization of the current block, after the transform and the quantization, and during the transform and the quantization. For example, the filtering can be applied to the residual samples obtained by performing the entropy decoding on the residual signal of the current block and then performing the inverse quantization and the inverse transform on the residual signal of the current block.

[0388] According to an embodiment of the present application, at least one filter can be applied to the reconstructed samples of the current block. The reconstructed samples can be samples generated by adding the prediction samples and the residual samples. Alternatively, the reconstructed samples can be samples to which at least one of a bilateral filter, a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) is applied.

[0389] The filtering can be applied to the reconstructed samples generated by adding the prediction samples and the residual samples of the current block.

[0390] In one example, the bilateral filter can be applied by using the filtered target samples and the neighboring samples of the reconstructed samples. The samples to which the filtering is applied can be used as reference samples of a block following a decoding order. The bilateral filter can be filtering using a distance between the filtered target samples and the neighboring samples and a value of each sample.

[0391] In another example, the deblocking filter can be applied to the reconstructed samples. Alternatively, the deblocking filter can be applied to the reconstructed samples to which the bilateral filter is applied.

[0392] In addition, when the deblocking filter is applied, the filter can be applied based on a size / shape of a block. For example, when the filter is applied to a vertical boundary, the number of samples to which the filter is applied and used can be determined based on a size / shape of a left block. Similarly, the number of samples to which the filter is applied and used can be determined based on a size / shape of a right block.

[0393] In another example, SAO or ALF can be applied to the reconstructed samples to which the deblocking filter is applied. The SAO can be to apply an offset to the filtered target samples, and the ALF can be to use a two-dimensional filter using the filtered target samples and neighboring samples.

[0394] Further, the filters can be selectively applied, and the filters can be applied in a predetermined order. For example, the bilateral filter, the deblocking filter, and the SAO can be sequentially applied. Alternatively, the bilateral filter, the SAO, and the ALF can be sequentially applied. Alternatively, the bilateral filter, the SAO, the deblocking filter, and the ALF can be sequentially applied. Here, information of the predetermined order can be determined in the encoder and signaled in the decoder.

[0395] The reconstructed samples can be samples generated by performing the motion compensation. Further, the reconstructed samples can be reconstructed samples of a neighboring block adjacent to the current block.

[0396] In another example, the filtering can be performed with respect to the reconstructed samples of the current block and the reconstructed samples of the neighboring block.

[0397] In another example, the filtering can be applied based on the transform information of the current block. For example, when the transform is skipped for the current block, at least one of the filters can not be applied.

[0398] According to an embodiment of the present application, when the filter is applied, it can be determined whether to apply the filter to the reference samples, the prediction samples, and the reconstructed samples, and applying the filter can be performed.

[0399] Figure 8 is a diagram illustrating determining whether to apply the filter to the reference samples, the prediction samples, and the reconstructed samples and applying the filter according to an embodiment of the present application.

[0400] The encoder / decoder can perform at least one of the following steps: determining whether to apply the filter to the reference samples and applying the filter at S801, determining whether to apply the filter to the prediction samples and applying the filter at S802, and determining whether to apply the filter to the reconstructed samples and applying the filter at S803. The order of performing each step can vary, and a part of the steps can be performed.

[0401] Here, the filter can be a first filter, a second filter, or a third filter, and another filter can be applied based on the filter information. For example, the second filter can be adaptively applied based on at least one of whether the first filter is applied and the filter type.

[0402] In an example, when the first filter is applied to the current block, the second filter can not be applied. Alternatively, on the contrary, when the first filter is applied, it can be determined to apply the second filter.

[0403] In another example, when the filter is applied to the reference samples of the current block, the filter can not be applied to the prediction samples.

[0404] In another example, when the filter is not applied to the reference samples of the current block, the filter can be applied to the prediction samples.

[0405] In another example, when the filter is applied to the reference samples of the current block, a first filter type can be applied to the prediction samples. Alternatively, when the filter is not applied to the reference samples, a second filter type can be applied to the prediction samples.

[0406] In another example, when a bilateral filter is applied to the reconstructed samples of the first block, the filter can not be applied to the reference samples for predicting the second block. The reference samples can be the reconstructed samples of the first block to which the bilateral filter is applied.

[0407] Figure 9 FIG. 1 is a diagram illustrating a picture coding method according to an embodiment of the present application.

[0408] Referring to Figure 9 At S901, the encoder can determine whether to apply a filter to a filter target sample.

[0409] The filter target sample can include at least one of a prediction sample, a reference sample, a reconstructed sample, and a residual sample.

[0410] Further, whether to apply the filter can be determined based on at least one of a coding mode, an intra prediction mode, an inter prediction mode, transform information, filter information, a size (a horizontal size or a vertical size), a shape, and a reference sample line of the current block, and a coding mode, an intra prediction mode, an inter prediction mode, transform information, filter information, a size (a horizontal size or a vertical size), a shape, and a reference sample line of a neighboring block.

[0411] Further, whether to apply the filter can be determined based on whether a shape of the current block is a non-square shape.

[0412] Further, whether to apply the filter can be determined based on whether an intra prediction mode of the current block is a predetermined direction mode.

[0413] Further, at S902, the encoder can determine a filter type based on a result of whether to apply the filter.

[0414] Further, the filter type can include at least one of a filter tap, a filter shape, and a filter coefficient.

[0415] Further, when it is determined to apply the filter to the filter target sample, the filter type can be determined.

[0416] Further, the filter type can be determined based on a position of the filter target sample.

[0417] Further, the filter type can be determined based on at least one of a size and a prediction mode of the current block.

[0418] Further, at S903, the encoder can perform filtering on the filter target sample by using the determined filter type.

[0419] Figure 10 FIG. 1 is a diagram illustrating a flowchart of an image encoding method according to an embodiment of the present application.

[0420] Referring to Figure 10 At S1001, the decoder can determine whether to apply filtering on a filter target sample.

[0421] The filter target sample can include at least one of a prediction sample, a reference sample, a reconstructed sample, and a residual sample.

[0422] Further, whether to apply filtering can be determined based on at least one of an encoding mode, an intra prediction mode, an inter prediction mode, transform information, filtering information, a size (horizontal size or vertical size), a shape, and a reference sample line of the current block, and an encoding mode, an intra prediction mode, an inter prediction mode, transform information, filtering information, a size (horizontal size or vertical size), a shape, and a reference sample line of a neighboring block.

[0423] Further, whether to apply filtering can be determined based on whether the current block has a non-square shape.

[0424] Further, whether to apply filtering can be determined based on whether an intra prediction mode of the current block is a predetermined direction mode.

[0425] Further, at S1002, the decoder can determine a filter type based on a result of whether to apply filtering.

[0426] Further, the filter type can include at least one of a filter tap, a filter shape, and a filter coefficient.

[0427] Further, when it is determined to apply filtering on the filter target sample, the filter type can be determined.

[0428] Further, the filter type can be determined based on a position of the filter target sample.

[0429] Further, the filter type can be determined based on at least one of a size and a prediction mode of the current block.

[0430] Further, at S1003, the decoder can perform filtering on the filter target sample by using the determined filter type.

[0431] The above-described embodiments can be performed in the same manner in the encoder and the decoder.

[0432] At least one of the above-described embodiments or a combination thereof can be used to encode / decode a video.

[0433] The order applied to the above-described embodiments between the encoder and the decoder can be different, or the order applied to the above-described embodiments in the encoder and the decoder can be the same.

[0434] The above-described embodiments can be performed for each of the luma signal and the chroma signal, or the above-described embodiments can be performed identically for the luma signal and the chroma signal.

[0435] The block form to which the above-described embodiments of the present application are applied can have a square form or a non-square form.

[0436] The above-described embodiments of the present application can be applied depending on the size of at least one of a coding block, a prediction block, a transform block, a block, a current block, a coding unit, a prediction unit, a transform unit, a unit, and a current unit. Here, the size can be defined as a minimum size or a maximum size or both the minimum size and the maximum size such that the above-described embodiments are applied, or can be defined as a fixed size to which the above-described embodiments are applied. In addition, in the above-described embodiments, a first embodiment can be applied to a first size, and a second embodiment can be applied to a second size. In other words, the above-described embodiments can be applied in combination depending on the size. In addition, when the size is equal to or greater than the minimum size and equal to or smaller than the maximum size, the above-described embodiments can be applied. In other words, when the block size is included in a certain range, the above-described embodiments can be applied.

[0437] For example, when the size of the current block is 8x8 or greater, the above-described embodiments can be applied. For example, when the size of the current block is 4x4 or greater, the above-described embodiments can be applied. For example, when the size of the current block is 16x16 or greater, the above-described embodiments can be applied. For example, when the size of the current block is equal to or greater than 16x16 and equal to or smaller than 64x64, the above-described embodiments can be applied.

[0438] The above-described embodiments of the present application can be applied depending on a temporal layer. In order to identify a temporal layer to which the above-described embodiments can be applied, a corresponding identifier can be signaled, and the above-described embodiments can be applied to a designated temporal layer identified by the corresponding identifier. Here, the identifier can be defined as a lowest layer or a highest layer or both the lowest layer and the highest layer to which the above-described embodiments can be applied, or can be defined as a specific layer indicating that the embodiments are applied. In addition, a fixed temporal layer to which the embodiments are applied can be defined.

[0439] For example, the above-described embodiment can be applied when the temporal layer of the current picture is the lowest layer. For example, the above-described embodiment can be applied when the temporal layer identifier of the current picture is 1. For example, the above-described embodiment can be applied when the temporal layer of the current picture is the highest layer.

[0440] A slice type or a parallel block group type to which the above-described embodiment of the present application is applied can be defined, and the above-described embodiment can be applied depending on the corresponding slice type or parallel block group type.

[0441] In the above-described embodiments, a method is described based on a flowchart having a series of steps or units, but the present application is not limited to the order of the steps, and some steps can be performed simultaneously with other steps or in a different order from other steps. In addition, it will be understood by those of ordinary skill in the art that the steps in the flowchart are not mutually exclusive, and other steps can be added to the flowchart or some steps can be deleted from the flowchart without affecting the scope of the present application.

[0442] The embodiments include examples of various aspects. All possible combinations of the various aspects can not be described, but one of ordinary skill in the art will be able to recognize different combinations. Accordingly, the present application can include all substitutions, modifications and changes within the scope of the claims.

[0443] Embodiments of the present application can be implemented in the form of program instructions, which can be executed by various computer components, and can be recorded in computer-readable recording media. The computer-readable recording media can include independent program instructions, data files, data structures, etc., or combinations thereof. The program instructions recorded in the computer-readable recording media can be specifically designed and constructed for the present application, or known to those skilled in the computer software technology. Examples of the computer-readable recording media include magnetic recording media such as hard disks, floppy disks and magnetic tapes; optical data storage media such as CD-ROMs or DVD-ROMs; magnetic-optical media such as floptical disks; and hardware devices specifically configured to store and implement program instructions, such as read-only memories (ROMs), random access memories (RAMs), flash memories, etc. Examples of the program instructions include not only machine language codes formatted by a compiler, but also high-level language codes that can be implemented by computers using an interpreter. The hardware devices can be configured to be operated by one or more software modules to perform processes according to the present application, or vice versa.

[0444] Although the present application has been described in connection with certain specific embodiments and in terms of certain specific items such as detailed elements, it is to be understood that they are only provided for the purpose of better comprehending the present application and that the present application is not limited to those embodiments. Various modifications and changes can be made by those skilled in the art to which the present application pertains without departing from the spirit and scope of the present application.

[0445] Therefore, the spirit of the present application should not be limited to the above-described embodiments, and the entire scope of the appended claims and their equivalents will fall within the scope and spirit of the present application.

[0446] Industrial applicability

[0447] The present application can be used when performing image encoding / decoding.

Claims

1. A method of video decoding, comprising: determining prediction samples of a current block based on an intra prediction mode of the current block; determining whether to filter the prediction samples of the current block; and filtering the prediction samples of the current block in response to determining to filter the prediction samples, wherein the filtering is performed based on a weighted sum of the prediction samples and at least one reference sample of the prediction samples, wherein the determination of whether to filter the prediction samples of the current block is based on a reference line index indicating a reference line of more than one reference line in a vicinity of the current block that is referenced by the current block, wherein the determination of not filtering the prediction samples of the current block is made when the reference line index is not "0", and wherein the reference line index is "0" when the reference line index indicates a first reference line that is immediately adjacent to the current block. 2.The method of Claim 1, wherein the weighted sum is calculated by using weights applied to the prediction samples and the at least one reference sample. 3.The method of Claim 2, wherein the weight of a reference sample of the at least one reference sample decreases and the weight of the prediction sample increases as a distance between the prediction sample and the reference sample increases. 4.The method of Claim 1, wherein the determination of whether to filter the prediction samples of the current block is based on at least one of the intra prediction mode, a block size, the reference line index. 5.The method of Claim 1, wherein the determination of filtering the prediction samples of the current block is made when the intra prediction mode is one of a planar mode, a DC mode, a horizontal mode, a vertical mode, or when an index of the intra prediction mode with a prediction direction is less than 10 or greater than 58. 6.The method of Claim 1, wherein the determination of not filtering the prediction samples of the current block is made when transform skip information indicates to skip a transform process of residual samples in the current block. 7.The method of Claim 2, wherein the at least one reference sample includes an above reference sample and a left reference sample, the above reference sample is located above the prediction sample, and the left reference sample is located left of the prediction sample. 8.The method of Claim 2, wherein the at least one reference sample includes an opposite direction reference sample determined based on an opposite direction line and a first reference line, and the first reference line is immediately adjacent to the current block and the opposite direction line has a direction opposite to a prediction direction of the intra prediction mode. 9.The method of Claim 8, wherein the opposite direction reference sample is located at an integer position closest to the opposite direction line. 10.A method of video encoding, comprising: determining prediction samples of a current block based on an intra prediction mode of the current block; determining whether to filter the prediction samples of the current block; and filtering the prediction samples of the current block in response to determining to filter the prediction samples, wherein the filtering is performed based on a weighted sum of the prediction samples and at least one reference sample of the prediction samples, wherein the determination of whether to filter the prediction samples of the current block is based on a reference line index indicating a reference line of more than one reference line in a vicinity of the current block that is referenced by the current block, wherein the determination of not filtering the prediction samples of the current block is made when the reference line index is not "0", and wherein the reference line index is "0" when the reference line index indicates a first reference line that is immediately adjacent to the current block. wherein, ​ ​ wherein ​ ​ wherein ​ ​ wherein, ​ ​ wherein ​ ​ wherein ​ ​ ​ ​ wherein, ​ ​ ​ wherein ​ ​ ​ ​ ​ ​ wherein the filtering is performed based on a weighted sum of the prediction sample and at least one reference sample of the prediction sample, wherein it is determined whether to filter the prediction sample of the current block based on a reference line index indicating a reference line of more than one reference line in a vicinity of the current block that is referenced by the current block, wherein it is determined not to filter the prediction sample of the current block when the reference line index is not "0", and wherein the reference line index is "0" when the reference line index indicates a first reference line immediately adjacent to the current block.

11. The video encoding method of claim 10, wherein the weighted sum is calculated by using weights applied to the prediction sample and the at least one reference sample.

12. The video encoding method of claim 11, wherein as a distance between the prediction sample and a reference sample of the at least one reference sample increases, a weight of the reference sample decreases and a weight of the prediction sample increases.

13. The video encoding method of claim 10, wherein it is determined whether to filter the prediction sample of the current block based on at least one of the intra prediction mode, a block size, the reference line index.

14. The video encoding method of claim 10, wherein it is determined to filter the prediction sample of the current block when the intra prediction mode is one of a planar mode, a DC mode, a horizontal mode, a vertical mode, or when an index of the intra prediction mode with a prediction direction is less than 10 or greater than 58.

15. The video encoding method of claim 10, wherein it is determined not to filter the prediction sample of the current block when transform skip information indicates to skip a transform process of a residual sample in the current block.

16. The video encoding method of claim 11, wherein the at least one reference sample includes an above reference sample and a left reference sample, the above reference sample is located above the prediction sample, and the left reference sample is located left of the prediction sample.

17. The video encoding method of claim 11, wherein, the at least one reference sample includes an opposite direction reference sample determined based on an opposite direction line and a first reference line, the first reference line is immediately adjacent to the current block and the opposite direction line has a direction opposite to a prediction direction of the intra prediction mode, and wherein the opposite direction reference sample is located at an integer position closest to the opposite direction line.

18. A method for transmitting a bitstream generated by an image encoding method, the method for transmitting comprising: transmitting the bitstream, wherein the image encoding method comprises: determining a prediction sample of a current block based on an intra prediction mode of the current block; determining whether to filter the prediction sample of the current block; and in response to determining to filter the prediction sample, filtering the prediction sample of the current block, wherein the filtering is performed based on a weighted sum of the prediction sample and at least one reference sample of the prediction sample, wherein it is determined whether to filter the prediction samples of the current block based on a reference line index indicating a reference line among more than one reference lines in the vicinity of the current block that is referenced by the current block, wherein it is determined not to filter the prediction samples of the current block when the reference line index is not "0", and wherein the reference line index is "0" when the reference line index indicates a first reference line that is immediately adjacent to the current block.

Images