Image encoding / decoding method and apparatus for performing reference sample filtering on basis of intra prediction mode, and method for transmitting bitstream

Abstract

To provide an image encoding / decoding method and apparatus.SOLUTION: An image decoding method performed by an image decoding apparatus may comprise: determining an intra prediction mode of a current block; performing planar intra prediction based on the intra prediction mode being a planar prediction mode; and obtaining a reconstructed sample of the current block based on a prediction sample obtained based on the planar intra prediction. The step of performing the planar intra prediction may comprise applying filtering to an intra prediction reference sample of the current block.SELECTED DRAWING: Figure 22

Description

[Technical Field]

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly to an image encoding / decoding method and apparatus that perform reference sample filtering based on an intra prediction mode, and a method for transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure. [Background technology]

[0002] Recently, demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As image data becomes higher in resolution and quality, the amount of information or bits to be transmitted increases relatively compared to conventional image data. The increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[0003] This requires highly efficient image compression techniques for effectively transmitting, storing, and reproducing high-resolution, high-quality image information. Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus that performs reference sample filtering based on intra prediction modes.

[0006] Another object of the present disclosure is to provide a method and apparatus for encoding / decoding an image that performs reference sample filtering based on one of a plurality of filters.

[0007] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus that performs reference sample filtering based on a specific intra prediction mode.

[0008] Another object of the present disclosure is to provide a method for transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0009] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0010] Another object of the present disclosure is to provide a recording medium storing a bitstream that is received by an image decoding device according to the present disclosure, decoded, and used to restore an image.

[0011] The technical problems to be solved by the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not mentioned above will be clearly understood by a person having ordinary skill in the technical field to which the present disclosure pertains from the following description. [Means for solving the problem]

[0012] According to one embodiment of the present disclosure, an image decoding method performed by an image decoding device includes a step of determining an intra prediction mode of a current block, a step of performing planar intra prediction based on the intra prediction mode being a planar prediction mode, and a step of obtaining a reconstructed sample of the current block based on a predicted sample obtained based on the planar intra prediction, wherein the step of performing the planar intra prediction may include a step of applying filtering to an intra prediction reference sample of the current block.

[0013] According to an embodiment of the present disclosure, the filtering can be performed by adaptively selecting one of the first filter and the second filter.

[0014] According to one embodiment of the present disclosure, the one filter may be selected based on the number of intra-prediction reference samples.

[0015] According to one embodiment of the present disclosure, the first filter may be a 3-tap filter and the second filter may be a 5-tap filter.

[0016] According to one embodiment of the present disclosure, the 3-tap filter may be [1,2,1].

[0017] According to one embodiment of the present disclosure, the 5-tap filter may be [1,4,6,4,1].

[0018] According to an embodiment of the present disclosure, the number of intra-prediction reference samples may be determined based on the size of the current block.

[0019] According to an embodiment of the present disclosure, the number of intra-prediction reference samples may be determined based on a comparison between the size of the current block and a specific value.

[0020] According to one embodiment of the present disclosure, the application of the filtering may be determined based on whether decoder side intra mode derivation (DIMD) is applied.

[0021] According to an embodiment of the present disclosure, the application of the filtering may be determined based on whether or not combined inter intra prediction (CIIP) is applied.

[0022] According to one embodiment of the present disclosure, the filtering can be applied regardless of the color components of the current block.

[0023] According to an embodiment of the present disclosure, the filtering can be applied regardless of the size of the current block.

[0024] According to one embodiment of the present disclosure, all of the filter coefficients used in the filtering may be positive numbers.

[0025] According to one embodiment of the present disclosure, an image decoding device includes a memory and at least one processor, wherein the at least one processor determines an intra prediction mode of a current block, performs planar intra prediction based on the intra prediction mode being a planar prediction mode, obtains reconstructed samples of the current block based on predicted samples obtained based on the planar intra prediction, and the planar intra prediction can be performed by applying filtering to intra prediction reference samples of the current block.

[0026] According to one embodiment of the present disclosure, an image encoding method performed by an image encoding device includes a step of determining an intra prediction mode of a current block, a step of performing planar intra prediction based on the intra prediction mode being a planar prediction mode, and a step of obtaining a reconstructed sample of the current block based on a prediction sample obtained based on the planar intra prediction, wherein the step of performing the planar intra prediction may include a step of applying filtering to an intra prediction reference sample of the current block.

[0027] According to an embodiment of the present disclosure, a bitstream generated by an image encoding device or an image encoding method can be transmitted.

[0028] According to one embodiment of the present disclosure, in a method for transmitting a bitstream generated by an image encoding method, the image encoding method includes a step of determining an intra prediction mode of a current block, a step of performing planar intra prediction based on the intra prediction mode being a planar prediction mode, and a step of obtaining a reconstructed sample of the current block based on a prediction sample obtained based on the planar intra prediction, wherein the step of performing the planar intra prediction may include a step of applying filtering to an intra prediction reference sample of the current block.

[0029] According to one embodiment of the present disclosure, the bitstream generated by the image encoding method can be stored or recorded on a computer-readable medium.

[0030] The features described above in this brief summary of the present disclosure are merely exemplary embodiments of the detailed description of the present disclosure that follows and are not intended to limit the scope of the present disclosure. [Effects of the Invention]

[0031] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0032] Furthermore, the present disclosure can provide an image encoding / decoding method and apparatus that performs reference sample filtering based on an intra prediction mode.

[0033] Furthermore, according to the present disclosure, an image encoding / decoding method and apparatus for performing reference sample filtering based on a planar intra prediction mode can be provided.

[0034] According to the present disclosure, a method for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure can be provided.

[0035] Furthermore, according to the present disclosure, a recording medium storing a bitstream generated by the image encoding method or apparatus according to the present disclosure can be provided.

[0036] Furthermore, according to the present disclosure, it is possible to provide a recording medium that stores a bitstream that is received by the image decoding device according to the present disclosure, decoded, and used to restore an image.

[0037] The effects obtained by the present disclosure are not limited to the effects described above, and other effects not described above will be clearly understood by those having ordinary skill in the art to which the present disclosure pertains from the following description. [Brief explanation of the drawings]

[0038] [Figure 1] 1 is a diagram illustrating a video coding system to which embodiments of the present disclosure can be applied;

[0039] [Figure 2] 1 is a diagram schematically illustrating an image encoding device to which an embodiment of the present disclosure can be applied.

[0040] [Figure 3] FIG. 1 is a diagram schematically illustrating an image decoding device to which an embodiment of the present disclosure can be applied.

[0041] [Figure 4] FIG. 10 is a diagram showing block division types according to a multi-type tree structure.

[0042] [Figure 5] FIG. 1 illustrates a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structure according to the present disclosure.

[0043] [Figure 6] 1 is a flowchart illustrating a video / image coding method based on intra-prediction.

[0044] [Figure 7] 10 is a diagram illustrating an example configuration of an intra prediction unit 185 according to the present disclosure. FIG.

[0045] [Figure 8] 1 is a flowchart illustrating a video / image decoding method based on intra prediction.

[0046] [Figure 9] FIG. 10 is a diagram illustrating an example configuration of an intra prediction unit 265 according to the present disclosure.

[0047] [Figure 10] 10 is a flowchart showing an intra-prediction mode signaling procedure in an image encoding device.

[0048] [Figure 11] 10 is a flowchart showing an intra-prediction mode determination procedure in the image decoding device.

[0049] [Figure 12] 10 is a flowchart illustrating the intra-prediction mode derivation procedure in more detail.

[0050] [Figure 13] FIG. 10 is a diagram illustrating intra-prediction directions according to one embodiment of the present disclosure.

[0051] [Figure 14] FIG. 10 is a diagram illustrating intra-prediction directions according to another embodiment of the present disclosure.

[0052] [Figure 15] FIG. 10 is a diagram illustrating reference samples for wide-angle intra-prediction modes according to one embodiment of the present disclosure.

[0053] [Figure 16] FIG. 10 illustrates some wide-angle intra-prediction directions according to one embodiment of the present disclosure.

[0054] [Figure 17] FIG. 1 is a diagram illustrating peripheral blocks that can be used in CIIP (Combined Inter and Intra Prediction).

[0055] [Figure 18] FIG. 1 is a diagram illustrating MRL (Multi Reference Line) intra prediction.

[0056] [Figure 19a] FIG. 1 is a diagram illustrating ISPs (Intra Subpartitions). [Figure 19b] FIG. 1 is a diagram illustrating ISPs (Intra Subpartitions).

[0057] [Figure 20] FIG. 10 illustrates a planar intra-prediction mode procedure according to one embodiment of the present disclosure.

[0058] [Figure 21] FIG. 10 is a diagram illustrating a planar intra-prediction mode procedure according to another embodiment of the present disclosure.

[0059] [Figure 22] FIG. 1 is a diagram illustrating an image encoding or decoding method that performs reference sample filtering based on an intra-prediction mode according to an embodiment of the present disclosure.

[0060] [Figure 23] FIG. 1 is a diagram illustrating an image encoding / decoding device according to an embodiment of the present disclosure.

[0061] [Figure 24] FIG. 1 illustrates a content streaming system to which an embodiment of the present disclosure can be applied. DETAILED DESCRIPTION OF THE INVENTION

[0062] The present disclosure will be described in detail below with reference to the accompanying drawings, so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein.

[0063] In describing the embodiments of the present disclosure, if it is determined that a detailed description of a known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted. In addition, in the drawings, parts that are not related to the description of the present disclosure will be omitted, and similar parts will be designated by similar reference numerals.

[0064] In this disclosure, when a component is referred to as being "coupled," "coupled," or "connected" to another component, this includes not only a direct connection, but also an indirect connection where another component exists between them. Furthermore, when a component is referred to as "including" or "having" another component, this does not mean that the other component is excluded, but that the component can further include the other component, unless otherwise specified.

[0065] In this disclosure, terms such as "first" and "second" are used only to distinguish one component from another component, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0066] In this disclosure, components that are distinguished from one another are used to clearly describe the characteristics of each component and do not necessarily mean that the components are separate. In other words, multiple components may be integrated into a single hardware or software unit, or a single component may be distributed into multiple hardware or software units. Therefore, even if not otherwise specified, such integrated or distributed embodiments are also included within the scope of this disclosure.

[0067] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, an embodiment consisting of a subset of the components described in one embodiment is also within the scope of this disclosure. Furthermore, an embodiment including other components in addition to the components described in various embodiments is also within the scope of this disclosure.

[0068] The present disclosure relates to image encoding and decoding, and terms used in this disclosure may have their ordinary meaning in the technical field to which the present disclosure belongs unless they are newly defined in this disclosure.

[0069] In this disclosure, a "picture" generally refers to a unit representing any one image in a specific time period, and a slice / tile is a coding unit constituting a part of a picture, and one picture may be composed of one or more slices / tiles. Furthermore, a slice / tile may include one or more coding tree units (CTUs).

[0070] In this disclosure, "pixel" or "pel" may refer to the smallest unit constituting one picture (or image). Also, "sample" may be used as a term corresponding to pixel. A sample may generally indicate a pixel or a pixel value, may indicate only a pixel / pixel value of a luma component, or may indicate only a pixel / pixel value of a chroma component.

[0071] In this disclosure, the term "unit" may refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information related to that region. The term "unit" may be used interchangeably with terms such as "sample array," "block," or "area," depending on the situation. In general, an M×N block may include a set (or array) of samples or transform coefficients consisting of M columns and N rows.

[0072] In the present disclosure, a "current block" may refer to any one of a "current coding block," a "current coding unit," a "block to be coded," a "block to be decoded," or a "block to be processed." When prediction is performed, a "current block" may refer to a "current predicted block" or a "block to be predicted." When transformation (inverse transformation) / quantization (inverse quantization) is performed, a "current block" may refer to a "current transformed block" or a "block to be transformed." When filtering is performed, a "current block" may refer to a "block to be filtered."

[0073] Furthermore, in this disclosure, unless explicitly stated as a chroma block, the term "current block" may refer to a block including both a luma component block and a chroma component block, or the "luma block of the current block." The chroma block of the current block may be explicitly expressed as a "chroma block" or a "current chroma block," including the explicit description of the chroma block.

[0074] In the present disclosure, " / " and "," can be interpreted as "and / or." For example, "A / B" and "A, B" can be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" can mean "at least one of A, B, and / or C."

[0075] In this disclosure, "or" can be interpreted as "and / or." For example, "A or B" can mean 1) only "A," 2) only "B," or 3) "A and B." Alternatively, in this disclosure, "or" can mean "additionally or alternatively."

[0076] Video Coding System Overview

[0077] FIG. 1 is a diagram illustrating a video coding system according to this disclosure.

[0078] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may transmit encoded video and / or image information or data to the decoding device 20 in a file or streaming format via a digital storage medium or a network.

[0079] An encoding device 10 according to an embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to an embodiment may include a reception unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The reception unit 21 may be included in the decoding unit 22. The rendering unit 23 may include a display unit, which may be configured as a separate device or an external component.

[0080] The video source generation unit 11 can acquire video / images through a video / image capture, synthesis, or generation process. The video source generation unit 11 can include a video / image capture device and / or a video / image generation device. The video / image capture device can include, for example, one or more cameras, a video / image archive containing previously captured video / images, etc. The video / image generation device can include, for example, a computer, a tablet, a smartphone, etc., and can (electronically) generate video / images. For example, virtual video / images can be generated via a computer, etc., in which case the video / image capture process can be replaced with a process in which related data is generated.

[0081] The encoder 12 may encode the input video / image. The encoder 12 may perform a series of steps such as prediction, transformation, and quantization for compression and coding efficiency. The encoder 12 may output the encoded data (encoded video / image information) in a bitstream format.

[0082] The transmitter 13 may transmit the encoded video / image information or data output in a bitstream format to the receiver 21 of the decoding device 20 in a file or streaming format via a digital storage medium or a network. The digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. The transmitter 13 may include elements for generating a media file in a predetermined file format and elements for transmitting via a broadcasting / communication network. The receiver 21 may extract / receive the bitstream from the storage medium or network and transmit it to the decoder 22.

[0083] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transformation, and prediction corresponding to the operations of the encoding unit 12.

[0084] The rendering unit 23 can render the decoded video / images, and the rendered video / images can be displayed via the display unit.

[0085] Overview of the image encoding device

[0086] FIG. 2 is a diagram schematically illustrating an image encoding device to which an embodiment of the present disclosure can be applied.

[0087] 2, the image encoding device 100 may include an image division unit 110, a subtraction unit 115, a transform unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transform unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter prediction unit 180, an intra prediction unit 185, and an entropy encoding unit 190. The inter prediction unit 180 and the intra prediction unit 185 may be collectively referred to as a "prediction unit." The transform unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transform unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0088] Depending on the embodiment, all or at least some of the components constituting the image encoding device 100 may be realized by a single hardware component (e.g., an encoder or a processor). Also, the memory 170 may include a decoded picture buffer (DPB) and may be realized by a digital storage medium.

[0089] The image division unit 110 may divide an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. As an example, the processing units may be called coding units (CUs). The coding units may be obtained by recursively dividing a coding tree unit (CTU) or a largest coding unit (LCU) using a QT / BT / TT (quad-tree / binary-tree / ternary-tree) structure. For example, one coding unit may be divided into multiple coding units at deeper depths based on a quad-tree structure, a binary-tree structure, and / or a ternary-tree structure. To divide the coding units, the quad-tree structure may be applied first, and then the binary-tree structure and / or the ternary-tree structure may be applied later. The coding procedure according to the present disclosure may be performed based on the final coding unit that is not further divided. The maximum coding unit may be used as the final coding unit, or a lower-depth coding unit obtained by dividing the maximum coding unit may be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or reconstruction, which will be described later. As another example, a processing unit of the coding procedure may be a prediction unit (PU) or a transform unit (TU). The prediction unit and the transform unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0090] The prediction unit (inter prediction unit 180 or intra prediction unit 185) may perform prediction on a current block (current block) to generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block or CU. The prediction unit may generate various information related to prediction of the current block and transmit it to the entropy coding unit 190. The prediction information may be coded by the entropy coding unit 190 and output in a bitstream format.

[0091] The intra prediction unit 185 may predict the current block by referring to samples in the current picture. The referenced samples may be located in the neighborhood of the current block or may be located far away from the current block according to the intra prediction mode and / or intra prediction technique. The intra prediction modes may include a plurality of non-directional modes and a plurality of directional modes. The non-directional modes may include, for example, a DC mode and a planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the granularity of the prediction direction. However, this is merely an example, and more or less directional prediction modes may be used depending on the settings. The intra prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to neighboring blocks.

[0092] The inter prediction unit 180 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation between the motion information of neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on the inter prediction direction (e.g., L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be called a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block may be called a collocated picture (colPic). For example, the inter predictor 180 may construct a motion information candidate list based on neighboring blocks and generate information indicating which candidate is used to derive a motion vector and / or a reference picture index for the current block. Inter prediction may be performed based on various prediction modes. For example, in the case of skip mode and merge mode, the inter predictor 180 may use motion information of neighboring blocks as motion information for the current block. In the case of skip mode, unlike in merge mode, a residual signal may not be transmitted.In the case of a motion vector prediction (MVP) mode, the motion vector of a neighboring block is used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding a motion vector difference and an indicator for the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[0093] The predictor may generate a prediction signal based on various prediction methods and / or prediction techniques, which will be described later. For example, the predictor may apply intra prediction or inter prediction to predict the current block, or may simultaneously apply intra prediction and inter prediction. A prediction method that simultaneously applies intra prediction and inter prediction to predict the current block may be referred to as combined inter and intra prediction (CIIP). The predictor may also perform intra block copy (IBC) to predict the current block. Intra block copy can be used for content image / video coding, such as screen content coding (SCC), for games. IBC is a method of predicting a current block using an already reconstructed reference block in a current picture that is located a predetermined distance away from the current block. When IBC is applied, the position of the reference block in the current picture may be coded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but can be performed similarly to inter prediction in that a reference block is derived within the current picture. That is, the IBC may use at least one of the inter prediction techniques described in this disclosure.

[0094] The prediction signal generated by the prediction unit may be used to generate a restored signal or a residual signal. The subtraction unit 115 may subtract the prediction signal (predicted block, predicted sample array) output from the prediction unit from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual block, residual sample array). The generated residual signal may be transmitted to the conversion unit 120.

[0095] The transform unit 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditionally non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph representing inter-pixel relationship information. The CNT refers to a transform obtained based on a predicted signal generated using all previously reconstructed pixels. The transform process may be applied to pixel blocks having the same square size or to non-square blocks of variable size.

[0096] The quantization unit 130 may quantize the transform coefficients and transmit the quantized transform coefficients to the entropy coding unit 190. The entropy coding unit 190 may encode the quantized signal (information about the quantized transform coefficients) and output the encoded signal in a bitstream format. The information about the quantized transform coefficients may be referred to as residual information. The quantization unit 130 may rearrange the quantized transform coefficients in a block format into a one-dimensional vector format based on a coefficient scan order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector format.

[0097] The entropy coding unit 190 may perform various coding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy coding unit 190 may also code information necessary for video / image restoration (e.g., values ​​of syntax elements) together with or separately from the quantized transform coefficients. The coded information (e.g., coded video / image information) may be transmitted or stored in a bitstream format in network abstraction layer (NAL) units. The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may also include general constraint information. The signaling information, transmitted information and / or syntax elements mentioned in this disclosure may be encoded through the above-described encoding procedure and included in the bitstream.

[0098] The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcasting network and / or a communication network, and the digital storage medium may include various storage media such as a USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitting unit (not shown) that transmits and / or a storing unit (not shown) that stores the signal output from the entropy encoding unit 190 may be provided as an internal / external element of the image encoding device 100, or the transmitting unit may be provided as a component of the entropy encoding unit 190.

[0099] The quantized transform coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying inverse quantization and inverse transform to the quantized transform coefficients via the inverse quantization unit 140 and the inverse transform unit 150.

[0100] The adder 155 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter prediction unit 180 or the intra prediction unit 185. When there is no residual for the current block to be processed, such as when a skip mode is applied, the predicted block may be used as the reconstructed block. The adder 155 may be referred to as a reconstruction unit or a reconstructed block generation unit. The generated reconstructed signal may be used for intra prediction of the next current block to be processed in the current picture, and may also be used for inter prediction of the next picture after filtering, as will be described later.

[0101] The filtering unit 160 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 160 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc. The filtering unit 160 may generate various information related to filtering and transmit it to the entropy coding unit 190, as will be described later in connection with each filtering method. The filtering information may be coded by the entropy coding unit 190 and output in a bitstream format.

[0102] The modified reconstructed picture transmitted to the memory 170 can be used as a reference picture in the inter prediction unit 180. When inter prediction is applied through this, the image encoding device 100 can avoid a prediction mismatch between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0103] The DPB in the memory 170 may store modified reconstructed pictures for use as reference pictures in the inter predictor 180. The memory 170 may store motion information of blocks from which motion information in the current picture is derived (or coded) and / or motion information of already reconstructed intra-picture blocks. The stored motion information may be transmitted to the inter predictor 180 to be used as motion information of spatially surrounding blocks or temporally surrounding blocks. The memory 170 may store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra predictor 185.

[0104] Overview of the image decoding device

[0105] FIG. 3 is a diagram schematically illustrating an image decoding device to which an embodiment of the present disclosure can be applied.

[0106] 3, the image decoding apparatus 200 may include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a "prediction unit." The inverse quantization unit 220 and the inverse transform unit 230 may be included in a residual processing unit.

[0107] Depending on the embodiment, all or at least some of the components constituting the image decoding device 200 may be realized by a single hardware component (e.g., a decoder or a processor). Also, the memory 170 may include a DPB and may be realized by a digital storage medium.

[0108] The image decoding device 200, which receives a bitstream including video / image information, can reconstruct an image by performing a process corresponding to the process performed by the image encoding device 100 of FIG. 2. For example, the image decoding device 200 can perform decoding using a processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The reconstructed image signal decoded and output by the image decoding device 200 can be reproduced by a reproduction device (not shown).

[0109] The image decoding apparatus 200 may receive a signal output from the image encoding apparatus of FIG. 2 in a bitstream format. The received signal may be decoded via an entropy decoding unit 210. For example, the entropy decoding unit 210 may parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information on various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). The video / image information may also include general constraint information. The image decoding apparatus may further use the information on the parameter sets and / or the general constraint information to decode an image. The signaling information, received information, and / or syntax elements referred to in the present disclosure may be obtained from the bitstream by being decoded via the decoding procedure. For example, the entropy decoding unit 210 may decode information in a bitstream based on a coding method such as Exponential-Golomb coding, CAVLC, or CABAC, and output values ​​of syntax elements required for image restoration and quantized values ​​of transform coefficients related to residuals. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using information on the syntax element to be decoded and decoded information on neighboring blocks and the block to be decoded, or information on symbols / bins decoded in a previous step, predicts the occurrence probability of the bins based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. After determining the context model, the CABAC entropy decoding method may update the context model using information on the decoded symbol / bin for the context model of the next symbol / bin.Among the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction units (inter prediction unit 260 and intra prediction unit 265), and residual values ​​entropy decoded by the entropy decoding unit 210, i.e., quantized transform coefficients and related parameter information, may be input to the inverse quantization unit 220. Also, among the information decoded by the entropy decoding unit 210, information related to filtering may be provided to the filtering unit 240. Meanwhile, a receiving unit (not shown) for receiving a signal output from the image encoding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0110] Meanwhile, the image decoding apparatus according to the present disclosure may be referred to as a video / image / picture decoding apparatus. The image decoding apparatus may include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder 235, a filtering unit 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265.

[0111] The inverse quantization unit 220 may inverse quantize the quantized transform coefficients and output the transform coefficients. The inverse quantization unit 220 may rearrange the quantized transform coefficients in a two-dimensional block format. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the image encoding device. The inverse quantization unit 220 may perform inverse quantization on the quantized transform coefficients using a quantization parameter (e.g., quantization step size information) to obtain transform coefficients.

[0112] The inverse transform unit 230 can inversely transform the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0113] The prediction unit may perform prediction on a current block and generate a predicted block including prediction samples for the current block. The prediction unit may determine whether intra prediction or inter prediction is applied to the current block based on information about the prediction output from the entropy decoding unit 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0114] The prediction unit can generate a prediction signal based on various prediction methods (techniques) described below, as described in the description of the prediction unit of the image encoding device 100.

[0115] The intra predictor 265 may predict the current block by referring to samples in the current picture. The description of the intra predictor 185 may also be applied to the intra predictor 265.

[0116] The inter prediction unit 260 may derive a predicted block for a current block based on a reference block (reference sample array) identified by a motion vector on a reference picture. To reduce the amount of motion information transmitted in inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on correlations between motion information of neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include information on an inter prediction direction (e.g., L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may construct a motion information candidate list based on the neighboring blocks and derive a motion vector and / or a reference picture index for the current block based on received candidate selection information. Inter prediction may be performed based on various prediction modes (techniques), and the prediction information may include information indicating the inter prediction mode (technique) for the current block.

[0117] The adder 235 may generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to a prediction signal (predicted block, predicted sample array) output from a prediction unit (including the inter prediction unit 260 and / or intra prediction unit 265). When there is no residual for the current block, such as when a skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 also applies to the adder 235. The adder 235 may also be referred to as a reconstruction unit or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next current block in the current picture, and may also be used for inter prediction of the next picture via filtering, as described below.

[0118] The filtering unit 240 may apply filtering to the reconstructed signal to improve subjective / objective image quality. For example, the filtering unit 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and may store the modified reconstructed picture in the memory 250, specifically, in a DPB of the memory 250. The various filtering methods may include, for example, deblocking filtering, sample adaptive offset, an adaptive loop filter, a bilateral filter, etc.

[0119] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter predictor 260. The memory 250 can store motion information of a block from which motion information in the current picture is derived (or decoded) and / or motion information of a block in an already reconstructed picture. The stored motion information can be transmitted to the inter predictor 260 to be used as motion information of a spatially surrounding block or a temporally surrounding block. The memory 250 can store reconstructed samples of reconstructed blocks in the current picture and transmit them to the intra predictor 265.

[0120] In this specification, the embodiments described for the filtering unit 160, inter prediction unit 180 and intra prediction unit 185 of the image encoding device 100 can also be applied in a similar or corresponding manner to the filtering unit 240, inter prediction unit 260 and intra prediction unit 265 of the image decoding device 200, respectively.

[0121] Overview of CTU division

[0122] As described above, a coding unit can be obtained by recursively dividing a coding tree unit (CTU) or a largest coding unit (LCU) according to a QT / BT / TT (quad-tree / binary-tree / ternary-tree) structure. For example, a CTU can be first divided into a quad-tree structure. Then, the leaf nodes of the quad-tree structure can be further divided according to a multi-type tree structure.

[0123] Quadtree division refers to dividing the current CU (or CTU) into four equal parts. By quadtree division, the current CU can be divided into four CUs with the same width and height. If the current CU is not further divided into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. A CU corresponding to a leaf node of the quadtree structure is not further divided and can be used as the final coding unit described above. Alternatively, a CU corresponding to a leaf node of the quadtree structure can be further divided into four parts according to a multi-type tree structure.

[0124] 4 is a diagram showing the types of division of a block by a multi-type tree structure. Division by a multi-type tree structure can include two divisions by a binary tree structure and two divisions by a ternary tree structure.

[0125] The two divisions based on the binary tree structure can include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) refers to a division that divides the current CU into two equal parts vertically. As shown in FIG. 4, vertical binary splitting can generate two CUs each having the same height as the current CU and half the width of the current CU. Horizontal binary splitting (SPLIT_BT_HOR) refers to a division that divides the current CU into two equal parts horizontally. As shown in FIG. 4, horizontal binary splitting can generate two CUs each having a height half the height of the current CU and a width equal to the width of the current CU.

[0126] The two divisions based on the ternary tree structure can include vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). Vertical ternary splitting (SPLIT_TT_VER) divides the current CU vertically at a ratio of 1:2:1. As shown in FIG. 4, the vertical ternary splitting can generate two CUs each having the same height as the current CU and a width equal to one-quarter of the current CU's width, and a CU each having the same height as the current CU and a width equal to half the current CU's width. Horizontal ternary splitting (SPLIT_TT_HOR) divides the current CU horizontally at a ratio of 1:2:1. As shown in FIG. 4, the horizontal ternary splitting can generate two CUs each having a height equal to one-quarter of the current CU's height and a width equal to one CU's width.

[0127] FIG. 5 is a diagram illustrating a signaling mechanism for partitioning information of a quadtree with nested multi-type tree structure according to the present disclosure.

[0128] Here, the CTU is treated as the root node of the quadtree and is first split into a quadtree structure. Information (e.g., qt_split_flag) indicating whether quadtree splitting is performed on the current CU (CTU or quadtree node (QT_node)) can be signaled. For example, if qt_split_flag is a first value (e.g., '1'), the current CU can be split into a quadtree. On the other hand, if qt_split_flag is a second value (e.g., '0'), the current CU is not split into a quadtree but becomes a quadtree leaf node (QT_leaf_node). Each quadtree leaf node can then be further split into a multitype tree structure. That is, the quadtree leaf node can become a multitype tree node (MTT_node). A first flag (e.g., mtt_split_cu_flag) can be signaled to indicate whether the current node is further split into a multitype tree structure. If the node is further split (e.g., the first flag is 1), a second flag (e.g., mtt_split_cu_vertical_flag) may be signaled to indicate the splitting direction. For example, if the second flag is 1, the splitting direction may be vertical, and if the second flag is 0, the splitting direction may be horizontal. Then, a third flag (e.g., mtt_split_cu_binary_flag) may be signaled to indicate whether the splitting type is a binary split type or a ternary split type. For example, if the third flag is 1, the splitting type may be a binary split type, and if the third flag is 0, the splitting type may be a ternary split type. A node of a multitype tree obtained by binary splitting or ternary splitting can be further split into a multitype tree structure. However, a node of a multitype tree cannot be split into a quadtree structure.If the first flag is 0, the corresponding node of the multitype tree is not further divided and becomes a leaf node (MTT_leaf_node) of the multitype tree. The CU corresponding to the leaf node of the multitype tree can be used as the final coding unit described above.

[0129] Based on the above mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1.

[0130] [Table 1]

[0131] One CTU may include a coding block of luma samples (hereinafter referred to as a "luma block") and two coding blocks of corresponding chroma samples (hereinafter referred to as "chroma blocks"). The above-mentioned coding tree scheme may be applied equally to the luma blocks and chroma blocks of the current CU, or may be applied separately. Specifically, the luma blocks and chroma blocks in one CTU may be divided into the same block tree structure, which may be referred to as a single tree (SINGLE_TREE). Alternatively, the luma blocks and chroma blocks in one CTU may be divided into separate block tree structures, which may be referred to as a dual tree (DUAL_TREE). In other words, when a CTU is divided into a dual tree, a block tree structure for the luma blocks and a block tree structure for the chroma blocks may exist separately. In this case, the block tree structure for the luma block may be referred to as a dual tree luma (DUAL_TREE_LUMA), and the block tree structure for the chroma block may be referred to as a dual tree chroma (DUAL_TREE_CHROMA). For P and B slices / tile groups, the luma block and the chroma block in one CTU may be restricted to have the same coding tree structure. However, for I slices / tile groups, the luma block and the chroma block may have separate block tree structures. If a separate block tree structure is applied, the luma CTB (Coding Tree Block) may be divided into CUs based on a specific coding tree structure, and the chroma CTB may be divided into chroma CUs based on another coding tree structure. That is, a CU in an I slice / tile group to which a separate block tree structure is applied may consist of a coding block of a luma component or a coding block of two chroma components.In addition, a CU in an I slice / tile group and a CU in a P or B slice / tile group to which the same block tree structure is applied can be composed of blocks of three color components (a luma component and two chroma components).

[0132] Although the above describes a quadtree coding tree structure with a multi-type tree, the structure in which a CU is divided is not limited to this. For example, the BT structure and the TT structure may be interpreted as concepts included in a Multiple Partitioning Tree (MPT) structure, and a CU may be interpreted as being divided by a QT structure and an MPT structure. In an example in which a CU is divided by a QT structure and an MPT structure, the division structure may be determined by signaling a syntax element (e.g., MPT_split_type) containing information on whether a leaf node of the QT structure is divided into several blocks and a syntax element (e.g., MPT_split_mode) containing information on whether the leaf node of the QT structure is divided vertically or horizontally.

[0133] In another example, CUs may be divided in a manner different from that of the QT structure, BT structure, or TT structure. That is, unlike the QT structure in which lower-depth CUs are divided into 1 / 4 the size of higher-depth CUs, the BT structure in which lower-depth CUs are divided into 1 / 2 the size of higher-depth CUs, or the TT structure in which lower-depth CUs are divided into 1 / 4 or 1 / 2 the size of higher-depth CUs, lower-depth CUs may be divided into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of higher-depth CUs, as the case may be, and the manner in which CUs are divided is not limited thereto.

[0134] Intra Prediction Overview

[0135] Intra prediction according to the present disclosure will be described below.

[0136] Intra prediction may refer to a prediction that generates prediction samples for a current block based on reference samples in a picture to which the current block belongs (hereinafter, referred to as the current picture). When intra prediction is applied to the current block, neighboring reference samples used for intra prediction of the current block may be derived. The neighboring reference samples of the current block may include samples adjacent to the left boundary of the current block having a size of nW×nH and a total of 2×nH samples adjacent to the bottom-left, samples adjacent to the top boundary of the current block and a total of 2×nW samples adjacent to the top-right, and one sample adjacent to the top-left. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0137] However, some of the neighboring reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder may construct neighboring reference samples to be used for prediction by substituting unavailable samples with available samples, or may construct neighboring reference samples to be used for prediction through interpolation of available samples.

[0138] When neighboring reference samples are derived, (i) a predicted sample can be derived based on an average or interpolation of neighboring reference samples of the current block, or (ii) the predicted sample can be derived based on a reference sample that exists in a specific (prediction) direction with respect to the predicted sample among the neighboring reference samples of the current block. (i) is sometimes called a non-directional mode or a non-angular mode, and (ii) is sometimes called a directional mode or an angular mode.

[0139] Alternatively, the predicted sample may be generated by interpolating a first neighboring sample located in a prediction direction of an intra prediction mode of the current block and a second neighboring sample located in the opposite direction based on a sample to be predicted of the current block among the neighboring reference samples. The above case may be referred to as linear interpolation intra prediction (LIP).

[0140] Alternatively, chroma prediction samples can be generated based on luma samples using a linear model, which is sometimes called LM (Linear Model) mode.

[0141] Alternatively, a temporal prediction sample of the current block may be derived based on filtered neighboring reference samples, and the prediction sample of the current block may be derived by weighted summing at least one reference sample derived according to the intra prediction mode among the existing neighboring reference samples, i.e., unfiltered neighboring reference samples, and the temporal prediction sample. In this case, it may be called Position Dependent Intra Prediction (PDPC).

[0142] In addition, a reference sample line with the highest prediction accuracy may be selected from multiple reference sample lines surrounding the current block, and a predicted sample may be derived using a reference sample located in a prediction direction from the selected line. In this case, information about the used reference sample line (e.g., intra_luma_ref_idx) may be coded into a bitstream and signaled. This may be referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction. If MRL is not applied, a reference sample may be derived from a reference sample line directly adjacent to the current block, and in this case, information about the reference sample line may not be signaled.

[0143] In addition, the current block may be divided into vertical or horizontal sub-partitions, and intra prediction may be performed for each sub-partition based on the same intra prediction mode. In this case, neighboring reference samples for intra prediction may be derived for each sub-partition. That is, in the encoding / decoding order, reconstructed samples of a previous sub-partition may be used as neighboring reference samples for the current sub-partition. In this case, the intra prediction mode for the current block is uniformly applied to the sub-partitions, but by deriving and using neighboring reference samples for each sub-partition, intra prediction performance may be improved as needed. This prediction method may be called intra sub-partitions (ISP) or ISP-based intra prediction.

[0144] The above-mentioned intra prediction techniques may be distinguished from directional or non-directional intra prediction modes and referred to by various terms such as the intra prediction type or additional intra prediction mode. For example, the intra prediction techniques (e.g., the intra prediction type or additional intra prediction mode) may include at least one of the above-mentioned LIP, LM, PDPC, MRL, and ISP. A general intra prediction method other than the specific intra prediction types such as LIP, LM, PDPC, MRL, and ISP may be referred to as a normal intra prediction type. The normal intra prediction type may be generally applied when the above-mentioned specific intra prediction types are not applied, and prediction may be performed based on the above-mentioned intra prediction mode. Meanwhile, post-processing filtering may be performed on the derived prediction samples as necessary.

[0145] Specifically, the intra prediction procedure may include an intra prediction mode / type determination step, a neighboring reference sample derivation step, and an intra prediction mode / type-based prediction sample derivation step. Furthermore, if necessary, a post-processing filtering step may be performed on the derived prediction samples.

[0146] FIG. 6 is a flow chart illustrating a video / image coding method based on intra prediction.

[0147] The encoding method of FIG. 6 may be performed by the image encoding apparatus of FIG. 2. Specifically, step S610 may be performed by the intra prediction unit 185, and step S620 may be performed by the residual processing unit. Specifically, step S620 may be performed by the subtraction unit 115. Step S630 may be performed by the entropy encoding unit 190. The prediction information of step S630 may be derived by the intra prediction unit 185, and the residual information of step S630 may be derived by the residual processing unit. The residual information is information about the residual sample. The residual information may include information about quantized transform coefficients for the residual sample. As described above, the residual sample may be derived into transform coefficients via the transform unit 120 of the image encoding apparatus, and the transform coefficients may be derived into quantized transform coefficients via the quantization unit 130. Information about the quantized transform coefficients may be coded by the entropy encoding unit 190 through a residual coding procedure.

[0148] The image encoding apparatus may perform intra prediction on a current block (S610). The image encoding apparatus may determine an intra prediction mode / type for the current block, derive neighboring reference samples for the current block, and then generate predicted samples within the current block based on the intra prediction mode / type and the neighboring reference samples. Here, the steps of determining the intra prediction mode / type, deriving neighboring reference samples, and generating predicted samples may be performed simultaneously, or one step may be performed before the other steps.

[0149] FIG. 7 is a diagram illustrating an example configuration of the intra prediction unit 185 according to the present disclosure.

[0150] 7, the intra prediction unit 185 of the image encoding device may include an intra prediction mode / type determination unit 186, a reference sample derivation unit 187, and / or a prediction sample derivation unit. The intra prediction mode / type determination unit 186 may determine the intra prediction mode / type for the current block. The reference sample derivation unit 187 may derive neighboring reference samples for the current block, and the prediction sample derivation unit 188 may derive prediction samples for the current block. Meanwhile, although not shown, if a prediction sample filtering procedure (not shown) is performed, the intra prediction unit 185 may further include a prediction sample filter unit (not shown).

[0151] The image encoding apparatus may determine a mode / type to be applied to the current block from among a plurality of intra prediction modes / types, and may compare rate-distortion costs (RD costs) for the intra prediction modes / types to determine an optimal intra prediction mode / type for the current block.

[0152] Meanwhile, the image coding apparatus may also perform a prediction sample filtering procedure. The prediction sample filtering may be called post-filtering. The prediction sample filtering procedure may filter some or all of the prediction samples. In some cases, the prediction sample filtering procedure may be omitted.

[0153] 6, the image encoding apparatus may generate residual samples for the current block based on predicted samples or filtered predicted samples (S620). The image encoding apparatus may derive the residual samples by subtracting the predicted samples from original samples of the current block. That is, the image encoding apparatus may derive residual sample values ​​by subtracting corresponding predicted sample values ​​from original sample values.

[0154] The image encoding apparatus may encode image information including information related to the intra prediction (prediction information) and residual information related to the residual samples (S630). The prediction information may include the intra prediction mode information and / or the intra prediction technique information. The image encoding apparatus may output the encoded image information in a bitstream format. The output bitstream may be transmitted to an image decoding apparatus via a storage medium or a network.

[0155] The residual information may include a residual coding syntax, which will be described later. The image encoding apparatus may transform / quantize the residual samples to derive quantized transform coefficients. The residual information may include information on the quantized transform coefficients.

[0156] Meanwhile, as described above, the image coding apparatus can generate a reconstructed picture (including reconstructed samples and reconstructed blocks). To this end, the image coding apparatus can again inverse-quantize / inverse-transform the quantized transform coefficients to derive (modified) residual samples. The reason for again performing inverse-quantization / inverse-transformation on the residual samples after transforming / quantizing them is to derive residual samples that are identical to the residual samples derived in the image decoding apparatus. The image coding apparatus can generate a reconstructed block including reconstructed samples for the current block based on the predicted samples and the (modified) residual samples. A reconstructed picture for the current picture can be generated based on the reconstructed block. As described above, an in-loop filtering procedure or the like can further be applied to the reconstructed picture.

[0157] FIG. 8 is a flowchart illustrating a video / image decoding method based on intra prediction.

[0158] The image decoding device can perform operations corresponding to those performed by the image coding device.

[0159] The decoding method of FIG. 8 may be performed by the image decoding apparatus of FIG. 3. Steps S810 to S830 may be performed by the intra prediction unit 265, and the prediction information of S810 and the residual information of S840 may be obtained from a bitstream by the entropy decoding unit 210. The residual processing unit of the image decoding apparatus may derive residual samples for the current block based on the residual information (S840). Specifically, the inverse quantization unit 220 of the residual processing unit may derive transform coefficients by performing inverse quantization based on the quantized transform coefficients derived based on the residual information, and the inverse transform unit 230 of the residual processing unit may derive residual samples for the current block by performing inverse transform on the transform coefficients. Step S850 may be performed by the adder 235 or a reconstruction unit.

[0160] Specifically, the image decoding apparatus may derive an intra prediction mode / type for a current block based on received prediction information (intra prediction mode / type information) (S810). The image decoding apparatus may also derive neighboring reference samples for the current block (S820). The image decoding apparatus may generate prediction samples within the current block based on the intra prediction mode / type and the neighboring reference samples (S830). In this case, the image decoding apparatus may perform a prediction sample filtering procedure. The prediction sample filtering may be referred to as post-filtering. Some or all of the prediction samples may be filtered by the prediction sample filtering procedure. In some cases, the prediction sample filtering procedure may be omitted.

[0161] The image decoding apparatus may generate residual samples for the current block based on the received residual information (S840). The image decoding apparatus may generate reconstructed samples for the current block based on the predicted samples and the residual samples, and derive a reconstructed block including the reconstructed samples (S850). A reconstructed picture for the current picture may be generated based on the reconstructed block. As described above, an in-loop filtering procedure may be further applied to the reconstructed picture.

[0162] FIG. 9 is a diagram illustrating an example configuration of the intra prediction unit 265 according to the present disclosure.

[0163] As shown in FIG. 9, the intra prediction unit 265 of the image decoding apparatus may include an intra prediction mode / type determination unit 266, a reference sample derivation unit 267, and a prediction sample derivation unit 268. The intra prediction mode / type determination unit 266 determines the intra prediction mode / type for the current block based on intra prediction mode / type information generated and signaled by the intra prediction mode / type determination unit 186 of the image encoding apparatus, and the reference sample derivation unit 266 may derive neighboring reference samples of the current block from a reconstructed reference region within the current picture. The prediction sample derivation unit 268 may derive prediction samples of the current block. Meanwhile, although not shown, if the above-described prediction sample filtering procedure is performed, the intra prediction unit 265 may further include a prediction sample filter unit (not shown).

[0164] The intra prediction mode information may include, for example, flag information (e.g., intra_luma_mpm_flag) indicating whether a most probable mode (MPM) or a remaining mode is applied to the current block. If the MPM is applied to the current block, the intra prediction mode information may further include index information (e.g., intra_luma_mpm_idx) indicating one of the intra prediction mode candidates (MPM candidates). The intra prediction mode candidates (MPM candidates) may be configured from an MPM candidate list or an MPM list. If the MPM is not applied to the current block, the intra prediction mode information may further include remaining mode information (e.g., intra_luma_mpm_remainder) indicating one of the remaining intra prediction modes excluding the intra prediction mode candidates (MPM candidates). The image decoding apparatus may determine the intra prediction mode of the current block based on the intra prediction mode information.

[0165] Furthermore, the intra prediction technique information may be implemented in various forms. For example, the intra prediction technique information may include intra prediction technique index information indicating one of the intra prediction techniques. For another example, the intra prediction technique information may include at least one of reference sample line information (e.g., intra_luma_ref_idx) indicating whether the MRL is applied to the current block and, if so, which reference sample line is used, ISP flag information (e.g., intra_subpartitions_mode_flag) indicating whether the ISP is applied to the current block, ISP type information (e.g., intra_subpartitions_split_flag) indicating a subpartition division type if the ISP is applied, flag information indicating whether PDPC is applied, or flag information indicating whether LIP is applied. In the present disclosure, the ISP flag information may be referred to as an ISP application indicator.

[0166] The intra prediction mode information and / or the intra prediction technique information may be encoded / decoded using a coding method described in this disclosure. For example, the intra prediction mode information and / or the intra prediction technique information may be encoded / decoded using entropy coding (e.g., CABAC, CAVLC) based on a truncated (rice) binary code.

[0167] The intra-prediction mode / type decision method according to the present disclosure will be described in more detail below.

[0168] When intra prediction is applied to the current block, the intra prediction mode applied to the current block may be determined using the intra prediction modes of neighboring blocks. For example, the image decoding apparatus may construct an MPM (most probable mode) list derived based on the intra prediction modes of neighboring blocks (e.g., left and / or upper neighboring blocks) of the current block and additional candidate modes, and select one of the MPM candidates in the MPM list based on the received MPM index. Alternatively, the image decoding apparatus may select one of the remaining intra prediction modes not included in the MPM list based on remaining intra prediction mode information. For example, whether the intra prediction mode applied to the current block is among the MPM candidates (i.e., included in the MPM list) or among the remaining modes may be indicated based on an MPM flag (e.g., intra_luma_mpm_flag). A value of 1 for the mpm flag may indicate that the intra prediction mode for the current block is included in the mpm candidates (mpm list), and a value of 0 for the mpm flag may indicate that the intra prediction mode for the current block is not included in the mpm candidates (mpm list). The mpm index may be signaled in the form of an mpm_idx or intra_luma_mpm_idx syntax element, and the remaining intra prediction mode information may be signaled in the form of a rem_intra_luma_pred_mode or intra_luma_mpm_remainder syntax element. For example, the remaining intra prediction mode information may indicate one of the remaining intra prediction modes that are not included in the mpm candidates (mpm list) among all intra prediction modes by indexing them in order of prediction mode number. The intra prediction mode may be an intra prediction mode for a luma component (sample).Hereinafter, the intra prediction mode information may include at least one of the mpm flag (e.g., intra_luma_mpm_flag), the mpm index (e.g., mpm_idx or intra_luma_mpm_idx), and the remaining intra prediction mode information (rem_intra_luma_pred_mode or intra_luma_mpm_remainder). In the present disclosure, the MPM list may be referred to by various terms such as an MPM candidate list, a candModeList, etc.

[0169] FIG. 10 is a flowchart showing an intra-prediction mode signaling procedure in an image encoding device.

[0170] 10, the image encoding apparatus may construct an MPM list for a current block (S1010). The MPM list may include candidate intra prediction modes (MPM candidates) that are likely to be applied to the current block. The MPM list may also include intra prediction modes of neighboring blocks, and may further include specific intra prediction modes according to a predetermined method.

[0171] The image encoding apparatus may determine an intra prediction mode for a current block (S1020). The image encoding apparatus may perform prediction based on various intra prediction modes and may perform rate-distortion optimization (RDO) based on the prediction to determine an optimal intra prediction mode. In this case, the image encoding apparatus may determine the optimal intra prediction mode using only MPM candidates included in the MPM list, or may determine the optimal intra prediction mode using not only the MPM candidates included in the MPM list but also the remaining intra prediction modes. Specifically, for example, if the intra prediction type of the current block is not a normal intra prediction type but a specific type (e.g., LIP, MRL, or ISP), the image encoding apparatus may determine the optimal intra prediction mode using only the MPM candidates. That is, in this case, the intra prediction mode for the current block can be determined only from the MPM candidates, and in this case, the mpm flag may not be coded / signaled. In the case of the specific type, the image decoding apparatus can infer that the mpm flag is 1 without receiving separate signaling of the mpm flag.

[0172] Meanwhile, in general, if the intra prediction mode of the current block is one of the MPM candidates in the MPM list, the image coding apparatus may generate an MPM index (mpm idx) indicating one of the MPM candidates. If the intra prediction mode of the current block is not in the MPM list, the image coding apparatus may generate remaining intra prediction mode information indicating the same mode as the intra prediction mode of the current block from the remaining intra prediction modes not included in the MPM list.

[0173] The image encoding apparatus may encode intra prediction mode information and output the encoded intra prediction mode information in a bitstream format (S1030). The intra prediction mode information may include the above-described mpm flag, mpm index, and / or remaining intra prediction mode information. Generally, the mpm index and remaining intra prediction mode information have an alternative relationship and are therefore not simultaneously signaled to indicate the intra prediction mode for one block. That is, when the mpm flag value is 1, the mpm index may be signaled, and when the mpm flag value is 0, the remaining intra prediction mode information may be signaled. However, as described above, when a specific intra prediction type is applied to the current block, the mpm flag may not be signaled, but its value may be inferred to 1, and only the mpm index may be signaled. That is, in this case, the intra prediction mode information may include only the mpm index.

[0174] In the example shown in FIG. 10, S1020 is illustrated as being performed after S1010, but this is just one example, and S1020 may be performed before S1010 or may be performed simultaneously with S1010.

[0175] FIG. 11 is a flowchart showing the procedure for determining an intra-prediction mode in the image decoding apparatus.

[0176] The image decoding apparatus can determine the intra prediction mode of the current block based on the intra prediction mode information determined and signaled by the image encoding apparatus.

[0177] 11, the image decoding apparatus may obtain intra prediction mode information from a bitstream (S1110). The intra prediction mode information may include at least one of an mpm flag, an mpm index, and a remaining intra prediction mode, as described above.

[0178] The image decoding apparatus may construct an MPM list (S1120). The MPM list is constructed in the same manner as the MPM list in the image encoding apparatus. That is, the MPM list may include intra prediction modes of neighboring blocks and may further include a specific intra prediction mode according to a predetermined method.

[0179] In the example shown in FIG. 11, S1120 is illustrated as being performed after S1110, but this is just one example, and S1120 may be performed before S1110 or may be performed simultaneously with S1110.

[0180] The image decoding apparatus determines the intra prediction mode of the current block based on the MPM list and the intra prediction mode information (S1130). Step S1130 will be described in more detail with reference to FIG.

[0181] FIG. 12 is a flowchart for explaining the intra-prediction mode derivation procedure in more detail.

[0182] Steps S1210 and S1220 in Fig. 12 can correspond to steps S1110 and S1120 in Fig. 11. Therefore, a detailed description of steps S1210 and S1220 will be omitted.

[0183] The image decoding apparatus obtains intra prediction mode information from a bitstream, constructs an MPM list (S1210, S1220), and then determines whether a predetermined condition exists (S1230). Specifically, as shown in FIG. 12, if the value of the mpm flag is 1 (Yes in S1230), the image decoding apparatus may derive the candidate indicated by the mpm index from among the MPM candidates in the MPM list as the intra prediction mode of the current block (S1240). As another example, if the value of the mpm flag is 0 (No in S1230), the image decoding apparatus may derive the intra prediction mode indicated by the remaining intra prediction mode information from among the remaining intra prediction modes not included in the MPM list as the intra prediction mode of the current block (S1250). On the other hand, as another example, if the intra prediction type of the current block is a specific type (e.g., LIP, MRL, or ISP, etc.) (Yes in S1230), the image decoding device can also derive the candidate indicated by the mpm index in the MPM list as the intra prediction mode of the current block without checking the mpm flag (S1240).

[0184] On the other hand, when intra prediction is performed on the current block, prediction can be performed on the luma component block (luma block) and prediction can be performed on the chroma component block (chroma block) of the current block, and in this case, the intra prediction mode for the chroma component (chroma block) can be set separately from the intra prediction mode for the luma component (luma block).

[0185] For example, an intra-prediction mode for a chroma component may be indicated based on intra-chroma prediction mode information, which may be signaled in the form of an Intra_chroma_pred_mode syntax element. As an example, the intra-chroma prediction mode information may indicate one of candidate modes including at least one of a Planar mode, a DC mode, a vertical mode, a Horizontal mode, a Derived Mode (DM), an L_CCLM mode, a T_CCLM mode, and a LT_CCLM mode. DM may also be referred to as a direct mode. CCLM may also be referred to as an LM mode.

[0186] FIG. 13 is a diagram illustrating intra-prediction directions according to one embodiment of the present disclosure.

[0187] For example, the intra prediction modes may include two non-directional intra prediction modes and 33 directional intra prediction modes. The non-directional intra prediction modes may include a planar intra prediction mode and a DC intra prediction mode, and the directional intra prediction modes may include intra prediction modes No. 2 to No. 34. The planar intra prediction mode may be referred to as a planar mode, and the DC intra prediction mode may be referred to as a DC mode.

[0188] Alternatively, in order to capture any edge direction presented in a natural video, the intra prediction modes may include two non-directional intra prediction modes and 65 extended directional intra prediction modes, as shown in Figure 14. The non-directional intra prediction modes may include a planar mode and a DC mode, and the directional intra prediction modes may include intra prediction modes 2 to 66. The extended intra prediction modes may be applied to blocks of any size and may be applied to both luma components (luma blocks) and chroma components (chroma blocks).

[0189] Alternatively, the intra prediction modes may include two non-directional intra prediction modes and 129 directional intra prediction modes. The non-directional intra prediction modes may include a planar mode and a DC mode, and the directional intra prediction modes may include intra prediction modes 2 to 130.

[0190] Meanwhile, the intra prediction modes may further include a cross-component linear model (CCLM) mode for chroma samples in addition to the above-described intra prediction modes. The CCLM modes are classified into L_CCLM, T_CCLM, and LT_CCLM depending on whether the left sample, the top sample, or both are considered to derive the LM parameters, and may be applied only to the chroma components.

[0191] The intra prediction modes can be indexed, for example, as shown in Table 2 below.

[0192] [Table 2]

[0193] FIG. 14 is a diagram illustrating intra-prediction directions according to another embodiment of the present disclosure. In FIG. 14, the dashed line direction indicates a wide-angle mode that is applied only to blocks other than squares. As shown in FIG. 14, in order to capture any edge direction presented in natural video, intra-prediction modes according to one embodiment may include 93 directional intra-prediction modes along with two non-directional intra-prediction modes. The non-directional intra-prediction modes may include a planar mode and a DC mode. The directional intra-prediction modes may include intra-prediction modes numbered 2 to 80 and -1 to -14, as indicated by the arrows in FIG. 15. The planar mode may be denoted as INTRA_PLANAR, and the DC mode may be denoted as INTRA_DC. The directional intra-prediction modes may be denoted as INTRA_ANGULAR-14 to INTRA_ANGULAR-1 and INTRA_ANGULAR2 to INTRA_ANGULAR80.

[0194] Alternatively, the MPM list may be configured to include N MPMs, where N may be 5 or 6.

[0195] To construct the MPM list, three modes can be considered, as described below.

[0196] -Default intra modes

[0197] -Neighbor intra modes

[0198] -Derived intra modes

[0199] For the peripheral intra mode, two peripheral blocks can be considered: a left peripheral block (A) and an upper peripheral block (B).

[0200] Also, to construct the MPM list, the following initialized default MPMs can be considered:

[0201] Default 6 MPM modes={A, Planar(0) or DC(1), Vertical(50), HOR(18), VER-4(46), VER+4(54)}

[0202] An MPM list can be constructed by performing a pruning process on the two surrounding intra modes. If the two surrounding intra modes are identical and the surrounding intra modes are greater than the DC(1) mode, the MPM list can include the {A, Planar, DC} mode and three derived intra modes. The three derived intra modes can be obtained by adding a predetermined offset value to the surrounding intra modes and / or performing a modulo operation. If the two surrounding intra modes are different, the two surrounding intra modes can be assigned to the first MPM mode and the second MPM mode, and the remaining four MPM modes can be derived from the default mode and / or the surrounding intra modes. In the MPM list generation process, a pruning process can be performed to prevent the same mode from being duplicated in the MPM list. A truncated binary code (TBC) can be used for entropy coding of modes other than the MPM modes.

[0203] FIG. 15 is a diagram illustrating reference samples for a wide-angle intra prediction mode according to one embodiment of the present disclosure.

[0204] As described above, the prediction direction of intra prediction may be defined as 45° to −135° clockwise. However, if the current block is a non-square block, some existing directional intra prediction modes may be adaptively replaced with wide-angle intra prediction modes. When the replaced wide-angle intra prediction is applied, information about the existing intra prediction may be signaled, and after parsing, the information may be remapped to an index of the wide-angle intra prediction mode. Therefore, the total number of intra prediction modes for a specific block (e.g., a non-square block of a specific size) may not be changed, i.e., the total number of intra prediction modes is 67, and the intra prediction mode coding for the specific block may not be changed.

[0205] As shown in Figure 15, an upper reference sample having a length of 2W+1 and a left reference sample having a length of 2H+1 may be defined to support the wide-angle prediction direction. Meanwhile, when wide-angle intra prediction is applied, the intra prediction mode substituted for the wide-angle intra prediction mode may vary depending on the aspect ratio of the current block. The intra prediction mode substituted for the wide-angle intra prediction mode according to the aspect ratio may be derived as shown in the following table.

[0206] [Table 3]

[0207] FIG. 16 is a diagram illustrating some wide-angle intra-prediction directions according to one embodiment of the present disclosure.

[0208] On the other hand, as shown in Figure 16, when wide-angle intra prediction exceeding 45 degrees is performed, two vertically adjacent prediction samples can be predicted based on two non-adjacent reference samples. Therefore, to reduce the negative effect of the increased gap Δpα, a low-pass reference sample filter and side smoothing can be applied to wide-angle prediction.

[0209] On the other hand, there may be cases where the wide-angle mode indicates a non-fractional offset. Wide-angle modes that satisfy this condition may include modes of [-14, -12, -10, -6, 72, 76, 78, 80]. As an example, when a block is predicted using the above-mentioned wide-angle mode, samples in the reference buffer may be directly copied without applying any interpolation. This can reduce the number of samples required for smoothing. Also, non-fractional modes can be designed between existing prediction modes and wide-angle modes.

[0210] Overview of derivation of peripheral reference samples

[0211] When intra prediction is applied to the current block, neighboring reference samples used for intra prediction of the current block may be derived. The neighboring reference samples of the current block may include a total of 2×nH samples adjacent to the left boundary and bottom-left of the current block having a size of nW×nH, a total of 2×nW samples adjacent to the top boundary and top-right of the current block, and one sample adjacent to the top-left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. The neighboring reference samples of the current block may also include a total of nH samples adjacent to the right boundary of the current block having a size of nW×nH, a total of nW samples adjacent to the bottom boundary of the current block, and one sample adjacent to the bottom-right of the current block.

[0212] On the other hand, when MRL is applied, the reference samples may be located on other lines, for example, lines 1 to 3, adjacent to the current block on the left / top side, instead of line 0. In this case, the number of surrounding reference samples may be further increased. Specific areas and numbers of surrounding reference samples will be described later.

[0213] Meanwhile, when the ISP described below is applied, the neighboring reference samples can be derived in units of sub-partitions.

[0214] Meanwhile, an interpolation filter can be applied to derive extended intra reference samples. Some of the neighboring reference samples of the current block may not yet be decoded or may not be available. In this case, the decoder can construct neighboring reference samples to be used for prediction through interpolation of available samples.

[0215] Meanwhile, extrapolation can be applied to derive extended intra reference samples. Some of the neighboring reference samples of the current block may not yet be decoded or available. In this case, the decoder can construct neighboring reference samples to be used for prediction through extrapolation of available samples. From the bottom left to the top right reference sample, the reference samples can be updated to the latest sample (last available sample), and pixels that have not yet been decoded or are not available can be constructed by substituting or padding with the last available sample.

[0216] Reference Sample Filtering Overview

[0217] Meanwhile, filtering may be applied to neighboring reference samples of the current block. This may be called pre-filtering because it is applied to neighboring reference samples before intra prediction, unlike post-filtering, which is filtering applied to predicted samples after intra prediction. The filtering of the neighboring reference samples may be called smoothing filtering, and may be performed, for example, as shown in the following equation.

[0218] the filtered sample values ​​p[x][y] with x = -1,y = -1..refH-1 and x = 0..refW-1,y = -1 are derived as follows:

[0219] p[-1][-1] = (refUnfilt[-1][0]+2*refUnfilt[-1][-1]+refUnfilt[0][-1]+2)>>2(1.3.4-1)

[0220] p[-1][y] = (refUnfilt[-1][y+1]+2*refUnfilt[-1][y]+refUnfilt[-1][y-1]+2)>>2

[0221] for y = 0..refH-2(1.3.4-2)

[0222] p[-1][refH-1] = refUnfilt[-1][refH-1](1.3.4-3)

[0223] p[x][-1] = (refUnfilt[x-1][-1]+2*refUnfilt[x][-1]+refUnfilt[x+1][-1]+2)>>2

[0224] for x = 0..refW-2(1.3.4-4)

[0225] p[refW-1][-1]=refUnfilt[refW-1][-1](1.3.4-5)

[0226] Here, refunfilt indicates a neighboring reference sample that has not yet been filtered, and [x][y] indicates the x and y coordinates of the sample. For example, this can indicate the coordinates when the top-left sample position coordinate of the current block is (0,0).

[0227] If filtering is applied to the surrounding reference samples, the filtered surrounding reference samples can be used as reference samples in the prediction sample derivation step, and if filtering is not applied to the surrounding reference samples, the unfiltered surrounding reference samples can be used as reference samples in the prediction sample derivation step.

[0228] The above-described neighboring reference sample filtering can be applied when, for example, some or all of the following specific conditions are satisfied.

[0229] -nTbW*nTbH is greater than 32

[0230] -cIdx is equal to 0

[0231] - IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT (IntraSubPartitionsSplitType is equal to ISP_NO_SPLIT)

[0232] -One or more of the following conditions are true:

[0233] -predModeIntra is equal to INTRA_PLANAR

[0234] -predModeIntra is equal to INTRA_ANGULAR34

[0235] -predModeIntra is equal to INTRA_ANGULAR2 and nTbH is greater than or equal to nTbW

[0236] -predModeIntra is equal to INTRA_ANGULAR66 and nTbW is greater than or equal to nTbH

[0237] For example, procedures such as determining an intra prediction mode / type, deriving neighboring reference samples, and deriving predicted samples may all be performed on a CU basis. For another example, determining an intra prediction mode / type may be performed on a CU basis, but deriving neighboring reference samples and predicted samples may be performed on a TU basis within the CU. In this case, TUs within the CU may share the same intra prediction mode / type. Therefore, taking this into consideration, whether to filter the neighboring reference samples may be determined taking into account the width and height, nTbW and nTbH, of the TU (or TB).

[0238] The prediction unit of the encoding device / decoding device can derive reference samples according to the intra prediction mode of the current block from the surrounding reference samples of the current block, and can generate predicted samples of the current block based on the reference samples.

[0239] For example, a prediction sample may be derived based on the average or interpolation of neighboring reference samples of the current block, or (ii) the prediction sample may be derived based on a reference sample located in a specific (prediction) direction relative to the prediction sample among the neighboring reference samples of the current block. (i) This may also be referred to as a non-directional mode or a non-angular mode, and (ii) this may also be referred to as a directional mode or an angular mode. Furthermore, the prediction sample may be generated by interpolating the first and second neighboring samples located in the opposite direction of the prediction direction of the intra prediction mode of the current block based on the prediction sample of the current block among the neighboring reference samples. The above-described case may also be referred to as linear interpolation intra prediction (LIP). Alternatively, a temporary prediction sample of the current block may be derived based on filtered neighboring reference samples, and the prediction sample of the current block may be derived by weighted summing the temporary prediction sample and at least one reference sample derived according to the intra prediction mode among the existing neighboring reference samples, i.e., unfiltered neighboring reference samples. The above case is sometimes called Position Dependent Intra Prediction (PDPC). In addition, intra prediction coding can be performed by selecting a reference sample line with the highest prediction accuracy from multiple reference sample lines surrounding the current block, deriving a prediction sample using a reference sample located in the prediction direction of the selected line, and signaling the used reference sample line to a decoding device. The above case is sometimes called multi-reference line intra prediction (MRL) or MRL-based intra prediction.In addition, the current block may be divided into vertical or horizontal sub-partitions and intra prediction may be performed based on the same intra prediction mode, with neighboring reference samples derived and used for each sub-partition. That is, in this case, the intra prediction mode for the current block is uniformly applied to the sub-partitions, but neighboring reference samples may be derived and used for each sub-partition, thereby improving intra prediction performance as needed. This prediction method is sometimes called intra sub-partitions (ISP) or ISP-based intra prediction. Specific details will be described later. In addition, when a prediction direction based on a prediction sample indicates a gap between neighboring reference samples, i.e., when the prediction direction indicates a fractional sample position, the value of the prediction sample may be derived through interpolation of multiple reference samples located around the edge direction (around the fractional sample position).

[0240] The above-described intra prediction methods may be referred to as intra prediction types, distinguished from the intra prediction modes in FIG. 13 and / or FIG. 14. The intra prediction types may be referred to by various terms, such as intra prediction techniques or additional intra prediction modes. For example, the intra prediction types (or additional intra prediction modes, etc.) may include at least one of the above-described LIP, PDPC, MRL, and ISP. Information about the intra prediction types may be coded by the coding device, included in a bitstream, and signaled to the coding device. The information about the intra prediction types may be implemented in various forms, such as flag information indicating whether each intra prediction type is applied, or index information indicating one of various intra prediction types.

[0241] The MPM list for deriving the intra prediction mode may be configured differently depending on the intra prediction type, or the MPM list may be configured in common regardless of the intra prediction type.

[0242] Meanwhile, when a prediction sample of a current block is generated through interpolation of a reference sample, an interpolation filter for the interpolation can be derived in various ways. For example, the interpolation filter can be determined based on a predetermined condition. For example, the interpolation filter can be determined based on the intra prediction mode of the current block and / or the size of the current block. The interpolation filter can include, for example, a Gaussian filter and a cubic filter. For example, when the intra prediction mode of the current block is the lower left diagonal intra prediction mode (#2), the upper left diagonal intra prediction mode (#34), or the upper right diagonal intra prediction mode (#66), it can be determined that the interpolation filter is not applied or that a Gaussian filter is applied instead of a cubic filter. Furthermore, for example, when the reference line index of the MRL is 0, a cubic filter is applied, and when the reference line index is greater than 0, it can be determined that the interpolation filter is not applied or that a Gaussian filter is applied. In addition, if the intra prediction mode is based on the position of the current prediction sample and the prediction direction according to the intra prediction mode indicates a fractional sample point rather than an integer sample point of the surrounding reference sample, an interpolation filter can also be applied to generate a reference sample value corresponding to the fractional sample point.

[0243] On the other hand, in the 6-tap interpolation filter of the enhanced compression model, the 4-tap cubic interpolation can be replaced with a 6-tap cubic interpolation filter to derive the predicted sample from the reference sample. For reference sample filtering, a 6-tap Gaussian filter is applied for larger blocks (W>=32 and H>=32), otherwise the existing VVC 4-tap Gaussian interpolation filter can be applied. The extended intra reference sample can be derived using a 4-tap interpolation filter instead of nearest neighbor rounding.

[0244] On the other hand, if the prediction direction according to the intra prediction mode based on the position of the current prediction sample (target prediction sample) in the current block indicates a fractional sample point rather than an integer sample point of the surrounding reference sample, an interpolation filter may be applied to generate a reference sample value corresponding to the fractional sample point.

[0245] Meanwhile, after performing intra prediction, a filter that relaxes block boundaries can be applied to reduce errors between predicted samples of the current block and already-reconstructed neighboring samples. For example, this filter can determine whether to apply a filter and the filter type depending on the predicted mode and the block size.

[0246] On the other hand, in decoder-side intra mode derivation (DIMD), intra prediction can be derived by a weighted average between planar and derived two directions. Two angle modes can be selected from a histogram of gradients (HoG) calculated for neighboring pixels of the current block. Once two modes are selected, the weighted average of the predictor of the selected mode and the planar predictor can be used as the final predictor for the block. To determine the weight, the amplitude of the HoG can be used for each of the two modes.

[0247] Since the induced intra mode can be included in the primary list of intra MPM (Most Probable Mode), the DIMD process can be performed before the MPM list is constructed. The primary induced intra mode of a DIMD block can be stored with the block and used to construct the MPM lists of surrounding blocks.

[0248] Overview of CIIP (combined inter and intra prediction)

[0249] Meanwhile, intra prediction can also be combined with inter prediction and applied to the current block, which is called a combined inter and intra prediction (CIIP) mode. Figure 17 is a diagram showing neighboring blocks that can be used in combined inter and intra prediction (CIIP). An additional flag (e.g., ciip_flag) can be signaled to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current block (e.g., CU). For example, when the current block is coded in merge mode, if the block includes at least 64 luma samples (i.e., block width*block height is equal to or greater than 64), and the block width and / or block height is less than 128 luma samples, an additional flag can be signaled to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current block. CIIP prediction can combine inter-prediction signals and intra-prediction signals, and the inter-prediction signal of CIIP mode P_inter can be derived using the same inter-prediction process applied to general merge mode. The intra-prediction signal (P_intra) can be derived by an intra-prediction process using planar mode. The intra- and inter-prediction signals can then be combined using a weighted average, where the weights can be calculated according to the coding modes of the upper and left neighboring blocks, as shown in FIG. 17.

[0250] As an example, if the top neighboring block is available and intra-coded, isIntraTop may be set to 1; otherwise, isIntraTop may be set to 0.

[0251] As another example, if the left neighboring block is available and intra-coded, isIntraLeft may be set to 1; otherwise, isIntraLeft may be set to 0.

[0252] As another example, if (isIntraLeft+isIntraLeft) is 2, then wt can be set to 3.

[0253] Otherwise, if (isIntraLeft+isIntraLeft) is 1, then wt can be set to 2.

[0254] Otherwise, wt can be set to 1.

[0255] Meanwhile, the CIIP prediction can be derived by the following formula:

[0256] JPEG2025183459000005.jpg9136

[0257] As an example, the left JPEG2025183459000006.jpg8119 is a CIIP signal, JPEG2025183459000007.jpg9130 indicates an inter prediction signal, JPEG2025183459000008.jpg10121 can indicate an intra-prediction signal.

[0258] MRL (Multi-Reference Line) intra prediction

[0259] Conventional intra prediction uses only the neighboring samples of the first line above and the first line to the left of a current block as reference samples for intra prediction. However, the Multiple-Reference Line (MRL) method can perform intra prediction using neighboring samples located on sample lines one to three samples away from the top and / or left of the current block as reference samples. FIG. 18 is a diagram illustrating MRL intra prediction, showing an example of multiple reference lines. Here, a multiple reference line index (e.g., mrl_idx) indicates which line of the current block is used for intra prediction. For example, the multiple reference line index can be signaled via a coding unit syntax as follows: The multiple reference line index can be configured in the form of an intra_luma_ref_idx syntax element.

[0260] [Table 4]

[0261] intra_luma_ref_idx[x0][y0] may specify the intra reference line index IntraLumaRefLineIdx[x0][y0] as specified in Table 5. If intra_luma_ref_idx[x0][y0] does not exist, it may be considered to be equal to 0.

[0262] The intra_luma_ref_idx is sometimes called the (intra) reference sample line index or mrl_idx, and the intra_luma_ref_idx is sometimes called the intra_luma_ref_line_idx.

[0263] [Table 5]

[0264] If intra_luma_mpm_flag[x0][y0] is not present, it can be considered the same as 1.

[0265] The MRL can be disabled for the first line (row) of a CTU to prevent extended reference lines outside the current CTU from being used, and the PDPC can be disabled if the additional reference lines are used.

[0266] Intra Sub-Partitions(ISP) prediction

[0267] In conventional intra prediction, a block to be currently coded is treated as a single coding unit and coded without division. However, the Intra Sub-Partitions (ISP) prediction method divides the block to be currently coded horizontally or vertically and then performs intra prediction coding. Figures 19a and 19b are diagrams illustrating the Intra Sub-Partitions (ISP) process according to the present disclosure. In this case, a reconstructed block is generated by coding / decoding the divided block unit, and the reconstructed block is used as a reference block for the next divided block. The current intra sub-partitions (ISP) are divided according to the block size as shown in Table 6.

[0268] [Table 6]

[0269] More specifically, Figure 19a is a diagram illustrating an example of division into 4x8 and 8x4 blocks (CU), and Figure 19b is a diagram illustrating an example of division into all remaining blocks excluding 4x8, 8x4, and 4x4 blocks (CU).

[0270] [Table 7]

[0271] The intra sub-partitioning method generates an MPM list for each partitioning method (horizontal partitioning and vertical partitioning) to reduce coding complexity, and compares suitable prediction modes from the generated MPM lists in terms of rate distortion optimization (RDO) to generate the optimal mode. Furthermore, when multiple reference line (MRL) intra prediction is used, the above-described intra sub-partitioning method cannot be used. That is, the intra sub-partitioning method is applied only when the 0th reference line is used (i.e., intra_luma_ref_idx value 0). Furthermore, when the above-described intra sub-partitioning method is used, the above-described PDPC cannot be used.

[0272] The intra subpartition method first transmits whether or not intra subpartitions are applied on a block-by-block basis, and if the current block uses intra subpartitions (intra_subpartitions_mode_flag), it again encodes / decodes information about whether the division is horizontal or vertical (intra_subpartitions_split_flag).

[0273] When an intra sub-partition method is applied, the intra prediction mode for the current block is applied to the sub-partitions in the same manner, and neighboring reference samples are derived and used in units of sub-partitions, thereby improving intra prediction performance. That is, when the intra sub-partition method is applied, a residual sample processing procedure is performed in units of sub-partitions. In other words, an intra prediction sample is derived for each sub-partition, and a residual signal (residual sample) for the corresponding sub-partition is added thereto to obtain a reconstructed sample. The residual signal (residual sample) may be derived through an inverse quantization / inverse transform procedure based on residual information (quantized transform coefficient information or residual coding syntax) in the bitstream. That is, a prediction sample and a residual sample may be derived for a first sub-partition, and then a reconstructed sample for the first sub-partition may be derived based on the derived prediction sample and residual sample. In this case, when a prediction sample for a second sub-partition is derived, some of the reconstructed samples in the first sub-partition (e.g., neighboring reference samples on the left or upper side of the second sub-partition) may be used as neighboring reference samples for the second sub-partition. Similarly, predicted samples and residual samples for the second subpartition may be derived, and based on the derivation, reconstructed samples for the second subpartition may be derived. In this case, when deriving predicted samples for the third subpartition, some of the reconstructed samples in the second subpartition (e.g., neighboring reference samples on the left or upper side of the third subpartition) may be used as neighboring reference samples for the third subpartition. The same applies hereinafter.

[0274] In intra prediction, reference sample filtering can be applied when the reference line index is 0, i.e., MRL (multi reference line) based intra prediction is not applied, the number of pixels in the block is 32 or more, the hue component of the current block is a luma hue component, ISP (Intra subpartitions) is not applied, and the intra prediction mode is one of 0, -14, -12, -10, -6, 2, 34, 66, 72, 76, 78, and 80.

[0275] That is, currently, intra reference sample filtering may not be applied to blocks below a certain size. For example, it is not applied to blocks such as 4x4, 4x8, and 8x4. Furthermore, reference sample filtering may not be applied when the intra prediction mode of the current block is MRL or ISP is applied. Furthermore, when the intra prediction mode is a planar mode or a specific directional mode, for example, intra reference sample filtering can only be applied to an integer directional mode or a specific wide-angle directional mode, which presents somewhat restrictive application conditions. Furthermore, according to ECM (enhanced compression model) and the like proposed for future codec standardization, as described above, a 6-tap cubic filter or a 6-tap Gaussian filter can be applied, significantly reducing the effectiveness of existing intra reference sample filtering.

[0276] Accordingly, the present disclosure proposes a method for applying intra reference sample filtering in a more sophisticated or simplified manner. According to an embodiment of the present disclosure, a filter to be applied to intra reference sample filtering may be adaptively selected from among a plurality of filters. For example, a filter may be selected according to specific conditions, including the size of the current block or the number of reference samples. According to another embodiment of the present disclosure, intra reference sample filtering may be applied only in a specific intra prediction mode, and reference sample filtering may be performed when generating intra-predicted pixels. Meanwhile, according to another embodiment of the present disclosure, intra reference sample filtering may or may not always be applied regardless of other conditions, including the intra prediction mode.

[0277] Example 1

[0278] In this embodiment, a method for applying filtering more sophisticatedly is proposed by diversifying the filters that can be applied to reference sample filtering. More specifically, this embodiment proposes one or more filters used for reference sample filtering, where each filter may have different filter coefficients or different numbers of filter coefficients, i.e., different numbers of taps, and all of the filter coefficients may be positive numbers.

[0279] For clarity of explanation, each filter will be referred to as an n-th filter, such as a first filter or a second filter, but this is not intended to limit the order or type of filters. Also, for clarity of explanation, the following description will be given using only a first filter and a second filter, but there are not necessarily only two filters, and more various filters may exist, so the present disclosure is not limited to this.

[0280] As an example, filters that can be used for reference sample filtering may include a 3-tap filter as a first filter. For example, the 3-tap filter may be a [1,2,1] filter. Also, the second filter may include an n-tap filter where n is not 3. For example, n may be 5 or 6. Alternatively, n may be an odd integer. When n is 6, the filter coefficients of a 6-tap filter may be [1,4,6,4,1,0]. As another example, when n is 5, the filter coefficients of a 5-tap filter may be [1,4,6,4,1]. As an example, the second filter may be a Gaussian filter that can be applied to integer sample positions. That is, it may be an n-tap Gaussian filter.

[0281] As an example, if a 5-tap filter is [1,4,6,4,1], it can be applied as follows:

[0282] -ref_filtered[x]=(ref[x-2]+4*ref[x-1]+6*ref[x]+4*ref[x+1]+ref[x+2]) / 16

[0283] In the above equation, ref[x] refers to the reference sample at the x position to which the current reference sample filtering is applied, and ref_filtered[x] refers to the pixel to which the reference sample filtering is applied. That is, by adaptively applying a more sophisticated [1,4,6,4,1] smoothing filter instead of a 3-tap filter (e.g., a [1,2,1] smoothing filter), encoding and decoding efficiency can be improved.

[0284] On the other hand, when selecting one of multiple filters, the reference sample filtering efficiency can be further improved by adaptively applying the first filter (e.g., a [1,2,1] filter) and the second filter (e.g., a [1,4,6,4,1] filter) according to the number of reference samples.

[0285] For example, intra reference sample filtering may always be performed using the second filter. That is, if the first filter is a 3-tap filter, intra reference sample filtering may be performed using a filter other than the first filter. In this case, if the other filter used is assumed to be the second filter, the second filter may be an n-tap filter where n>3. For example, if the second filter is a [1,4,6,4,1] filter, the second filter may always be applied.

[0286] As another example, when performing reference sample filtering, a filter may be selected taking into account the size or area of ​​the block. For example, if the width and / or height of the current block is equal to or greater than a specific value (e.g., 32 or 16), one filter is used; otherwise, another filter may be used. In this case, for reference sample filtering of a current block whose size or area is equal to or greater than a specific value, any k-tap filter may be selected; otherwise, any j-tap filter may be selected, where k>j. That is, the size of the block may be used as a filter selection criterion. For example, if the width and / or height of the current block is equal to or greater than a specific value, a 5-tap filter (e.g., [1,4,6,4,1]) may be applied; otherwise, a 3-tap filter (e.g., [1,2,1]) may be applied. As another example, if the size or area of ​​the block (e.g., the product of the width and block height) is equal to or greater than a specific value (e.g., 1024 or 256), a 5-tap filter (e.g., [1,4,6,4,1]) may be applied; otherwise, a 3-tap filter (e.g., [1,2,1]) may be applied.

[0287] According to this embodiment, by adaptively selecting a filter and performing reference sample filtering, it is possible to improve filtering performance.

[0288] Meanwhile, the reference sample filtering application conditions may include conditions on whether ISP / MRL is applied, the color components of the current block, or the intra prediction mode, as described above, but may also follow modified reference sample filtering application conditions. Meanwhile, the modified reference sample filtering application conditions may include some of the above-described conditions added or modified, or some conditions excluded. The modified reference sample filtering application conditions will be described in detail below.

[0289] Example 2

[0290] In this embodiment, a method for simplifying the conditions for applying reference sample filtering is proposed. Table 8 below shows an example of performance when a [1,2,1] filter is used and reference sample filtering is not applied.

[0291] [Table 8]

[0292] The experimental results show the performance in all intra predictions. As can be seen from the table above, the performance of intra [1,2,1] reference sample filtering is shown to be 0.03% for the Y component.

[0293] On the other hand, when the reference sample filtering conditions are changed to apply reference sample filtering only in a specific intra mode (e.g., planar intra mode), a performance improvement of 0.02% is observed for the Y component. In other words, considering the results of the previous experiment described above, intra reference sample filtering can be expected to improve performance even when applied in a specific intra mode.

[0294] Therefore, in this embodiment, a method of applying reference sample filtering only in a specific intra mode (for example, planar intra mode) is proposed as follows.

[0295] As an example, intra reference sample filtering may be applied when the reference line index is 0 (i.e., MRL is not applied), the number of pixels in the current block is 32 or more, the color component of the current block is a luma color component, ISP is not applied to the current block, and the intra prediction mode index is 0, i.e., the intra direction is 0 (Planar intra mode).

[0296] Meanwhile, among the above application conditions, if the last condition, intra direction, is number 0, the remaining conditions except for the condition may be removed or added to improve coding performance and reduce complexity.

[0297] For example, to obtain higher coding performance, a condition for applying reference sample filtering may be applied regardless of the block size. For example, reference sample filtering may be applied when the reference line index is 0 (MRL not applied), the color component of the current block is a luma color component, ISP is not applied to the current block, and the intra direction is planar intra mode.

[0298] As another example, to obtain higher coding performance, a condition for applying reference sample filtering may be applied regardless of the color component of the block. For example, reference sample filtering may be applied when the reference line index is 0 (MRL not applied), the number of pixels in the current block is 32 or more, ISP is not applied to the current block, and the intra direction is planar intra mode.

[0299] As another example, to obtain higher coding performance, a condition for applying reference sample filtering may be applied taking into account whether the above-mentioned DIMD and / or CIIP are applied. Whether to perform reference sample filtering may be determined based on whether planar intra prediction is performed in the above-mentioned DIMD and CIIP. For example, reference sample filtering may be applied when the reference line index is 0 (MRL not applied condition), the number of pixels of the current block is 32 or more, ISP is not applied to the current block, DIMD is not applied, and the intra direction is planar intra mode. As another example, reference sample filtering may be applied when the reference line index is 0 (MRL not applied condition), the number of pixels of the current block is 32 or more, ISP is not applied to the current block, CIIP is not applied, and the intra direction is planar intra mode. Alternatively, reference sample filtering may be applied only when neither DIMD nor CIIP is applied.

[0300] As another example, contrary to the above example, reference sample filtering may be applied when CIIP is applied to the current block, when DIMD is applied, or only when CIIP or DIMD is applied. As an example, in this case, the intra prediction mode may be planar intra prediction.

[0301] As another example, when an intra predictor (eg, a planar intra predictor) is combined with other intra / inter predictions, such as CIIP or DIMD, it may be determined whether to apply reference sample filtering in intra prediction.

[0302] Meanwhile, in one embodiment, when reference sample filtering is applied only in planar intra mode, a method is proposed in which the reference sample filtering process is integrated with intra planar prediction. That is, reference sample filtering does not exist as a separate process, but can be an internal process of planar prediction. Figures 20 and 21 show examples of two specifications in which a reference sample filtering process is added to planar prediction. As an example, in planar intra prediction mode, a reference sample width and a reference sample height are input and can be used for reference sample filtering.

[0303] Meanwhile, when applying reference sample filtering only in the planar intra mode, only a single filter may be used, or multiple filters may be adaptively applied as in the previous embodiment 1. For example, when adaptively applying a first filter (e.g., a [1,2,1] filter) and a second filter (e.g., a [1,4,6,4,1] filter), the first filter (e.g., a [1,2,1] filter) may always be applied, or the second filter (e.g., a [1,4,6,4,1] filter) may always be applied, or the second filter may be applied if the block width and / or height is equal to or greater than a specific value (e.g., 32 or 16) and the first filter may be applied otherwise, or the second filter may be applied if the block size or area (e.g., block width x block height) is equal to or greater than a specific value (e.g., 1024 or 256) and the first filter may be applied otherwise.

[0304] According to the above embodiment, the performance of reference sample filtering can be improved, thereby improving the encoding and decoding efficiency.

[0305] Example 3

[0306] In this embodiment, another method for simplifying the application conditions for reference sample filtering is proposed.

[0307] For example, as described above, reference sample filtering can always be applied when intra prediction is applied to the current block, regardless of the intra prediction mode, color components of the current block, or whether ISP / MRL is applied.

[0308] As another example, as mentioned above, reference sample filtering does not always need to be applied when intra prediction is applied to the current block, regardless of the intra prediction mode, color components of the current block, or whether ISP / MRL is applied.

[0309] Meanwhile, as an example, when reference sample filtering is always applied, the reference sample filter may be a single filter (e.g., a [1,2,1] filter), or in addition to a single filter, multiple filters proposed above may be considered. For example, a first filter (e.g., a [1,2,1] filter) and a second filter (e.g., a [1,4,6,4,1] filter) may be considered. Meanwhile, even if the reference sample filtering itself is performed regardless of any conditions, when one of multiple filters is adaptively selected and applied, other conditions such as the size of the current block and the intra prediction mode described above may be considered to select a filter.

[0310] When reference sample filtering is always applied or not applied, there is no need to check conditions such as block size conditions, whether ISP / MRL is used, intra prediction direction conditions, etc. In other words, when intra prediction is applied regardless of the above conditions, reference sample filtering can always be performed. Therefore, the complexity of the overall intra prediction algorithm and overhead related to whether reference sample filtering is applied can be reduced, thereby resulting in increased efficiency and speed of image encoding / decoding.

[0311] 22 is a diagram illustrating an image encoding or decoding process for performing reference sample filtering based on an intra prediction mode according to an embodiment of the present disclosure. For example, the process illustrated in FIG. 22 can be performed by an image encoding device or an image decoding device.

[0312] For example, when an image encoding or decoding device performs image encoding or decoding, it may determine an intra prediction mode of a current block (S2201). When determining the intra prediction mode, the intra prediction mode may be determined based on information for determining the intra prediction mode, obtained from a bitstream, or derived based on other information. For example, the information for determining the intra prediction mode may include an MPM list index, as described above.

[0313] Meanwhile, once the intra prediction mode is determined, reference samples used for intra prediction may be derived (S2202). As an example, the reference samples may be derived based on the determined intra prediction mode, and this derivation process may be included in the process of performing intra prediction. More specifically, in the embodiment of FIG. 22, although they are expressed as separate steps for clarity of explanation, the processes of deriving reference samples (S2202), determining whether to perform reference sample filtering on the derived reference samples (S2203), and / or performing reference sample filtering based on whether to perform reference sample filtering (S2204) may be implemented in a manner that is included in the process of performing intra prediction (S2205), but the present disclosure is not limited thereto.

[0314] For example, if the determined intra prediction mode is a specific intra prediction mode (e.g., a planar intra prediction mode), the step of performing reference sample filtering (S2204) may be included in the step of performing intra prediction (S2205). That is, reference sample filtering may be included in the intra prediction process itself based on the specific intra prediction mode. Meanwhile, as an example, as long as intra prediction is applied to the current block, reference sample filtering (S2204) may be applied regardless of other conditions (e.g., intra prediction direction, block size, etc.). That is, the step of determining whether to perform reference sample filtering on the reference sample (S2203) may be omitted depending on the case. Meanwhile, if step (S2203) is not omitted and whether to perform reference sample filtering on the reference sample (S2203) is determined, the reference sample filtering may be determined based on information signaled from the bitstream or other information, or may be determined based on conditions under which reference sample filtering can be applied. For example, the conditions for reference sample filtering described in the above-mentioned embodiments may be applied.

[0315] Meanwhile, when reference sample filtering (S2204) is applied, for example, the reference sample filtering may be performed by adaptively selecting one of a plurality of filters. The filter may also be adaptively selected based on the number of intra-prediction reference samples. The number of intra-prediction reference samples may be determined based on the size of the current block. For example, the number of intra-prediction reference samples may be determined based on a comparison between the size of the current block and a specific value, as described in the above embodiment. Meanwhile, one of the plurality of filters (e.g., the first filter) may be a 3-tap filter, and the 3-tap filter may be configured as [1, 2, 1]. Another filter (e.g., the second filter) may be an n-tap filter, where n is a value greater than 3. For example, the other filter may be a 5-tap filter or a 6-tap filter. For example, the 5-tap filter may be [1, 4, 6, 4, 1], and the 6-tap filter may be [1, 4, 6, 4, 1, 0]. For example, all filter coefficients used for filtering may be positive numbers. Meanwhile, application conditions for reference sample filtering may include whether decoder side intra mode derivation (DIMD) and / or combined inter intra prediction (CIIP) are applied. That is, whether reference sample filtering is applied may be determined based on whether decoder side intra mode derivation (DIMD) and / or combined inter intra prediction (CIIP) are applied. Meanwhile, reference sample filtering may be applied regardless of the color components of the current block and regardless of the size of the current block.

[0316] When intra prediction is performed based on the intra prediction mode (S2205), intra prediction samples may be generated. Based on the generated intra prediction samples, reconstructed samples for the current block may be obtained. As described above, the reconstructed samples may be generated based on the residual samples and the predicted samples.

[0317] Furthermore, since the image encoding and decoding method described with reference to Figure 22 corresponds to one embodiment of the present disclosure, further steps that combine the embodiments described above may be added, and the order of some steps may be changed or deleted.

[0318] For example, in the case of an image encoding method, the method may further include a step of encoding residual samples (i.e., residual signals) obtained based on the predicted samples and the current block into a bitstream, or a step of encoding information related to an intra-prediction mode into a bitstream, and may further include a step of transmitting the bitstream to an image decoding device, etc.

[0319] 23 is a diagram illustrating an image encoding / decoding device according to an embodiment of the present disclosure. As an example, the image encoding / decoding device may include a memory 2302 and at least one processor 2303.

[0320] As an example, at least one processor performs the above-described embodiments and image encoding / decoding methods, but some steps may be performed in parallel, and if possible, some steps may be reordered or omitted.

[0321] As an example, at least one processor may determine an intra prediction mode of the current block, perform intra prediction based on the intra prediction mode, and obtain reconstructed samples based on the prediction samples generated by the intra prediction.

[0322] As an example, the at least one processor performs planar intra prediction based on the fact that the intra prediction mode is a planar prediction mode, and the planar intra prediction may include a filtering process of intra prediction reference samples of the current block.

[0323] According to the present disclosure, by changing or simplifying the conditions for intra-prediction reference sample filtering, it is possible to reduce the overhead of intra-prediction reference sample filtering and improve the encoding or decoding efficiency.

[0324] Various embodiments according to the present disclosure may be used alone or in combination with other embodiments.

[0325] Although the exemplary methods of the present disclosure are expressed as a series of operations for clarity of explanation, this is not intended to limit the order in which the steps are performed, and the steps may be performed simultaneously or in a different order, if necessary. To achieve a method according to the present disclosure, the steps illustrated may include other steps, or some steps may be omitted and the remaining steps may be included, or some steps may be omitted and additional other steps may be included.

[0326] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) can perform the operation (step) to check the execution conditions and circumstances of the operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or the image decoding device can perform the predetermined operation after performing an operation to check whether the predetermined condition is satisfied.

[0327] The various embodiments of the present disclosure are not intended to enumerate all possible combinations, but are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combination of two or more.

[0328] Additionally, various embodiments of the present disclosure may be implemented using hardware, firmware, software, or a combination thereof, etc. In the case of a hardware implementation, the implementation may be using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general processors, controllers, microcontrollers, microprocessors, etc.

[0329] In addition, an image decoding apparatus and an image encoding apparatus to which an embodiment of the present disclosure is applied may be included in a multimedia broadcast transmitting / receiving apparatus, a mobile communication terminal, a home cinema video apparatus, a digital cinema video apparatus, a surveillance camera, a video conversation apparatus, a real-time communication apparatus such as video communication, a mobile streaming apparatus, a storage medium, a camcorder, a video on demand (VoD) service providing apparatus, an over-the-top (OTT) video apparatus, an internet streaming service providing apparatus, a three-dimensional (3D) video apparatus, an image telephone video apparatus, a medical video apparatus, etc., and may be used to process a video signal or a data signal. For example, an over-the-top (OTT) video apparatus may include a game console, a Blu-ray player, an internet-connected TV, a home theater system, a smartphone, a tablet PC, a digital video recorder (DVR), etc.

[0330] FIG. 24 is a diagram illustrating a content streaming system to which the embodiments of the present disclosure can be applied.

[0331] As shown in FIG. 24, a content streaming system to which an embodiment of the present disclosure is applied can broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0332] The encoding server compresses content input from a multimedia input device such as a smartphone, camera, or camcorder into digital data to generate a bitstream and transmits the bitstream to the streaming server. As another example, if a multimedia input device such as a smartphone, camera, or video camera directly generates a bitstream, the server can be omitted.

[0333] The bitstream can be generated by an image encoding method and / or image encoding device to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0334] The streaming server transmits multimedia data to a user device based on a user request via a web server, and the web server serves as an intermediary for informing the user of available services. When a user requests a desired service from the web server, the web server transmits the request to the streaming server, which then transmits the multimedia data to the user. In this case, the content streaming system may include a separate control server, which may control commands and responses between devices in the content streaming system.

[0335] The streaming server may receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content may be received in real time. In this case, the streaming server may store the bitstream for a certain period of time to provide a smooth streaming service.

[0336] Examples of the user device include a mobile phone, a smartphone, a laptop computer, a digital broadcasting terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation system, a slate PC, a tablet PC, an ultrabook, a wearable device such as a smartwatch, smart glass, a head mounted display (HMD), a digital TV, a desktop computer, and digital signage.

[0337] Each server in the content streaming system can be operated as a distributed server, in which case data received from each server can be processed in a distributed manner.

[0338] The scope of the present disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands can be stored and executed on a device or computer. [Industrial Applicability]

[0339] The embodiments of the present disclosure can be used to encode / decode images.

Claims

1. An image decoding method performed by an image decoding device, comprising: determining an intra-prediction mode for a current block; performing planar intra prediction based on the fact that the intra prediction mode is a planar prediction mode; obtaining a reconstructed sample of the current block based on the predicted sample obtained based on the planar intra prediction; An image decoding method, wherein filtering to intra-prediction reference samples of the current block is performed only when the intra-prediction mode is a planar prediction mode.

2. The image decoding method according to claim 1 , wherein the filtering is performed by adaptively selecting one filter from a plurality of filters including a first filter or a second filter.

3. The image decoding method according to claim 2 , wherein the selected one filter is selected based on a number of intra-prediction reference samples.

4. The image decoding method according to claim 3 , wherein the number of intra-prediction reference samples is determined based on the size of the current block.

5. The image decoding method of claim 4 , wherein the number of intra-prediction reference samples is determined based on a comparison between the size of the current block and a specific value.

6. The image decoding method according to claim 2 , wherein the first filter is a 3-tap filter and the second filter is a 5-tap filter.

7. The image decoding method according to claim 6 , wherein the 3-tap filter is [1, 2, 1].

8. The image decoding method according to claim 6 , wherein the 5-tap filter is [1, 4, 6, 4, 1].

9. 2. The image decoding method according to claim 1, wherein application of the filtering is determined based on at least one of whether to apply decoder side intra mode derivation (DIMD) or whether to apply combined inter intra prediction (CIIP).

10. The image decoding method of claim 1 , wherein the filtering is applied regardless of the size of the current block.

11. The image decoding method of claim 1 , wherein the filtering is applied regardless of the color components of the current block.

12. The filtering may include: Based on a [1,2,1] filter, and The image decoding method according to claim 1 , wherein the decoding is performed based on at least one of the size of the current block, the color components of the current block, or whether intra subpartition (ISP) is applied.

13. An image coding method performed by an image coding device, comprising: determining an intra-prediction mode for a current block; performing planar intra prediction based on the fact that the intra prediction mode is a planar prediction mode; obtaining a reconstructed sample of the current block based on the predicted sample obtained from the planar intra prediction; An image coding method, wherein filtering to intra-prediction reference samples of the current block is performed only when the intra-prediction mode is a planar prediction mode.

14. A method for transmitting a bitstream generated by an image coding method, comprising: The image encoding method includes: determining an intra-prediction mode for a current block; performing planar intra prediction based on the fact that the intra prediction mode is a planar prediction mode; obtaining a reconstructed sample of the current block based on the predicted sample obtained from the planar intra prediction; A method in which filtering to intra-prediction reference samples of the current block is performed only when the intra-prediction mode is a planar prediction mode.

Images