Image encoding / decoding method and apparatus using intra prediction based on prediction combination, and method for transmitting bitstream

Abstract

An image encoding / decoding method and apparatus are provided. An image decoding method performed by an image decoding device according to one embodiment of the present disclosure may comprise the steps of: obtaining template-based intra prediction information of a current block; generating two or more prediction blocks for the current block according to the intra prediction information based on the template; and restoring the current block based on the weighted sum of the two or more prediction blocks, and determining a weight for each prediction block used in the weighted sum according to a cost based on template matching.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to an image encoding / decoding method and apparatus using intra prediction based on a combination between prediction blocks, and a method of transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0002] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data increase, the amount of information or bits to be transmitted relatively increases compared to existing image data. The increase in the amount of information or bits to be transmitted results in an increase in transmission costs and storage costs.

[0003] Therefore, an efficient image compression technique is needed to effectively transmit, store, and reproduce information on high-resolution and high-quality images. Summary of the Invention

[0004] [Technical Problem]

[0005] The present disclosure aims to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0006] The present disclosure also aims to provide a method and apparatus for encoding / decoding a still image or video.

[0007] The present disclosure also aims to provide an image compilation technique using intra prediction based on a prediction combination.

[0008] The present disclosure also aims to provide a technique for combining several prediction blocks that predict using intra modes derived from a template-based intra mode derivation (TIMD) mode.

[0009] The present disclosure also aims to provide a technique for combining a prediction block predicted using a planar mode with a prediction block predicted using a mode other than the planar mode.

[0010] The present disclosure also aims to provide a method and apparatus for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0011] The present disclosure also aims to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0012] The present disclosure also aims to provide a recording medium storing a bitstream that is received and decoded by an image decoding apparatus according to the present disclosure and is used to reconstruct an image.

[0013] The technical objectives of the present disclosure are not limited to the above, and those skilled in the art to which the present disclosure pertains will clearly understand other technical objectives not described above from the following description.

[0014] [Technical Solutions]

[0015] One aspect of the present disclosure provides a method for decoding an image, including: obtaining template-based intra prediction information for a current block, generating two or more prediction blocks for the current block according to the template-based intra prediction information, and reconstructing the current block based on a weighted sum of the two or more prediction blocks, wherein weights for each prediction block used for the weighted sum are determined according to a cost of template matching.

[0016] Another aspect of the present disclosure provides an image encoding method, including: determining a current block to undergo template-based intra prediction, generating two or more prediction blocks for the current block according to the template-based intra prediction, and calculating a weighted sum of the two or more prediction blocks for the current block, wherein weights for each prediction block used for the weighted sum are determined according to a cost of template matching.

[0017] Another aspect of the present disclosure provides a method for transmitting a bitstream, including: transmitting a bitstream generated using the image encoding method, wherein the image encoding method includes: determining a current block to undergo template-based intra prediction, generating two or more prediction blocks for the current block according to the template-based intra prediction, and calculating a weighted sum of the two or more prediction blocks for the current block, wherein weights for each prediction block used for the weighted sum are determined according to a cost of template matching.

[0018] It is possible to transmit a bitstream generated using an image encoding device or an image encoding method.

[0019] A bitstream generated using the image encoding method can be stored or recorded on a computer-readable medium.

[0020] A bitstream generated using the image encoding method can be transmitted through a bitstream transmission device.

[0021] The features of the present invention briefly summarized above are merely illustrative aspects of the detailed description of the present invention and do not limit the scope of the present invention.

[0022] [Advantageous Effects]

[0023] According to the present disclosure, it is possible to provide an image encoding / decoding method and device with improved encoding / decoding efficiency.

[0024] According to the present disclosure, it is possible to provide an image encoding / decoding technique with improved compilation quality.

[0025] According to the present disclosure, the prediction performance can be improved by combining several prediction blocks when performing in-picture encoding on a still image or a video.

[0026] According to the present invention, there can be provided a method or apparatus for transmitting a bitstream generated using an image encoding method or apparatus according to the present disclosure.

[0027] According to the present invention, there can be provided a recording medium storing a bitstream generated using an image encoding method or apparatus according to the present disclosure.

[0028] According to the present invention, there can be provided a recording medium storing a bitstream that is received and decoded by an image decoding apparatus according to the present disclosure and is used for reconstructing an image.

[0029] The effects of the present disclosure are not limited to the above effects, and those skilled in the art to which the present disclosure pertains will clearly understand other effects not described above from the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a view schematically showing a video compilation system to which an embodiment of the present disclosure is applicable.

[0031] Figure 2 is a view schematically showing an image encoding apparatus to which an embodiment of the present disclosure is applicable.

[0032] Figure 3 is a view schematically showing an image decoding apparatus to which an embodiment of the present disclosure is applicable.

[0033] Figure 4 is a diagram schematically showing a template-based intra-mode derivation (TIMD) technique to which an embodiment of the present disclosure is applicable.

[0034] Figure 5 and Figure 6 is a diagram schematically showing a decoder-side intra-mode derivation (DIMD) technique to which an embodiment of the present disclosure is applicable.

[0035] Figure 7 is a diagram schematically showing an intra-prediction mode to which an embodiment of the present disclosure is applicable.

[0036] Figure 8 is a flowchart schematically showing a prediction combination method according to an embodiment of the present disclosure.

[0037] Figure 9 is a flowchart schematically showing an image decoding method according to an embodiment of the present disclosure.

[0038] Figure 10is a flowchart schematically showing an image encoding method according to an embodiment of the present disclosure.

[0039] Figure 11 is a view showing a content streaming system to which the embodiment of the present disclosure is applicable. Detailed Description of the Invention

[0040] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0041] When describing the present disclosure, if the detailed description of related known functions or configurations makes the scope of the present disclosure unnecessarily ambiguous, the detailed description thereof will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and similar reference numerals are assigned to similar parts.

[0042] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with an intermediate component present. Additionally, when a component "includes" or "has" other components, unless otherwise specified, it means that other components may also be included, rather than excluding other components.

[0043] In the present disclosure, terms such as first and second are only used for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise specified. Accordingly, within the scope of the present disclosure, the first component in one embodiment may be referred to as the second component in another embodiment, and similarly, the second component in one embodiment may be referred to as the first component in another embodiment.

[0044] In the present disclosure, the components that are mutually distinguished are intended to clearly describe each feature, and do not mean that the components must be separated. That is, multiple components can be implemented integrated in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even without special description, the embodiments in which these components are integrated or distributed are included within the scope of the present disclosure.

[0045] In the present disclosure, the components described in each embodiment are not necessarily essential components, and some components may be optional components. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included within the scope of the present disclosure. In addition, embodiments including other components in addition to the components described in various embodiments are included within the scope of the present disclosure.

[0046] The present disclosure relates to the encoding and decoding of images. Unless redefined in the present disclosure, the terms used in the present disclosure may have the general meanings commonly used in the technical field to which the present disclosure belongs.

[0047] In the present disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that forms part of a picture, and a picture can be composed of one or more slices / tiles. In addition, a slice / tile can include one or more coding tree units (CTUs).

[0048] In the present disclosure, a "pixel" or "pel" can mean the smallest individual that constitutes a picture (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample generally can represent a pixel or the value of a pixel, or can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0049] In the present disclosure, a "unit" can represent a basic unit of image processing. The unit can include at least one of a specific area of a picture and information related to that area. In some cases, the unit can be used interchangeably with terms such as "sample array", "block", or "region". Generally, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients in M columns and N rows.

[0050] In the present disclosure, a "current block" can mean one of "current coding block", "current coding unit", "coding target block", "decoding target block", or "processing target block". When prediction is performed, a "current block" can mean "current prediction block" or "prediction target block". When transform (inverse transform) / quantization (dequantization) is performed, a "current block" can mean "current transform block" or "transform target block". When filtering is performed, a "current block" can mean "filtering target block".

[0051] In addition, in the present disclosure, unless explicitly stated as a chrominance block, a "current block" can mean a block including both a luminance component block and a chrominance component block or the "luminance block of the current block". The "chrominance component block of the current block" can be expressed by explicitly describing a chrominance component block including terms such as "chrominance block" or "current chrominance block".

[0052] In the present disclosure, the slashes " / " or "," can be interpreted as indicating "and / or". For example, "A / B" and "A,B" can mean "A and / or B". In addition, "A / B / C" and "A / B / C" can mean "at least one of A, B, and / or C".

[0053] In the present disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in the present disclosure, "or" should be interpreted to indicate "additionally or alternatively".

[0054] Overview of the video compilation system

[0055] Figure 1 is a view showing the video compilation system to which the embodiments of the present disclosure are applicable.

[0056] The video compilation system according to an embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver encoded video and / or image information or data in the form of a file or stream transmission to the decoding device 20 via a digital storage medium or a network.

[0057] The encoding device 10 according to an embodiment may include a video source generator 11, an encoder 12, and a transmitter 13. The decoding device 20 according to an embodiment may include a receiver 21, a decoder 22, and a renderer 23. The encoder 12 may be referred to as a video / image encoding unit, and the decoder 22 may be referred to as a video / image decoding device. The transmitter 13 may be included in the encoder 12. The receiver 21 may be included in the decoder 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0058] The video source generator 11 may acquire video / images through the process of capturing, synthesizing, or generating video / images. The video source generator 11 may include a video / image capturing device and / or a video / image generating device. The video / image capturing device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generating device may include, for example, a computer, a tablet computer, and a smart phone, and may generate video / images (electronically). For example, virtual video / images may be generated by a computer or the like. In this case, the video / image capturing process may be replaced by a process of generating relevant data.

[0059] The encoder 12 may encode the input video / images. For compression and compilation efficiency, the encoder 12 may perform a series of processes such as prediction, transformation, and quantization. The encoder 12 is capable of outputting encoded data (encoded video / image information) in the form of a bitstream.

[0060] The transmitter 13 can transmit the encoded video / image information in the form of a file or a stream transmission through a digital storage medium or a network, or output the data in the form of a bitstream to the receiver 21 of the decoding device 20. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 can include components for generating a media file in a predetermined file format and can include components for transmitting through a broadcast / communication network. The transmitter 13 can be provided as a transmission device separate from the encoder 12. In this case, the transmission device includes: at least one processor that acquires the encoded video / image information or the data output in the form of a bitstream; and a transmitter for delivering it in the form of a file or a stream transmission. The receiver 21 can extract / receive the bitstream from the storage medium or the network and send the bitstream to the decoder 22.

[0061] The decoder 22 can decode the video / image by performing a series of processes corresponding to the operations of the encoder 12, such as dequantization, inverse transformation, and prediction.

[0062] The renderer 23 can render the decoded video / image. The rendered video / image can be displayed through a display.

[0063] Overview of the image encoding device

[0064] Figure 2 is a view schematically showing an image encoding device to which an embodiment of the present disclosure can be applied.

[0065] As Figure 2 shown, the image encoding device 100 can include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 can be collectively referred to as a "prediction unit". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 can be included in a residual processor. The residual processor can also include the subtractor 115.

[0066] In some embodiments, all or at least some of the multiple components configuring the image encoding device 100 can be configured by one hardware component (e.g., the image encoding device 100 or a processor). In addition, the memory 170 can include a decoded picture buffer (DPB) and can be configured by a digital storage medium.

[0067] The image partitioner 110 may partition an input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). Coding units may be obtained by recursively partitioning a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree binary tree ternary tree (QT / BT / TT) structure. For example, a coding unit may be partitioned into multiple coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the partitioning of coding units, a quadtree structure may be applied first, and then a binary tree structure and / or a ternary tree structure may be applied. The coding process according to the present disclosure may be performed based on the final coding units that are no longer partitioned. The largest coding unit may be used as the final coding unit, or the coding units of a deeper depth obtained by partitioning the largest coding unit may be used as the final coding units. Here, the coding process may include processes of prediction, transformation, and reconstruction that will be described later. As another example, the processing unit of the coding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0068] The prediction unit (inter-frame prediction unit 180 or intra-frame prediction unit 185) may perform prediction on a block to be processed (current block) and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The prediction unit may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0069] The intra-frame prediction unit 185 may predict the current block by referring to samples in the current picture. According to the intra-frame prediction mode and / or intra-frame prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. The intra-frame prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional modes may include, for example, the DC mode and the planar mode. According to the level of detail of the prediction direction, the directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes. However, this is only an example, and more or fewer directional prediction modes may be used according to the settings. The intra-frame prediction unit 185 may determine the prediction mode applied to the current block by using the prediction mode applied to neighboring blocks.

[0070] The inter - frame prediction unit 180 may derive a prediction block of a current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter - frame prediction mode, the motion information may be predicted in units of blocks, sub - blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include inter - frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter - frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc. The reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter - frame prediction unit 180 may configure a motion information candidate list based on neighboring blocks and generate information specifying which candidate to use to derive the motion vector and / or reference picture index of the current block. Inter - frame prediction may be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter - frame prediction unit 180 may use the motion information of neighboring blocks as the motion information of the current block. In the case of the skip mode, different from the merge mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of a neighboring block may be used as a motion vector predictor, and the motion vector of the current block may be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference may mean the difference between the motion vector of the current block and the motion vector predictor.

[0071] The prediction unit may generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the prediction unit may not only apply intra - frame prediction or inter - frame prediction, but also apply intra - frame prediction and inter - frame prediction simultaneously to predict the current block. The prediction method of applying both intra - frame prediction and inter - frame prediction simultaneously to predict the current block may be referred to as combined inter - frame and intra - frame prediction (CIIP). In addition, the prediction unit may perform intra - block copy (IBC) to predict the current block. Intra - block copy may be used for content image / video compilation such as games, for example, screen content compilation (SCC). IBC is a method of predicting a current picture using a previously reconstructed reference block in the current picture at a position separated from the current block by a predetermined distance. When IBC is applied, the position of the reference block in the current picture may be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction in the current picture, but may be performed similar to inter - frame prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter - frame prediction techniques described in the present disclosure.

[0072] The prediction signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be transmitted to the transformer 120.

[0073] The transformer 120 can generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique can include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a karhunen-loève transform (KLT), a graph-based transform (GBT), or a conditional non-linear transform (CNT). Here, the GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. The CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Additionally, the transform process can be applied to square pixel blocks of the same size or can be applied to blocks that are not square but have a variable size.

[0074] The quantizer 130 can quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 can encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients can be referred to as residual information. The quantizer 130 can rearrange the block-form quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and generate information about the quantized transform coefficients based on the quantized transform coefficients in one-dimensional vector form.

[0075] The entropy encoder 190 can perform various encoding methods, such as exponential Golomb, context-adaptive variable length coding (CAVLC), context-adaptive binary arithmetic coding (CABAC), etc. The entropy encoder 190 can encode, together or separately, the information required for video / image reconstruction other than the quantized transform coefficients (e.g., the values of syntax elements, etc.). The encoded information (e.g., the encoded video / image information) can be transmitted or stored in the form of a bitstream in units of a network abstraction layer (NAL). The video / image information can also include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Additionally, the video / image information can also include general constraint information. The information signaled, transmitted, and / or syntax elements described in this disclosure can be encoded through the above encoding process and included in the bitstream.

[0076] The bitstream can be transmitted over a network or stored in a digital storage medium. The network can include a broadcast network and / or a communication network, and the digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. An emitter (not shown) that transmits the signal output from the entropy encoder 190 and / or a storage unit (not shown) that stores the signal can be included as internal / external elements of the image encoding device 100. Alternatively, an emitter can be provided as a component of the entropy encoder 190.

[0077] The quantized transform coefficients output from the quantizer 130 can be used to generate a residual signal. For example, the quantized transform coefficients can be dequantized and inverse-transformed by the dequantizer 140 and the inverse-transformer 150 to reconstruct the residual signal (residual block or residual samples).

[0078] The adder 155 adds the reconstructed residual signal to the prediction signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If the block to be processed has no residual, for example, in the case of applying the skip mode, the predicted block can be used as the reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0079] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to the filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to the filtering can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0080] The modified reconstructed picture transmitted to the memory 170 can be used as a reference picture in the inter-frame prediction unit 180. When inter-frame prediction is applied by the image encoding device 100, prediction mismatch between the image encoding device 100 and the image decoding device can be avoided and the encoding efficiency can be improved.

[0081] The DPB of the memory 170 may store the modified reconstructed picture for use as a reference picture in the inter - prediction unit 180. The memory 170 may store the motion information of the blocks from which the motion information in the current picture is derived (or encoded) and / or the motion information of the blocks that have been reconstructed in the picture. The stored motion information may be transmitted to the inter - prediction unit 180 and used as the motion information of spatially neighboring blocks or temporally neighboring blocks. The memory 170 may store the reconstructed samples of the reconstructed blocks in the current picture and may transmit the reconstructed samples to the intra - prediction unit 185.

[0082] Overview of the image decoding device

[0083] Figure 3 is a view schematically showing an image decoding apparatus to which embodiments of the present disclosure are applicable.

[0084] As Figure 3 shown, the image decoding apparatus 200 may include an entropy decoder 210, a de - quantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter - prediction unit 260, and an intra - prediction unit 265. The inter - prediction unit 260 and the intra - prediction unit 265 may be collectively referred to as a "prediction unit". The de - quantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0085] According to an embodiment, all or at least some of the multiple components configuring the image decoding apparatus 200 may be configured by hardware components (e.g., the image decoding apparatus 200 or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0086] The image decoding apparatus 200 that has received a bitstream including video / image information may reconstruct an image by performing processing corresponding to the processing performed by the Figure 2 image encoding apparatus 100. For example, the image decoding apparatus 200 may perform decoding using the processing units applied in the image encoding apparatus 100. Thus, the decoding processing unit may be, for example, a compilation unit. The compilation unit may be obtained through a partitioned compilation tree unit or a maximum compilation unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 may be reproduced by a reproduction apparatus (not shown).

[0087] The image decoding apparatus 200 may receive, in the form of a bitstream, from Figure 2The signal output by the image encoding device 100. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information can also include information about various parameter sets, such as the Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information can also include general constraint information. The image decoding device 200 can also decode the picture based on the parameter set information and / or general constraint information. The information and / or syntax elements signaled / received described in this disclosure can be decoded through the decoding process and obtained from the bitstream. For example, the entropy decoder 210 decodes the information in the bitstream based on an encoding method such as Exponential Golomb coding, CAVLC, or CABAC, and outputs the values of the syntax elements required for image reconstruction and the quantization values of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive the bins corresponding to each syntax element in the bitstream, use the decoding target syntax element information, neighboring blocks, and decoding information of the decoding target block or information of the symbols / bins decoded in the previous stage to determine the context model, perform arithmetic decoding on the bins by predicting the occurrence probability of the bins according to the determined context model, and generate symbols corresponding to the values of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The information related to prediction in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual values on which entropy decoding is performed in the entropy decoder 210, that is, the quantized transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the information about filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, the receiver (not shown) for receiving the signal output from the image encoding device can be further configured as an internal / external component of the image decoding device 200, or the receiver can be a component of the entropy decoder 210.

[0088] Meanwhile, the image decoding device 200 according to the present disclosure can be referred to as a video / image / picture decoding device. The image decoding device 200 can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoder 210. The sample decoder can include at least one of the dequantizer 220, inverse transformer 230, adder 235, filter 240, memory 250, inter-frame prediction unit 160, or intra-frame prediction unit 265.

[0089] The dequantizer 220 can dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order executed in the image coding device 100. The dequantizer 220 can dequantize the quantized transform coefficients by using quantization parameters (e.g., quantization step information) and obtain the transform coefficients.

[0090] The inverse transformer 230 can perform an inverse transform on the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0091] The prediction unit can perform prediction on the current block and generate a prediction block including prediction samples of the current block. The prediction unit can determine whether to apply intra prediction or inter prediction to the current block based on the information about the prediction output from the entropy decoder 210, and can determine a specific intra / inter prediction mode (prediction technique).

[0092] Similar to that described in the prediction unit of the image coding device 100, the prediction unit can generate a prediction signal based on various prediction methods (techniques) described later.

[0093] The intra prediction unit 265 can predict the current block by referring to samples in the current picture. The description of the intra prediction unit 185 also applies to the intra prediction unit 265.

[0094] The inter prediction unit 260 can derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include a motion vector and a reference picture index. The motion information can also include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks existing in the current picture and temporal neighboring blocks existing in the reference picture. For example, the inter prediction unit 260 can configure a motion information candidate list based on neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. The inter prediction can be performed based on various prediction modes, and the information about the prediction can include information indicating the inter prediction mode of the current block.

[0095] The adder 235 can generate a reconstructed block by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-frame prediction unit 260 and / or the intra-frame prediction unit 265). If the block to be processed has no residual, such as when the skip mode is applied, the prediction block can be used as the reconstructed block. The description of the adder 155 equally applies to the adder 235. The adder 235 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0096] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0097] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame prediction unit 260. The memory 250 can store the motion information of the blocks from which the motion information in the current picture is derived (or decoded) and / or the motion information of the blocks that have been reconstructed in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 260 to be used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 250 can store the reconstructed samples of the reconstructed blocks in the current picture and transmit the reconstructed samples to the intra-frame prediction unit 265.

[0098] In the present disclosure, the embodiments described in the filter 160, the inter-frame prediction unit 180, and the intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, the inter-frame prediction unit 260, and the intra-frame prediction unit 265 of the image decoding device 200.

[0099] Overview of CTU partitioning

[0100] As described above, the CUs can be obtained by recursively partitioning the CTU or the largest coding unit (LCU) according to a quadtree / binary tree / trinary tree (QT / BT / TT) structure. For example, the CTU can be first partitioned according to the QT structure. Subsequently, the leaf nodes of the QT structure can be further partitioned according to a multi-type tree structure.

[0101] Partitioning according to QT involves dividing the current CU (or CTU) into four equal parts. According to QT, the current CU can be partitioned into four CUs with the same width and the same height. When the current CU is no longer partitioned according to the QT structure, the current CU corresponds to a leaf node of the QT structure. The CU corresponding to the leaf node of the QT structure is no longer partitioned and can be used as the above-mentioned final compilation unit. Alternatively, the CU corresponding to the leaf node of the QT structure can be further partitioned according to a multi-type tree structure.

[0102] Generation of the MPM candidate list

[0103] Generally, when an image is partitioned into blocks, the current block to be compiled and neighboring blocks have similar image features. Therefore, the current block and neighboring blocks are likely to have the same or similar intra prediction modes. Thus, the encoder can use the intra prediction mode of neighboring blocks to encode the intra prediction mode of the current block.

[0104] For example, the encoder / decoder can generate a most probable mode (MPM) list for the current block. This MPM list can be referred to as an MPM candidate list. Here, the MPM can be a mode used to improve the compilation efficiency during the compilation of intra prediction modes considering the similarity between the current block and neighboring blocks. As described above, the MPM list may or may not include the planar mode. For example, when the MPM list includes the planar mode, the number of candidates in the MPM list can be 6. When the MPM list does not include the planar mode, the number of candidates in the MPM list can be 5.

[0105] The encoder / decoder can generate an MPM list including six MPMs.

[0106] To generate the MPM list, the following types of modes can be considered.

[0107] Default intra mode

[0108] Neighboring intra mode

[0109] Intra prediction mode (IPM) derived from neighboring inter modes

[0110] Decoder-side intra mode derivation (DIMD) mode

[0111] Derived intra mode

[0112] For the neighboring intra mode, neighboring blocks can be considered, i.e., the left neighboring block, the upper neighboring block, the lower left neighboring block, the upper right neighboring block, the upper left neighboring block, etc. When the neighboring intra mode is used as an MPM, the input order can vary according to the size information of the current block. For example, when the height of the block is greater than or equal to its width, the intra mode of the upper neighboring block can be considered first, and then the intra mode of the left neighboring block can be considered.

[0113] Intra-mode information can be obtained from the IPM buffer even if neighboring blocks have been coded in inter-mode rather than intra-mode. When the motion vectors of neighboring inter blocks indicate the position of an intra-mode, the intra-mode can be stored in the IPM buffer. The intra-mode stored in the IPM buffer can be used as the MPM mode for neighboring blocks.

[0114] MPM candidates can be generated using DIMD already described in 1.4.10. When the current block is not in DIMD mode, the intra-model exported using DIMD can be used as MPM candidates.

[0115] As described above, when the generated MPM list does not include the planar mode, the planar mode can be excluded from the list and the number of MPM list candidates can be set to five.

[0116] When matrix-based intra prediction (MIP) is not applied to the current block, the above-described MPM list generation method can be used. For example, the above-described MPM list generation method can be used to derive an intra prediction mode for linear interpolation intra prediction (LIP), position-dependent intra prediction combination (PDPC), multi-reference line (MRL), or intra sub-partition (ISP) intra prediction or conventional intra prediction. At the same time, the left neighboring block or the upper neighboring block can be coded based on the above MIP. In other words, MIP can be applied to the coding of the left neighboring block or the upper neighboring block. In this case, it is not appropriate to use the MIP intra prediction mode number of the neighboring block (left neighboring block / upper neighboring block) to which MIP has been applied without changing the MPM list of the current block to which MPM has not been applied. Therefore, for example, the intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP has been applied can be regarded as a direct current (DC) mode or a planar mode. As another example, the intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP has been applied can be mapped to a general intra prediction mode based on a mapping table and used for MPM list generation. In this case, the mapping can be performed based on the block size type of the current block. For example, the mapping table can be provided as follows.

[0117] [Table 1]

[0118]

[0119] Here, MIP IntraPredMode[xNbX][yNbX] indicates the MIP intra prediction mode of the neighboring block (left neighboring block / upper neighboring block), and the block size type MipSizeId indicates the block size type of the neighboring block or the current block. The numbers 0, 1, and 2 under the block size type value indicate the general intra prediction modes to which the MIP intra prediction mode is mapped in the case of the corresponding block size type.

[0120] For example, when the current block has a block size type of 0 and the MIP intra prediction mode number of the neighboring block is 10, the mapped general intra prediction mode number can be 18.

[0121] However, the mapping relationship is only an example and may change.

[0122] Meanwhile, when MIP is applied to the current block, an MPM list for the current block to which MIP is applied can be generated separately. This MPM list can be referred to by a name such as MIP MPM list (or MPM list for MIP, candMipModeList) to distinguish it from the MPM list in the case of the current block to which MIP is not applied. Hereinafter, to distinguish it from the latter, this MPM list is referred to as the "MIP MPM list", but it can also be referred to as the "MPM list".

[0123] The MIP MPM list can include n candidate blocks, and for example, n can be 3. The MIP MPM list can be generated based on the left neighboring block and the upper neighboring block of the current block. Among them, the left neighboring block can be the uppermost one among the neighboring blocks adjacent to the left boundary of the current block. In addition, the upper neighboring block can be the leftmost one among the neighboring blocks adjacent to the upper boundary of the current block.

[0124] For example, when MIP is applied to the left neighboring block (and when the block size type of the left neighboring block is the same as that of the current block), the first candidate intra prediction mode (or candMipModeA) can be set to be the same as the MIP intra prediction mode of the left neighboring block. In addition, for example, when MIP is applied to the upper neighboring block (and when the block size type of the upper neighboring block is the same as that of the current block), the second candidate intra prediction mode (or candMipModeB) can be set to be the same as the MIP intra prediction mode of the upper neighboring block. Meanwhile, the left neighboring block or the upper neighboring block can be compiled based on intra prediction rather than MIP. In other words, when compiling the left neighboring block or the upper neighboring block, an intra prediction type other than MIP can be applied. In this case, it is inappropriate to use the general intra prediction mode number of the neighboring block (left neighboring block / upper neighboring block) to which MIP has not been applied without any change as the candidate intra prediction model of the current block to which MPM has been applied. Therefore, for example, the MIP intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP has not been applied can be regarded as the MIP intra prediction mode of a specific value (such as 0, 1, 2, etc.). As another example, the general intra prediction mode of the neighboring block (left neighboring block / upper neighboring block) to which MIP has not been applied can be mapped to the MIP intra prediction mode based on a mapping table and used for generating the MIP MPM list. In this case, the mapping can be performed based on the block size type of the current block. For example, the mapping table can be provided as follows.

[0125] [Table 2]

[0126]

[0127] Here, IntraPredModeY[xNbX][yNbX] indicates the intra prediction mode of the neighboring block (left neighboring block / upper neighboring block). The intra prediction mode of the neighboring block can be the intra prediction mode of the luminance component (sample), that is, the intra luminance prediction mode. The block size type MipSizeId indicates the block size type of the neighboring block or the current block. The numbers 0, 1, and 2 under the block size type value indicate the MIP intra prediction modes to which the general intra prediction mode is mapped in the case of the corresponding block size type.

[0128] In addition, the neighboring block (e.g., left neighboring block / upper neighboring block) may be unavailable (e.g., outside the current picture, outside the current tile / slice, etc.), or even when MIP has been applied, an MIP intra prediction mode that is not available for the current block according to the block size type may have been used. In this case, the specific MIP intra prediction mode predefined for the first candidate and / or the second candidate can be used for the first candidate intra prediction mode or the second candidate intra prediction mode. In addition, the specific MIP intra prediction mode predefined for the third candidate can be used as the third candidate intra prediction mode.

[0129] For example, specific MIP intra prediction modes can be provided as shown in the following table. The specific MIP intra prediction mode can be referred to as the "MIP default candidate mode".

[0130] [Table 3]

[0131]

[0132] The MIP MPM list can be generated based on the first candidate intra prediction mode and the second candidate intra prediction mode. For example, when the first candidate intra prediction mode and the second candidate intra prediction mode are different from each other, the first candidate intra prediction mode can be set as the 0th candidate of the MIP MPM list (e.g., candMipModeList[0]), and the second candidate intra prediction mode can be set as the 1st candidate of the MIP MPM list (e.g., candMipModeList[1]). The specific MIP intra prediction mode predefined can be used as the 2nd candidate of the MIP MPM list (e.g., candMipModeList[2]).

[0133] On the other hand, when the first candidate in-frame prediction mode and the second candidate in-frame prediction mode are the same as each other, one of the first candidate in-frame prediction model and the second candidate in-frame prediction mode can be set as the 0th candidate of the MIP MPM list (e.g., candMipModeList[0]), and a predefined specific in-frame prediction mode for MIP can be used as the 1st candidate of the MIP MPM list (e.g., candMipModeList[1]) and the 2nd candidate of the MIP MPM list (e.g., candMipModeList[2]).

[0134] As described above, the in-frame prediction mode for MIP of the current block can be derived based on the MIP MPM list. In this case, as described above, the MPM flag, MPM index, and remaining in-frame prediction mode information that can be included in the in-frame prediction mode information for MIP can be referred to as "intra_mip_mpm_flag", "intra_mip_mpm_idx", and "intra_mip_mpm_remainder", respectively. The process for deriving the in-frame prediction mode for MIP from the MIP MPM list can be performed as described above in Figure 1 .2.1-1 and Figure 1 .2.1-2.

[0135] Generation of the secondary MPM candidate list

[0136] The encoder / decoder can generate a secondary MPM list for the current block. The secondary MPM list can also be referred to as the "secondary MPM candidate list". In addition, the secondary MPM list can be generated by excluding the in-frame modes of the MPM candidate list. For example, when the MPM list contains mode 50, the secondary MPM list may not include mode 50.

[0137] The encoder / decoder can generate M secondary MPM lists, and M can be 16.

[0138] To generate the secondary MPM list, the following types of modes can be considered.

[0139] Default in-frame mode

[0140] Neighboring in-frame mode

[0141] IPM derived from neighboring inter-frame mode

[0142] DIMD mode

[0143] Derived in-frame mode

[0144] For the adjacent intra mode, adjacent blocks can be considered, i.e., the left adjacent block, the upper adjacent block, the lower-left adjacent block, the upper-right adjacent block, the upper-left adjacent block, etc. When the adjacent intra mode is used as the secondary MPM, the input order can vary according to the size information of the current block. For example, when the height of the block is greater than or equal to the width of the block, the intra mode of the upper adjacent block can be considered first, and then the intra mode of the left adjacent block can be considered.

[0145] Even when the adjacent blocks have been coded in the inter mode rather than the intra mode, the intra mode information can be obtained from the IPM buffer. When the motion vector of the adjacent inter block indicates the position of the intra mode, the intra mode can be stored in the IPM buffer. The intra mode stored in the IPM buffer can be used as the secondary MPM mode of the adjacent block.

[0146] Template-based intra mode derivation (TIMD)

[0147] Figure 4 is a diagram showing the templates and reference samples used in TIMD. Refer to Figure 4 , the sum of absolute transform differences (SATD) between the predicted block predicted from the template region and the actual reconstructed samples can be calculated for the IPM intra mode of the adjacent adjacent intra blocks and inter blocks, and then the mode with the minimum SATD can be selected as the intra mode of the current block. Alternatively, two modes with the two minimum SATDs can be selected, and then the weighted sum of the predicted blocks for the two predicted modes can be calculated for mixing so that the result of the mixing can be used as the predicted block of the current block.

[0148] When the following expression is satisfied, the method of mixing two modes can be applied.

[0149] [Expression 1]

[0150]

[0151] When the above conditions are satisfied, two modes can be mixed together to generate a predicted block, and otherwise, only one mode with the minimum SATD value can be selected. The weight ratio for mixing two predicted blocks can be calculated according to the following Expression 2.

[0152] [Expression 2]

[0153]

[0154]

[0155] Decoder-side intra mode derivation ( DIMD)

[0156] Figure 5is a diagram schematically showing a Histogram of Oriented Gradients (HoG), and Figure 6 is a diagram schematically showing prediction block generation when the DIMD mode is applied.

[0157] In the DIMD mode, the intra prediction mode information itself may not be transmitted, but may be derived and used by the encoder and decoder. First, the horizontal gradient and the vertical gradient are calculated from the second neighboring sample columns and rows, and the HoG may be generated from the horizontal gradient and the vertical gradient. The HoG may be generated as Figure 5 shown. The HoG may be obtained by applying a Sobel filter to the L-shaped 3-pixel rows and columns adjacent to the current block. When the boundary of a block exists in different CTUs, texture analysis is not used.

[0158] Subsequently, as Figure 6 shown, two intra modes with the maximum histogram amplitudes may be selected, and then the prediction blocks predicted using the selected intra modes are mixed with the planar mode to generate the final prediction block. Weights may be introduced from the amplitudes of the histogram. In addition, a DIMD flag may be transmitted for each block to determine whether to use DIMD.

[0159] Embodiment

[0160] As seen from the DIMD technique and the TIMD technique, the prediction blocks generated using various modes may be combined to generate the final prediction block, and such a combined block may improve the intra prediction performance. In particular, in the case of the DIMD mode, the planar mode, the first intra mode (Intra Mode 1) and the second intra mode (Intra Mode 2), i.e., three prediction blocks, may be combined, which may significantly improve the intra prediction performance.

[0161] According to the present disclosure, based on the TIMD mode which is a template-based in-picture prediction mode, a new prediction block is generated by combining one or more prediction blocks, and the intra prediction performance may be further improved by combining multiple prediction blocks.

[0162] According to the present disclosure, as seen from the DIMD technique and the TIMD technique, the final prediction block may be generated by combining the prediction blocks generated using various modes, and such a combined block may result in an improvement in the intra prediction performance. In particular, in the case of the DIMD mode, the planar mode, the first intra mode and the second intra mode, i.e., three prediction blocks, may be combined, which may significantly improve the intra prediction performance.

[0163] Meanwhile, the following description of the prediction direction of the intra prediction mode and the prediction mode is based on Figure 7 the mode numbers shown in. However, this is for clarity of description, and the present disclosure is not limited thereto.

[0164] In the following, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0165] Figure 8 is a flowchart showing a process of deriving a TIMD mode according to an embodiment of the present disclosure. When TIMD information (e.g., TIMD flag) that can indicate whether to apply TIMD technology indicates that the TIMD mode is applied, TIMD can be applied. In other words, when TIMD is selected (S801), operations can be performed in the following order. Each operation can be omitted, and the order of operations can be changed.

[0166] When the TIMD mode is applied, an intra-mode candidate list can be obtained (obtain an intra-mode candidate list) (S802). In operation S802, an intra-mode candidate list can be generated. The above MPM list and / or supplementary MPM list can be used to generate the intra-mode candidate list. For example, the intra-mode candidate list can be generated from the intra-modes of only the MPM list, the intra-modes of only the supplementary MPM list, or the intra-modes of both lists. At the same time, when the DC mode, horizontal direction mode, or vertical direction mode does not exist in the MPM list and / or supplementary MPM list, an intra-mode candidate list can be generated to include this mode. In addition, nearby modes ipm - k to ipm + k (excluding imp) within a range k (k is a natural number) starting from the intra-mode ipm included in the MPM list and / or supplementary MPM list can be included in the intra-mode candidate list. For example, k can be 2. As an example, when a specific mode (e.g., mode 18 (horizontal direction mode)) is included in the MPM list and / or supplementary MPM list, nearby modes (e.g., modes 16, 17, 19, 20, etc.) of the specific mode (e.g., mode 18) can be included in the intra-mode candidate list.

[0167] Subsequently, the template cost of the mode can be calculated (obtain the template cost of the mode) (S803). For example, in operation S803, the intra-modes of the intra-mode candidate list obtained in the previous operation can be utilized. As an example, based on the intra-modes of the intra-mode candidate list, the SATD value or sum of absolute differences (SAD) value between the predicted samples generated using the intra-mode of the left template region and / or upper template region of the current block and the reconstructed region can be calculated and compared to calculate the template cost.

[0168] Subsequently, the modes can be sorted based on the calculated cost (sort the modes based on the cost) (S804). As an example, in operation S804, the intra-modes can be sorted in ascending order of the template cost. At the same time, operation S804 can be performed while calculating the template cost of the mode. In other words, operation S804 can be included in operation S803.

[0169] Subsequently, N modes can be selected (selecting N modes) (S805). For example, N (N is a natural number) intra modes can be selected in ascending order of template cost, and the final N intra modes can be used to generate a prediction block. As an example, a fixed specific number of modes can be selected and used according to a predefined rule between the encoder and the decoder. N can be an integer such as 2, 3, etc. According to whether the cost value of each intra candidate mode is lower than a specific threshold, only the modes with a cost lower than the threshold can be selected. In other words, when the N modes have a cost lower than the threshold, the N modes can be selectively used. As an example, the threshold can be signaled as a bitstream or predefined as a commitment between the encoder and the decoder and used. In this case, the threshold (e.g., variable threshold) can be defined according to the following expression.

[0170] [Expression 3]

[0171]

[0172] For example, the variable can be any value that is a natural number, and the variable iTempWidth represents the horizontal length of the left template area and can be 0 when the left template area is not used. For example, the variable iTempWidth can vary according to the size of the current prediction block. As an example, when the horizontal length of the current prediction block is a specific value (e.g., 8) or less, the variable iTempWidth can be a specific value (e.g., 2), and otherwise can be defined as another specific value (e.g., 4). The variable iTempHeight represents the vertical length of the upper template area and can be 0 when the upper template area is not used. For example, the variable iTempHeight can vary according to the size of the current prediction block. As an example, when the vertical length of the current prediction block is a specific value (e.g., 8) or less, the variable iTempHeight can be a specific value (e.g., 2), and otherwise can be defined as another specific value (e.g., 4). Additionally, the variable width can be the horizontal length of the current block, and the variable height can be the vertical length of the current block.

[0173] Meanwhile, the following can be defined for the N selected intra modes.

[0174] 1. The variable uiBestCost can be defined. The variable uiBestCost can be the lowest template cost value in the intra mode candidate list. In addition, the value of uiBestCost can be changed in the operation before the operation S806 of obtaining the fusion information (e.g., fusion flag) indicating whether to perform fusion (i.e., combination). For example, when the variable uiBestCost is greater than the maximum cost value MaxCost, the value of uiBestCost can be changed as follows. As an example, the maximum cost value MaxCost can be a predefined value or a value determined by the encoder.

[0175] (1) The template cost value can be additionally calculated for the neighboring intra modes of the intra mode with the value of the variable uiBestCost. For example, when the intra mode with the value of the variable uiBestCost is a specific mode (e.g., mode 50), the template cost can be additionally calculated for the neighboring modes (e.g., mode 49, mode 51, etc.). Therefore, when the template cost value of the neighboring mode is lower than the template cost value of the intra mode with the value of the variable uiBestCost, the variable uiBestCost can be changed to a lower (i.e., smaller) value. Here, the method of calculating the cost value can be the same as the operation S803 of calculating the template cost of the mode described above.

[0176] 2. The variable uiSecondaryCost can be defined. The variable uiSecondaryCost can be the second lowest template cost value in the intra mode candidate list.

[0177] In addition, the value of uiSecondaryCost can be changed in the operation before the operation S806 of obtaining the fusion information (e.g., fusion flag) indicating whether to perform fusion (i.e., combination). For example, when the variable uiSecondaryCost is greater than the variable MaxCost which is the maximum cost value, the value of uiSecondaryCost can be changed as follows. As an example, the maximum cost value MaxCost can be a predefined value or a value determined by the encoder.

[0178] (1) The template cost value can be additionally calculated for neighboring intra - modes of the intra - mode with the uiSecondaryCost value. For example, when the intra - mode with the uiSecondaryCost value is a specific mode (e.g., mode 34), the template cost value can be additionally calculated for neighboring modes (e.g., mode 33, mode 35, etc.). Thus, when the template cost value of the neighboring mode is less than the template cost value of the intra - mode with the value of the variable uiSecondaryCost, the variable uiSecondaryCost can be changed to a smaller value. During this process, the variable uiSecondaryCost can have a value smaller than the variable uiBestCost. Here, the method of calculating the cost value can be the same as the method S803 for calculating the template cost of the mode described above.

[0179] 3. A variable can be defined . The variable can indicate the k - th minimum template cost value for multiple (e.g., N) intra - modes. Additionally, in the operation before the operation S806 of obtaining the fusion information (e.g., fusion flag) indicating whether to perform prediction combination (i.e., fusion), the value of the variable can be changed as follows. For example, when the variable is greater than the maximum cost value (e.g., MaxCost), the value of the variable can be changed as follows. (1 ≤ k ≤ N)

[0180] (1) Meanwhile, the template cost value can be additionally calculated for neighboring intra - modes of the intra - mode with the value of the variable . For example, when the intra - mode with the value is a specific mode (e.g., mode 50), the template cost can be additionally calculated for neighboring modes (e.g., mode 49, mode 51, etc.). Thus, when the template cost value of the neighboring mode is less than the template cost value of the intra - mode with the value, can be changed to a smaller value. Here, the method of calculating the cost value can be the same as the method S803 for calculating the template cost of the mode described above.

[0181] 4. A variable MaxCost can be defined. MaxCost can be the maximum cost value, which can be any value. For example, the variable MaxCost can be calculated according to the following expression.

[0182] [Expression 4]

[0183]

[0184] The variables iTempWidth, iTempHeight, height, and width used in Expression 4 have been described above, and thus redundant descriptions thereof will be omitted.

[0185] Subsequently, it can be determined whether to perform intra-frame fusion, that is, intra-frame prediction fusion (S806). For example, it can be determined whether to perform intra-frame prediction fusion based on fusion information (e.g., a fusion flag). In other words, in operation S806, it can be determined whether to perform prediction combination. For example, it can be determined whether to perform prediction combination according to specific conditions. As an example, it can be determined whether to perform prediction combination under the following conditions.

[0186] [Expression 5]

[0187]

[0188]

[0189]

[0190] When at least one of the above expressions is satisfied, it can be determined to perform prediction combination. For example, in Expression 5, uiBestCost can be the best cost value, which is the same as the uiBestCost defined during operation S805 of selecting N modes. Therefore, redundant descriptions thereof will be omitted. In addition, the variable uiSecondaryCost indicating the secondary cost value is the same as the uiSecondaryCost defined during operation S805 of selecting N modes, and thus redundant descriptions thereof will be omitted. At the same time, p and q can be any values. For example, p and q can be different specific values. As an example, p and q can be determined to be 1 and 2 respectively.

[0191] Subsequently, based on the application of prediction combination (e.g., when the value of the fusion flag is true) (S807), the prediction blocks predicted using N selected intra-frame modes can be combined, that is, undergo intra-frame prediction fusion, and the result of the intra-frame prediction fusion can be used as the final prediction block. After generating the prediction mode using the selected mode, PDPC is applicable, or after combining the prediction blocks, PDPC can be applied to the final prediction block. At the same time, general intra-frame prediction can be performed based on the non-application of prediction combination (e.g., when the value of the fusion flag is false) (S808). According to general intra-frame prediction, the prediction block generated in a single intra-frame mode can be used as the final prediction block.

[0192] According to an embodiment, in the intra-frame prediction fusion operation S807, N prediction blocks can be generated in multiple modes (i.e., N modes). For example, the prediction blocks can be combined based on the following expression.

[0193] [Expression 6]

[0194]

[0195]

[0196] In the above expression, can be the final prediction block. Additionally, can be a prediction block predicted using an intra mode with the k-th smallest template cost. That is, multiple prediction blocks can be sorted in the order of template cost and are referred to as the first prediction block, the n-th prediction block, etc. Additionally, can be any integer, which can be a weight applied to each prediction block, and can be maintained. Here, l can be any natural number. N can be the number of combined prediction blocks, and k can be a natural number equal to or greater than 1 and equal to or less than N. According to this embodiment, can be calculated based on the following look-up table.

[0197] [Expression 7]

[0198]

[0199]

[0200]

[0201]

[0202]

[0203]

[0204]

[0205]

[0206] For example, in order to apply a larger with a lower computational template cost, i.e., a larger weight, s1 and s0 in the above expression can be calculated as follows.

[0207] [Expression 8]

[0208]

[0209]

[0210] For example, N can be the number of combined prediction blocks, and It can be the k-th lowest template cost value for multiple (e.g., N) intra modes. k can be a natural number equal to or greater than 1 and equal to or less than N.

[0211] As another example, in order to apply a larger with a lower computational template cost, that is, a larger weight, s1 and s0 can be calculated as follows.

[0212] [Expression 9]

[0213]

[0214]

[0215] For example, in Expression 9, N can be the number of combined prediction blocks, and can be the k-th lowest template cost value for N intra modes. k can be a natural number equal to or greater than 1 and equal to or less than N.

[0216] As another example, in order to apply a larger with a lower computational template cost, that is, a larger weight, s1 and s0 can be calculated as follows.

[0217] [Expression 10]

[0218]

[0219]

[0220] For example, in the above Expression 10, the variable MaxCostValue can be any cost value larger than and the N template costs can always have values smaller than MaxCostValue. In other words, the variable MaxCostValue can represent any maximum cost value, and for example, MaxCostValue can be equal to . In addition, N can be the number of combined prediction blocks, that is, the number of blocks undergoing prediction combination (fusion), can be the k-th lowest template cost value for N intra modes, and k can be a natural number equal to or greater than 1 and equal to or less than N.

[0221] Meanwhile, can be used by way of example Calculated according to the embodiment, in the TIMD mode, the prediction blocks based on the planar mode can always be combined regardless of the template cost of the planar mode. Additionally, the planar mode can have a fixed weight value. For example, in order to generate the final prediction block, a fixed weight value corresponding to a specific ratio (e.g., 1 / 4 or 1 / 8) of the total weight can be applied to the planar prediction block. As an example, when the total weight sum is 64, 16 which is 1 / 4 of the total weight or 8 which is 1 / 8 of the total weight can be used as the fixed weight value.

[0222] Additionally, in the TIMD mode, the weight values derived based on the template cost can be used. The weight values derived based on the template cost can be calculated in the same manner as , s0 and s1.

[0223] According to this embodiment, in the TIMD mode, the modes can be sorted in ascending order of the template cost. For example, when one or a selective combination of the following conditions is satisfied, the planar mode can be selected. In other words, when at least one of the following conditions is satisfied, the planar mode can be used as a candidate prediction mode for combination. That is, the prediction block based on the planar mode can be used as a candidate prediction block for prediction combination.

[0224] [Expression 11]

[0225]

[0226]

[0227]

[0228]

[0229]

[0230] The variable uiBestCost is a variable of the cost value, which is the same as the variable uiBestCost described in the operation S805 of selecting N modes, and thus its redundant description will be omitted. The variable uiSecondaryCost is also the same as the above variable uiSecondaryCost, and thus its redundant description will be omitted. Additionally, the variable planarCost related to the template cost of the planar mode can be the cost of the template area calculated for the planar mode in the operation S803 of calculating the template cost. p and q can be any specific values, and for example, can be determined as 1 and 2 respectively.

[0231] Meanwhile, when it is determined in operation S806 of determining whether to perform prediction combination that prediction combination is to be applied (for example, the fusion flag is true), at least one of the conditions in Expression 11 may be a condition for combination between a prediction block based on a planar mode and another prediction block. In other words, at least one condition may be applied as a condition for additional combination with the planar mode, and in the operation of determining whether to perform fusion (prediction combination), it may be determined whether a condition including at least one condition is satisfied.

[0232] Meanwhile, in condition 5 of Expression 11, the variable Threshold may be any value and may be a threshold value. Further, for example, the variable Threshold may be determined according to the following expression.

[0233] [Expression 12]

[0234]

[0235] For example, the variable β may be any natural number, and the variable iTempWidth may be the horizontal length of the left template region and may be 0 when the left template region is not used. The variable iTempWidth may vary according to the size of the current prediction block. For example, when the horizontal length of the current prediction block is a specific value (e.g., 8) or less, iTempWidth may be a specific value (e.g., 2), and otherwise may be defined as another specific value (e.g., 4). The variable iTempHeight represents the vertical length of the upper template region and may be 0 when the upper template region is not used. The variable iTempHeight may vary according to the size of the current prediction block. For example, when the vertical length of the current prediction block is a specific value (e.g., 8) or less, the variable TempHeight may be a specific value (e.g., 2), and otherwise may be defined as another specific value (e.g., 4). Additionally, the variable width may be the horizontal length of the current block, and the variable height may be the vertical length of the current block.

[0236] According to this embodiment, a prediction block based on a planar mode can be combined with another prediction block only when the size of the current block is greater than, less than, or equal to a specific size. In other words, when applying prediction combination (fusion), the planar mode can be selected only when a specific condition is satisfied. For example, the planar mode can be additionally combined with another prediction block based on the comparison result between the size of the current block (e.g., either the width or the height, or the product of the width and the height (e.g., width * height)) and a specific value (e.g., 16) (e.g., when the size of the current block is greater than the specific value or equal to or greater than the specific value). As another example, the planar mode can be additionally combined with another prediction block based on the comparison result between the size of the current block (e.g., either the width or the height, or the product of the width and the height (e.g., width * height)) and a specific value (e.g., 1024) (e.g., when the size of the current block is less than the specific value or equal to or less than the specific value). This condition can be additionally used in combination with conditions 1 to 5 of Expression 11.

[0237] Meanwhile, according to this embodiment, assume that in the TIMD mode, the cost value for a specific intra mode is the k-th cost value. In this case, when the k-th cost value satisfies at least one of the following conditions, the prediction block of the specific intra mode can be used as a candidate prediction block for combination. In other words, each prediction block of all intra modes that satisfy the following conditions can be used as a candidate for combination, and the number of candidate blocks for combination can be limited to be less than a specific number (e.g., M is a natural number).

[0238] [Expression 13]

[0239]

[0240]

[0241]

[0242]

[0243]

[0244]

[0245]

[0246]

[0247] The variable uiBestCost can be the same as the variable uiBestCost described in the operation S805 of selecting N modes. The variable uiSecondaryCost and the variable Similarly, they are the same as the variable uiSecondaryCost and the variable described in the operation S805 of selecting N modes, respectively. Therefore, redundant descriptions thereof will be omitted. The same. Therefore, redundant descriptions thereof will be omitted.

[0248] When the template cost value of the neighboring mode of the intra mode with a value is lower than the template cost value of the intra mode with a value, it can be changed to the template cost value of the neighboring node, that is, the smaller of the two values. In this process, the cost value can have a value smaller than the variable uiBestCost and / or the variable uiSecondaryCost. p and q can be any values and can be calculated as different values. For example, p can be determined as a specific value (e.g., 1), and q can be determined as another specific value (e.g., 2). For example, when k is an integer greater than or equal to 2, conditions 7 and 9 of the above Expression 13 can be applied. Meanwhile, when the information indicating whether to perform fusion (e.g., fusion flag) in the operation S806 of determining whether to perform prediction combination is true (i.e., when the information indicating whether to perform fusion represents applying fusion), the condition can subsequently be applied as a condition for additional combination with another intra mode, and in the operation S806 of determining whether to perform fusion, it can also be determined whether the condition is satisfied.

[0249] Meanwhile, when performing prediction block combination, plane prediction blocks, that is, blocks predicted using the plane mode, can be combined. Here, the blocks predicted using the plane mode can be blocks predicted using the horizontal plane mode or the vertical plane mode. For example, the process of generating a prediction block by applying the horizontal plane mode can be related to the following expression.

[0250] [Expression 14]

[0251]

[0252] For example, the process of generating a prediction block by applying the vertical plane mode can be related to the following expression.

[0253] [Expression 15]

[0254]

[0255] In Expressions 14 and 15, the predicted sample of the current block can be represented as pred(x, y). In this case, for example, when the upper left pixel of the current block is at position (0, 0), x can be the x-axis coordinate of the current block, and y can be the y-axis coordinate of the current block. In addition, rec(x, y) can be the prereconstructed reference sample at position (x, y), W can be the width of the current block, and H can be the height of the current block.

[0256] Meanwhile, for example, in the TIMD mode, when generating a prediction block using a vertical direction mode or a neighboring mode of the vertical direction mode (e.g., modes 34 to 66, etc.), a prediction mode can be generated using a vertical plane mode, and the prediction block based on the vertical plane mode can be used for prediction combination.

[0257] As another example, in the TIMD mode, when generating a prediction block using a horizontal direction mode or a neighboring mode of the horizontal direction mode (e.g., modes 2 to 33, etc.), a prediction mode can be generated using a horizontal plane mode, and the prediction block based on the horizontal plane mode can be used for prediction combination.

[0258] According to an embodiment of the present disclosure, both the compilation quality and efficiency can be improved by performing intra prediction combination (intra fusion) and selecting a prediction mode used in the prediction combination based on a template cost, etc.

[0259] Meanwhile, according to an embodiment, when all or at least one of the following conditions are satisfied, a block predicted based on a plane mode can be combined with another prediction block.

[0260] 1. When compiling a pre-reconstructed block adjacent to the current block in a plane mode and the intra prediction mode of the current block is not a plane mode.

[0261] 2. When the size of the current block is greater than or less than a specific size.

[0262] (1) For example, only when the size of the current block (e.g., at least one of the width and height of the current block or the product of the width and height of the current block (e.g., width * height)) is greater than a specific value (e.g., 16) or equal to or greater than a specific value, the plane prediction block can be combined with another prediction block.

[0263] (2) For example, only when the size of the current block (e.g., at least one of the width and height of the current block or the product of the width and height of the current block (e.g., width * height)) is less than a specific value (e.g., 1024) or equal to or less than a specific value, the plane prediction block can be combined with another prediction block.

[0264] 3. When additional information (e.g., plane combination flag) indicating whether to combine the plane prediction block is signaled and the value of the flag is true.

[0265] 4. When at least one of the above other combination conditions (e.g., Expression 11, Expression 13, etc.) is satisfied in the TIMD mode.

[0266] According to this embodiment, a prediction block predicted using the planar mode can be combined with another prediction block. The other prediction block combined with the prediction block predicted using the planar mode can be the following prediction blocks.

[0267] 1. A prediction block predicted using an intra prediction mode selected from MPM, secondary MPM, and non-MPM.

[0268] (1) Non-MPM can be the remaining intra mode that is not among the candidates in the MPM list and the secondary MPM list.

[0269] 2. In the case of the MRL mode, non-adjacent reference samples and an intra prediction mode are used to predict the prediction block.

[0270] (1) Non-adjacent reference samples or adjacent reference samples can be used to predict the planar prediction block to be combined.

[0271] 3. In the case of the ISP mode, the prediction block is predicted in sub-block units.

[0272] (1) The planar prediction block to be combined can be generated in sub-block units or CU units. When the planar prediction block is generated in sub-block units, the sub-block units can be combined. Alternatively, the sub-block units can not be combined, and an intra mode different from the planar mode can be generated in the sub-block units and then combined in the CU units.

[0273] (2) Additionally, the planar prediction block can be combined with only a specific sub-block. For example, when the sub-blocks halved in the vertical or horizontal direction are SB1 and SB2, the planar prediction block can be combined with only sub-block SB1 or SB2.

[0274] 4. A prediction block generated in the TIMD mode

[0275] (1) The prediction blocks can be combined regardless of the existence of the variable timdIsBlended related to TIMD mixing, or the prediction blocks can be combined only when the variable timdIsBlended is 0 or 1.

[0276] 1) In the TIMD mode, timdIsBlended indicates 1 (= true) when mixing prediction blocks, and indicates 0 (= false) in the case of a single prediction block without mixing.

[0277] 5. A prediction block generated in the MIP mode

[0278] Another prediction block combined with a prediction block predicted using the planar mode may be a single prediction block or a prediction block obtained by combining two or more prediction blocks. For example, a prediction block predicted using an intra prediction mode selected by MPM, secondary MPM, or non-MPM can be obtained by combining prediction block A predicted using a specific intra prediction mode (e.g., m) and the k-th reference sample row and prediction block B predicted using the intra prediction mode m and the (k + 1)-th reference sample row.

[0279] For example, in the case of combining a planar prediction block with another prediction block, a fixed weight value can be applied to the planar prediction block. For example, in order to generate a final prediction block, a fixed weight value corresponding to a specific ratio (e.g., w) of the total weight can be used for the planar mode. w can be any value and, as an example, can be 1 / 4 or 1 / 8.

[0280] For example, in the case of combining a planar prediction block with another block, a cost value can be derived from a template region adjacent to the current block, and a weight value can be used. For example, the method of calculating the weight can be the same as described above, and thus its redundant description will be omitted.

[0281] In addition, when compiling a pre-reconstructed block adjacent to the current block in the planar mode, the weight for predicting the combined planar prediction block can be increased and applied. For example, when a specific ratio (e.g., 1 / 4) of the total weight is used for a prediction block predicted using the planar mode and a pre-reconstructed block adjacent to the current block is predicted using a planar node, a value (e.g., 1 / 2) higher than the above specific ratio (e.g., 1 / 4) of the total weight can be used as the weight.

[0282] When the cost value of the planar mode is within a specific range, that is, when the planar mode is selectable due to its low cost value, during the process of generating another prediction block to be combined with the planar prediction block, the weight of the planar prediction block can be increased and applied. For example, when combining two prediction blocks in the TIMD mode, but the planar mode corresponds to the third lowest cost value, the weight of the planar prediction block can be increased more than before.

[0283] As described above, a planar prediction block can be a block predicted using the planar mode. Additionally, a block predicted using a horizontal planar mode or a vertical planar mode can be referred to as a planar prediction block. Meanwhile, a planar prediction block can be based on the horizontal planar mode and combined with another prediction block, and the predicted sample pred(x, y) generated based on the horizontal planar mode can be calculated as follows.

[0284] [Expression 16]

[0285]

[0286] The predicted sample pred(x, y) generated based on the vertical plane mode can be calculated as follows.

[0287] [Expression 17]

[0288]

[0289] In the above expression, when the upper-left pixel of the current block is at position (0, 0), x can be the x-axis coordinate of the current block, and y can be the y-axis coordinate of the current block. Additionally, rec(x, y) can be the reconstructed sample at position (x, y), which is a reference sample, W can be the width of the block, and H can be the height of the block.

[0290] Meanwhile, as an example, when using a specific intra-frame mode to generate another predicted block to be combined with a planar prediction block, the planar prediction block can be a vertical planar prediction block. In other words, the planar prediction block can be a block predicted using the vertical plane mode. For example, the specific intra-frame mode can include the vertical direction mode and the nearby modes of the vertical direction mode (e.g., modes 34 to 66). As another example, in the case of generating another predicted block to be combined with a planar prediction block using a specific intra-frame mode, the planar prediction block can be a horizontal planar prediction block. In other words, the planar prediction block can be a block predicted using the horizontal plane mode. For example, the specific intra-frame mode can include the horizontal direction mode and the nearby modes of the horizontal direction mode (e.g., modes 2 to 33).

[0291] According to the present disclosure, a predicted block predicted using a planar mode can be combined with a predicted block predicted using a mode other than the planar mode, and the prediction performance can be improved by combining the planar prediction block with the predicted block.

[0292] Figure 9 is a flowchart showing an image decoding method that can be executed by an image decoding apparatus according to an embodiment of the present disclosure. Since Figure 9 the image decoding method can be based on the above embodiment, redundant descriptions will not be repeated here.

[0293] For example, template-based intra prediction information of a current block can be obtained for image decoding (S901). Additionally, two or more prediction blocks of the current block can be generated based on the template-based intra prediction information (S902). Additionally, the current block can be reconstructed based on a weighted sum of the two or more prediction blocks (S903). For example, the weight for each prediction block used for the weighted sum can be determined according to the cost of template matching. Additionally, the weight can be determined based on the sum of the number of prediction blocks and the cost of template matching. Additionally, the weight can be determined based on a value calculated by subtracting the cost of template matching from a predefined value. Meanwhile, for example, a planar mode can be used to predict the first prediction block among the two or more prediction blocks (the first prediction block and the nth prediction block). Additionally, the first prediction block can be predicted using the planar mode based on the size of the current block. Additionally, the weight of the first prediction block can be calculated as a predefined fixed weight value. Additionally, the weighted sum can be calculated only from some of the two or more prediction blocks based on predefined conditions. Additionally, the predefined conditions can include a condition of the size of the current block. Additionally, the predefined conditions can include a condition of the intra prediction mode of the reconstructed block of the current block. Meanwhile, for example, the weight of the first prediction block can be calculated based on the intra prediction mode of the reconstructed block (another pre-reconstructed block). Additionally, a horizontal or vertical planar mode can be used to predict the first prediction block among the two or more prediction blocks.

[0294] Meanwhile, Figure 8 The image decoding method disclosed in Figure 8 corresponds to an embodiment of the present disclosure. Therefore, some operations can be changed or removed, and the order of some operations can be changed. These embodiments are also included in the embodiments of the present disclosure.

[0295] Figure 10 is a flowchart showing an image encoding method that can be performed by an image encoding apparatus according to an embodiment of the present disclosure. Since Figure 10 the image encoding method of Figure 10 can be based on the above embodiments, and thus redundant descriptions will not be repeated.

[0296] For example, it may be determined that the current block undergoes template-based intra prediction for image coding (S1001). In addition, two or more prediction blocks for the current block may be generated based on the template-based intra prediction (S1002). Subsequently, a weighted sum of the two or more prediction blocks for the current block may be calculated (S1003). For example, the weight for each prediction block used for the weighted sum may be determined according to the cost based on template matching. In addition, the weight may be determined based on the sum of the number of prediction blocks and the cost based on template matching. In addition, the weight may be determined based on the value calculated by subtracting the cost based on template matching from a predefined value. Meanwhile, for example, the planar mode may be used to predict the first prediction block among the two or more prediction blocks (the first prediction block and the nth prediction block). In addition, the planar mode may be used to predict the first prediction block based on the size of the current block. In addition, the weight of the first prediction block may be calculated as a predefined fixed weight value. In addition, the weighted sum may be calculated only from some of the two or more prediction blocks based on a predefined condition. In addition, the predefined condition may include a condition of the size of the current block. In addition, the predefined condition may include a condition of the intra prediction mode of the reconstructed block of the current block. Meanwhile, for example, the weight of the first prediction block may be determined based on the intra prediction mode of the reconstructed block (another pre-reconstructed block). In addition, the horizontal or vertical planar mode may be used to predict the first prediction block among the two or more prediction blocks.

[0297] Meanwhile, Figure 9 The image decoding method disclosed in Figure 9 corresponds to an embodiment of the present disclosure. Therefore, some operations may be changed or removed, and the order of some operations may be changed. These embodiments are also included in the embodiments of the present disclosure.

[0298] Various embodiments of the present disclosure may be used alone or in combination with other embodiments.

[0299] For clarity of description, the exemplary methods of the present disclosure are presented as a series of operations, but this is not intended to limit the order of execution of the steps, and if necessary, the steps may be performed simultaneously or in a different order. To implement the method according to the present disclosure, in addition to the illustrated steps, other steps may be included, steps excluding some steps may be included, or some steps may be excluded and other additional steps may be included.

[0300] In the present disclosure, an image coding device or an image decoding device that performs a specific operation (step) may perform an operation (step) of checking the conditions or circumstances for performing the operation (step). For example, when it is described that a certain operation is performed when a certain condition is satisfied, the image coding device or the image decoding device may perform an operation of determining whether the certain condition is satisfied, and then perform the certain operation.

[0301] The various embodiments of the present disclosure are intended to describe representative aspects of the present disclosure rather than list all possible combinations, and the details described in the various embodiments can be applied independently or in combinations of two or more.

[0302] In addition, the various embodiments of the present disclosure can be implemented by hardware, firmware, software, or a combination thereof. To be implemented by hardware, the various embodiments of the present disclosure can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0303] Furthermore, the image decoding device and the image encoding device to which the embodiments of the present disclosure are applied can be included in a multimedia broadcast transceiver, a mobile communication terminal, a home movie video device, a digital movie video device, a surveillance camera, a video chat device, a device for real-time communication such as video communication, a mobile streaming device, a storage medium, a camera, a video-on-demand (VoD) service provider, an over-the-top (OTT) video device, an Internet streaming service provider, a three-dimensional (3D) video device, a videophone, a medical video device, etc., and can be used to process video signals or data signals. For example, the OTT video device can include a game console, a Blu-ray player, an Internet-connected television (TV), a home theater system, a smart phone, a tablet personal computer (PC), a digital video recorder (DVR), etc.

[0304] Figure 11 is a view showing a content streaming system to which the embodiments of the present disclosure can be applied.

[0305] As Figure 11 shown, the content streaming system to which the embodiments of the present disclosure are applied can mainly include an encoding server, a streaming server, a network server, a media storage, a user device, and a multimedia input device.

[0306] The encoding server compresses the content input from a multimedia input device such as a smart phone, a camera, a portable video camera, etc. into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, a camera, a video camera, etc. directly generates a bitstream, the encoding server can be omitted.

[0307] The bitstream can be generated by the image encoding method or the image encoding device of the embodiments of the present disclosure, and the streaming server can temporarily store the bitstream during the process of sending or receiving the bitstream.

[0308] The streaming server sends multimedia data to the user device based on a request from the user via the network server, and the network server serves as a medium for notifying the user of the service. When the user requests a desired service from the network server, the network server can deliver it to the streaming server, and the streaming server can send multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves as controlling commands / responses between devices in the content streaming system.

[0309] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream within a predetermined time.

[0310] Examples of the user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a tablet computer, a superbook, a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display), a digital TV, a desktop computer, a digital signage, etc.

[0311] Each server in the content streaming system can operate as a distributed server, and in this case, the data received from each server can be distributed.

[0312] The scope of the present disclosure includes software or executable commands (e.g., an operating system, an application, firmware, a program, etc.) for enabling the operation of a method according to various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or commands stored thereon and executable on the device or the computer.

[0313] Industrial Applicability

[0314] Embodiments of the present disclosure can be used for encoding or decoding an image.

Claims

1. An image decoding method, comprising: obtaining template-based intra prediction information for a current block; generating two or more prediction blocks for the current block according to the template-based intra prediction information; and reconstructing the current block according to a weighted sum of the two or more prediction blocks, wherein weights for each prediction block used for the weighted sum are determined according to a cost based on template matching.

2. The image decoding method according to claim 1, wherein the weights are determined according to the number of prediction blocks and a sum of costs based on template matching.

3. The image decoding method according to claim 1, wherein the weights are determined according to a value calculated by subtracting the cost based on template matching from a predefined value.

4. The image decoding method according to claim 1, wherein a planar mode is used to predict a first prediction block among the two or more prediction blocks.

5. The image decoding method according to claim 4, wherein according to the size of the current block, the planar mode is used to predict the first prediction block.

6. The image decoding method according to claim 4, wherein the weight of the first prediction block is calculated as a predefined fixed weight value.

7. The image decoding method according to claim 1, wherein according to predefined conditions, a weighted sum is calculated only from some of the two or more prediction blocks.

8. The image decoding method according to claim 7, wherein the predefined conditions include conditions for the size of the current block.

9. The image decoding method according to claim 7, wherein the predefined conditions include conditions for an intra prediction mode of a reconstructed block of the current block.

10. The image decoding method according to claim 1, wherein weights of a first prediction block are calculated according to an intra prediction mode of a reconstructed block.

11. The image decoding method according to claim 1, wherein a horizontal or vertical planar mode is used to predict a first prediction block among the two or more prediction blocks.

12. An image encoding method, comprising: determining a current block to undergo template-based intra prediction; generating two or more prediction blocks for the current block according to the template-based intra prediction; and calculating a weighted sum of the two or more prediction blocks for the current block, wherein weights for each prediction block used for the weighted sum are determined according to a cost based on template matching.

13. A computer-readable recording medium storing a bitstream generated using the image encoding method of claim 12.

14. A method for transmitting a bitstream, comprising transmitting a bitstream generated using an image encoding method, wherein the image encoding method comprises: determining a current block to undergo template-based intra prediction; generating two or more prediction blocks for the current block according to the template-based intra prediction; and calculating a weighted sum of the two or more prediction blocks for the current block, wherein weights for each prediction block used for the weighted sum are determined according to a cost based on template matching.

Images