Image encoding / decoding method and apparatus, and recording medium storing bitstream therein

Abstract

An image decoding / encoding method and apparatus according to the present disclosure may: derive an intra prediction mode for a current block; constructing a reference sample for the current block; generating a prediction sample for the current block based on the intra prediction mode and the reference sample; and decoding / encoding the current block based on the prediction sample for the current block.

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus and a recording medium having a bit stream stored therein. Background Art

[0002] Recently, demands for high-resolution and high-quality images such as HD (High Definition) images and UHD (Ultra High Definition) images have been increasing in various application fields, and therefore, efficient image compression technology is being discussed.

[0003] There are various technologies, such as inter-frame prediction technology that uses video compression technology to predict pixel values ​​included in the current picture from pictures before or after the current picture, intra-frame prediction technology that predicts pixel values ​​included in the current picture by using pixel information in the current picture, entropy coding technology that assigns short symbols to values ​​with high occurrence frequency and assigns long symbols to values ​​with low occurrence frequency, etc., and these image compression technologies can be used to effectively compress image data and transmit or store it. Summary of the invention

[0004] Technical issues

[0005] The present disclosure is directed to providing a method and apparatus for defining various plane modes and performing prediction based on the plane modes.

[0006] The present disclosure is directed to providing a method and apparatus for deriving a planar mode using decoder-side intra mode derivation (DIMD).

[0007] The present disclosure is directed to providing a method and apparatus for deriving a planar pattern based on template matching.

[0008] Technical Solution

[0009] The image decoding method and apparatus according to the present disclosure derive an intra prediction mode of a current block, construct a reference sample of the current block, and generate a prediction sample of the current block based on the intra prediction mode and the reference sample.

[0010] In the image decoding method and apparatus according to the present disclosure, the intra prediction mode may be derived from a plurality of predefined intra prediction modes.

[0011] In the image decoding method and apparatus according to the present disclosure, the plurality of predefined intra prediction modes may include a normal plane mode, a horizontal plane mode, and a vertical plane mode.

[0012] In the image decoding method and device according to the present disclosure, the horizontal plane mode may represent a mode for generating a predicted sample of the current sample in the current block using a horizontal sample and an upper-right sample of the current sample, the horizontal sample may represent a sample among the reference samples that is horizontally adjacent to the current sample, and the upper-right sample may represent a sample among the reference samples that is adjacent to the upper-right corner of the current block.

[0013] In the image decoding method and device according to the present disclosure, the vertical plane mode may represent a mode for generating a predicted sample of the current sample in the current block using the vertical sample and the lower-left sample of the current sample, and the vertical sample may represent a sample vertically adjacent to the current sample among the reference samples, and the lower-left sample may represent a sample adjacent to the lower-left corner of the current block among the reference samples.

[0014] The image decoding method and apparatus according to the present disclosure may obtain a first flag indicating whether the normal plane mode is used for the current block.

[0015] When the first flag indicates that the normal plane mode is not used for the current block, the image decoding method and apparatus according to the present disclosure may obtain a second flag indicating an intra prediction mode used for the current block among the horizontal plane mode or the vertical plane mode.

[0016] In the image decoding method and apparatus according to the present disclosure, whether to obtain the first flag may be determined based on whether intra sub-partitioning (ISP) is applied to the current block.

[0017] The image decoding method and apparatus according to the present disclosure may derive an intra derivation mode based on decoder-side intra mode derivation (DIMD), and select one of a horizontal plane mode and a vertical plane mode based on the intra derivation mode.

[0018] In the image decoding method and device according to the present disclosure, when the intra-frame derivation mode is a horizontal orientation mode, the intra-frame prediction mode of the current block can be derived as a horizontal plane mode, and when the intra-frame derivation mode is a vertical orientation mode, the intra-frame prediction mode of the current block can be derived as a vertical plane mode.

[0019] The image decoding method and apparatus according to the present disclosure may sort the order of a plurality of predefined intra prediction modes based on the cost calculated by the template matching method.

[0020] The image encoding method and apparatus according to the present disclosure derive an intra prediction mode of a current block, construct a reference sample of the current block, and generate a prediction sample of the current block based on the intra prediction mode and the reference sample.

[0021] In the image encoding method and apparatus according to the present disclosure, the intra prediction mode may be derived from a plurality of predefined intra prediction modes.

[0022] In the image encoding method and apparatus according to the present disclosure, the plurality of predefined intra prediction modes may include a normal plane mode, a horizontal plane mode, and a vertical plane mode.

[0023] In the image encoding method and device according to the present disclosure, the horizontal plane mode may represent a mode for generating a predicted sample of the current sample in the current block using the horizontal sample and the upper-right sample of the current sample, and the horizontal sample may represent a sample horizontally adjacent to the current sample among the reference samples, and the upper-right sample may represent a sample adjacent to the upper-right corner of the current block among the reference samples.

[0024] In the image encoding method and device according to the present disclosure, the vertical plane mode can represent a mode for generating a predicted sample of the current sample in the current block using the vertical sample and the lower-left sample of the current sample, the vertical sample can represent a sample vertically adjacent to the current sample among the reference samples, and the lower-left sample can represent a sample adjacent to the lower-left corner of the current block among the reference samples.

[0025] According to the image encoding method and apparatus of the present disclosure, a first flag indicating whether a normal planar mode is used for a current block can be obtained.

[0026] When the first flag indicates that the normal plane mode is not used for the current block, the image encoding method and apparatus according to the present disclosure may obtain a second flag indicating an intra prediction mode used for the current block among the horizontal plane mode or the vertical plane mode.

[0027] In the image encoding method and apparatus according to the present disclosure, whether to obtain the first flag may be determined based on whether intra sub-partitioning (ISP) is applied to the current block.

[0028] The image encoding method and apparatus according to the present disclosure may derive an intra derivation mode based on DIMD, and select one of a horizontal plane mode and a vertical plane mode based on the intra derivation mode.

[0029] In the image encoding method and device according to the present disclosure, when the intra-frame derivation mode is a horizontal orientation mode, the intra-frame prediction mode of the current block can be derived as a horizontal plane mode, and when the intra-frame derivation mode is a vertical orientation mode, the intra-frame prediction mode of the current block can be derived as a vertical plane mode.

[0030] The image encoding method and apparatus according to the present disclosure may sort the order of a plurality of predefined intra prediction modes based on the cost calculated by the template matching method.

[0031] A computer-readable digital storage medium is provided, in which encoded video / image information for causing a decoding device according to the present disclosure to perform an image decoding method is stored.

[0032] A computer-readable digital storage medium is provided, in which video / image information generated according to the image encoding method according to the present disclosure is stored.

[0033] A method and apparatus for transmitting video / image information generated according to an image encoding method according to the present disclosure are provided.

[0034] Beneficial Effects

[0035] According to the present disclosure, by defining various plane modes and performing prediction based on the plane modes, more diverse prediction methods can be considered for prediction, the accuracy of prediction can be increased, and compression performance can be improved.

[0036] According to the present disclosure, by efficiently deriving a planar mode using decoder-side intra mode derivation (DIMD), signaling overhead can be reduced and compression efficiency can be improved.

[0037] According to the present disclosure, by effectively deriving a plane pattern based on template matching, signaling overhead can be reduced and compression efficiency can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A video / image compilation system according to the present disclosure.

[0039] Figure 2 is a rough block diagram of an encoding device to which an embodiment of the present disclosure can be applied and which performs encoding of a video / image signal.

[0040] Figure 3 is a rough block diagram of a decoding device to which an embodiment of the present disclosure can be applied and which performs decoding of a video / image signal.

[0041] Figure 4 An intra prediction method performed by the decoding device 300 according to an embodiment of the present disclosure is illustrated.

[0042] Figure 5 is a diagram for describing a prediction mode determination method based on template matching according to an embodiment of the present disclosure.

[0043] Figure 6 The diagram illustrates a schematic configuration of the intra prediction unit 331 that performs the intra prediction method according to the present disclosure.

[0044] Figure 7 An intra prediction method performed by the encoding apparatus 200 according to an embodiment of the present disclosure is illustrated.

[0045] Figure 8 The configuration of the intra prediction unit 222 that performs the inter prediction method according to the present disclosure is schematically illustrated.

[0046] Fig. 9 is an example of a content streaming system to which embodiments of the present disclosure can be applied. DETAILED DESCRIPTION

[0047] Because the present disclosure can make various changes and has several embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, it is not intended to limit the present disclosure to specific embodiments, and it should be understood to include all changes, equivalents and substitutes included in the spirit and technical scope of the present disclosure. While describing each of the accompanying drawings, similar reference numerals are used for similar components.

[0048] Terms such as first, second, etc. can be used to describe various components, but components should not be limited by these terms. These terms are only used to distinguish one component from other components. For example, without departing from the scope of the present disclosure, a first component can be referred to as a second component, and similarly, a second component can also be referred to as a first component. Terms and / or combinations of any one or more related statement items in a plurality of related statement items are included.

[0049] When a component is referred to as being "connected" or "linked" to another component, it should be understood that it can be directly connected or linked to another component, but another component may also exist in between. On the other hand, when a component is referred to as being "directly connected" or "directly linked" to another component, it should be understood that another component does not exist in between.

[0050] The terms used in this application are only used to describe specific embodiments and are not intended to limit the present disclosure. Unless the context clearly indicates otherwise, singular expressions include plural expressions. In this application, it should be understood that terms such as "including" or "having" are intended to designate the existence of features, numbers, steps, operations, components, parts or combinations thereof described in the specification, but do not exclude the possibility of the existence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof in advance.

[0051] The present disclosure relates to video / image coding. For example, the methods / embodiments disclosed herein may be applied to methods disclosed in the Universal Video Coding (VVC) standard. In addition, the methods / embodiments disclosed herein may be applied to methods disclosed in the Basic Video Coding (EVC) standard, the AOMedia Video 1 (AV1) standard, the second generation audio video coding standard (AVS2), or the next generation video / image coding standard (e.g., H.267 or H.268, etc.).

[0052] This specification proposes various embodiments of video / image coding, and unless otherwise stated, these embodiments may be performed in combination with each other.

[0053] Here, video may refer to a collection of a series of images over time. A picture generally refers to a unit representing an image within a specific time period, and a slice / tile is a unit that forms a part of a picture in coding. A slice / tile may include at least one coding tree unit (CTU). A picture may consist of at least one slice / tile. A tile is a rectangular area consisting of multiple CTUs within a specific tile column and a specific tile row of a picture. A tile column is a rectangular area of ​​a CTU having the same height as the picture and a width assigned by the syntax requirements of the picture parameter set. A tile row is a rectangular area of ​​a CTU having a height assigned by the picture parameter set and the same width as the width of the picture. The CTU within a tile may be arranged continuously according to a CTU raster scan, and the tiles within a picture may be arranged continuously according to a raster scan of the tile. A slice may include an integer number of complete tiles or an integer number of continuous complete CTU rows within a tile of a picture that may be exclusively included in a single NAL unit. At the same time, a picture may be divided into at least two sub-pictures. A sub-picture may be a rectangular area of ​​at least one slice within a picture.

[0054] A pixel or a picture element may refer to the smallest unit constituting a picture (or image). In addition, "sample" may be used as a term corresponding to a pixel. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luminance component, or only a pixel / pixel value of a chrominance component.

[0055] A unit may represent a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with the corresponding region. A unit may include a luminance block and two chrominance (e.g., cb, cr) blocks. In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may include a set (or array) of transform coefficients or samples (or sample arrays) consisting of M columns and N rows.

[0056] Here, "A or B" may refer to "only A", "only B", or "both A and B". In other words, herein, "A or B" may be interpreted as "A and / or B". For example, herein, "A, B or C" may refer to "only A", "only B", "only C", or "any combination of A, B, and C".

[0057] As used herein, a slash ( / ) or a comma may mean "and / or". For example, "A / B" may mean "A and / or B". Thus, "A / B" may mean "only A", "only B", or "both A and B". For example, "A, B, C" may mean "A, B, or C".

[0058] Here, "at least one of A and B" may refer to "only A", "only B", or "both A and B". In addition, herein, expressions such as "at least one of A or B" or "at least one of A and / or B" can be interpreted in the same manner as "at least one of A and B".

[0059] In addition, herein, “at least one of A, B, and C” may refer to “only A”, “only B”, “only C”, or “any combination of A, B, and C”. In addition, “at least one of A, B, or C” or “at least one of A, B and / or C” may refer to “at least one of A, B, and C”.

[0060] In addition, the brackets used herein may refer to "for example". Specifically, when the indication is "prediction (intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction". In other words, the "prediction" here is not limited to "intra-frame prediction", and "intra-frame prediction" may be proposed as an example of "prediction". In addition, even when the indication is "prediction (ie, intra-frame prediction)", "intra-frame prediction" may be proposed as an example of "prediction".

[0061] Here, technical features described individually in one drawing may be implemented individually or simultaneously.

[0062] Figure 1 A video / image compilation system according to the present disclosure is shown.

[0063] refer to Figure 1 , a video / image coding system may include a first device (source device) and a second device (receiving device).

[0064] The source device may send the encoded video / image information or data to the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The source device may include a video source, an encoding device, and a sending unit. The receiving device may include a receiving unit, a decoding device, and a renderer. The encoding device may be referred to as a video / image encoding device, and the decoding device may be referred to as a video / image decoding device. The transmitter may be included in the encoding device. The receiver may be included in the decoding device. The renderer may include a display unit, and the display unit may be composed of a separate device or an external component.

[0065] The video source may obtain the video / image through the process of capturing, synthesizing or generating the video / image. The video source may include a device for capturing the video / image and a device for generating the video / image. The device for capturing the video / image may include at least one camera, a video / image archive including previously captured videos / images, etc. The device for generating the video / image may include a computer, a tablet computer, a smart phone, etc., and may (electronically) generate the video / image. For example, a virtual video / image may be generated by a computer, etc., and in this case, the process of capturing the video / image may be replaced by the process of generating the relevant data.

[0066] The encoding device can encode the input video / image. The encoding device can perform a series of processes such as prediction, transformation, quantization, etc. for compression and coding efficiency. The encoded data (encoded video / image information) can be output in the form of a bitstream.

[0067] The sending unit may send the encoded video / image information or data output in the form of a bit stream to the receiving unit of the receiving device in the form of a file or stream transmission through a digital storage medium or a network. The digital storage medium may include various storage media, such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The sending unit may include an element for generating a media file in a predetermined file format and may include an element for transmitting through a broadcast / communication network. The receiving unit may receive / extract a bit stream and send it to a decoding device.

[0068] The decoding device may decode the video / image by performing a series of processes corresponding to the operations of the encoding device, such as inverse quantization, inverse transformation, prediction, etc.

[0069] The renderer may render the decoded video / image. The rendered video / image may be displayed through a display unit.

[0070] Figure 2 A rough block diagram of an encoding device to which an embodiment of the present disclosure can be applied and which performs encoding of a video / image signal is shown.

[0071] refer to Figure 2, the encoding device 200 may be composed of an image segmenter 210, a predictor 220, a residual processor 230, an entropy encoder 240, an adder 250, a filter 260, and a memory 270. The predictor 220 may include an inter-frame predictor 221 and an intra-frame predictor 222. The residual processor 230 may include a transformer 232, a quantizer 233, an inverse quantizer 234, and an inverse transformer 235. The residual processor 230 may further include a subtractor 231. The adder 250 may be referred to as a reconstructor or a reconstructed block generator. According to an embodiment, the above-mentioned image segmenter 210, the predictor 220, the residual processor 230, the entropy encoder 240, the adder 250, and the filter 260 may be configured by at least one hardware component (e.g., an encoder chipset or processor). In addition, the memory 270 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include a memory 270 as an internal / external component.

[0072] The image divider 210 may divide the input image (or picture, frame) input to the encoding device 200 into at least one processing unit. As an example, the processing unit may be referred to as a coding unit (CU). In this case, the coding unit may be recursively divided from a coding tree unit (CTU) or a maximum coding unit (LCU) according to a quadtree binary tree ternary tree (QTBTTT) structure.

[0073] For example, one coding unit may be segmented into a plurality of coding units having a deeper depth based on a quadtree structure, a binary tree structure, and / or a ternary structure. In this case, for example, the quadtree structure may be applied first, and the binary tree structure and / or the ternary structure may be applied later. Alternatively, the binary tree structure may be applied before the quadtree structure. The coding process according to this specification may be performed based on a final coding unit that is no longer segmented. In this case, based on coding efficiency according to image characteristics, etc., the maximum coding unit may be directly used as the final coding unit, or if necessary, the coding unit may be recursively segmented into coding units of a deeper depth, and the coding unit with the best size may be used as the final coding unit. Here, the coding process may include processes such as prediction, transformation, and reconstruction described later.

[0074] As another example, the processing unit may further include a prediction unit (PU) or a transform unit (TU). In this case, the prediction unit and the transform unit may be divided or partitioned from the above-mentioned final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transform unit may be a unit for deriving a transform coefficient and / or a unit for deriving a residual signal from the transform coefficient.

[0075] In some cases, a unit may be used interchangeably with terms such as a block or region. In general, an MxN block may represent a set of transform coefficients or samples consisting of M columns and N rows. A sample may generally represent a pixel or a pixel value, and may represent only a pixel / pixel value of a luma component, or only a pixel / pixel value of a chroma component. A sample may be used as a term to make a picture (or image) correspond to a pixel or a picture element.

[0076] The encoding device 200 may subtract the prediction signal (prediction block, prediction sample array) output from the inter predictor 221 or the intra predictor 222 from the input image signal (original block, original sample array) to generate a residual signal (residual signal, residual sample array), and the generated residual signal is transmitted to the transformer 232. In this case, a unit that subtracts the prediction signal (prediction block, prediction sample array) from the input image signal (original block, original sample array) within the encoding device 200 may be referred to as a subtractor 231.

[0077] The predictor 220 may perform prediction on a block to be processed (hereinafter referred to as a current block) and generate a predicted block including a prediction sample for the current block. The predictor 220 may determine whether intra prediction or inter prediction is applied in units of a current block or CU. The predictor 220 may generate various information about the prediction, such as prediction mode information, and send it to the entropy encoder 240, as described later in the description of each prediction mode. The information about the prediction may be encoded in the entropy encoder 240 and output in the form of a bitstream.

[0078] The intra-frame predictor 222 can predict the current block by referring to the samples in the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode may include at least one non-directional mode and multiple directional modes. The non-directional mode may include at least one of the DC mode or the plane mode. Depending on the detail level of the prediction direction, the directional mode may include 33 directional modes or 65 directional modes. However, this is only an example, and more or less directional modes may be used depending on the configuration. The intra-frame predictor 222 may determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring blocks.

[0079] The inter-frame predictor 221 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring block may be the same or different. The temporal neighboring block may be referred to as a collocated reference block, a collocated CU (colCU), etc., and the reference picture including the temporal neighboring block may be referred to as a collocated picture (colPic). For example, the inter-frame predictor 221 may configure a motion information candidate list based on the neighboring blocks, and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Inter prediction may be performed based on various prediction modes, and for example, for skip mode and merge mode, the inter predictor 221 may use motion information of a neighboring block as motion information of a current block. For skip mode, unlike merge mode, a residual signal may not be transmitted. For motion vector prediction (MVP) mode, a motion vector of a neighboring block is used as a motion vector predictor, and a motion difference is signaled to indicate a motion vector of a current block.

[0080] The predictor 220 may generate a prediction signal based on various prediction methods described later. For example, the predictor may not only apply intra prediction or inter prediction to predict a block, but may also apply intra prediction and inter prediction at the same time. It may be referred to as a combined inter and intra prediction (CIIP) mode. In addition, the predictor may be based on an intra block copy (IBC) prediction mode or may be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode may be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it may be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC may use at least one of the inter prediction techniques described herein. The palette mode may be considered an example of intra coding or intra prediction. When the palette mode is applied, the sample values ​​within the picture may be signaled based on information about the palette table and the palette index. The prediction signal generated by the predictor 220 may be used to generate a reconstructed signal or a residual signal.

[0081] The transformer 232 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is expressed as a graph. CNT refers to a transform obtained based on generating a prediction signal using all previously reconstructed pixels. In addition, the transform process may be applied to square pixel blocks of the same size or may be applied to non-square blocks of variable size.

[0082] The quantizer 233 may quantize the transform coefficients and send them to the entropy encoder 240, and the entropy encoder 240 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 233 may rearrange the quantized transform coefficients in the block form into a one-dimensional vector form based on the coefficient scanning order, and may generate information about the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0083] The entropy encoder 240 may perform various encoding methods such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), etc. The entropy encoder 240 may encode information necessary for video / video image reconstruction (e.g., values ​​of syntax elements, etc.) in addition to transform coefficients quantized together or individually.

[0084] The encoded information (e.g., encoded video / image information) can be transmitted or stored in units of network abstraction layer (NAL) units in the form of a bitstream. The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. Here, information and / or syntax elements transmitted / signaled from the encoding device to the decoding device may be included in the video / image information. The video / image information may be encoded and included in the bitstream through the above-mentioned encoding process. The bitstream may be transmitted through a network or may be stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmission unit (not shown) for transmission and / or a storage unit (not shown) for storing a signal output from the entropy encoder 240 may be configured as an internal / external element of the encoding device 200, or the transmission unit may also be included in the entropy encoder 240.

[0085] The quantized transform coefficients output from the quantizer 233 can be used to generate a prediction signal. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the inverse quantizer 234 and the inverse transformer 235. The adder 250 can add the reconstructed residual signal to the prediction signal output from the inter-frame predictor 221 or the intra-frame predictor 222 to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block can be used as a reconstructed block. The adder 250 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture, and can also be used for inter-frame prediction of the next picture through filtering described later. At the same time, luminance mapping with chroma scaling (LMCS) can be applied in the picture encoding and / or reconstruction process.

[0086] The filter 260 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 260 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and the modified reconstructed picture can be stored in the memory 270, specifically in the DPB of the memory 270. Various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 260 can generate various information about filtering and send it to the entropy encoder 240. The information about filtering can be encoded in the entropy encoder 240 and output in the form of a bit stream.

[0087] The modified reconstructed picture transmitted to the memory 270 may be used as a reference picture in the inter predictor 221. When inter prediction is applied therethrough, the encoding apparatus can avoid prediction mismatch in the encoding apparatus 200 and the decoding apparatus, and can also improve encoding efficiency.

[0088] The DPB of the memory 270 may store the modified reconstructed picture to be used as a reference picture in the inter-frame predictor 221. The memory 270 may store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the block in the pre-reconstructed picture. The stored motion information may be sent to the inter-frame predictor 221 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 270 may store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 222.

[0089] Figure 3 A rough block diagram of a decoding device to which an embodiment of the present disclosure can be applied and which performs decoding of a video / image signal is shown.

[0090] refer to Figure 3 , the decoding device 300 may be configured by including an entropy decoder 310, a residual processor 320, a predictor 330, an adder 340, a filter 350, and a memory 360. The predictor 330 may include an inter-frame predictor 332 and an intra-frame predictor 331. The residual processor 320 may include an inverse quantizer 321 and an inverse transformer 322.

[0091] According to an embodiment, the above-mentioned entropy decoder 310, residual processor 320, predictor 330, adder 340 and filter 350 may be configured by one hardware component (e.g., decoder chipset or processor). In addition, the memory 360 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium. The hardware component may further include the memory 360 as an internal / external component.

[0092] When a bit stream including video / image information is input, the decoding device 300 may respond to the Figure 2 The image is reconstructed by a process of processing video / image information in an encoding device. For example, the decoding device 300 can derive a unit / block based on relevant information of block segmentation obtained from a bitstream. The decoding device 300 can perform decoding by using a processing unit applied in the encoding device. Therefore, the processing unit of decoding can be a coding unit, and the coding unit can be divided from a coding tree unit or a maximum coding unit according to a quadtree structure, a binary tree structure and / or a ternary tree structure. At least one transform unit can be derived from the coding unit. And, the reconstructed image signal decoded and output by the decoding device 300 can be played by a playback device.

[0093] The decoding device 300 may receive the bit stream from Figure 2 The received signal can be decoded by the entropy decoder 310. For example, the entropy decoder 310 can parse the bitstream to derive information (e.g., video / image information) necessary for image reconstruction (or picture reconstruction). The video / image information may further include information about various parameter sets such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The decoding device may further decode the picture based on the information about the parameter set and / or the general constraint information. The signaled / received information and / or the syntax elements described later in this document may be decoded and obtained from the bitstream by a decoding process. For example, the entropy decoder 310 may decode the information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, CABAC, etc., and output the values ​​of the syntax elements necessary for image reconstruction and the quantized values ​​of the transform coefficients of the residual. In more detail, the CABAC entropy decoding method can receive a bin corresponding to each syntax element from a bitstream, determine a context model by using information of syntax elements to be decoded, decoded information of neighboring blocks and blocks to be decoded, or information of symbols / bins decoded in the previous step, perform arithmetic decoding on the bins by predicting the probability of occurrence of the bins according to the determined context model, and generate symbols corresponding to the values ​​of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using information about the decoded symbols / bins of the context model for the next symbol / bin. Among the information decoded in the entropy decoder 310, information about prediction is provided to the predictor (inter-frame predictor 332 and intra-frame predictor 331), and the residual value, that is, the quantized transform coefficient and related parameter information, which is entropy decoded in the entropy decoder 310, can be input to the residual processor 320. The residual processor 320 can derive a residual signal (residual block, residual sample, residual sample array). In addition, information about filtering among the information decoded in the entropy decoder 310 can be provided to the filter 350. Meanwhile, a receiving unit (not shown) that receives a signal output from the encoding device may be further configured as an internal / external element of the decoding device 300 or the receiving unit may be a component of the entropy decoder 310 .

[0094] Meanwhile, the decoding device according to this specification may be referred to as a video / image / picture decoding device, and the decoding device may be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 310, and the sample decoder may include at least one of an inverse quantizer 321, an inverse transformer 322, an adder 340, a filter 350, a memory 360, an inter-frame predictor 332, and an intra-frame predictor 331.

[0095] The inverse quantizer 321 may inverse quantize the quantized transform coefficient and output the transform coefficient. The inverse quantizer 321 may rearrange the quantized transform coefficient into a two-dimensional block form. In this case, the rearrangement may be performed based on the coefficient scanning order performed in the encoding device. The inverse quantizer 321 may perform inverse quantization on the quantized transform coefficient by using a quantization parameter (e.g., quantization step size information) and obtain the transform coefficient.

[0096] The inverse transformer 322 inversely transforms the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0097] The predictor 320 may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor 320 may determine whether to apply intra prediction or inter prediction to the current block based on the information on prediction output from the entropy decoder 310, and determine a specific intra / inter prediction mode.

[0098] The predictor 320 can generate a prediction signal based on various prediction methods described later. For example, the predictor 320 can not only apply intra prediction or inter prediction to predict a block, but also apply intra prediction and inter prediction at the same time. It can be called a combined inter and intra prediction (CIIP) mode. In addition, the predictor can be based on an intra block copy (IBC) prediction mode or can be based on a palette mode for prediction of a block. The IBC prediction mode or the palette mode can be used for content image / video coding of games, such as screen content coding (SCC), etc. IBC basically performs prediction within the current picture, but it can be performed similarly to inter prediction because it derives a reference block within the current picture. In other words, IBC can use at least one of the inter prediction techniques described herein. The palette mode can be considered as an example of intra coding or intra prediction. When the palette mode is applied, information about the palette table and the palette index can be included in the video / image information and sent with a signal.

[0099] The intra-frame predictor 331 can predict the current block by referring to samples within the current picture. Depending on the prediction mode, the referenced sample can be located near the current block or can be located at a specific distance away from the current block. In intra-frame prediction, the prediction mode can include at least one non-directional mode and multiple directional modes. The intra-frame predictor 331 can determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0100] The inter-frame predictor 332 may derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information sent in the inter-frame prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between the neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may further include inter-frame prediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). For inter-frame prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter-frame predictor 332 may configure a motion information candidate list based on the neighboring blocks, and derive a motion vector and / or a reference picture index for the current block based on the received candidate selection information. Inter-frame prediction may be performed based on various prediction modes, and information about the prediction may include information indicating an inter-frame prediction mode for the current block.

[0101] The adder 340 may add the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 332 and / or the intra-frame predictor 331) to generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array). When there is no residual of the block to be processed, such as when the skip mode is applied, the prediction block may be used as the reconstructed block.

[0102] The adder 340 may be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal may be used for intra prediction of the next block to be processed in the current picture, may be output through filtering described later, or may be used for inter prediction of the next picture. At the same time, luminance mapping with chroma scaling (LMCS) may be applied during picture decoding.

[0103] The filter 350 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 350 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture, and send the modified reconstructed picture to the memory 360, specifically the DPB of the memory 360. The various filtering methods may include deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0104] The (modified) reconstructed picture stored in the DPB of the memory 360 can be used as a reference picture in the inter-frame prediction unit 332. The memory 360 can be derived from the motion information in its current picture (or decoded) The motion information of the block and / or the motion information of the block in the pre-reconstructed picture. The stored motion information can be sent to the inter-frame predictor 260 to be used as the motion information of the spatial neighboring block or the motion information of the temporal neighboring block. The memory 360 can store the reconstructed samples of the reconstructed block in the current picture and send them to the intra-frame predictor 331.

[0105] Here, the embodiments described in the filter 260, the inter-frame predictor 221, and the intra-frame predictor 222 of the encoding device 200 may also be equally or correspondingly applied to the filter 350, the inter-frame predictor 332, and the intra-frame predictor 331 of the decoding device 300, respectively.

[0106] Figure 4 An intra prediction method performed by the decoding device 300 according to an embodiment of the present disclosure is illustrated.

[0107] refer to Figure 4 , the decoding device may derive an intra prediction mode of the current block (S400). The intra prediction mode of the current block may be derived from a plurality of predefined intra prediction modes.

[0108] According to an embodiment of the present disclosure, in addition to the normal plane mode, the plurality of predefined intra-frame prediction modes may also include a horizontal plane mode and / or a vertical plane mode. In an embodiment, the horizontal plane mode and / or the vertical plane mode may be a prediction mode that utilizes only a portion of the process of generating (or deriving) prediction samples in the normal plane mode. In the present disclosure, the horizontal plane mode may be referred to as a horizontal plane prediction mode, and the vertical plane mode may be referred to as a vertical plane prediction mode. The normal plane mode is a plane mode used in general image compression technology, and in the present disclosure, the normal plane mode may be referred to as a general plane mode or a normal plane mode.

[0109] As an example, the normal planar mode may use the following formula 1 to generate prediction samples.

[0110] [Formula 1]

[0111]

[0112]

[0113]

[0114] Referring to Formula 1, the normal plane mode may construct a prediction mode to perform a mixture of a vertical plane prediction sample predV and a horizontal plane prediction sample predH. In the present disclosure, the mixture may also be referred to as a combination, a weighted sum, and a weighted prediction.

[0115] According to an embodiment of the present disclosure, predV and predH of Formula 1 may be added as a separate prediction mode. In other words, predV and predH of Formula 1 may be used for intra prediction as a separate intra prediction mode.

[0116] As an example, the vertical plane mode may be defined as the following Formula 2.

[0117] [Formula 2]

[0118]

[0119] Referring to Formula 2, the vertical plane mode can generate prediction samples using the vertical samples of the current sample in the current block and its lower left sample. The vertical sample represents the sample vertically adjacent to the current sample among the reference samples of the current block, and the lower left sample represents the sample adjacent to the lower left corner of the current block among the reference samples of the current block. The vertical sample and the lower left sample can be weighted and summed based on the position (or vertical coordinate) of the current sample and the height of the current block. The weighted sum value can be shifted by taking the logarithmic value of the width of the current block as the base 2. In addition, the horizontal plane mode can be defined as the following Formula 3.

[0120] [Formula 3]

[0121]

[0122] Referring to Formula 3, the horizontal plane mode can generate a prediction sample using a horizontal sample in the current block and an upper right sample of the current sample. The horizontal sample represents a sample horizontally adjacent to the current sample among the reference samples of the current block, and the upper right sample represents a sample adjacent to the upper right corner of the current block among the reference samples of the current block. A weighted sum can be performed on the horizontal sample and the upper right sample based on the position (or horizontal coordinate) of the current sample and the width of the current block. A shift operation can be performed on the weighted sum value by taking the logarithmic value of the height of the current block as the base 2.

[0123] According to an embodiment of the present disclosure, a vertical plane mode and a horizontal plane mode may be signaled as one of the plane modes. As an embodiment, a syntax structure such as the following Table 1 may be defined. The syntax (or syntax structure) defined in the present disclosure is described based on the VVC standard, which is a current video compression standard, but the present invention is not limited thereto. In view of the essence of the present invention, the syntax may be modified within substantially the same scope.

[0124] [Table 1]

[0125]

[0126] Referring to Table 1, intra_luma_mpm_flag is a syntax element indicating whether the most probable mode (MPM) is applied to the current block. When the MPM is applied to the current block, whether the intra prediction mode of the current block is the planar mode may be indicated by intra_luma_not_planar_flag.

[0127] According to an embodiment of the present disclosure, as shown in Table 1, when intra_luma_not_planar_flag indicates that the planar mode is used, a syntax element planar_flag indicating whether the normal planar mode is used as the intra prediction mode may be signaled. When planar_flag indicates that the normal planar mode is not used, a syntax element planar_dir_flag indicating whether it is a horizontal planar mode or a vertical planar mode may be signaled.

[0128] As an embodiment, the method proposed in the present disclosure can be applied only when the current block is not in intra sub-partitioning (ISP) mode. In other words, a directional plane mode other than a normal plane mode can be used only when the ISP mode is not applied to the current block. That is, whether to obtain planar_flag can be determined based on whether the ISP is applied to the current block. When planar_flag is not signaled, that is, when planar_flag does not exist, planar_flag can be inferred to be 1.

[0129] As an embodiment, the mode information may be signaled independently of the normal planar mode transmission method. As an example, a syntax structure such as Table 2 below may be defined.

[0130] [Table 2]

[0131]

[0132] Referring to Table 2, the syntax element planar_horver_flag indicating whether the directional plane mode is applied to the current block may be signaled before the MPM-based intra prediction mode derivation process. Specifically, when the planar_horver_flag value is 1, it may indicate that the intra prediction mode of the current block (ie, the current coding unit, the current coding block) is a horizontal plane mode or a vertical plane mode. When the planar_horver_flag value is 0, it may indicate that the intra prediction mode of the current block is neither a horizontal plane mode nor a vertical plane mode. When planar_horver_flag does not exist, its value may be inferred to be 0.

[0133] When the planar_horver_flag value is 1, the syntax element planar_dir_flag indicating the horizontal or vertical plane mode may be signaled. For example, when the planar_dir_flag value is 1, it may indicate that the intra prediction mode of the current block is the horizontal plane mode, and when the planar_dir_flag value is 0, it may indicate that the intra prediction mode of the current block is the vertical plane mode.

[0134] In addition, the horizontal / vertical plane mode flag planar_horver_flag and the direction indication flag planar_dir_flag may precede the ISP mode flag. As an example, a syntax structure such as the following Table 3 may be defined.

[0135] [Table 3]

[0136]

[0137] Referring to Table 3, whether to resolve the ISP mode may be determined depending on whether the directional plane mode is applied. In Table 3, the contents described above with reference to Tables 1 and 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0138] In addition, the horizontal / vertical plane mode flag planar_horver_flag and the direction indication flag planar_dir_flag may precede the multiple reference line (MRL) mode flag. As an example, a syntax structure as in Table 4 below may be defined.

[0139] [Table 4]

[0140]

[0141] In Table 4, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0142] In addition, the horizontal / vertical plane mode flag planar_horver_flag and the direction indication flag planar_dir_flag may precede the matrix-based intra prediction (MIP) mode flag. As an example, a syntax structure such as Table 5 may be defined.

[0143] [Table 5]

[0144]

[0145] In Table 5, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0146] In addition, when the flat mode is applied, the flag may be parsed without considering whether ISP is applied. A syntax structure such as the following Table 6 may be defined.

[0147] [Table 6]

[0148]

[0149] In Table 6, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0150] Alternatively, the proposed mode information may be signaled independently of the flat mode signaling method.A syntax structure such as Table 7 below may be defined.

[0151] [Table 7]

[0152]

[0153] In Table 7, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0154] In addition, the horizontal / vertical plane mode flag planar_horver_flag and the direction indication flag planar_dir_flag may precede the MRL mode flag. As an example, a syntax structure such as the following Table 8 may be defined.

[0155] [Table 8]

[0156]

[0157] In Table 8, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0158] In addition, the horizontal / vertical plane mode flag planar_horver_flag and the direction indication flag planar_dir_flag may precede the MIP mode flag. As an example, a syntax structure such as the following Table 9 may be defined.

[0159] [Table 9]

[0160]

[0161] In Table 9, the contents described above with reference to Table 1 and Table 2 can be applied in the same manner. Redundant descriptions related thereto are omitted.

[0162] In addition, the horizontal / vertical plane mode flag planar_horver_flag may be subject to context model coding. In addition, the direction indication flag planar_dir_flag may be subject to context model coding. Alternatively, the horizontal / vertical plane mode flag may be subject to context model coding, and the direction indication flag may be subject to bypass coding.

[0163] In the example of the above-mentioned syntax, in order to express the relationship with other intra-frame prediction modes, a higher-level switch (ie, on / off) is not described separately, but the prediction mode proposed in the embodiment of the present disclosure may be signaled whether the prediction mode is activated in a higher-level syntax such as a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), a picture header, a slice header, etc. For example, in the SPS, whether the direction indication flag is applied may be first signaled, and then in a lower-level syntax (e.g., a coding unit syntax), whether the prediction mode is activated at a higher level may be first checked, and then it may be determined whether to parse the flag.

[0164] In the above-mentioned embodiment, an example is given for the case where the intra prediction method is applied to the luminance component, but this prediction method can be applied to the chrominance component in the same manner. For example, when the chrominance component is coded, when the prediction mode of the corresponding luminance component is the horizontal plane mode or the vertical plane mode, the same prediction mode can be selected and coded by the direct mode (DM). Alternatively, additional information about which plane mode among the normal plane mode, the vertical plane mode, and the horizontal plane mode is applied to the chrominance component can be sent by a signal.

[0165] In addition, this embodiment proposes a signaling method for the above-mentioned multiple plane modes. As an embodiment, the plane mode used for the current block among the multiple plane modes may be specified based on a mode derived by decoder-side intra mode derivation (DIMD).

[0166] DIMD is a method of calculating a histogram of gradients (HoG) based on reconstructed neighboring samples, and then predicting the current block using N prediction modes with high histogram magnitudes. When DIMD is applied, a gradient may be calculated based on at least two samples of a neighboring area belonging to the current block. Here, the gradient may include at least one of a gradient in a horizontal direction or a gradient in a vertical direction. Based on at least one of the calculated gradient or the magnitude of the gradient, an intra-frame prediction mode of the current block may be derived. Here, the magnitude of the gradient may be determined based on the sum of the gradient in the horizontal direction and the gradient in the vertical direction. Through this derivation method, one intra-frame prediction mode may be derived for the current block, or two or more intra-frame prediction modes may be derived.

[0167] As an embodiment, when a gradient-based derivation method is applied, one or more intra-frame prediction modes may be derived from reconstructed neighboring samples. In this case, according to an embodiment of the present disclosure, one or more prediction modes may be derived based on the gradient derived from the reconstructed neighboring samples. For example, one or more intra-frame prediction modes may be derived from the reconstructed neighboring samples. As an example, the maximum number of intra-frame prediction modes derived by the gradient-based derivation method may be predefined in an encoder / decoder. As an example, the maximum number of intra-frame prediction modes derived by the gradient-based derivation method may be N. For example, N may be defined as 2, 3, 4, 5, 6, or 7.

[0168] As an example, the predictor obtained by the derived intra prediction mode (which may be referred to as a prediction sample or prediction block) may be combined with the predictor obtained by the planar mode (which may be abbreviated as the planar mode predictor). For example, the predictor obtained by the derived intra prediction mode may be weighted summed with the planar mode predictor. In such a case, as an example, the weights used in the weighted summation may be determined based on the gradients calculated from the neighboring samples reconstructed according to an embodiment of the present disclosure.

[0169] In addition, as an example, when the number of intra prediction modes derived by the gradient-based derivation method is two or more, a predictor obtained by the derived intra prediction mode may be combined with a planar mode predictor. As an example, when the number of intra prediction modes derived by the gradient-based derivation method is one, a predictor obtained by one derived intra prediction mode may be output as a prediction sample or a prediction block by the gradient-based derivation method without being combined (or weighted-summed) with the planar mode predictor.

[0170] In addition, as an example, the gradient can be calculated in units of windows of a predetermined size. Based on the calculated gradient, the angle indicating the directionality of the sample within the corresponding window can be calculated. The calculated angle can correspond to any one of the above-mentioned multiple predefined intra-frame prediction modes. The magnitude of the gradient can be stored / updated for the intra-frame prediction mode corresponding to the calculated angle. In the present disclosure, the magnitude of the gradient can be referred to as the magnitude of the gradient, the histogram magnitude, the histogram size, etc. Through this process, the intra-frame prediction mode corresponding to the calculated gradient can be determined for each window. The magnitude of the gradient can be stored / updated for the determined intra-frame prediction mode. The first T intra-frame prediction modes with the largest magnitude among the stored gradient magnitudes are selected, and the selected intra-frame prediction mode can be set as the intra-frame prediction mode of the current block. Here, T can be an integer of 1, 2, 3 or more.

[0171] As an example, the neighboring area of ​​the current block can be the left, upper left and upper areas of the current block. For example, the gradient in the horizontal direction or the gradient in the vertical direction can be derived from the rows and columns of the second neighboring samples. The rows and / or columns of the second neighboring samples can represent the rows and / or columns of the rows and columns of the neighboring samples next to the current block. As an example, HoG can be derived from the gradient in the horizontal direction and the gradient in the vertical direction. The gradient or histogram of the gradient can be derived by applying a window of L-shaped rows and columns using 3 pixels (or pixel lines) around the current block. In this case, the window can be defined as having a size of 3x3. However, as an example, the present disclosure is not limited thereto, and the window can be defined as having a size of 2x2, 4x4, 5x5, etc. In addition, as an example, the window can be a Sobel filter.

[0172] As an example, two or more intra prediction modes with the largest gradient magnitude (or magnitude) may be selected. The final prediction block may be generated by mixing (or combining, weighted summing) the prediction block predicted using the selected intra prediction mode and the prediction block predicted using the planar mode. In this case, the weight applied to each prediction block may be derived based on the histogram magnitude.

[0173] In an embodiment of the present disclosure, when the plane mode used for the current block is not the normal plane mode, in order to determine which of the horizontal / vertical plane modes is the prediction mode of the current block, a prediction mode with the highest HoG magnitude obtained in the DIMD process (hereinafter referred to as DIMD mode) may be used. That is, in the above-mentioned embodiment, planar_flag is signaled, but in the case where planar_dir_flag is not signaled, the plane mode used for the current block among the horizontal / vertical plane modes may be determined based on the DIMD prediction mode.

[0174] As an example, when the DIMD mode (i.e., the mode with the highest HoG magnitude) is a horizontal orientation mode, the intra prediction mode of the current block may be derived as a horizontal plane mode. When the DIMD mode is a vertical orientation mode, the intra prediction mode of the current block may be derived as a vertical plane mode. When the DIMD mode is less than the diagonal mode (mode 34 in the VVC standard), the DIMD mode may be determined as a horizontal orientation mode. When the DIMD mode is greater than or equal to the diagonal mode, the DIMD mode may be determined as a vertical orientation mode.

[0175] Alternatively, as an example, when the current block is in planar mode, the prediction mode with the highest HoG magnitude obtained in the DIMD process can be used to determine which mode among the normal planar mode, vertical planar mode and horizontal planar mode is the prediction mode of the current block.

[0176] That is, in the above embodiment, without signaling planar_flag and planar_dir_flag, the prediction mode of the current block may be determined as one of the plane modes for the current block among the normal plane mode, the vertical plane mode, and the horizontal plane mode based on the mode derived from DIMD.

[0177] For example, when the DIMD mode is a mode within a specific range from the horizontal mode, the plane mode for the current block can be determined as the horizontal plane mode. When the DIMD mode is a mode within a specific range from the vertical mode, the plane mode for the current block can be determined as the vertical plane mode. In other cases, the plane mode for the current block can be determined as the normal plane mode.

[0178] Specifically, when the DIMD mode is ±M mode relative to the horizontal mode, the current prediction mode can be derived as the horizontal plane mode. For example, when M is 5 and the DIMD mode is 16, because 13≤DIMD mode≤23, the intra prediction mode of the current block can be derived as the horizontal plane mode.

[0179] Similarly, when the DIMD mode is ±M mode relative to the vertical mode, the current prediction mode can be derived as the vertical plane mode. In other cases, the current prediction mode can be derived as the normal plane mode. In this case, if there are 65 intra-frame prediction directions, M can be an integer from 0 to 16 or less. Alternatively, when there are K intra-frame prediction directions, M can be an integer ranging from 0 to K / 4.

[0180] The decoding apparatus may construct a reference sample of the current block (S410). The decoding apparatus may generate a prediction sample of the current block based on the determined intra prediction mode of the current block and the constructed reference sample (S420).

[0181] Hereinafter, a method of selecting a prediction mode based on prediction accuracy in a predefined template region is described.

[0182] Figure 5 is a diagram for describing a prediction mode determination method based on template matching according to an embodiment of the present disclosure.

[0183] According to an embodiment of the present disclosure, a plane mode applied to a current block among multiple plane modes may be determined using template matching. A plane mode used for the current block among a normal plane mode, a vertical plane mode, and a horizontal plane mode may be determined using template matching, and a plane mode used for the current block among a vertical plane mode and a horizontal plane mode may be determined using template matching.

[0184] In the latter case, when a planar mode is used for the current block, a flag indicating whether it is a normal planar mode or a directional planar mode may be signaled (see above). Figure 4 When the normal plane mode is not applied, that is, when the directional plane mode is applied, the plane mode for the current block among the vertical plane mode and the horizontal plane mode may be determined using the cost calculated by using the template area.

[0185] As an example, a vertical / horizontal plane pattern may be used to generate Figure 5 The prediction samples (or prediction blocks) of the template area illustrated in FIG. In this case, the template reference area can be used. The cost can be calculated based on the difference between the generated prediction samples and the reconstructed samples. In this case, the sum of absolute differences (SAD), the sum of absolute transform differences (SATD), etc. can be used for cost calculation. For example, the horizontal or vertical plane mode can be used in Figure 5 Prediction is performed in the template area illustrated in , and the mode with the smallest cost among the two modes can be selected as the prediction mode for the current block.

[0186] As another example, as a template-based mode selection method, Figure 4 The planar_dir_flag described in the embodiment is signaled, but the order of the vertical / horizontal plane mode candidates is sorted in cost order by template-based SAD and SATD, and the intra prediction mode of the current block can be determined based on the list of sorted candidates.

[0187] In this case, the direction indication flag indicating the vertical / horizontal plane mode may be subjected to context coding. In this case, the accuracy of the direction indication flag is improved, and thus it can be expected that the coding performance is improved. In addition, as an embodiment, the template area may be selectively utilized depending on the vertical / horizontal plane mode. As described above with reference to Figure 4 As in the described formula, the vertical plane mode mainly uses the upper reference samples, and the horizontal plane mode mainly uses the left reference samples.

[0188] Therefore, in the case of horizontal plane mode, deriving prediction samples from the left template region can lead to more accurate predictions. Similarly, in the case of horizontal plane mode, deriving prediction samples from the upper reference samples can lead to more accurate predictions. In the case of horizontal plane mode, prediction samples can be generated from the left template region, and in the case of vertical plane mode, prediction samples can be generated from the upper template region.

[0189] In addition, as an embodiment, when the sizes of the left template and the upper template are different, the prediction mode may be determined by normalizing the cost by using the number of pixels in each template area and comparing the derived results.

[0190] In the former case, only information indicating whether the current block is in the planar mode is signaled, but the planar mode for the current block among the normal planar mode, the vertical planar mode, and the horizontal planar mode may be determined using a cost calculated by using a template area.

[0191] As an example, the normal / vertical / horizontal plane mode may be used to generate Figure 5 The prediction samples (or prediction blocks) of the template area shown in . The cost can be calculated based on the difference between the generated prediction samples and the reconstructed samples. In this case, SAD, SATD, etc. can be used for cost calculation. For example, in normal, horizontal or vertical plane mode Figure 5 Prediction is performed in the template area shown in , and the mode with the smallest cost among the three modes can be selected as the intra prediction mode of the current block.

[0192] As another embodiment, as a template-based mode selection method, the order of normal / vertical / horizontal plane mode candidates may be sorted in order of lowest cost by template-based SAD and SATD, and the intra prediction mode of the current block may be determined based on the sorted candidate list. Information indicating the candidate used for the current block within the sorted candidate list may be signaled.

[0193] In addition, as an embodiment, the template area can be selectively utilized according to the vertical / horizontal plane mode. Figure 4 As in the formula described above, the vertical plane mode mainly uses the upper reference samples, and the horizontal plane mode mainly uses the left reference samples.

[0194] Therefore, in the case of horizontal plane mode, deriving prediction samples from the left template region can lead to more accurate predictions. Similarly, in the case of horizontal plane mode, deriving prediction samples from the upper reference samples can lead to more accurate predictions. In the case of horizontal plane mode, prediction samples can be generated from the left template region, and in the case of vertical plane mode, prediction samples can be generated from the upper template region.

[0195] In addition, as an embodiment, when the sizes of the left template and the upper template are different, the prediction mode may be determined by normalizing the cost by using the number of pixels in each template area and comparing the derived results.

[0196] In addition, according to an embodiment of the present disclosure, position-dependent intra prediction (PDPC) may be applied to a prediction block generated according to the prediction mode proposed in the above embodiment. In the present disclosure, PDPC may be referred to as position-based intra prediction sample filtering. PDPC is a filter for alleviating discontinuities between a prediction block and adjacent reconstructed samples.

[0197] Similar to other conventional prediction blocks, PDPC can also be applied to the reference Figure 4 and Figure 5 Prediction block constructed in the above method. As an embodiment, since the vertical / horizontal plane mode has similar performance to the normal plane mode, the PDPC method applied to the conventional plane prediction block may be applied to the vertical / horizontal plane mode.

[0198] Alternatively, since in the horizontal plane mode, prediction is performed by utilizing a relatively large number of left reference samples although no upper reference samples are utilized except for the upper right reference samples, by considering this, the PDPC method for the horizontal prediction mode having similar characteristics can be applied. Similarly, since in the vertical plane mode, prediction is performed by utilizing a relatively large number of upper reference samples although no left reference samples are utilized except for the lower left reference samples, by considering this, the PDPC method for the vertical prediction mode having similar characteristics can be applied.

[0199] Meanwhile, PDPC is an additional process for predicting blocks and is applied in units of pixels, and therefore it may affect the complexity of the codec. With this in mind, PDPC application conditions for horizontal / vertical plane prediction blocks may be defined. For example, for horizontal / vertical plane mode, whether PDPC is applied to it may be determined depending on the size of the block and the ratio of its width and height. Alternatively, for horizontal / vertical plane mode, whether PDPC is applied to it may be determined depending on the prediction mode of the neighboring block. Alternatively, PDPC may not be applied to horizontal / vertical plane mode.

[0200] Figure 6 FIG. 3 illustrates a schematic configuration of an intra prediction unit 331 that performs an intra prediction method according to the present disclosure.

[0201] refer to Figure 6 , the intra prediction unit 331 may include an intra prediction mode deriver 600 , a reference sample constructor 610 , and a prediction sample generator 620 .

[0202] The intra prediction mode deriver 600 may derive the intra prediction mode of the current block. The intra prediction mode of the current block may be derived from a plurality of predefined intra prediction modes.

[0203] As described above, in addition to the normal planar mode, the predefined multiple intra prediction modes may also include a horizontal planar mode and / or a vertical planar mode. In an embodiment, the horizontal planar mode and / or the vertical planar mode may be a prediction mode that utilizes only a portion of the process of generating (or deriving) prediction samples in the normal planar mode.

[0204] As described above, when the horizontal plane mode is applied, the horizontal sample and the upper right sample of the current sample in the current block can be used to generate the prediction sample of the current sample. When the vertical plane mode is applied, the vertical sample and the lower left sample of the current sample in the current block can be used to generate the prediction sample of the current sample.

[0205] In addition, as described above with reference to Tables 1 to 9, a syntax element indicating whether a plane mode is used for a current block, a syntax indicating a plane mode applied to a current block among a plurality of plane modes, and the like may be signaled.

[0206] In addition, as described above, the mode proposed in the present disclosure may be determined based on whether ISP is applied to the current block. As an example, the directional plane mode proposed in the present disclosure may be used for intra prediction of the current block only when ISP is not applied.

[0207] In addition, as described above, a plane mode used for the current block among a plurality of plane modes can be specified based on a mode derived from DIMD.

[0208] In addition, as mentioned above Figure 5 As described, template matching can be used to determine the plane mode applied to the current block among multiple plane modes. Template matching can be used to determine the plane mode used for the current block among the normal plane mode, the vertical plane mode, and the horizontal plane mode, and template matching can be used to determine the plane mode used for the current block among the vertical plane mode and the horizontal plane mode.

[0209] The reference sample constructor 610 may construct a reference sample of the current block. The prediction sample generator 620 may generate a prediction sample of the current block based on the determined intra prediction mode of the current block and the constructed reference sample.

[0210] In addition, as described above, position-based intra-prediction sample filtering may be applied to a prediction block generated according to the planar mode proposed in the present disclosure.

[0211] Figure 7 An intra prediction method performed by the encoding apparatus 200 according to an embodiment of the present disclosure is illustrated.

[0212] In the embodiment of the present disclosure, a planar mode-based intra prediction method performed by an encoding device is described. Figures 4 to 6 The embodiments are described, and redundant descriptions are omitted here.

[0213] refer to Figure 7 , the encoding apparatus may determine an intra prediction mode of a current block (S700). The intra prediction mode of the current block may be determined from a plurality of predefined intra prediction modes.

[0214] As described above, in addition to the normal planar mode, the predefined multiple intra prediction modes also include a horizontal planar mode and / or a vertical planar mode. In an embodiment, the horizontal planar mode and / or the vertical planar mode may be a prediction mode that utilizes only a portion of the process of generating (or deriving) prediction samples in the normal planar mode.

[0215] As described above, when the horizontal plane mode is applied, the horizontal sample and the upper right sample of the current sample in the current block may be used to generate the prediction sample of the current sample. When the vertical plane mode is applied, the vertical sample and the lower left sample of the current sample in the current block may be used to generate the prediction sample of the current sample.

[0216] In addition, as described above with reference to Tables 1 to 9, a syntax element indicating whether a plane mode is used for a current block, a syntax indicating a plane mode applied to a current block among a plurality of plane modes, and the like may be signaled.

[0217] In addition, as described above, the mode proposed in the present disclosure may be determined based on whether ISP is applied to the current block. As an example, the directional plane mode proposed in the present disclosure may be used for intra prediction of the current block only when ISP is not applied.

[0218] In addition, as described above, a plane mode used for a current block among a plurality of plane modes can be specified based on a mode derived by DIMD.

[0219] In addition, as mentioned above Figure 5 As described, template matching can be used to determine the plane mode applied to the current block among multiple plane modes. Template matching can be used to determine the plane mode used for the current block among the normal plane mode, the vertical plane mode, and the horizontal plane mode, and template matching can be used to determine the plane mode used for the current block among the vertical plane mode and the horizontal plane mode.

[0220] The encoding apparatus may construct a reference sample of the current block (S710). The encoding apparatus may generate a prediction sample of the current block based on the determined intra prediction mode of the current block and the constructed reference sample (S720).

[0221] In addition, as described above, position-based intra-prediction sample filtering may be applied to a prediction block generated according to the planar mode proposed in the present disclosure.

[0222] Figure 8 The configuration of the intra prediction unit 222 that performs the inter prediction method according to the present disclosure is schematically illustrated.

[0223] refer to Figure 8 , the intra prediction unit 222 may include an intra prediction mode determiner 800 , a reference sample constructor 810 , and a prediction sample generator 820 .

[0224] Specifically, the intra prediction mode determiner 800 may determine the intra prediction mode of the current block. The intra prediction mode of the current block may be determined from a plurality of predefined intra prediction modes.

[0225] As described above, in addition to the normal planar mode, the plurality of predefined intra prediction modes may also include a horizontal planar mode and / or a vertical planar mode. In an embodiment, the horizontal planar mode and / or the vertical planar mode may be a prediction mode that utilizes only a portion of the process of generating (or deriving) prediction samples in the normal planar mode.

[0226] As described above, when the horizontal plane mode is applied, the horizontal sample and the upper right sample of the current sample in the current block can be used to generate the prediction sample of the current sample. When the vertical plane mode is applied, the vertical sample and the lower left sample of the current sample in the current block can be used to generate the prediction sample of the current sample.

[0227] In addition, as described above with reference to Tables 1 to 9, a syntax element indicating whether a plane mode is used for a current block, a syntax indicating which of a plurality of plane modes is applied to a current block, etc. may be signaled.

[0228] In addition, as described above, the mode proposed in the present disclosure may be determined based on whether ISP is applied to the current block. As an example, the directional plane mode proposed in the present disclosure may be used for intra prediction of the current block only when ISP is not applied.

[0229] In addition, as described above, a plane mode used for the current block among a plurality of plane modes can be specified based on a mode derived from DIMD.

[0230] In addition, as mentioned above Figure 5 As described, template matching can be used to determine the plane mode applied to the current block among multiple plane modes. Template matching can be used to determine the plane mode used for the current block among the normal plane mode, the vertical plane mode, and the horizontal plane mode, and template matching can be used to determine the plane mode used for the current block among the vertical plane mode and the horizontal plane mode.

[0231] The reference sample constructor 810 may construct a reference sample of the current block. The prediction sample generator 820 may generate a prediction sample of the current block based on the determined intra prediction mode of the current block and the constructed reference sample.

[0232] In addition, as described above, position-based intra-prediction sample filtering may be applied to the prediction block generated according to the planar mode proposed in the present disclosure.

[0233] In the above embodiments, the method is described as a series of steps or boxes based on the flowchart, but the corresponding embodiments are not limited to the order of the steps, and some steps may occur simultaneously or in a different order than other steps described above. In addition, those skilled in the art will appreciate that the steps shown in the flowchart are not exclusive, and other steps may be included or one or more steps in the flowchart may be deleted without affecting the scope of the embodiments of the present disclosure.

[0234] The above-mentioned method according to the embodiment of the present disclosure can be implemented in the form of software, and the encoding device and / or decoding device according to the present disclosure can be included in a device that performs image processing, such as a TV, a computer, a smart phone, a set-top box, a display device, etc.

[0235] In the present disclosure, when the embodiment is implemented as software, the above method can be implemented as a module (process, function, etc.) that performs the above functions. The module can be stored in a memory and can be executed by a processor. The memory can be located inside or outside the processor and can be connected to the processor by various well-known means. The processor may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and / or a data processing device. The memory may include a read-only memory (ROM), a random access memory (RAM), a flash memory, a memory card, a storage medium, and / or other storage devices. In other words, the embodiments described herein can be implemented on a processor, a microprocessor, a controller, or a chip. For example, the functional unit shown in each of the figures can be implemented on a computer, a processor, a microprocessor, a controller, or a chip. In this case, information (e.g., information about instructions) or an algorithm for implementation can be stored in a digital storage medium.

[0236] In addition, the decoding device and the encoding device of the embodiment of the present disclosure can be included in multimedia broadcast sending and receiving devices, mobile communication terminals, home theater video devices, digital theater video devices, surveillance cameras, video conversation devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, devices for providing video on demand (VoD) services, over-the-top video (OTT) devices, devices for providing Internet streaming services, three-dimensional (3D) video devices, virtual reality (VR) devices, augmented reality (AR) devices, videophone video devices, transportation terminals (e.g., vehicle (including autonomous driving vehicle) terminals, aircraft terminals, ship terminals, etc.) and medical video devices, etc., and can be used to process video signals or data signals. For example, over-the-top video (OTT) devices can include game consoles, Blu-ray players, networked TVs, home theater systems, smart phones, tablet computers, digital video recorders (DVRs), etc.

[0237] In addition, the processing method of the embodiment of the present disclosure can be generated in the form of a program executed by a computer and can be stored in a computer-readable recording medium. Multimedia data having a data structure according to an embodiment of the present disclosure can also be stored in a computer-readable recording medium. Computer-readable recording media include all types of storage devices and distributed storage devices that store computer-readable data. Computer-readable recording media may include, for example, Blu-ray discs (BD), universal serial buses (USB), ROMs, PROMs, EPROMs, EEPROMs, RAMs, CD-ROMs, tapes, floppy disks, and optical media storage devices. In addition, computer-readable recording media include media implemented in the form of carrier waves (e.g., transmitted via the Internet). In addition, the bit stream generated by the encoding method may be stored in a computer-readable recording medium or may be sent via a wired or wireless communication network.

[0238] In addition, the embodiments of the present disclosure may be implemented by a computer program product through a program code, and the program code may be executed on a computer by the embodiments of the present disclosure. The program code may be stored on a computer-readable carrier.

[0239] Fig. 9 is an example of a content streaming system to which embodiments of the present disclosure can be applied.

[0240] refer to Fig. 9 The content streaming transmission system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming transmission server, a web server, a media storage, a user device, and a multimedia input device.

[0241] The encoding server generates a bitstream by compressing content input from a multimedia input device such as a smartphone, a camera, a camcorder, etc. into digital data and transmits it to the streaming server. As another example, when a multimedia input device such as a smartphone, a camera, a camcorder, etc. directly generates a bitstream, the encoding server may be omitted.

[0242] A bitstream may be generated by applying the encoding method or the bitstream generating method of the embodiment of the present disclosure, and the streaming server may temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0243] The streaming server sends multimedia data to the user device through the web server based on the user's request, and the web server serves as a medium to inform the user what services are available. When the user requests the required service from the web server, the web server delivers it to the streaming server, and the streaming server sends the multimedia data to the user. In this case, the content streaming system may include a separate control server, and in this case, the control server controls the command / response between each device in the content streaming system.

[0244] The streaming server may receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a certain period of time.

[0245] Examples of user devices may include mobile phones, smart phones, laptop computers, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, tablet computers, ultrabooks, wearable devices (e.g., smart watches, smart glasses, head-mounted displays (HMDs), digital televisions, desktops, digital signage, etc.).

[0246] Each server in the content streaming system may be operated as a distributed server, and in this case, data received from each server may be distributed and processed.

[0247] The claims set forth herein can be combined in various ways. For example, the technical features of the method claims of the present disclosure can be combined and implemented as a device, and the technical features of the device claims of the present disclosure can be combined and implemented as a method. In addition, the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a device, and the technical features of the method claims of the present disclosure and the technical features of the device claims can be combined and implemented as a method.

Claims

1. An image decoding method, comprising: deriving the intra prediction mode of the current block; Constructing a reference sample of the current block; as well as generating a prediction sample of the current block based on the intra prediction mode and the reference sample, wherein the intra prediction mode is derived from a plurality of predefined intra prediction modes, and The plurality of predefined intra prediction modes include a normal plane mode, a horizontal plane mode, and a vertical plane mode.

2. The image decoding method according to claim 1, wherein: The horizontal plane mode indicates a mode for generating a prediction sample of a current sample in the current block using a horizontal sample and an upper right sample of the current sample, The horizontal samples represent samples among the reference samples that are horizontally adjacent to the current sample, and The upper right sample represents a sample adjacent to the upper right corner of the current block among the reference samples.

3. The image decoding method according to claim 1, wherein: The vertical plane mode indicates a mode for generating a prediction sample of a current sample in the current block using a vertical sample and a lower left sample of the current sample, The vertical samples represent samples among the reference samples that are vertically adjacent to the current sample, and The lower left sample represents a sample adjacent to the lower left corner of the current block among the reference samples.

4. The image decoding method according to claim 1, wherein: The derivation of the intra prediction mode of the current block includes obtaining a first flag indicating whether the normal planar mode is used for the current block.

5. The image decoding method according to claim 4, wherein: The derivation of the intra-frame prediction mode of the current block further includes: when the first flag indicates that the normal plane mode is not used for the current block, obtaining a second flag indicating the intra-frame prediction mode of the horizontal plane mode or the vertical plane mode used for the current block.

6. The image decoding method according to claim 4, wherein: Whether to obtain the first flag is determined based on whether intra sub partitioning (ISP) is applied to the current block.

7. The image decoding method according to claim 1, wherein: The derivation of the intra prediction mode of the current block comprises: deriving an intra mode derivation based on a decoder-side intra mode derivation (DIMD); and One of the horizontal plane mode and the vertical plane mode is selected based on the intra derivation mode.

8. The image decoding method according to claim 7, wherein: When the intra derivation mode is a horizontal directional mode, the intra prediction mode of the current block is derived as the horizontal plane mode, and When the intra derivation mode is a vertical orientation mode, the intra prediction mode of the current block is derived as the vertical plane mode.

9. The image decoding method according to claim 1, wherein: The derivation of the intra prediction mode of the current block includes sorting the order of the plurality of predefined intra prediction modes based on costs calculated by a template matching method.

10. An image encoding method, comprising: Determining an intra prediction mode for a current block; Constructing a reference sample of the current block; as well as generating a prediction sample of the current block based on the intra prediction mode and the reference sample, wherein the intra-frame prediction mode is determined from a plurality of predefined intra-frame prediction modes, and The plurality of predefined intra prediction modes include a normal plane mode, a horizontal plane mode, and a vertical plane mode. 11 . A computer-readable storage medium configured to store a bit stream generated by the image encoding method according to claim 10 .

12. A method for transmitting data related to image information, comprising: Determining an intra prediction mode for a current block; Constructing a reference sample of the current block; generating a prediction sample of the current block based on the intra prediction mode and the reference sample; Based on the prediction samples, generating a bitstream by encoding the current block; as well as sending data comprising said bit stream, wherein the intra prediction mode is derived from a plurality of predefined intra prediction modes, and The plurality of predefined intra prediction modes include a normal plane mode, a horizontal plane mode, and a vertical plane mode.

Images