Image encoding / decoding method based on intra prediction, method for transmitting bit stream, and recording medium storing bit stream

Abstract

An image encoding / decoding method and apparatus are provided. An image decoding method according to the present disclosure comprises the steps of: determining an intra prediction mode of a current block; and generating a prediction block of the current block by performing intra prediction based on the determined intra prediction mode, where the intra prediction may be constrained to be performed by using only reference samples within a predetermined range among available reference samples from the current block.

Description

Technical Field

[0001] The present disclosure relates to an intra-prediction-based image encoding / decoding method, a method for transmitting a bitstream, and a recording medium storing the bitstream, and more particularly, to image coding based on an intra-prediction mode through reference sample constraints. Background Art

[0002] Recently, demand for higher-resolution and higher-quality images, such as high-definition (HD) and ultra-high-definition (UHD), has increased across various fields. As image data becomes higher-resolution and higher-quality, the amount of information transmitted, or the bit rate, increases relative to conventional image data. This increase in transmitted information or bit rate leads to increased transmission and storage costs.

[0003] Therefore, an efficient image compression technology is needed to effectively transmit, store, and reproduce information of high-resolution and high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The present disclosure aims to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] In addition, the present disclosure aims to provide an image encoding / decoding method and apparatus based on intra-frame prediction and reference sample constraints.

[0007] In addition, the present disclosure aims to provide an image encoding / decoding method and apparatus for improving the efficiency of enhanced compression model (ECM) intra-frame prediction.

[0008] The present disclosure is to provide a non-transitory computer-readable recording medium for storing a bit stream generated by the image encoding method or apparatus according to the present disclosure.

[0009] The present disclosure is to provide a non-transitory computer-readable recording medium for storing a bit stream received and decoded by an image decoding apparatus according to the present disclosure and used for image reconstruction.

[0010] The present disclosure is to provide a method for transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0011] The technical problems to be achieved in the present disclosure are not limited to the above-mentioned technical problems, and other technical problems not described can be clearly understood by those skilled in the art from the following description.

[0012] Technical Solution

[0013] According to one aspect of the present disclosure, an image decoding method includes: determining an intra-frame prediction mode of a current block; and generating a prediction block of the current block by performing intra-frame prediction based on the determined intra-frame prediction mode, wherein the intra-frame prediction can be constrained to be performed by using only reference samples within a predetermined range among available reference samples of the current block.

[0014] According to one aspect of the present disclosure, an image encoding method includes: determining an intra-frame prediction mode of a current block, and encoding prediction mode information of the current block based on the determined intra-frame prediction mode, wherein the intra-frame prediction can be determined by using only reference samples within a predetermined range among available reference samples of the current block.

[0015] According to another aspect of the present disclosure, a computer-readable recording medium may store a bitstream generated by the image encoding method or the image encoding device of the present disclosure.

[0016] A transmission method according to another embodiment of the present disclosure may transmit a bit stream generated by an image encoding apparatus or an image encoding method of the present disclosure.

[0017] The features of the present disclosure briefly summarized above are merely exemplary embodiments of the detailed description that follows and are not intended to limit the scope of the present disclosure.

[0018] Beneficial effects

[0019] According to the present disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0020] In addition, according to the present disclosure, an image encoding / decoding method and apparatus based on intra-frame prediction and reference sample constraint may be provided.

[0021] In addition, according to the present disclosure, an image encoding / decoding method and apparatus that can generate an adaptive intra prediction block in a reference sample region in intra prediction can be provided.

[0022] In addition, according to the present disclosure, an image encoding / decoding method and apparatus that improves the efficiency of enhanced compression model (ECM) intra prediction can be provided.

[0023] According to the present disclosure, a non-transitory computer-readable recording medium for storing a bit stream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0024] According to the present disclosure, there may be provided a non-transitory computer-readable recording medium for storing a bit stream received and decoded by the image decoding apparatus according to the present disclosure and used for image reconstruction.

[0025] According to the present disclosure, a method of transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure may be provided.

[0026] Effects obtainable by the present disclosure are not limited to the above-described effects, and other effects that are not described can be clearly understood from the following description by those having ordinary skill in the art. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 FIG2 shows a schematic diagram of a video coding system to which an embodiment of the present disclosure can be applied.

[0028] Figure 2 A schematic diagram showing an image encoding apparatus to which an embodiment of the present disclosure can be applied is shown.

[0029] Figure 3 A schematic diagram showing an image decoding apparatus to which an embodiment of the present disclosure can be applied is shown.

[0030] Figure 4 A flowchart illustrating an example of an intra prediction mode signaling method in an encoding device.

[0031] Figure 5 A flowchart showing an example of a method for determining an intra prediction mode in a decoding device.

[0032] Figure 6 A diagram showing an example of neighboring blocks used to derive an MPM list.

[0033] Figure 7 is a diagram illustrating an example of an intra prediction method that can be applied to the present disclosure.

[0034] Figure 8 is a diagram illustrating an example of a block for vertical plane prediction according to an embodiment of the present disclosure.

[0035] Figure 9 is a diagram illustrating an example of a neighboring block search position based on an MPM mode in an 8×8 block according to an embodiment of the present disclosure.

[0036] Figure 10 2 is a diagram illustrating an image encoding method and / or an image decoding method according to an embodiment of the present disclosure.

[0037] Figure 11 is a diagram illustrating an example of an intra prediction method that can be applied to the present disclosure.

[0038] Figure 12 is a diagram illustrating an example of a block for horizontal plane prediction according to an embodiment of the present disclosure.

[0039] Figure 13is a diagram for describing exemplary search positions of neighboring blocks for an MPM when the embodiment of the present disclosure is applied in an 8x8 block according to an embodiment of the present disclosure.

[0040] Figure 14 2 is a diagram illustrating an image encoding method and / or an image decoding method according to an embodiment of the present disclosure.

[0041] Figure 15 is a diagram for describing an example of a method for combining an intra prediction method for performing prediction mainly based on an upper reference sample and an intra prediction method for performing prediction mainly based on a left reference sample according to an embodiment of the present disclosure.

[0042] Figure 16 is a diagram for describing an example of an intra-prediction block and its neighboring sample regions according to an embodiment of the present disclosure.

[0043] Figure 17 is a diagram illustrating an example of neighboring block search positions for an MPM when the proposed method is applied in an 8x8 block according to an embodiment of the present disclosure.

[0044] Figure 18 A diagram illustrating an image encoding method and / or an image decoding method according to an embodiment of the present disclosure.

[0045] Figure 19 is a diagram for describing an image decoding method that can be performed by an image decoding device according to an embodiment of the present disclosure.

[0046] Figure 20 is a diagram for describing an image encoding method that can be performed by an image encoding device according to an embodiment of the present disclosure.

[0047] Figure 21 An exemplary diagram showing a content streaming system to which embodiments of the present disclosure can be applied. DETAILED DESCRIPTION

[0048] Hereinafter, in order for those skilled in the art to easily implement them, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. However, the present disclosure can be implemented in various forms and is not limited to the embodiments described herein.

[0049] When describing the embodiments of the present disclosure, when well-known configurations or functions are considered to obscure the main points of the present disclosure, their detailed explanation will be omitted. In addition, parts that are not related to the description of the present disclosure are omitted from the accompanying drawings, and similar reference numerals have been assigned to similar parts.

[0050] In the present disclosure, when certain components are described as being “connected,” “coupled,” or “linked” to another component, this may include not only direct connections but also indirect connections with another component interposed therebetween. In addition, when a component is described as “including” or “having” another component, this means that, unless expressly stated otherwise, it does not exclude other components but may further include additional components.

[0051] In this disclosure, unless otherwise expressly stated, the terms first, second, etc. are used only to distinguish one component from another and do not limit the order or importance of the components. Therefore, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0052] In this disclosure, distinguishable components are described to clearly explain their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components can be integrated into a single hardware or software unit, or a single component can be distributed across multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure without explicitly describing them.

[0053] In the present disclosure, the components described in the various embodiments do not necessarily mean required components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of the present disclosure. In addition, embodiments including additional components in addition to the components described in the various embodiments are also included in the scope of the present disclosure.

[0054] The present disclosure relates to encoding and decoding of images, and terms used herein may have ordinary meanings commonly used in the technical field to which the present disclosure belongs, unless these terms are newly defined in the present disclosure.

[0055] In this disclosure, a "picture" generally refers to a unit representing a single image at a specific point in time. A slice / tile is a coding unit that constitutes part of a picture, and a picture can be composed of one or more slices / tiles. In addition, a slice / tile can include one or more coding tree units (CTUs).

[0056] In this disclosure, "pixel" or "picture element" may refer to the smallest unit that constitutes a picture (or image). Furthermore, the term "sample" may be used as a corresponding term for a pixel. A sample may generally represent a pixel or a pixel value, and may indicate only the pixel / pixel value of the luma component or only the pixel / pixel value of the chroma component.

[0057] In this disclosure, a "unit" may refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture or information related to that region. Depending on the context, the term "unit" may be used interchangeably with "sample array," "block," "region," and the like. Typically, an M×N block may include a set (or array) of samples (or sample array) or a set (or array) of transform coefficients, consisting of M columns and N rows.

[0058] In the present disclosure, the term "current block" may refer to one of the following: "current coding block," "current coding unit," "encoding target block," "decoding target block," or "processing target block." When prediction is performed, the "current block" may refer to the "current prediction block" or the "prediction target block." When transform (inverse transform) / quantization (dequantization) is performed, the "current block" may refer to the "current transform block" or the "transform target block." When filtering is performed, the "current block" may refer to the "filtering target block."

[0059] In the present disclosure, unless explicitly stated as a chroma block, the term "current block" may refer to a block including both a luma component block and a chroma component block, or may refer to a "luma block of the current block." The luma component block of the current block may be explicitly expressed with terms such as "luma block" or "current luma block," clearly indicating that it is a luma component block. Additionally, the chroma component block of the current block may be explicitly expressed with terms such as "chroma block" or "current chroma block," clearly indicating that it is a chroma component block.

[0060] In the present disclosure, " / " and "," may refer to "and / or". For example, "A / B" and "A, B" may refer to "A and / or B". In addition, "A / B / C" and "A, B, C" may refer to "at least one of A, B, and / or C".

[0061] In the present disclosure, "or" may mean "and / or". For example, "A or B" may mean 1) only "A", 2) only "B", or 3) "A and B". Alternatively, in the present disclosure, "or" may also mean "in addition or alternatively".

[0062] Overview of the video compilation system

[0063] Figure 1 A schematic diagram illustrating a video coding system to which embodiments of the present disclosure can be applied is shown.

[0064] The video coding system according to an embodiment may include an encoder device 10 and a decoder device 20. The encoder device 10 may transmit encoded video and / or image information or data to the decoder device 20 in the form of a file or stream via a digital storage medium or a network.

[0065] The encoder device 10 according to the embodiment may include a video source generator 11, an encoder 12, and a transmitter 13. The decoder device 20 according to the embodiment may include a receiver 21, a decoder 22, and a renderer 23. The encoder 12 may be referred to as a video / image encoder, and the decoder 22 may be referred to as a video / image decoder. The transmitter 13 may be included in the encoder 12. The receiver 21 may be included in the decoder 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0066] Video source generator 11 can obtain videos / images through a process of capturing, synthesizing, or generating videos / images. Video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive containing previously captured videos / images, etc. The video / image generation device may include, for example, a computer, tablet computer, or smartphone, and may (electronically) generate videos / images. For example, virtual videos / images may be generated by a computer, etc. In this case, the video / image capture process may be replaced by a process of generating relevant data.

[0067] The encoder 12 can encode the input video / image. The encoder 12 can perform a series of processes such as prediction, transformation, and quantization for compression and coding efficiency. The encoder 12 can output the encoded data (encoded video / image information) in the form of a bitstream.

[0068] Transmitter 13 can obtain the encoded video / image information or data output in the form of a bitstream and transmit it to receiver 21 of decoder device 20 or another external device in the form of a file or stream via a digital storage medium or network. Digital storage media can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. Transmitter 13 can include components for generating media files using a predetermined file format and components for transmission via a broadcast / communication network. Transmitter 13 can be provided as a transmission device separate from encoder 12. In this case, the transmission device can include at least one processor for obtaining the encoded video / image information or data in the form of a bitstream and a transmitter for delivering it in the form of a file or stream. Receiver 21 can extract / receive the bitstream from the storage medium or network and transmit it to decoder 22.

[0069] The decoder 22 may decode a video / image by performing a series of processes such as dequantization, inverse transformation, prediction, etc. corresponding to the operation of the encoder 12 .

[0070] The renderer 23 may render the decoded video / image. The rendered video / image may be displayed through a display unit.

[0071] Overview of image encoding apparatus

[0072] Figure 2 A schematic diagram showing an image encoding apparatus to which an embodiment of the present disclosure can be applied is shown.

[0073] like Figure 2 As described in

[15] , the image encoding apparatus 100 may include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame predictor 180, an intra-frame predictor 185, and an entropy encoder 190. The inter-frame predictor 180 and the intra-frame predictor 185 may be collectively referred to as a "predictor." The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may further include a subtractor 115.

[0074] Depending on the embodiment, all or at least some of the components constituting the image encoding apparatus 100 may be implemented as a single hardware component (ie, an encoder or a processor). In addition, the memory 170 may include a decoded picture buffer (DPB) and may be implemented by a digital storage medium.

[0075] The image partitioner 110 may partition the input image (or picture, or frame) input to the image encoding device 100 into at least one processing unit. For example, a processing unit may be referred to as a coding unit (CU). A coding unit may be obtained by recursively partitioning a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree, binary tree, or ternary tree (QT / BT / TT) structure. For example, a coding unit may be divided into coding units of greater depth based on a quadtree, binary tree, and / or ternary tree structure. The coding unit partitioning may first be applied using a quadtree structure, followed by a binary tree and / or ternary tree structure. The coding process according to the present disclosure may be performed based on a final coding unit without further partitioning the final coding unit. The largest coding unit may be used directly as the final coding unit, or a coding unit of greater depth obtained by partitioning the largest coding unit may be used as the final coding unit. The coding process may include processes such as prediction, transform, and / or reconstruction, which will be described later. As another example, the processing unit used in the coding process may be a prediction unit (PU) or a transform unit (TU). The prediction unit and the transform unit may each be divided or partitioned from the final coding unit. The prediction unit may be a unit for sample prediction, and the transform unit may be a unit for deriving a transform coefficient from a transform coefficient and / or deriving a residual signal from the transform coefficient.

[0076] The predictor (inter-frame predictor 180 or intra-frame predictor 185) can perform prediction on the target block (current block) and generate a prediction block including prediction samples for the current block. The predictor can determine whether to apply intra-frame prediction or inter-frame prediction to the current block or coding unit (CU). The predictor can generate various information related to the prediction of the current block and send it to the entropy encoder 190. The prediction-related information can be encoded by the entropy encoder 190 and can be output in the form of a bitstream.

[0077] The intra-frame predictor 185 can predict the current block by referring to samples within the current picture. The referenced samples may be located in a neighboring area of the current block, or may be located at a farther position, depending on the intra-frame prediction mode and / or the intra-frame prediction method. The intra-frame prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional mode may include, for example, a DC mode and a planar mode. The directional mode may include, for example, 33 directional prediction modes or 65 directional prediction modes depending on the granularity of the predicted orientation, depending on the granularity of the predicted orientation. However, this is only an example, and a greater or lesser number of directional prediction modes may be used depending on the configuration. The intra-frame predictor 185 may also determine the prediction mode applied to the current block by using the prediction mode applied to the neighboring block.

[0078] The inter-frame predictor 180 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector in a reference picture. To reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation of motion information between neighboring blocks and the current block. Motion information can include a motion vector and a reference picture index. The motion information can further include information regarding the inter-frame prediction direction (i.e., L0 prediction, L1 prediction, Bi prediction, etc.). In inter-frame prediction, neighboring blocks can include spatially neighboring blocks within the current picture and temporally neighboring blocks within a reference picture. The reference picture containing the reference block and the reference picture containing the temporally neighboring block can be the same or different. Temporally neighboring blocks can be referred to as collocated reference blocks or collocated coding units (colCUs). The reference picture containing temporally neighboring blocks can be referred to as collocated pictures (colPics). For example, the inter-frame predictor 180 can construct a motion information candidate list based on the neighboring blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index for the current block. Inter-frame prediction can be performed based on various prediction modes, and for example, in skip mode and merge mode, the inter-frame predictor 180 can use the motion information of a neighboring block as the motion information of the current block. In skip mode, unlike merge mode, a residual signal may not be transmitted. In motion vector prediction (MVP) mode, the motion vector of a neighboring block can be used as a motion vector predictor, and the motion vector of the current block can be signaled by encoding a motion vector difference and an indicator for the motion vector predictor. The motion vector difference may refer to the difference between the motion vector of the current block and the motion vector predictor.

[0079] The predictor can generate a prediction signal based on various prediction methods and / or prediction techniques described below. For example, the predictor can apply intra prediction or inter prediction to the prediction of the current block, or it can apply both intra and inter prediction simultaneously. A prediction method that applies both intra and inter prediction to the prediction of the current block is referred to as combined inter and intra prediction (CIIP). Furthermore, the predictor can perform intra block copying (IBC) on the prediction of the current block. Intra block copying can be used, for example, for screen content coding (SCC) in applications such as game content image / video coding. IBC is a method that predicts the current block using a pre-reconstructed reference block within the current picture, which is located at a predetermined distance from the current block. When applying IBC, the location of the reference block within the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction within the current picture, but because it derives the reference block within the current picture, it can operate similarly to inter prediction. In other words, IBC can use at least one of the inter prediction methods described in this disclosure.

[0080] The prediction signal generated by the predictor can be used to generate a reconstructed signal or a residual signal. The subtractor 115 can generate a residual signal (residual block, residual sample array) by subtracting the prediction signal (prediction block, prediction sample array) output from the predictor from the input image signal (original block, original sample array). The generated residual signal can be sent to the transformer 120.

[0081] Transformer 120 can generate transform coefficients by applying a transform method to the residual signal. For example, the transform method may include at least one of a discrete cosine transform (DCT), a discrete sine transform (DST), a Karhunen-Loeve transform (KLT), a graph-based transform (GBT), or a conditional nonlinear transform (CNT). Here, GBT refers to a transform obtained from a graph when relationship information between pixels is represented as a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. The transform process can be applied to pixel blocks of the same square size or to non-square, variable-sized blocks.

[0082] The quantizer 130 may quantize the transform coefficients and transmit them to the entropy encoder 190. The entropy encoder 190 may encode the quantized signal (information about the quantized transform coefficients) and output it as a bitstream. The information about the quantized transform coefficients may be referred to as residual information. The quantizer 130 may rearrange the block-shaped quantized transform coefficients into a one-dimensional vector based on a coefficient scanning order and may generate information about the quantized transform coefficients based on the one-dimensional vector of the quantized transform coefficients.

[0083] The entropy encoder 190 can perform various encoding methods, such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), or Context-Adaptive Binary Arithmetic Coding (CABAC). The entropy encoder 190 can not only encode the quantized transform coefficients, but also encode information necessary for video / image reconstruction (i.e., syntax element values) together with or separately from the quantized transform coefficients. The encoded information (i.e., the encoded video / image information) can be transmitted or stored in a network abstraction layer (NAL) unit in the form of a bitstream. The video / image information may further include information about various parameter sets, such as an adaptation parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). In addition, the video / image information may further include general constraint information. The signaling information, transmitted information, and / or syntax elements described in this disclosure can be included in the bitstream by being encoded through the above-described encoding process.

[0084] The bitstream may be transmitted via a network or stored in a digital storage medium. Here, the network may include a broadcast network and / or a communication network, etc., and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be provided as an internal / external element of the image encoding device 100, or the transmitter may be configured as a component of the entropy encoder 190.

[0085] The quantized transform coefficients output from the quantizer 130 may be used to generate a residual signal. For example, a residual signal (residual block or residual sample) may be reconstructed by applying dequantization and inverse transform to the quantized transform coefficients through the dequantizer 140 and the inverse transformer 150.

[0086] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, or reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-frame predictor 180 or the intra-frame predictor 185. When there is no residual for the target block, such as when skip mode is applied, the prediction block can be used as the reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next target block in the current picture, and as described later, after filtering, it can also be used for inter-frame prediction of the next picture.

[0087] The filter 160 can apply filtering to the reconstructed signal to enhance the subjective / objective quality. For example, the filter 160 can apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and the modified reconstructed picture can be stored in the memory 170, specifically in the DPB of the memory 170. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc. The filter 160 can generate various filtering related information, as described in the explanation of each filtering method later, and can send it to the entropy encoder 190. The filtering related information can be encoded by the entropy encoder 190 and output in the form of a bit stream.

[0088] The modified reconstructed picture sent to the memory 170 can be used as a reference picture in the inter-frame predictor 180. When applying inter-frame prediction in this case, the image encoding device 100 can avoid prediction mismatch between the image encoding device 100 and the image decoding device, and can improve encoding efficiency.

[0089] The DPB in memory 170 can store the modified reconstructed picture for use as a reference picture in the inter-frame predictor 180. Memory 170 can store motion information for blocks in the current picture for which motion information has been derived (or encoded) and / or motion information for blocks in reconstructed pictures. The stored motion information can be sent to the inter-frame predictor 180 for use as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of the reconstructed blocks in the current picture and send them to the intra-frame predictor 185.

[0090] Overview of image decoding device

[0091] Figure 3 A schematic diagram illustrating an image decoding device to which embodiments of the present disclosure can be applied.

[0092] like Figure 3 As shown, the image decoding apparatus 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, and an intra-frame predictor 265. The inter-frame predictor 260 and the intra-frame predictor 265 may be collectively referred to as a "predictor." The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0093] All or at least some of the plurality of components constituting the image decoding apparatus 200 may be implemented as a single hardware component (ie, a decoder or a processor), depending on the embodiment. In addition, the memory 170 may include a DPB and may be implemented by a digital storage medium.

[0094] The image decoding apparatus 200 that receives a bit stream containing video / image information may perform the same Figure 2 The image is reconstructed using a process corresponding to the process performed by the image encoding device 100 in the decoder. For example, the image decoding device 200 can perform decoding using the processing units used in the image encoding device. Thus, the processing units used for decoding can be, for example, coding units. Coding units can be coding tree units or can be obtained by splitting a maximum coding unit. Furthermore, the reconstructed image signal decoded and output by the image decoding device 200 can be played back via a playback device (not shown).

[0095] The image decoding apparatus 200 can receive the image in the form of a bit stream. Figure 2The received signal may be decoded by the entropy decoder 210. For example, the entropy decoder 210 may parse the bitstream to extract information necessary for image reconstruction (or picture reconstruction) (i.e., video / image information). The video / image information may further include information about various parameter sets, such as an adaptive parameter set (APS), a picture parameter set (PPS), a sequence parameter set (SPS), or a video parameter set (VPS). Furthermore, the video / image information may further include general constraint information. The image decoding device may additionally use the parameter set information and / or general constraint information to decode the image. The signaling information, received information, and / or syntax elements described in this disclosure may be obtained from the bitstream through a decoding process. For example, the entropy decoder 210 may decode the information in the bitstream based on a coding method such as Exponential Golomb coding, CAVLC, or CABAC, and may output syntax element values necessary for image reconstruction and quantized values of transform coefficients associated with the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to syntax elements in a bitstream, determines a context model using information about the target syntax element, decoded information about neighboring blocks and the target block, or information about previously decoded symbols / bins, predicts the probability of a bin's occurrence based on the determined context model, and performs arithmetic decoding on the bins to generate the symbol corresponding to each syntax element. After determining the context model, the CABAC entropy decoding method uses the decoded symbol / bin information to update the context model for the next symbol / bin. Prediction-related information in the decoded information from the entropy decoder 210 is provided to the predictor (inter-frame predictor 260 and intra-frame predictor 265), and the residual values entropy-decoded by the entropy decoder 210, in other words, the quantized transform coefficients and related parameter information, are input to the dequantizer 220. Furthermore, filtering-related information in the decoded information from the entropy decoder 210 is provided to the filter 240. Meanwhile, a receiver (not shown) that receives a signal output from the image encoding device may be additionally configured as an internal / external element of the image decoding device 200 , or the receiver may be configured as a component of the entropy decoder 210 .

[0096] Meanwhile, the image decoding apparatus according to the present disclosure may also be referred to as a video / image / picture decoding apparatus. The image decoding apparatus may include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoder 210, and the sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame predictor 260, or an intra-frame predictor 265.

[0097] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients into two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scanning order used in the image encoding device. The dequantizer 220 may dequantize the quantized transform coefficients using a quantization parameter (i.e., quantization step size information) and obtain the transform coefficients.

[0098] The inverse transformer 230 may perform inverse transform on the transform coefficients to obtain a residual signal (a residual block or a residual sample array).

[0099] The predictor may perform prediction on the current block and generate a prediction block including prediction samples for the current block. The predictor may determine whether to apply intra prediction or inter prediction to the current block based on the prediction-related information output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction method).

[0100] The predictor can generate a prediction signal based on various prediction methods (techniques) to be described later, which is the same as described in the explanation of the predictor in the image encoding device 100 .

[0101] The intra predictor 265 may predict the current block by referring to samples within the current picture. The explanation of the intra predictor 185 can also be applied to the intra predictor 265 in the same manner.

[0102] The inter-frame predictor 260 can derive a prediction block for the current block based on a reference block (reference sample array) specified by a motion vector in a reference picture. To reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted at the block, sub-block, or sample level based on the correlation of motion information between neighboring blocks and the current block. Motion information can include a motion vector and a reference picture index. The motion information can further include information regarding the inter-frame prediction direction (i.e., L0 prediction, L1 prediction, Bi prediction, etc.). In inter-frame prediction, neighboring blocks can include spatially neighboring blocks within the current picture and temporally neighboring blocks in reference pictures. For example, the inter-frame predictor 260 can construct a motion information candidate list based on the neighboring blocks and derive the motion vector and / or reference picture index for the current block based on received candidate selection information. Inter-frame prediction can be performed based on various prediction modes (methods), and prediction-related information can include information indicating the inter-frame prediction mode (method) applied to the current block.

[0103] The adder 235 can generate a reconstruction signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the predictor (including the inter-frame predictor 260 and / or the intra-frame predictor 265). When there is no residual for the target block, such as when skip mode is applied, the prediction block can be used as a reconstructed block. The explanation of the adder 155 can also be applied to the adder 235 in the same manner. The adder 235 can be called a reconstructor or a reconstructed block generator. The generated reconstruction signal can be used for intra-frame prediction of the next target block in the current picture, and as described later, can also be used for inter-frame prediction of the next picture after filtering.

[0104] The filter 240 may apply filtering to the reconstructed signal to enhance the subjective / objective quality. For example, the filter 240 may apply various filtering methods to the reconstructed picture to generate a modified reconstructed picture, and the modified reconstructed picture may be stored in the memory 250, specifically in the DPB of the memory 250. Various filtering methods may include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, bilateral filter, etc.

[0105] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame predictor 260. The memory 250 can store motion information for blocks in the current picture for which motion information has been derived (or decoded) and / or motion information for blocks in the reconstructed picture. The stored motion information can be sent to the inter-frame predictor 260 to be used as motion information for spatially or temporally neighboring blocks. The memory 250 can store reconstructed samples of the reconstructed blocks in the current picture and send them to the intra-frame predictor 265.

[0106] In this specification, the embodiments described for the filter 160, the inter-frame predictor 180, and the intra-frame predictor 185 of the image encoding device 100 can be applied to the filter 240, the inter-frame predictor 260, and the intra-frame predictor 265 of the image decoding device 200 in the same or corresponding manner.

[0107] Intra-frame prediction mode / type determination

[0108] When intra prediction is applied, the intra prediction mode to be applied to the current block can be determined by using the intra prediction mode of the neighboring blocks. For example, the decoding device can select one of the most probable mode (MPM) candidates in the MPM list, which is derived based on the intra prediction modes of the neighboring blocks (e.g., the left neighboring block and / or the upper neighboring block) of the current block and the additional candidate modes based on the received MPM index. Alternatively, one of the remaining intra prediction modes (as well as the planar mode) not included in the MPM candidates can be selected based on the remaining intra prediction mode information. The MPM list can be configured to include or exclude the planar mode as a candidate. For example, when the MPM list includes the planar mode as a candidate, the MPM list can have six candidates, and when the MPM list excludes the planar mode as a candidate, the MPM list can have three candidates. When the MPM list excludes the planar mode as a candidate, a non-planar flag (i.e., a non-planar flag) indicating whether the intra prediction mode of the current block is not the planar mode can be signaled. ). For example, the MPM flag may be signaled first, and when the value of the MPM flag is 1, the MPM index and the non-planar flag may be signaled. In addition, when the value of the non-planar flag is 1, the MPM index may be signaled. Here, the configuration in which the MPM list excludes the planar mode as a candidate does not mean that the planar mode is not considered as the MPM, but because the planar mode is always considered as the MPM, the non-planar flag is first signaled to determine whether the mode is the planar mode.

[0109] For example, one can use the MPM flags (e.g. ) to indicate whether the intra prediction mode applied to the current block is among the MPM candidates (and planar mode) or among the remaining modes. A value of 1 for the MPM flag may indicate that the intra prediction mode of the current block is within the MPM candidates (and planar mode), and a value of 0 for the MPM flag may indicate that the intra prediction mode of the current block is not within the MPM candidates (and planar mode). A value of 0 for the non-planar flag (i.e., ) can indicate that the intra prediction mode of the current block is planar mode, and the value 1 of the non-planar flag can indicate that the intra prediction mode of the current block is not planar mode. The MPM index can be expressed in terms such as or The remaining intra prediction mode information can be signaled in the form of a syntax element such as or For example, the remaining intra prediction mode information may indicate one of the remaining intra prediction modes that is not included in the MPM candidates (and the planar mode) and indexed in the order of the prediction mode numbers in all intra prediction modes. The intra prediction mode may be an intra prediction mode for the luma component (sample). Hereinafter, the intra prediction mode information may include the MPM flag (i.e., ), non-planar signs (i.e. ), MPM index (i.e., or ) and the remaining intra prediction mode information (i.e., or ). In this disclosure, the MPM list may also be referred to by various terms, such as MPM candidate list or When a MIP is applied to the current block, a separate MPM flag for the MIP may be signaled (i.e., ), MPM index (i.e., ) and the remaining intra prediction mode information (i.e., ), and non-planar marks may not be signaled.

[0110] The following takes the intra-frame prediction mode signaling process in the encoding device and the intra-frame prediction mode determination process in the decoding device as examples.

[0111] Figure 4 A flowchart illustrating an example of an intra prediction mode signaling method in an encoding device.

[0112] refer to Figure 4 The encoding apparatus constructs an MPM list for the current block ( S400 ). The MPM list may include candidate intra prediction modes (MPM candidates) with a high likelihood of being applied to the current block. The MPM list may include intra prediction modes of neighboring blocks and may also include specific intra prediction modes according to a predetermined method. A specific method for configuring the MPM list will be described later.

[0113] The encoding device determines an intra prediction mode for the current block ( S410 ). The encoding device may perform prediction based on various intra prediction modes and may determine the optimal intra prediction mode based on rate-distortion optimization (RDO). In this case, the encoding device may determine the optimal intra prediction mode by using only the MPM candidates and planar mode included in the MPM list, or by using the remaining intra prediction modes in addition to the MPM candidates and planar mode included in the MPM list. Specifically, for example, when the intra prediction type of the current block is a specific type (i.e., LIP, MRL, or ISP) rather than a normal intra prediction type, the encoding device may determine the optimal intra prediction mode by considering only the MPM candidates and planar mode as intra prediction mode candidates for the current block. In other words, in this case, the intra prediction mode for the current block may be determined solely from the MPM candidates and planar mode, and the MPM flag may not be encoded / signaled. In this case, the decoder may estimate the MPM flag to be 1 without separately receiving the MPM flag.

[0114] Meanwhile, generally, when the intra prediction mode of the current block is not the planar mode and is one of the MPM candidates in the MPM list, the encoding apparatus generates an MPM index ( ). When the intra prediction mode of the current block is also not included in the MPM list, the encoding device generates remaining intra prediction mode information indicating a mode among the remaining intra prediction modes (and planar modes) not included in the MPM list, which is the same as the intra prediction mode of the current block.

[0115] The encoding device can encode the intra-frame prediction mode information and output it in the form of a bitstream. The intra-frame prediction mode information may include the above-mentioned MPM flag, non-plane flag, MPM index and / or remaining intra-frame prediction mode information. Generally, the MPM index and the remaining intra-frame prediction mode information are in an alternative relationship and may not be signaled at the same time to indicate the intra-frame prediction mode of the block. In other words, the MPM flag value 1 can be signaled together with the non-plane flag or the MPM index, and the MPM flag value 0 can be signaled together with the remaining intra-frame prediction mode information. However, as described above, when a specific intra-frame prediction type is applied to the current block, the MPM flag may not be signaled, and only the non-plane flag and / or the MPM index may be signaled. In other words, in this case, the intra-frame prediction mode information may only include the non-plane flag and / or the MPM index.

[0116] The decoding device may determine the intra prediction mode corresponding to the intra prediction mode information determined and signaled by the encoding device.

[0117] Figure 5 A flowchart showing an example of a method for determining an intra prediction mode in a decoding device.

[0118] refer to Figure 5 , the decoding apparatus obtains intra prediction mode information from the bitstream S500. As described above, the intra prediction mode information may include at least one of an MPM flag, a non-plane flag, an MPM index, and a remaining intra prediction mode.

[0119] The decoding apparatus constructs an MPM list (S510). The MPM list is configured in the same manner as the encoding apparatus. In other words, the MPM list may include intra prediction modes for neighboring blocks and may further include a specific intra prediction mode according to a predetermined method. The specific method for configuring the MPM list will be described later.

[0120] Although S510 is illustrated as being performed after S500 , this is merely an example, and S510 may be performed before or simultaneously with S500 .

[0121] The decoding apparatus determines an intra-prediction mode for the current block based on the MPM list and the intra-prediction mode information (S520). As an example, when the value of the MPM flag is 1, the decoding apparatus may derive the planar mode as the intra-prediction mode for the current block (based on the non-planar flag), or may derive the candidate indicated by the MPM index among the MPM candidates in the MPM list as the intra-prediction mode for the current block. In another example, when the value of the MPM flag is 0, the decoding apparatus may derive the intra-prediction mode indicated by the remaining intra-prediction mode information from the remaining intra-prediction modes and planar modes not included in the MPM list as the intra-prediction mode for the current block. Meanwhile, in another example, when the intra-prediction type of the current block is a specific type (i.e., LIP, MRL, IST, etc.), the decoding apparatus may derive the planar mode in the MPM list or the candidate indicated by the MPM index as the intra-prediction mode for the current block without checking the MPM flag.

[0122] For example, when MRL is not applied to the current block (i.e., when ), the non-planar flag can be signaled, and when MRL is applied to the current block (i.e., when When the non-plane flag is omitted, its value may be estimated to be 1 by the decoding device.

[0123] Meanwhile, the intra prediction mode may include two directional intra prediction modes and 65 directional prediction modes. The non-directional intra prediction mode may include a plane intra prediction mode and a DC intra prediction mode, and the directional intra prediction mode may include intra prediction modes numbered from 2 to 66. The extended directional intra prediction mode may be applied to blocks of all sizes and may be applied to both luma and chroma components.

[0124] Meanwhile, in addition to the above intra prediction modes, intra prediction modes may further include a cross-component linear model (CCLM) mode for chroma samples. Depending on whether the left sample, the top sample, or both are considered for deriving LM parameters, the CCLM mode can be classified as LT_CCLM, L_CCLM, and T_CCLM, and it can be applied only to chroma components.

[0125] For example, the intra prediction modes may be indexed as shown in Table 1 below.

[0126] [Table 1]

[0127]

[0128] Meanwhile, as described above, the intra prediction type (or additional intra prediction mode, etc.) may include at least one of LIP, PDPC, MRL, ISP, and MIP. The intra prediction type may be indicated based on the intra prediction type information, and the intra prediction type information may be implemented in various forms. As an example, the intra prediction type information may include intra prediction type index information indicating one of the intra prediction types. In another example, the intra prediction type information may include at least one of the following: reference sample line information indicating whether MRL is applied to the current block and which reference sample line is used when applying (i.e., ); ISP flag indicating whether to apply ISP to the current block (i.e., ); ISP type information indicating the split type of the sub-partition when ISP is applied (ie, ); flag information indicating whether PDPC is applied; or flag information indicating whether LIP is applied. In addition, the intra prediction type information may include an MIP flag (which may be referred to as an MIP flag) indicating whether MIP is applied to the current block. ).

[0129] Meanwhile, as described above, when MIP is applied to the current block (i.e., when The value of is 1), the MPM list for MIP can be configured separately, and the intra prediction mode information for MIP can include the so-called The MPM logo is called The MPM index is also called The remaining intra prediction mode information.

[0130] In addition, various prediction modes can be used for MIP, and the matrix and offset for the MIP can be derived based on the intra prediction mode used for the MIP. As described above, the matrix can be referred to as a (MIP) weight matrix, and the offset can be referred to as a (MIP) offset vector or a (MIP) deviation vector. The number of intra prediction modes for the MIP can be set differently based on the size of the current block. For example, i) when the height and width of the current block (i.e., CB or TB) are both 4, 35 intra prediction modes (i.e., intra prediction modes 0 to 34) may be available, ii) when the height and width of the current block are both equal to or less than 8, 19 intra prediction modes (i.e., intra prediction modes 0 to 18) may be available, and iii) in other cases, 11 intra prediction modes (i.e., intra prediction modes 0 to 10) may be available. For example, when the height and width of the current block are both 4, it can be referred to as block size type 0, when the height and width of the current block are both equal to or less than 8, it can be referred to as block size type 1, and in other cases, it can be referred to as block size type 2, and the number of intra-frame prediction modes for MIP can be organized in the following table. However, this is only an example, and the block size type and the number of available intra-frame prediction modes can be changed. In the present disclosure, the intra-frame prediction mode for MIP can be referred to as MIP intra-frame prediction mode, MIP prediction mode, or MIP mode.

[0131] [Table 2]

[0132]

[0133] Meanwhile, in the Enhanced Compression Model (ECM), a secondary MPM list has been introduced. The conventional primary MPM (PMPM) list includes six entries, and the secondary MPM (SMPM) list includes sixteen entries. First, a general MPM list having twenty-two entries is configured, and the first six entries in the general MPM list are included in the PMPM list, and the remaining entries are included in the SMPM list. The first entry in the general MPM list is a planar mode, and the remaining entries consist of intra modes of neighboring blocks on the left (L), top (A), bottom left (BL), top right (AR), and top left (AL), directional modes with offsets added from the first two available directional modes of the neighboring blocks, and a default mode, as Figure 6 As described in.

[0134] When the CU block is in a vertical orientation, the order of neighboring blocks may be top (A), left (L), bottom left (BL), top right (AR), and top left (AL). Otherwise, the order may be left (L), top (A), bottom left (BL), top right (AR), and top left (AL).

[0135] The PMPM flag is parsed, and when its value is 1, the PMPM index may be parsed to determine which entry in the PMPM list to select. Otherwise, the SPMPM flag may be parsed to determine whether to parse the SPMPM index or the remaining mode.

[0136] Neighboring reference sample export

[0137] When intra prediction is applied to the current block, neighboring reference samples to be used for intra prediction of the current block can be derived. The neighboring reference samples of the current block may include a total of 2×nH samples adjacent to the left boundary of the current block of size nW×nH and adjacent to its lower left corner, a total of 2×nW samples adjacent to the upper boundary of the current block and adjacent to its upper right corner, and one sample adjacent to the upper left corner of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right corner of the current block.

[0138] At the same time, when applying the MRL described below, the reference samples may be located in rows 1 to 3 instead of row 0 adjacent to the left / top of the current block, and in this case, the number of neighboring reference samples may be further increased. The specific area and number of neighboring reference samples are described below.

[0139] Meanwhile, when the below-described ISP is applied, neighboring reference samples can be derived in units of sub-partitions.

[0140] Some neighboring reference samples of the current block may not have been decoded yet, or may not be available. In this case, the decoder can configure the neighboring reference samples to be used for prediction by interpolation of available samples.

[0141] Some neighboring reference samples for the current block may not have been decoded yet or may not be available. In this case, the decoder can configure neighboring reference samples to be used for prediction by extrapolating the available samples. Starting from the lower left corner and continuing to the upper right corner, the reference samples are configured by replacing or filling the pixels that have not been decoded or are not available with the last available sample, while updating the reference samples to the latest sample (the last available sample).

[0142] Prediction sample derivation based on intra prediction mode / type

[0143] The predictor of the encoding apparatus / decoding apparatus may derive reference samples from the intra prediction mode of the current block among neighboring reference samples of the current block, and may generate prediction samples of the current block based on the reference samples.

[0144] For example, (i) the prediction sample can be derived based on an average or interpolation of neighboring reference samples of the current block, and (ii) the prediction sample can be derived based on a reference sample located at a specific (prediction) orientation among the neighboring reference samples of the current block. Case (i) can be referred to as non-directional mode or non-angle mode, and case (ii) can be referred to as directional mode or angle mode. Alternatively, the prediction sample can be generated by interpolating between a first neighboring sample and a second neighboring sample, where the second neighboring sample is located at an orientation opposite to the prediction orientation of the intra prediction mode for the current block relative to the prediction sample of the current block among the neighboring reference samples. This case can be referred to as linear interpolated intra prediction (LIP). Alternatively, the temporary prediction sample of the current block can be derived based on filtered neighboring reference samples, and the prediction sample of the current block can be derived by performing a weighted sum of the temporary prediction sample and at least one reference sample derived by an intra prediction mode using conventional neighboring reference samples (in other words, unfiltered neighboring reference samples). This case can be referred to as position-dependent intra prediction (PDPC). Furthermore, among multiple neighboring reference samples of the current block, a reference sample line with the highest prediction accuracy can be selected. Predicted samples can be derived using reference samples on the selected line located in the prediction direction. In this case, intra-frame prediction encoding can be performed by signaling the reference samples used to the decoding device. This scenario is referred to as multi-reference line intra prediction (MRL) or MRL-based intra prediction. Furthermore, the current block can be divided into vertical or horizontal sub-partitions, and intra prediction can be performed based on the same intra prediction mode, with neighboring reference samples derived and used within each sub-partition. In this case, the intra prediction mode of the current block is applied to the sub-partitions in the same manner, improving intra prediction performance by deriving and using neighboring reference samples within each sub-partition. This prediction method is referred to as intra sub-partitioning (IPS) or IPS-based intra prediction. This will be described in detail later. Furthermore, when the prediction direction based on the predicted sample points to a position between neighboring reference samples, in other words, when the prediction direction points to a fractional sample position, the value of the predicted sample can be derived by interpolating multiple reference samples located around the corresponding prediction direction (i.e., adjacent to the corresponding fractional sample position).

[0145] The above-mentioned intra-frame prediction method may be referred to as an intra-frame prediction type, which is distinguished from the intra-frame prediction mode. The intra-frame prediction type may also be referred to by various terms, such as an intra-frame prediction technique or an additional intra-frame prediction mode. For example, the intra-frame prediction type (or additional intra-frame prediction mode, etc.) may include at least one of the above-mentioned LIP, PDPC, MRL, or ISP. Information about the intra-frame prediction type may be encoded by the encoding device and included in a bitstream to be signaled to the decoding device. The information about the intra-frame prediction type can be implemented in various forms, such as flag information indicating whether each intra-frame prediction type is applied, or index information indicating one intra-frame prediction type among multiple intra-frame prediction types.

[0146] The MPM list for deriving the above-mentioned intra prediction mode may be configured differently according to the intra prediction type. Alternatively, the MPM list may be configured in common regardless of the intra prediction type.

[0147] Neighboring reference sample export

[0148] When intra prediction is applied to the current block, neighboring reference samples to be used for intra prediction of the current block can be derived. The neighboring reference samples of the current block may include a total of 2×nH samples adjacent to the left boundary of the current block of size nW×nH and adjacent to its lower left corner, a total of 2×nW samples adjacent to the upper boundary of the current block and adjacent to its upper right corner, and one sample adjacent to the upper left corner of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include a total of nH samples adjacent to the right boundary of the current block of size nW×nH, a total of nW samples adjacent to the lower boundary of the current block, and one sample adjacent to the lower right corner of the current block.

[0149] Meanwhile, when multi-reference line (MRL) is applied, the reference samples may be located in rows 1 to 3 instead of row 0 adjacent to the current block on the left / top, and in this case, the number of neighboring reference samples may be further increased. The area and number of specific neighboring reference samples are described below.

[0150] Meanwhile, when intra sub-partitioning (ISP) is applied, neighboring reference samples can be derived in units of sub-partitions.

[0151] DIMD (Decoder-side Intra Mode Derivation)

[0152] In DIMD, intra prediction can be derived as a planar orientation and a weighted average of two derived orientations. To achieve this, two angular modes can be selected from the Histogram of Gradients (HoG) calculated from the neighboring pixels of the current block. When two modes are selected, their predictors (prediction blocks) and the planar predictor can be calculated normally, and the corresponding weighted average can be used as the final predictor (final prediction block) for the current block. In this case, to determine the weights, the corresponding amplitude in the HoG can be used for each of the two modes.

[0153] Since the derived intra modes are included in the master list of intra MPMs, the DIMD process can be performed before configuring the MPM list.The master derived intra modes of a DIMD block can be stored with the block and can be used to configure the MPM lists of neighboring blocks.

[0154] In DIMD color mode, if Figure 12 As shown, the DIMD derivation method can be used to derive the chroma intra prediction mode of the current block based on the pre-reconstructed adjacent Y, Cb, and Cr samples in the second adjacent row and column. Specifically, to construct the HoG, the horizontal gradient and vertical gradient can be calculated not only for the pre-reconstructed Cb and Cr samples but also for each collocated pre-reconstructed luma sample of the current chroma block. Then, the intra prediction mode with the highest histogram magnitude can be used to perform chroma intra prediction for the current chroma block.

[0155] When the intra prediction mode derived from the DIMD chroma mode is the same as the intra prediction mode derived from the DM mode, the intra prediction mode with the second highest histogram magnitude may be used as the DIMD chroma mode. In order to indicate whether the above DIMD chroma mode is applied, a predetermined CU-level flag may be signaled.

[0156] TIMD (fusion for template-based intra-mode derivation)

[0157] For each intra prediction mode in the MPM, the SATD between the predicted sample and the reconstructed sample of the template can be calculated. Then, the first two intra prediction modes with the smallest SATD can be selected as the TIMD mode. These two TIMD modes can be fused according to the weights, and this weighted intra prediction can be used to code the current CU. The derivation of the TIMD mode can include the above-mentioned position-dependent intra prediction combination (PDPC).

[0158] The costs of the two selected modes may be compared to a predetermined threshold, and a cost factor of 2 may be applied, as shown in Equation 1 below.

[0159] [Formula 1]

[0160]

[0161] When the condition of Formula 1 is true, the above fusion can be applied. On the contrary, when the condition of Formula 1 is false, only Mode 1 can be used.

[0162] Meanwhile, the weights of the modes can be calculated according to their corresponding SATD costs, as shown in Equation 2 below.

[0163] [Formula 2]

[0164]

[0165]

[0166] Intra-frame prediction fusion

[0167] Intra-frame prediction methods derive prediction samples by weightedly combining multiple predictors (prediction blocks) generated from different reference lines. In this process, multiple intra-frame predictors are generated and fused using a weighted average. The process for deriving the predictors used in the fusion process is as follows.

[0168] Intra-frame prediction methods derive prediction samples by weightedly combining multiple predictors (prediction blocks) generated from different reference lines. During this process, multiple intra-frame predictors are generated and fused using a weighted average. The process for deriving the predictors used in the fusion process is as follows.

[0169] 1. For the single mode directional intra prediction mode including TIMD and DIMD, the proposed method utilizes The weights obtained from multiple reference lines derive intra prediction. Here, is an intra prediction obtained from the default reference line, and is the forecast obtained from the line above the default reference line. is set to 3 / 4, and the weight is set to 1 / 4.

[0170] 2. For TIMD mode that performs blending, For the first mode (weight is 1, is 0), and is used in the second mode (weight is 0, is 1).

[0171] 3. For DIMD mode that performs hybrid, the number of predictors (prediction blocks) used for weighted averaging is increased from 3 to 6.

[0172] When the directional intra prediction mode (angular intra mode) has non-integer gradients (reference sample interpolation is necessary) and the block size is greater than 16, the intra prediction fusion method is applied to the luma block and used with MRL, but not applied to ISP-coded blocks. In addition, PDPC is applied to the intra prediction mode by using the reference line closest to the current block.

[0173] The present disclosure relates to intra-frame prediction, and more specifically, to a technique for adaptively selecting an intra-frame prediction mode and applying it to a reference sample region in both directional and non-directional intra-frame prediction. Furthermore, according to an embodiment of the present disclosure, a technique for efficiently combining multiple intra-frame prediction blocks generated using different reference sample regions to improve coding efficiency can be proposed.

[0174] As an example, when the intra prediction mode of the current block is based on DIMD, a new intra prediction block may be generated by combining two prediction blocks in the intra prediction mode and a planar intra prediction block through HoG distribution analysis of reference samples of the current block.

[0175] As another example, when the intra prediction mode of the current block is based on TIMD, a new intra prediction block may be generated by combining two intra prediction blocks selected by template matching on an extended reference sample line of the current block.

[0176] As another example, when the intra-frame prediction mode of the current block is based on intra-frame prediction fusion, the selected reference sample line of the current block can be used to generate an intra-frame prediction block, and the reference sample line directly above the selected reference sample line can be used to generate a new prediction block by applying the same intra-frame prediction mode, so that a new prediction block can be generated by combining the two prediction blocks.

[0177] At the same time, according to embodiments of the present disclosure, a prediction block can be generated by selecting only an intra-frame prediction mode that primarily utilizes the upper reference samples of the current block. Similarly, a prediction block can also be generated by selecting only an intra-frame prediction mode that primarily utilizes the left reference samples of the current block. Furthermore, the prediction blocks generated in this manner can be combined based on weights to generate a (final) prediction block.

[0178] In other words, in the examples of the present disclosure, when using an intra-prediction mode that performs prediction primarily based on the upper reference sample, only the intra-prediction mode adaptively selected based on this can be selected to apply the intra-prediction mode to the current block. As another example, when using an intra-prediction mode that performs prediction primarily based on the left reference sample, only the intra-prediction mode adaptively selected based on this can be selected to apply the intra-prediction mode to the current block. In addition, by adaptively combining multiple reference blocks generated in this manner, intra-prediction accuracy can be improved.

[0179] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0180] Example #1

[0181] This embodiment proposes a method for generating an intra-frame prediction block by using only an intra-frame prediction mode that performs prediction based mainly on an upper reference sample of a current block. Figure 10 and other drawings are described.

[0182] As an example, intra prediction, i.e., intra prediction, can be performed by using a partially selected mode. The partially selected mode may mean a mode selected based on the position (e.g., above) of a reference sample, for example, a mode constrained to a specific range. To this end, an intra prediction mode candidate associated with an upper reference sample may be first set S1010. After S1010, although Figure 10 Not shown in the figure, the decoder can determine the intra prediction mode by obtaining information about the intra prediction mode from the bitstream. At the same time, the encoder can select and determine the intra prediction mode from the intra prediction mode candidates, and encode the information about the selected intra prediction mode into the bitstream. Afterwards, it can be determined whether the intra prediction mode of the current block is a specific mode (for example, planar mode or DC mode) S1020. When the intra prediction mode of the selected current block is planar mode or DC mode, intra prediction can be applied to the current block based on the improved DC mode or plane (vertical plane) prediction mode S1030, and when the intra prediction mode of the selected current block is not planar mode or DC mode, the selected intra prediction mode can be applied to the current block as is S1040. By reference Figures 7 to 9 Provide a detailed description.

[0183] Figure 7 FIG shows an example of an intra-frame prediction method that can be applied to the present disclosure. More specifically, Figure 7 : is an example of a method for performing intra prediction by using only an upper reference sample, wherein the left side, i.e., (A) is a diagram showing an example of intra prediction based on an intra prediction direction in an 8x8 block (e.g., for VVC-based, number 59), and the right side, (B) is a diagram showing the number of the intra prediction direction and the VVC.

[0184] Figure 7 (A) is an example of an intra prediction direction (e.g., intra prediction mode 59 for VVC, see Figure 7 (B)), this intra prediction direction only uses the upper reference sample to predict the current block. Figure 7As shown in , in vertical intra prediction (e.g., 50 for VVC), other intra prediction modes (e.g., intra prediction mode 66 for VVC) can perform intra prediction by using only the upper reference sample above the current block. Meanwhile, as another example, when wide-angle intra prediction of VVC is applied, modes 50 to 80 may be modes that use only the upper reference sample.

[0185] In this embodiment, intra prediction can be performed by combining only intra prediction modes that perform prediction primarily based on the upper reference sample, as in the example above. In this case, the encoder can transmit the corresponding intra prediction information to the decoder. In other words, according to this embodiment, intra prediction can be performed by adaptively selecting only intra prediction modes that perform prediction primarily based on the upper reference sample to generate a prediction block.

[0186] According to an embodiment of the present disclosure, when prediction is performed mainly based on the upper reference sample of the current block, only intra prediction modes within a specific range (e.g., p to q) may be used:

[0187] 1. As an example, when vertical intra prediction is applied (e.g., mode 50 for VVC), only intra prediction modes greater than or equal to a specific mode (e.g., mode 50 for VVC) may be used. In other words, intra prediction may be performed using intra prediction modes within a specific range (e.g., prediction modes 50 to 66 for VVC, and prediction modes 50 to 80 when the wide-angle intra prediction mode of VVC is applied).

[0188] 2. As another example, when diagonal intra prediction is applied, only intra prediction modes exceeding a specific mode (e.g., 34 for VVC) can be used. In other words, intra prediction can be performed using intra prediction modes within a specific range (e.g., prediction modes 35 to 66 for VVC, and prediction modes 35 to 80 when VVC's wide-angle intra prediction mode is applied).

[0189] 3. As another example, when horizontal intra prediction is applied, only intra prediction modes exceeding a specific mode (eg, number 18 for VVC) may be used.

[0190] 4. As another example, when vertical intra prediction is applied, only intra prediction modes less than or equal to a specific mode (e.g., number 50 for VVC) can be used. In other words, for VVC, only intra prediction modes within a specific range (e.g., 19 to 50) can be used.

[0191] In other words, as in the above example, intra prediction may not be performed using only intra prediction modes within a specific range (e.g., intra prediction modes p to q). For example, p and q may have values within a specific range. For example, p and q may have values from 0 to 80, and p ≤ q may be satisfied.

[0192] At the same time, according to this embodiment, the wide-angle intra prediction mode may or may not be applied. In other words, intra predictions exceeding the highest-numbered mode among general intra prediction modes (for example, number 66 for VVC) may be considered, or only the intra prediction mode with the highest number of corresponding modes may be used.

[0193] In addition, although only directional intra prediction modes are used as examples above, they correspond to embodiments of the present disclosure, so according to other embodiments, non-directional intra prediction modes such as PLANAR and DC can be used. Therefore, when DC mode is applied, according to the present disclosure, the DC value can be calculated using only the upper reference sample of the current block. In addition, when planar mode is applied, according to the present disclosure, vertical plane mode can be applied by using the following formula. Figure 8 2 is an example illustrating a block for vertical plane prediction according to an embodiment of the present disclosure.

[0194] [Formula 3]

[0195]

[0196] In the above formula, H may refer to the height of the block, and (x, y) may refer to the coordinates within the block. In addition, rec(x, y) may refer to the reference sample at the position (x, y), and (x, y) may mean the value predicted by the vertical plane at the (x, y) position.

[0197] Meanwhile, when constraining the intra prediction mode and signaling the intra prediction mode information according to this embodiment, the following exemplary method may be used:

[0198] The intra prediction mode adaptively selected based on the number X of available intra prediction modes may be encoded and decoded through appropriate binarization.

[0199] In this method, X may refer to the number of available intra prediction modes according to this embodiment. For example, for VVC-based, only intra prediction modes 50 to 66 are used, and when vertical plane mode and DC mode are included among the planar modes, X may be 19.

[0200] In this case, for signaling of the intra prediction mode selected in this embodiment, the intra prediction mode may be encoded by truncated binary coding based on the number X of intra prediction mode candidates and signaled as intra prediction mode information.

[0201] Alternatively, as in existing video coding standards including HEVC / VVC, after first selecting an MPM intra prediction mode candidate through intra prediction mode search of neighboring blocks, information about the selected intra prediction mode can be adaptively signaled based on MPM and non-MPM lists. The following example relates to an example of dividing the intra prediction mode into MPM and non-MPM modes and performing encoding and decoding. Figure 9 is a diagram illustrating an example of a neighboring block search position based on an MPM mode in an 8×8 block according to an embodiment of the present disclosure.

[0202] As an example, a specific number (hereinafter, β) of MPM intra-frame prediction modes can be preferentially selected using the intra-frame prediction modes of neighboring blocks to select β intra-frame prediction mode candidates. In this case, according to this embodiment, when selecting candidate prediction modes based on the intra-frame prediction modes of neighboring blocks, only intra-frame prediction modes included in the intra-frame prediction mode range (for example, prediction modes p to q) can be selected as candidate prediction modes. As an example, β can be any number within X (which can mean the number of intra-frame prediction modes that can be used according to this embodiment).

[0203] Meanwhile, in this case, the MPM index can be binarized by truncated unary binarization and encoded and decoded. Meanwhile, as an example, when an intra-frame prediction mode candidate is selected without being based on MPM (non-MPM), the selected intra-frame prediction mode candidate (e.g., multiple) can be encoded and decoded by truncated binarization. Meanwhile, the default mode for MPM-based can be determined in the order of high selection frequency among any number of selected intra-frame prediction mode candidates such as planar mode (e.g., number 0 for VVC), vertical mode (e.g., number 50 for VVC), DC mode (e.g., number 1 for VVC), and other special modes (e.g., number 66 for VVC).

[0204] As an example, the selection order and number of intra prediction mode candidates using neighboring blocks may follow the method used for selection in HEVC, VV, or ECM. (A) Figure 9 , on the left side, is a diagram showing an example of a neighboring block search position for MPM selection in the ECM according to an embodiment of the present disclosure. (B) Figure 9 The right side of FIG is a diagram showing an example of setting the neighboring block search position according to an embodiment of the present disclosure. As an example, when three MPMs are selected according to this embodiment, it can be as follows (B), Figure 9 The intra prediction candidate is searched and selected in the order of blocks including A -> AR -> AL -> AC -> TL pixels on the right side of . In the above, AC may correspond to a reference sample having an x-coordinate having a value obtained by dividing the width value of the current block by 2.

[0205] Meanwhile, the pixel position and search order used for the above-mentioned MPM search are not limited to this example, and intra-frame prediction candidates can be selected in various ways according to the intra-frame prediction mode of the upper reference sample and the left reference sample. In addition, in order to select more intra-frame prediction candidate modes, the intra-frame prediction candidate mode can be selected by searching the upper reference samples and the left reference samples of multiple reference sample lines.

[0206] Alternatively, the primary and secondary MPM intra prediction mode candidates may be preferentially selected through intra prediction mode search of neighboring blocks such as ECM. Hereinafter, an example of dividing the intra prediction mode into primary, secondary MPM, and non-MPM modes and performing encoding and decoding is described.

[0207] -β main MPM intra prediction modes can be preferentially selected based on the intra prediction modes of neighboring blocks to configure β intra prediction mode candidates. In this case, according to this embodiment, only intra prediction modes included in the intra prediction mode range (for example, specific modes p and q) can be selected. Here, β can be any number within X (which can mean the number of intra prediction modes that can be used according to this embodiment).

[0208] After selecting all β primary MPM intra prediction modes, γ secondary MPM intra prediction modes can be prioritized based on the intra prediction modes of neighboring blocks to configure γ intra prediction mode candidates. In this case, the intra prediction modes included in the secondary MPMs can be selected so as not to overlap with the intra prediction modes in the primary MPMs. In addition, when selecting candidates, only intra prediction modes included in a range of intra prediction modes (for example, between specific modes p and q) can be selected. Here, γ can be any number within the range of X-β.

[0209] - Primary and secondary MPM indices can be binarized by truncated unary binarization and encoded and decoded. Non-MPM X-β-γ intra prediction modes can be encoded and decoded by truncated binary binarization.

[0210] - The default mode in the MPM may be determined based on, for example, VVC in the order of high selection frequency among the selected X intra prediction modes such as planar mode (number 0), vertical mode (number 50), DC mode, and other special modes (for example, number 66).

[0211] At the same time, the selection order and number of intra prediction mode candidates of neighboring blocks can follow the method selected in ECM. Figure 9 (A) and (B) are the same as described above, so repeated description is omitted.

[0212] Meanwhile, the pixel positions and search order for the primary and secondary MPM searches are not limited to this example, and intra-frame prediction candidates may be selected in various ways according to the intra-frame prediction mode in the upper reference sample and the left reference sample. In addition, as described above, in order to select more intra-frame prediction candidates, intra-frame prediction candidates may also be selected by searching multiple rows of upper reference samples and left reference samples.

[0213] According to this embodiment, when encoding and decoding the intra prediction mode, it is not limited to this example, and various mode encoding and decoding methods can be applied according to the intra prediction mode X used.

[0214] As an example, it is also possible to determine an intra prediction mode candidate for the current block or determine the intra prediction mode for the current block based on DIMD or TIMD. In other words, the intra prediction mode candidate or intra prediction mode for the corresponding block can be inferred and applied by utilizing neighboring reference samples such as DIMD or TIMD.

[0215] As an example, the mode candidate inference method used in DIMD can be applied to predict the intra prediction mode of the current block. In other words, pixel gradients can be calculated using only the upper reference sample area, and the first intra prediction mode or the second prediction mode obtained by the corresponding gradient can be set as the intra prediction mode candidate for the current block. Here, one of the first intra prediction mode or the second prediction mode can be used as is, or, as described in the example, can be excluded when it is not included in the X intra prediction modes. When the first and second intra prediction modes are not included in the X intra prediction modes, only vertical plane prediction can be applied.

[0216] As another example, the mode inference method used in TIMD can be applied to predict the intra prediction mode of the current block. In other words, TIMD template matching can be performed using only the upper reference sample area, and as described above, only X intra prediction modes can be considered. In this embodiment, the intra prediction mode candidate to which TIMD template matching will be applied can be selected based on the MPM, the primary MPM, or the secondary MPM, or all X intra prediction modes can be selected, or the selection can be made in the same manner as the method currently used in TIMD.

[0217] As another example, the existing DIMD mode and the method proposed in this embodiment may be combined to select and apply the intra prediction mode of the corresponding block while applying the method proposed in this embodiment:

[0218] - The existing DIMD's planar mode can be replaced with a vertical planar mode.

[0219] - The DC of the existing DIMD can be replaced by a method using only the upper reference sample.

[0220] - Only upper reference samples can be used to calculate gradients and select prediction modes only within X intra prediction mode candidates.

[0221] As another example, the existing TIMD mode may be combined with the method proposed in this embodiment to select and apply the intra prediction mode of the corresponding block while applying the method proposed in this embodiment:

[0222] - The existing DIMD's planar mode can be replaced with a vertical planar mode.

[0223] - The DC of the existing DIMD can be replaced by a method using only the upper reference sample.

[0224] - TIMD template matching can be performed by using only the upper reference sample region.

[0225] - In this embodiment, the intra prediction mode candidates to which TIMD template matching will be applied may be selected based on MPM, primary MPM or secondary MPM, or all X intra prediction modes may be selected, or may be selected in the same manner as the method currently used in TIMD.

[0226] At the same time, whether the embodiment of the present disclosure is applied can be signaled as specific information in a VPS, SPS, PPS, picture header, slice header, DCI, etc. that is a high-level syntax (HLS). As an example, in order to determine whether the method proposed in this embodiment is applied in units of PPS, information on whether the method proposed in this embodiment is applied in the PPS can be signaled.

[0227] In addition, the method proposed in this embodiment can adaptively select whether to apply without sending additional information to the decoder, as in this method, or additional information about whether to apply the method proposed in this embodiment can be sent to the decoder to determine whether to apply it. As an example, it can be signaled by using a 1-bit flag indicating whether the method proposed in this embodiment is applied in units of CTU or CU. Alternatively, when the method proposed in this embodiment is applied based on the size or shape of a specific block or whether a specific condition exists, it can be signaled using a 1-bit flag indicating whether the method proposed in this embodiment is applied only in this case. As an example, when the block height is a specific multiple (for example, 4 times) or higher of the block width, the method proposed in this embodiment may not be applied, and therefore, the signaling of additional information about whether to apply can be omitted.

[0228] At the same time, under certain conditions, it is possible to implicitly infer whether the method proposed in this embodiment is applied. For example, when the left reference sample of the current block does not exist at the left boundary of the image, or when the left reference sample may not be used, it can be determined to omit the signaling information (e.g., a flag) indicating whether the method proposed in this embodiment is applied, and the method proposed in this embodiment is always applied. In addition, when the left reference sample of the current block is at a CTU boundary / tile boundary / slice boundary / sub-picture boundary, it can be determined to apply the method proposed in this embodiment after omitting the signaling information (e.g., a flag) indicating whether the method proposed in this embodiment is applied.

[0229] At the same time, whether to signal the application of the method proposed in this embodiment at a low level (e.g., a coding unit) can be adaptively determined based on information indicating whether the method proposed in this embodiment is applied at a high level defined in a high-level syntax (HLS). As an example, when the value of the information indicating whether the method proposed in this embodiment is applied in an SPS is false (e.g., 0), that is, when the method proposed in this embodiment is not used in units of SPSs, it is determined that the method proposed in this embodiment is not applied even in the low-level coding units, and signaling of the information indicating whether the method is applied may not be performed.

[0230] According to Embodiment 1 of the present disclosure, intra prediction efficiency can be improved.

[0231] Example #2

[0232] This embodiment proposes a method for generating an intra-frame prediction block by using only an intra-frame prediction mode that performs prediction mainly based on the left side of the current block, that is, the left reference sample. Figure 14 and other drawings for description.

[0233] First, as an example, Figure 14 S1420 and S1440 shown in FIG may correspond to Figure 10 As described above, intra prediction can be performed by using a partially selected mode, that is, intra prediction. The partially selected mode may mean a mode selected based on the position (e.g., left) of the reference sample, for example, a mode constrained to a specific range. To this end, an intra prediction mode candidate S1410 associated with the left reference sample of the current block may be set. After S1410, although Figure 14Not shown in the figure, the decoder can determine the intra prediction mode by obtaining information about the intra prediction mode from the bitstream. At the same time, the encoder can select and determine the intra prediction mode from the intra prediction mode candidates, and encode the information of the selected intra prediction mode into the bitstream. Afterwards, it can be determined whether the intra prediction mode of the current block is a specific mode (for example, planar mode or DC mode) S1420. When the intra prediction mode of the selected current block is planar mode or DC mode, the intra prediction can be applied to the current block based on the modified DC or plane (horizontal plane) prediction mode S1430, and when the intra prediction mode of the selected current block is not planar mode or DC mode, the selected intra prediction mode can be applied to the current block as is S1440. By reference Figures 11 to 13 Provide a detailed description.

[0234] Figure 11 is a diagram showing an example of an intra-frame prediction method that can be applied to the present disclosure. More specifically, Figure 11 This is an example of a method for performing intra prediction using only the left reference sample. The left side (A) shows an example of intra prediction based on intra prediction directions in an 8x8 block, and the right side (B) shows intra prediction directions and VVC numbers.

[0235] Figure 11 (A) is the intra prediction direction of the current block by using only the left reference sample (e.g., intra prediction mode 6 for VVC, reference Figure 11 (B)) Example. Figure 11 As shown in , in horizontal intra prediction (e.g., number 6 for VVC), other intra prediction modes (e.g., intra prediction mode 18 for VVC) can perform intra prediction by using only the left reference sample located on the left side of the current block. Meanwhile, as another example, when wide-angle intra prediction of VVC is applied, modes 14 to 18 may be modes that use only the left reference sample.

[0236] In the following embodiments of the present disclosure, a technique is proposed that performs intra-frame prediction by combining only intra-frame prediction that performs prediction primarily based on left reference samples and signal prediction information, as in the above example. In other words, when applying the method proposed in this embodiment, intra-frame prediction can be applied by adaptively selecting only intra-frame prediction that performs prediction primarily based on left reference samples, as in the following example.

[0237] As an example, when prediction is performed primarily based on the left reference sample, it may be possible to constrain the use of only intra prediction modes within a specific range (e.g., p to q):

[0238] 1. Only intra prediction modes less than or equal to the horizontal intra prediction mode number (e.g., 18 for VVC-based) can be used. For example, for VVC-based, it is possible to constrain the use of only intra prediction modes 2 to 18 (-14 to 18 when wide angle mode is applied).

[0239] 2. Only intra prediction modes with a number less than the diagonal intra prediction mode number (e.g., 34 for VVC-based) can be used. For example, for VVC-based, only intra prediction modes 2 to 33 (-14 to 33 when wide-angle mode is applied) can be used.

[0240] 3. There may be a restriction to use only intra prediction modes that are greater than or equal to the horizontal intra prediction mode number (e.g., 18 for VVC-based) but less than the vertical intra prediction mode number (e.g., 50 for VVC-based). For example, for VVC-based, it may be constrained to use only intra prediction modes 18 to 49.

[0241] As in the above example, intra prediction can be constrained to be performed by using only intra prediction modes within a specific range (eg, intra prediction modes p to q). As an example, p and q can have values from 0 to 80, and p≤q.

[0242] At the same time, when applying the method proposed in this embodiment, the wide-angle intra-frame prediction mode may or may not be applied. In other words, intra-frame prediction modes with negative numbers may be considered (for example, -1 to -14 for VVC-based), or only intra-frame prediction modes with a specific number or higher may be used (for example, intra-frame prediction mode number 2 or higher for VVC-based).

[0243] In addition, this embodiment can also be applied to non-directional intra prediction such as PLANAR and DC. When the DC mode is applied to the method proposed in this embodiment, the DC value can be calculated by using only the left reference sample of the current block. As another example, when the planar mode is applied to the method proposed in this embodiment, the horizontal plane mode can be applied as shown in the following formula. Figure 12 is a diagram illustrating an example of a block for horizontal plane prediction according to an embodiment of the present disclosure.

[0244] [Formula 4]

[0245]

[0246] In the above formula, W may refer to the width of the block, and (x, y) may refer to the coordinates within the block. rec(x, y) may refer to the reference sample at the (x, y) position, and It can mean the value predicted by the horizontal plane at the (x, y) position.

[0247] Meanwhile, when signaling mode information of the intra prediction mode applied in the method proposed in this embodiment, the method in the following example may be used:

[0248] - Encoding and decoding can be performed by appropriately binarizing the intra prediction mode adaptively selected based on the number X of available intra prediction modes.

[0249] In this method, X may refer to the number of intra prediction modes that can be used when applying the method proposed in this embodiment. For example, for VVC-based, only intra prediction modes 2 to 18 are used in the example, and when planar (horizontal plane) and DC modes are included, X may be 19.

[0250] As an example, for signaling of the intra prediction mode selected in this embodiment described above, truncated binary coding may be performed on the intra prediction mode based on X (the number of intra prediction mode candidates) and may be signaled.

[0251] Alternatively, after first selecting an MPM intra prediction mode candidate through intra prediction mode search of neighboring blocks as in HEVC / VVC, information about the selected intra prediction mode can be adaptively transmitted based on MPM and non-MPM lists. The following example relates to an example of dividing the intra prediction mode into MPM and non-MPM modes and performing encoding and decoding. Figure 12 is a diagram illustrating an example of a neighboring block search position based on an MPM mode in an 8×8 block according to an embodiment of the present disclosure.

[0252] - β intra prediction mode candidates can be preferentially selected by selecting the intra prediction mode (MPM) of a neighboring block. When selecting intra prediction mode candidates for a neighboring block, only intra prediction modes included in the intra prediction mode range proposed in this embodiment (e.g., between p and q) can be selected. β can be any number within X (which can mean the number of intra prediction modes that can be used according to this embodiment).

[0253] - MPM indexes can be binarized by truncated unary binarization and encoded, signaled, and decoded. Non-MPM X-β intra prediction modes can be encoded and decoded by truncated binary binarization.

[0254] - The default mode in the MPM may be determined in the order of high selection frequency among the selected X intra prediction modes such as planar (number 0 for VVC-based), horizontal (number 18 for VVC-based), DC (number 1 for VVC-based), or other intra prediction modes (for example, number 2 for VVC-based).

[0255] As an example, the selection order and number of intra prediction mode candidates using neighboring blocks may follow a method for selection in HEVC, VV, or ECM. Figure 13 is a diagram for describing exemplary search positions of neighboring blocks for MPM when applying the embodiment of the present disclosure in an 8x8 block according to an embodiment of the present disclosure. More specifically, Figure 13 (A) is a diagram for describing an example of a neighboring block search position for MPM selection in ECM according to an embodiment of the present disclosure, and Figure 13 (B) is a diagram for describing an example of neighboring block search position setting according to an embodiment of the present disclosure. As an example, when three MPMs are selected for the method proposed in this embodiment, the following can be used: Figure 13 (B) Intra prediction candidates are searched and selected in the order of blocks of L->BL->LA->TL->LC pixels. In the above, LC may mean a reference sample having a y coordinate having a value obtained by dividing the width value of the current block by 2.

[0256] Meanwhile, the pixel position and search order for the MPM search are not limited to this example, and intra prediction candidates may be selected in various ways according to the intra prediction mode in the upper reference sample and the left reference sample. In addition, in order to select more intra prediction candidates, intra prediction candidates may be selected by searching the upper reference samples and the left reference samples of multiple reference sample lines.

[0257] Alternatively, the primary and secondary MPM intra prediction mode candidates can be preferentially selected by searching for intra prediction modes of neighboring blocks such as ECM. The following example relates to an example of dividing the intra prediction mode into primary MPM, secondary MPM, and non-MPM modes and performing encoding and decoding:

[0258] - β intra-frame prediction mode candidates can be selected by preferentially selecting β main MPM intra-frame prediction modes based on the intra-frame prediction modes of neighboring blocks. When selecting intra-frame prediction mode candidates for neighboring blocks, only intra-frame prediction modes included in the intra-frame prediction mode range proposed in this embodiment (for example, between p and q) can be selected. In this case, β can be any number within X (which can mean the number of intra-frame prediction modes that can be used according to this embodiment).

[0259] After selecting all β primary MPM intra prediction modes, γ secondary MPM intra prediction modes can be selected by preferentially selecting them based on the intra prediction modes of neighboring blocks. The intra prediction modes included in the secondary MPMs do not need to overlap with those in the primary MPMs. As an example, when selecting intra prediction mode candidates for neighboring blocks, only intra prediction modes included in the intra prediction mode range proposed in this embodiment (e.g., between p and q) can be selected. γ can be any number within the range X-β.

[0260] - Primary and secondary MPM indices can be binarized by truncated unary binarization and encoded and decoded. Non-MPM X-β-γ intra prediction modes can be encoded and decoded by truncated binary binarization.

[0261] - It can be determined in the order of high selection frequency among X intra prediction modes selected in the MPM, such as planar (e.g., 0 for VVC), horizontal (e.g., 18 for VVC), DC mode, or other specific intra prediction modes (e.g., 2 for VVC).

[0262] As an example, the selection order and number of intra prediction mode candidates of neighboring blocks may follow the method selected in the ECM. Figure 13 (A) may relate to an example of a neighboring block search position for primary and secondary MPM selection in an ECM according to an embodiment of the present disclosure. Figure 13 (B) may relate to an example of setting a neighboring block search position according to an embodiment of the present disclosure. As an example, when three MPMs are selected for the method proposed in this embodiment, the following may be used: Figure 13 (B) Sequential search and selection of intra prediction candidates for blocks including L->BL->LA->TL->LC pixels. In the above, LC may mean a reference sample having a y coordinate having a value obtained by dividing a height value of a current block by 2.

[0263] Meanwhile, the pixel positions and search order for the primary and secondary MPM searches are not limited to this example, and intra prediction candidates may be selected in various ways according to the intra prediction mode in the upper reference sample and the left reference sample. In addition, in order to select more intra prediction candidates, intra prediction candidates may be selected by searching the upper reference sample and the left reference sample from multiple sample lines.

[0264] When the intra prediction mode is encoded and decoded by the method proposed in this embodiment, it is not limited to this example, and various mode encoding and decoding methods may be applied according to the intra prediction mode used.

[0265] Alternatively, it is possible to determine an intra-prediction mode candidate of the current block such as DIMD or TIMD, or to determine the intra-prediction mode of the current block. In other words, the intra-prediction mode of the current block can be determined by inferring the intra-prediction mode candidate or the intra-prediction mode of the corresponding block using neighboring reference samples such as DIMD or TIMD.

[0266] The mode candidate inference method used in DIMD can be applied to the mode prediction in the method proposed in this embodiment. In other words, only the left reference sample area can be used to calculate the pixel gradient, and the first intra-frame prediction mode or the second prediction mode obtained by the corresponding gradient can be set as the intra-frame prediction mode candidate for the current block. Here, the first intra-frame prediction mode or the second prediction mode can be used as is, or can be excluded when the first intra-frame prediction mode or the second prediction mode is not included in the X intra-frame prediction modes in the example. When the first and second intra-frame prediction modes are not included in the X intra-frame prediction modes, only horizontal plane prediction can be applied.

[0267] At the same time, as an example, the mode inference method used in TIMD can be applied to the intra mode prediction described above. In other words, by performing TIMD template matching using only the left reference sample area, only the X intra prediction modes in the example can be considered. The intra prediction mode candidate to which TIMD template matching will be applied can be selected, such as the MPM, primary MPM, or secondary MPM in this embodiment, or all X intra prediction modes can be selected, or they can be selected in the same manner as the method currently used in TIMD.

[0268] In addition, the DIMD mode can be combined with the method proposed in this embodiment to select and apply the intra prediction mode of the corresponding block while applying the method proposed in this embodiment:

[0269] - The plane of the existing DIMD can be replaced by a horizontal plane.

[0270] - The DC of the existing DIMD can be replaced by a method that only uses the left reference sample.

[0271] - Only left reference samples may be used to compute gradients and select a mode only within X intra prediction mode candidates.

[0272] In addition, the existing TIMD mode can be combined with the method proposed in this embodiment to select and apply the intra prediction mode of the corresponding block while applying the method proposed in this embodiment:

[0273] - The plane of the existing TIMD can be replaced by a horizontal plane.

[0274] - The DC of the existing TIMD can be replaced by a method that only uses the left reference sample.

[0275] - TIMD template matching can be performed by using only the left reference sample area.

[0276] - An intra prediction mode candidate to which TIMD template matching will be applied may be selected, such as MPM, primary MPM, or secondary MPM in this embodiment, or all X intra prediction modes may be selected, or may be selected in the same manner as the method currently used in TIMD.

[0277] Meanwhile, information indicating whether the intra-prediction mode selection method according to this embodiment is applied may be transmitted in a VPS, SPS, PPS, picture header, slice header, or DCI as a high-level syntax (HLS). As an example, to determine whether the method proposed in this embodiment is applied per PPS, information on whether the method proposed in this embodiment is applied may be transmitted in the PPS.

[0278] Furthermore, the information indicating whether to apply the method proposed in this embodiment may not be signaled to the decoder as additional information, but may be adaptively determined. Alternatively, additional information indicating whether to apply the method proposed in this embodiment may be explicitly signaled to the decoder, and application may be determined based on the corresponding information. As an example, a 1-bit flag may be used to indicate whether the method proposed in this embodiment is applied per CTU or CU, and this may be signaled to the decoder. Alternatively, when the method proposed in this embodiment is applied based on the size or shape of a specific block or the presence of a specific condition, specific information (e.g., a 1-bit flag) may be signaled regarding whether the method proposed in this embodiment is applied only in that case. As an example, when the width of the current block is a specific multiple (e.g., 4 times) of the block height, the method proposed in this embodiment may not be applied, and thus, signaling of the information indicating whether to apply the method may be omitted.

[0279] As an example, whether to apply the method proposed in this embodiment can be implicitly inferred by the specific conditions described above. As an example, when there is no left reference sample for the current block, such as at the left boundary of an image, or when the left reference sample may not be used, the method proposed in this embodiment can be always performed after omitting the signaling of information indicating whether to apply the method proposed in this embodiment. In other words, the intra-frame prediction mode selection method proposed in this embodiment can be applied. As an example, when the left reference sample of the current block is at a CTU boundary / tile boundary / slice boundary / sub-picture boundary, the method proposed in this embodiment can be always applied after omitting the signaling of information indicating whether to apply the method proposed in this embodiment.

[0280] For example, when information indicating whether the method proposed in this embodiment is applied at a high level is defined in the HLS, it may be adaptively determined whether to signal information indicating whether the method proposed in this embodiment is applied at a low level (e.g., a coding unit). For example, when the information indicating whether the method proposed in this embodiment is applied at the SPS level is false (e.g., 0), that is, when it is determined that the method proposed in this embodiment is not applied in units of SPSs, the method proposed in this embodiment is not applied in the coding unit, and therefore, signaling of information regarding whether the method is applied may not be performed.

[0281] According to Embodiment 2 of the present disclosure, intra prediction efficiency can be improved.

[0282] Example #3

[0283] This embodiment proposes a method for combining the intra prediction mode proposed in the above embodiment 1, which performs prediction mainly based on the upper reference sample, and the intra prediction mode proposed in the above embodiment 2, which performs prediction mainly based on the left reference sample. Figure 18 and other drawings for description.

[0284] First, as an example, as described above, intra prediction (i.e., intra-frame prediction) can be performed using a partially selected mode. A partially selected mode may refer to a mode selected based on the location of a reference sample (e.g., to the left), for example, a mode constrained to a specific range. Subsequently, an intra predictor (i.e., a prediction block) may be generated based on reference samples constrained to a specific location (e.g., the top or left region of the current block) and the associated intra prediction mode (S1810). For example, the intra prediction block generated based on the above reference sample of the current block and its associated intra prediction mode may be an above prediction block (e.g., the first prediction block, pred_above), and the intra prediction block generated based on the left reference sample of the current block and its associated intra prediction mode may be a left prediction block (e.g., the first prediction block, pred_left). Subsequently, weights w0 (e.g., the first weight) and w1 (e.g., the second weight) to be applied to each prediction block are determined (S1820). The generated intra prediction blocks may then be merged based on the determined weights (S1830). In this case, the fusion can be derived as the value obtained by multiplying each prediction block by the weight and adding them together (e.g., final prediction block pred = w0 * pred_above + w1 * pred_left).

[0285] At the same time, for the sake of clarity, Figure 18Assume two prediction blocks and two weights, but this is only an embodiment of the present disclosure. In other words, in other embodiments, there may be at least two prediction blocks and at least two weights. In addition, in the embodiment to be described below, a weight may be determined for each pixel in the prediction block. By referring to the following Figures 15 to 17 Describe in more detail.

[0286] Figure 15 is a diagram for describing an example of combining an intra prediction method for performing prediction mainly based on an upper reference sample and an intra prediction method for performing prediction mainly based on a left reference sample according to an embodiment of the present disclosure. More specifically, Figure 15 An example of combining prediction blocks generated by an intra prediction method is shown, in which the left side (A) mainly uses an upper reference sample and the right side (B) mainly uses a left reference sample.

[0287] As an example, the intra prediction block generated based on Example 1 and the intra prediction block generated based on Example 2 may be combined in a method as in the following example.

[0288] [Formula 5]

[0289]

[0290] In the formula, It may refer to the combined prediction block proposed in this embodiment, and It may mean a prediction block (for example, a first prediction block) predicted by using an intra prediction mode in which prediction is performed mainly based on an upper reference sample (Embodiment 1). w0 may refer to a prediction block (eg, a second prediction block) predicted by using an intra prediction mode that performs prediction mainly based on a left reference sample (Example 2). w0 may refer to a prediction block The weight (eg, the first weight) of the prediction block, and w1 may mean When combining two prediction blocks, the prediction block and Each pixel at the same position in can generate a combined predicted pixel at the corresponding position by using the first weight w0 and the second weight w1.

[0291] In the embodiment, the above and can be determined as a fixed value as follows.

[0292] [Formula 6]

[0293]

[0294] According to Formula 6, in an embodiment of the present disclosure, the first weight and the second weight may be the same. As an example, the first weight and the second weight may be 1 / 2, and the two prediction blocks and The equal combination may always be performed at a ratio of 1: 1. As an example, the second weight may be determined based on the first weight.

[0295] According to another embodiment, the first weight and the second weight may be determined as follows.

[0296] [Formula 7]

[0297]

[0298] In Equation 7, the first weight can have a fixed specific value, and the second weight Can have fixed values In this formula, the specific value Can be any value selected from all numbers. Alternatively, The first weight and the second weight may be different values. The second weight may be determined based on the first weight.

[0299] As another example, the first weight and the second weight may be variably determined as follows: As an example, the first weight and the second weight may be determined based on the size (eg, width and / or height) of the current block.

[0300] [Formula 8]

[0301] ,

[0302] In the above formula 8, variable width can mean the width of the current block, and variable height can mean the height of the current block. For example, for a 16x4 block, when the weight is derived by formula 8, the first weight can be changed to 16 / 20 = 0.8, and the second weight It can become 4 / 20 = 0.2. As an example, the second weight can be determined based on the first weight.

[0303] According to another embodiment, the first weight and the second weight can be variably determined as follows.

[0304] [Formula 9]

[0305] ,

[0306] The first weight and the second weight are applied to the first prediction block and the second prediction block, respectively, and thus they can be derived based on the number of reference samples used to generate the first prediction block and the second prediction block, respectively. In the above formula 9, the variable can mean the number of upper reference samples used for prediction, and It may mean the number of left reference samples used for prediction.As an example, the second weight may be determined based on the first weight.

[0307] According to another embodiment, the first weight and the second weight Can be determined variably as follows. By referring to Equation 10 to Figure 16 Provide a description. Figure 16 is a diagram for describing an example of an intra-prediction block and its neighboring reference sample region according to an embodiment of the present disclosure.

[0308] [Equation 10]

[0309] ,

[0310] As an example, the first weight w0 and the second weight w1 may be derived based on an error value derived by using a reference sample value. , as in the template matching method of TIMD, may be a prediction block derived by using the above-mentioned upper reference sample The value derived by calculating the error value between the actual reference sample value and the intra prediction mode applied in the reference sample area. In this case, when using neighboring reference samples to calculate When, such as Figure 16 As shown in , adjacent reference samples (e.g., top / upper left / upper right / left / lower left, etc.) can be used. Alternatively, only some of the reference samples in the five areas can be used. For example, only the upper reference sample can be used, or only the upper left reference sample can be used, or only the upper right reference sample can be used, or only the left reference sample can be used, or only the lower left reference sample can be used. Alternatively, some of the reference samples in the five areas can be combined and used, and the error value can be calculated based on this. In addition, as for the number of reference sample lines, 4 lines can be used in the same manner as in the existing TIMD method, or more or fewer (e.g., 3 lines, or 5 lines, etc.) reference sample lines can be used. According to an embodiment, only the left reference sample can be used, or only the upper reference sample can be used to calculate the variable Alternatively, the left reference sample and the top reference sample may be used, or the top, top right, and top left reference samples may be used. Alternatively, all of the left, top, top right, bottom left, and top left reference samples may be used. Meanwhile, in the formula, the variable , as in the template matching method of TIMD, can be achieved by applying the prediction block generated by using the left reference sample mentioned above After the intra prediction mode of is applied to the reference sample area, the value derived by calculating the error value with the actual reference sample value is calculated. As an example, As described in , the neighboring reference samples are used to calculate When , adjacent reference samples (e.g., top / upper left / upper right / left / lower left, etc.) can be used. Alternatively, only some of the reference samples in the five regions can be used. For example, only the top reference sample can be used, or only the upper left reference sample can be used, or only the upper right reference sample can be used, or only the left reference sample can be used, or only the lower left reference sample can be used. Alternatively, some of the reference samples in the five regions can be combined and used, and based on this, the error value can be calculated. In addition, as for the number of reference sample lines, 4 lines can be used in the same manner as the existing TIMD method, or more or fewer (e.g., 3 lines or 5 lines, etc.) reference sample lines can be used. In addition, only a portion of the reference samples in the five regions can be used, or they can be combined to derive the error value.

[0311] In this embodiment, the variable and When , the error value can be derived by using the mean square error (MSE), the sum of absolute transformed differences (SATD), or the sum of absolute differences (SAD) and , and when used to calculate and When the number of reference samples is different, the error value can be further coordinated through the normalization process.

[0312] Alternatively, in this embodiment, prediction blocks generated by an intra prediction method that performs prediction mainly based on an upper reference sample can be combined in a method as in the following equation ( Figure 15 (A)), and a prediction block generated by an intra prediction method that performs prediction based on a left reference sample ( Figure 15 (B)).

[0313] [Equation 11]

[0314]

[0315] In the formula, Can mean the coordinates within the block used for prediction. For example, may be a prediction block (eg, a first prediction block) generated by using an upper reference sample, and It may be a prediction block (eg, a second prediction block) generated by using the left reference sample. The first weight can be applied to the first prediction block, and the second weight can be applied to the second prediction block, from which the final prediction block pred can be derived. In this case, different weights and can be adaptively applied to each pixel position within the prediction block. In other words, the weight values may be based on But different.

[0316] For example, a method for adaptively selecting a first weight at a pixel position is proposed as follows: and the second weight method.

[0317] [Equation 12]

[0318]

[0319]

[0320] if ,but

[0321] if ,but

[0322] ,otherwise

[0323]

[0324] In the formula, width and height can refer to the width and height of the current block respectively. In addition, as shown in the embodiment, the variable It can be a value obtained by calculating the error value size using the actual block neighboring reference sample value after predicting the neighboring reference sample value of the current block by using the intra prediction block obtained by using the main reference sample, and the variable It may be a value obtained by calculating the magnitude of the error value using actual block neighboring reference sample values after predicting neighboring reference sample values of the current block by using an intra prediction block obtained by mainly using left reference samples.

[0325] According to the three equations proposed above, a combination weight inversely proportional to the distance from each pixel position to the left reference sample and the upper reference sample can be selected, and thus, a prediction block can be generated in a more complex manner. This can improve coding efficiency.

[0326] Meanwhile, in this embodiment, since it is possible to explicitly signal information about a prediction block generated by an intra prediction method that performs prediction mainly based on an upper reference sample ( Figure 15(A)) and a prediction block generated by an intra prediction method that performs prediction mainly based on the left reference sample ( Figure 15 Since the method proposed in Embodiment #1 and the method proposed in Embodiment #2 can be combined to signal the corresponding information, the information about the two intra-frame prediction modes for each of the two prediction blocks can be explicitly signaled separately.

[0327] Alternatively, as another example, the method for signaling information about intra-frame prediction modes according to Embodiment #1 and the method for signaling information about intra-frame prediction modes according to Embodiment #2 may be combined and used, as in the following example. In other words, information about multiple (e.g., two) intra-frame prediction modes may be signaled in one intra-frame prediction mode candidate group (e.g., based on Embodiment #1 or based on Embodiment #2, etc.).

[0328] - Intra prediction mode information can be encoded and decoded by truncated binary binarization of the intra prediction mode selected based on X (the number of available intra prediction modes).

[0329] In this method, X, the number of available intra-frame prediction modes, can be determined as the sum of the number of intra-frame prediction modes available in the intra-frame prediction method in embodiment #1 that performs prediction mainly based on the upper reference sample and the number of intra-frame prediction modes available in the intra-frame prediction method in embodiment #2 that performs prediction mainly based on the left reference sample.

[0330] Alternatively, after first selecting an MPM intra prediction mode candidate through an intra prediction mode search of neighboring blocks as in VC / VVC, information on the selected intra prediction mode may be adaptively signaled based on the MPM and non-MPM lists. The example in which the intra prediction mode is divided into the MPM and non-MPM modes and encoded and decoded is the same as the example described with reference to the above-mentioned Embodiments 1 and 2, and thus a description thereof is omitted.

[0331] at the same time, Figure 17 : This figure shows an example of neighboring block search positions for MPM when applying the proposed method in an 8x8 block according to an embodiment of the present disclosure. The selection order and number of intra prediction mode candidates of neighboring blocks can follow the method used for selection in HEVC, VV or ECM. Figure 17 (A) shows an example of a neighboring block search position for MPM selection in ECM. Alternatively, the neighboring block search position for the method proposed in this embodiment can be as follows Figure 17 (B) As mentioned above, when three MPMs are selected for the method proposed in this embodiment, the following can be used: Figure 17(B) Intra prediction candidates are searched and selected in the order of blocks including L->A->BL->AR->TL->LC->AC->LA->AL pixels. In the above, AC may refer to a reference sample coordinate whose x coordinate is a value obtained by dividing the width value of the current block by 2; and LC may refer to a reference sample coordinate whose y coordinate is a value obtained by dividing the height value of the current block by 2.

[0332] The pixel position and search order for the MPM search are not limited to this example, and intra-frame prediction candidates can be selected in various ways according to the intra-frame prediction mode of the upper reference sample and the left reference sample. In addition, in order to select more intra-frame prediction candidates, intra-frame prediction candidates can be selected by searching the upper reference samples and the left reference samples of multiple reference sample lines.

[0333] Alternatively, the main MPM and auxiliary MPM intra prediction mode candidates can be preferentially selected by searching the intra prediction mode of a neighboring block such as an ECM. The intra prediction mode can be divided into main MPM, auxiliary MPM, and non-MPM modes, and encoded and decoded, which is the same as described by referring to the above-mentioned embodiments 1 and 2, so repeated description is omitted.

[0334] For example, the selection order and number of intra prediction mode candidates of neighboring blocks may follow the method selected in the ECM. Figure 17 (A) is a diagram for an example of a neighboring block search position for primary and secondary MPM selection in an ECM according to an embodiment of the present disclosure, and Figure 17 (B) is a diagram showing an example of the neighboring block search position setting for the method proposed in this embodiment. As described above, when three MPMs are selected for the method proposed in this embodiment, the following can be used: Figure 17 (B) Intra prediction candidates are searched and selected in the order of blocks including L->A->BL->AR->TL->LC->AC->LA->AL pixels. In the above, AC may mean a reference sample coordinate whose x coordinate is a value obtained by dividing the width value of the current block by 2, and LC may mean a reference sample coordinate whose y coordinate is a value obtained by dividing the height value of the current block by 2.

[0335] Meanwhile, the pixel positions and search order for the primary and secondary MPM searches are not limited to this example, and intra-frame prediction candidates can be selected in various ways according to the intra-frame prediction mode in the upper reference sample and the left reference sample. In addition, in order to select more intra-frame prediction candidates, intra-frame prediction candidates can be selected by searching the upper reference sample and the left reference sample using multiple reference sample lines.

[0336] While the intra prediction mode is encoded and decoded in the method proposed in this embodiment, it is not limited to this example, and various mode encoding and decoding methods may be applied according to the intra prediction mode X used.

[0337] Meanwhile, as another example, the signaling of information regarding additional intra-frame prediction modes, such as DIMD or TIMD, can be eliminated. In other words, intra-frame prediction mode candidates for the corresponding block can be predicted by utilizing neighboring reference samples, such as DIMD or TIMD. In this embodiment, the mode candidate inference method used in DIMD or TIMD can be applied to predict the intra-frame prediction mode, which is the same as that described in Embodiments 1 and 2 above, and therefore its repeated description is omitted.

[0338] In addition, by combining the existing DIMD mode and the method proposed in this embodiment, the intra-frame prediction mode of the corresponding block can be selected and applied while applying the method proposed in this embodiment, which is the same as described in the above-mentioned embodiment #1 and embodiment #2, so its repeated description is omitted.

[0339] In addition, by combining the existing TIMD mode and the method proposed in this embodiment, the intra-frame prediction mode of the corresponding block can be selected and applied while applying the method proposed in this embodiment, which is the same as described in the above-mentioned embodiment #1 and embodiment #2, so its repeated description is omitted.

[0340] When combining the methods proposed in this embodiment, the number of combinations is not limited to this example. In other words, the combination between blocks to which intra prediction is applied, which is primarily based on multiple upper reference samples (Embodiment #1), or the combination between blocks to which intra prediction is applied, which is primarily based on multiple left reference samples (Embodiment 2) can be applied. Alternatively, combinations can be performed between blocks to which intra prediction is applied, which is primarily based on multiple reference samples, and between blocks to which intra prediction is applied, which is primarily based on multiple left reference samples.

[0341] The following example shows the combination between the three blocks to which the method proposed in the present invention is applied.

[0342] [Equation 13]

[0343]

[0344]

[0345] In the formula, pred may mean the combined (final) prediction block proposed in this embodiment, and pred_above may mean a prediction block (for example, a first prediction block) predicted by using an intra-frame prediction mode (the method in Example 1) that performs prediction mainly based on the upper reference sample. pred_left and pred_left2 may mean prediction blocks (for example, at least two second prediction blocks, or a second prediction block and a third prediction block, respectively) predicted by using an intra-frame prediction mode (the method in Example 2) that performs prediction mainly based on the left reference sample. At the same time, above, it is described that two prediction blocks are generated based on the left reference sample and one prediction block is generated based on the upper reference sample, but this only corresponds to the embodiment of the present disclosure. Therefore, at least two or one prediction blocks may be generated based on the left reference sample, or at least two prediction blocks may be generated based on the upper reference sample. In this case as well, weights may be determined separately for the prediction blocks or samples in the prediction blocks, and the description may be extended and applied. In the formula, It can mean the intra-block coordinates used for prediction. In this case, the first weight , second weight and the third weight It can also be adaptively applied to pixel positions in the prediction block. Here, the weights can be different values.

[0346] In this example, the prediction block pred_left and the prediction block pred_left2 (for example, at least two second prediction blocks, or the second prediction block and the third prediction block) may have different intra prediction mode ranges to which they may be applied. As an example, in the formula, pred_left may be a block generated by using only intra prediction modes 2 to 18 for VCC, pred_left2 may be a block generated by using only intra prediction modes 18 to 49 for VCC, and pred_above may be a block generated by using only intra prediction modes 50 to 66 for VCC. In this example, weights may be similarly applied by the weight setting method described in the extended example. 、 、 .

[0347] As an example, three weights 、 、 They may all be the same value, or may be determined in proportion to the number of intra prediction mode candidates used. 、 、 Can be different values. In addition, by extending the method proposed above, the method for signaling information about the intra prediction mode when generating three prediction blocks can also be similarly applied.

[0348] At the same time, whether the method proposed in this embodiment is applied can be signaled in the VPS, SPS, PPS, picture header, slice header, and DCI as high-level syntax (HLS). For example, to determine whether the method proposed in this embodiment is applied in units of PPS, information on whether the method proposed in this embodiment is applied can be signaled in the PPS.

[0349] In addition, according to this embodiment, whether to apply the embodiment of the present disclosure can be adaptively determined without explicitly signaling as additional information, as in this method. Alternatively, additional information about whether to apply the embodiment of the present disclosure can be explicitly signaled to determine whether to apply it, as in this method. As an example, whether the method proposed in this embodiment is applied in units of CTU or CU can be signaled by using a 1-bit flag. Alternatively, when the method proposed in this embodiment can be applied depending on the size or shape of a specific block or whether a specific condition exists, only in this case can the method proposed in this embodiment be signaled by using a 1-bit flag.

[0350] As an example, under certain conditions, it is possible to implicitly infer whether to apply the method proposed in this embodiment. For example, when the left reference sample of the current block such as the left boundary of the image does not exist, or when the left reference sample cannot be used, when applying this embodiment, the first weight Set to 1, and the second weight Set to 0, or only intra prediction mode that performs prediction primarily based on left reference samples can be used.

[0351] In addition, as another example, when the upper reference sample of the current block such as the upper boundary of the image does not exist, or when the upper reference sample cannot be used, when this embodiment is applied, the second weight Set to 1, and the first weight Set to 0, or only intra prediction mode that performs prediction primarily based on left reference samples can be used.

[0352] At the same time, after omitting the signaling of information indicating whether this embodiment is applied, the present disclosure may be preset to always apply or not apply. For example, when the upper / left reference sample of the current block is a CTU boundary / tile boundary / slice boundary / sub-picture boundary, the present disclosure may be preset to always apply or not apply after omitting the corresponding information.

[0353] At the same time, whether to signal information about whether the method proposed in this embodiment is applied at a low level (e.g., a coding unit) can be adaptively determined based on information about whether the method proposed in this embodiment is applied at a high level defined in the HLS. As an example, when the information about whether the method proposed in this embodiment is applied in an SPS is false (e.g., 0) (i.e., when the method proposed in this embodiment is not used in units of SPSs), it can be determined that the method proposed in this embodiment is not applied at a low level (e.g., a coding unit), and therefore, signaling of information about whether to use it can be omitted.

[0354] According to Embodiment 3 of the present disclosure, intra prediction efficiency can be improved.

[0355] Embodiments of image encoding / decoding methods

[0356] Figure 19 is a diagram for describing an image encoding / decoding method that can be performed by an image encoding / decoding apparatus according to an embodiment of the present disclosure. Figure 19 It can be performed by the above-mentioned image decoding device, or it can be performed by the image encoding device. In addition, all or part of the above-mentioned embodiments #1, #2 and / or #3 can be applied to this embodiment.

[0357] Based on this description, according to an embodiment of the present disclosure, an intra prediction mode for a current block is determined (S1910), and intra prediction is performed based on the determined intra prediction mode to generate a prediction block for the current block (S1920). However, the intra prediction may be constrained to be performed using only reference samples within a predetermined range among the available reference samples of the current block. In this case, the intra prediction mode may be determined to be one of the directional modes within the predetermined range based on the reference samples within the predetermined range. Furthermore, the predetermined range may be determined based on whether the intra prediction is based on wide-angle intra prediction. Furthermore, the predetermined range may be any one of the directional modes 50 to 66, 35 to 66, or 19 to 34. Furthermore, the predetermined range may be any one of the directional modes 2 to 18, 2 to 33, or 18 to 49. Furthermore, the prediction block may be generated by weighted summing a first prediction block obtained using reference samples within a first range and a second prediction block obtained using reference samples within a second range. Here, the first range and the second range may not overlap. Furthermore, the first weight applied to the first prediction block may be determined differently from the weight applied to the second prediction block. In other words, the first weight can be determined based on the size of the current block. Alternatively, the first weight and the second weight can be predefined values. Furthermore, since it can be used alone or in combination with the embodiments described in Embodiments #1 to #3, a repeated description thereof will be omitted.

[0358] According to the present disclosure, when intra-frame prediction is performed on an image, an adaptive intra-frame prediction block can be generated in a reference sample region, and multiple intra-frame prediction blocks can be efficiently fused and encoded / decoded. In addition, the encoding / decoding performance of intra-frame prediction can be improved.

[0359] At the same time, because Figure 19 Corresponding to the embodiments of the present disclosure, some steps may be changed, deleted, or added, and the order of execution may be changed, and it is obvious that this is also included in the scope of the present disclosure.

[0360] Embodiments of image encoding methods

[0361] Figure 20 is a diagram for describing an image encoding method that can be performed by an image encoding device according to an embodiment of the present disclosure. Figure 20 It can be performed by the above-mentioned image encoding apparatus. In addition, all or part of the above-mentioned embodiments #1, #2 and / or #3 can be applied to this embodiment.

[0362] Based on this description, according to an embodiment of the present disclosure, the intra prediction mode of the current block may be determined S2010, and prediction mode information of the current block may be encoded S2020 based on the determined intra prediction mode. In this case, the intra prediction mode may be determined by using only reference samples within a predetermined range among available reference samples of the current block. This is different from the above method of encoding the intra prediction mode by reference to the reference sample including Figure 19 The descriptions of other drawings are the same as those of the drawings, and thus repeated descriptions are omitted.

[0363] In the description Figure 20 When describing the image coding method in

[15] , some overlapping descriptions are omitted. According to the present disclosure, when performing intra-frame prediction on an image, adaptive intra-frame prediction blocks can be generated in a reference sample region, and multiple intra-frame prediction blocks can be efficiently fused and encoded. Furthermore, the coding performance of intra-frame prediction can be improved.

[0364] At the same time, because Figure 21 Corresponding to the embodiments of the present disclosure, some steps may be changed, deleted, or added, and the order of execution may be changed, and it is obvious that this is also included in the scope of the present disclosure.

[0365] For clarity of explanation, the exemplary methods of the present disclosure are described as a series of operations, but this is not intended to limit the order in which the steps are performed, and each step can be performed simultaneously or in a different order when necessary. In order to implement the methods according to the present disclosure, in addition to the steps shown, additional steps may be included, some steps may be omitted while the remaining steps are included, or some steps may be omitted while the additional steps are included.

[0366] In the present disclosure, an image encoding device or image decoding device that performs a predetermined operation (step) may perform the operation (step) to check the conditions or circumstances under which the corresponding operation (step) is to be performed. For example, when a description is given of performing a predetermined operation when a predetermined condition is satisfied, the image encoding device or image decoding device may perform an operation to check whether the predetermined condition is satisfied before performing the predetermined operation.

[0367] The various embodiments of the present disclosure are not a list of all possible combinations but are provided to illustrate representative aspects of the present disclosure, and the elements described in the various embodiments may be applied independently or in combination of two or more elements.

[0368] In addition, various embodiments of the present disclosure may be implemented using hardware, firmware, software, or a combination thereof. When implemented in hardware, the various embodiments may be implemented using one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, microcontrollers, or microprocessors.

[0369] In addition, the image decoding apparatus and the image encoding apparatus to which the embodiments of the present disclosure are applied may be included in various devices, such as multimedia broadcast transmission / reception devices, mobile communication terminals, home movie video devices, digital movie video devices, surveillance cameras, video conferencing devices, real-time communication devices such as video communication devices, mobile streaming devices, storage media, cameras, video on demand (VoD) service providers, over-the-top (OTT) devices, Internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, and medical video devices, and may be used to process video signals or data signals. For example, over-the-top (OTT) devices may include game consoles, Blu-ray players, Internet-connected TVs, home theater systems, smartphones, tablet computers, and digital video recorders (DVRs).

[0370] Figure 21 An exemplary diagram showing a content streaming system to which embodiments of the present disclosure can be applied is shown.

[0371] like Figure 21 As shown, the content streaming system to which the embodiments of the present disclosure are applied may broadly include an encoding server, a streaming server, a web server, a media storage, a user device, and a multimedia input device.

[0372] The encoding server compresses the content input from a multimedia input device such as a smartphone, camera, or camcorder into digital data, generates a bitstream, and sends it to the streaming server. As another example, when the multimedia input device such as a smartphone, camera, or camcorder directly generates the bitstream, the encoding server can be omitted.

[0373] A bitstream may be generated by applying the video encoding method and / or the image encoding apparatus according to the embodiments of the present disclosure, and a streaming server may temporarily store the bitstream during a process of transmitting or receiving the bitstream.

[0374] The streaming server can transmit multimedia data to a user device via a web server based on a user request, and the web server can act as an intermediary to inform the user of available services. When the user requests a desired service from the web server, the web server can transmit the request to the streaming server, and the streaming server can transmit the multimedia data to the user. In this case, the content streaming system can include a separate control server, and in this case, the control server can play a role in controlling the command / response exchange between devices within the content streaming system.

[0375] The streaming server can receive content from a media storage and / or encoding server. For example, when receiving content from an encoding server, the content can be received in real time. In this case, in order to provide a seamless streaming service, the streaming server can store the bitstream for a certain period of time.

[0376] Examples of user devices may include mobile phones, smartphones, laptops, digital broadcast terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet PCs, tablet computers, ultrabooks, wearable devices (i.e., smart watches, smart glasses, head-mounted displays (HMDs)), digital televisions, desktop computers, and digital signage.

[0377] Each server in the content streaming system can operate as a distributed server, in which case data received by each server can be processed in a distributed manner.

[0378] The scope of the present disclosure includes software or machine-executable instructions (i.e., operating systems, applications, firmware, programs, etc.) that enable the operations of the methods according to various embodiments to be performed on a device or computer, as well as non-transitory computer-readable media in which such software or instructions are stored and executable on a device or computer.

[0379] [Industrial Applicability]

[0380] The embodiments of the present disclosure may be used to encode / decode images.

Claims

1. An image decoding method performed by an image decoding apparatus, comprising: determining an intra prediction mode for a current block; as well as Based on the determined intra prediction mode, generating a prediction block of the current block by performing intra prediction, wherein the intra prediction mode is determined based on range constraint information indicating whether the intra prediction mode is constrained within a specific range; and In which, based on the range constraint information indicating that the intra-frame prediction mode is constrained within the specific range, based on the intra-frame prediction mode determined to be one of the intra-frame prediction modes within the specific range, the intra-frame prediction is performed mainly using reference samples within a predetermined range among the available reference samples of the current block.

2. The method according to claim 1, wherein Based on the fact that the reference samples within the predetermined range are mainly used, the intra prediction mode is determined to be a directional mode within the specific range.

3. The method according to claim 2, wherein: The specific range is determined based on whether the intra prediction is based on wide-angle intra prediction.

4. The method according to claim 2, wherein: The specific range is any one of the directional mode ranges 50 to 66, 35 to 66, or 19 to 34.

5. The method according to claim 2, wherein: The specific range is any one of the directional mode ranges 2 to 18, 2 to 33, or 18 to 49.

6. The method according to claim 1, wherein The prediction block is generated by performing a weighted summation of a first prediction block obtained by mainly using reference samples in a first range and a second prediction block obtained by mainly using reference samples in a second range.

7. The method according to claim 6, wherein: The first range and the second range do not overlap with each other.

8. The method according to claim 6, wherein: A first weight applied to the first prediction block is determined differently from a weight applied to the second prediction block.

9. The method according to claim 8, wherein The first weight is determined based on the size of the current block.

10. The method according to claim 8, wherein The first weight and the second weight are predefined values.

11. An image encoding method performed by an image encoding apparatus, comprising: determining an intra prediction mode for a current block; as well as Based on the determined intra-frame prediction mode, encoding the prediction mode information of the current block, wherein the intra-frame prediction mode is determined based on whether the intra-frame prediction mode is constrained within a specific range, and In which, based on the intra-frame prediction mode being constrained within the specific range, based on the intra-frame prediction mode determined to be one of the intra-frame prediction modes within the specific range, intra-frame prediction of the current block is mainly performed using reference samples within a predetermined range among the available reference samples of the current block.

12. A computer-readable medium storing a bit stream generated by the image encoding method of claim 11.

13. A method for transmitting a bitstream generated by an image encoding method, the image encoding method comprising: determining an intra prediction mode for a current block; as well as Based on the determined intra-frame prediction mode, encoding the prediction mode information of the current block, wherein the intra-frame prediction mode is determined based on whether the intra-frame prediction mode is constrained within a specific range, and In which, based on the intra-frame prediction mode being constrained within the specific range, based on the intra-frame prediction mode determined to be one of the intra-frame prediction modes within the specific range, the intra-frame prediction of the current block is mainly performed using reference samples within a predetermined range among the available reference samples of the current block.

Images