Video encoding / decoding method and apparatus performing pdpc and method of transmitting bitstream

Abstract

A video encoding / decoding method and apparatus performing PDPC and a method of transmitting a bitstream are provided. A video decoding method performed by a video decoding apparatus according to the disclosure can include the steps of generating a prediction block by performing intra prediction on a current block, determining whether to apply PDPC to the prediction block, and generating a final prediction block of the current block by applying PDPC to the prediction block based on the determination. The step of determining whether to apply PDPC to the prediction block can include determining whether a size of the current block satisfies a predetermined condition, determining to apply PDPC to the prediction block based on the size of the current block satisfying the predetermined condition, and skipping the determination of the color component of the current block when the size of the current block does not satisfy the predetermined condition and determining not to apply PDPC to the prediction block.

Description

Technical Field

[0001] This disclosure relates to image encoding / decoding methods and apparatus, and more specifically, to image encoding / decoding methods and apparatus for performing position-dependent intra-frame prediction (PDPC) and methods for transmitting bitstreams generated by the image encoding methods / apparatus of this disclosure. Background Technology

[0002] Recently, the demand for high-resolution and high-quality images (such as high-definition (HD) and ultra-high-definition (UHD) images) is increasing across various fields. With the improvement in image data resolution and quality, the amount of information or bits transmitted increases relatively compared to existing image data. This increase in the amount of information or bits transmitted leads to increased transmission and storage costs.

[0003] Therefore, efficient image compression technology is needed to effectively send, store, and reproduce information about high-resolution and high-quality images. Summary of the Invention

[0004] Technical issues

[0005] The purpose of this disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0006] Another object of this disclosure is to provide an image encoding / decoding method and apparatus that can improve encoding / decoding efficiency by making the PDPC application requirements for chroma blocks and luma blocks the same.

[0007] Another object of this disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or device according to this disclosure.

[0008] Another object of this disclosure is to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0009] Another object of this disclosure is to provide a recording medium that stores a bitstream that is received, decoded, and used to reconstruct an image by an image decoding device according to this disclosure.

[0010] The technical problems solved by this disclosure are not limited to those described above. Other technical problems not described herein will become clear to those skilled in the art through the following description.

[0011] Technical solution

[0012] An image decoding method performed by an image decoding device according to one aspect of this disclosure may include the following steps: generating a prediction block by performing intra-frame prediction on a current block; determining whether to apply position-related intra-frame prediction combination (PDPC) to the prediction block; and generating a final prediction block of the current block by applying PDPC to the prediction block based on the determination. The step of determining whether to apply PDPC to the prediction block may include: determining whether the size of the current block satisfies a predetermined condition. Based on the fact that the size of the current block satisfies the predetermined condition, it may be determined that PDPC will be applied to the prediction block. Based on the fact that the size of the current block does not satisfy the predetermined condition, the determination of the color components of the current block may be skipped, and it may be determined that PDPC will not be applied to the prediction block.

[0013] In the image decoding method according to this disclosure, the predetermined condition is that the size of the current block can be greater than or equal to a predetermined threshold.

[0014] In the image decoding method according to this disclosure, the predetermined condition can be satisfied based on the fact that the width of the current block is greater than or equal to the predetermined threshold and the height of the current block is greater than or equal to the predetermined threshold.

[0015] In the image decoding method according to this disclosure, the predetermined threshold may be 4.

[0016] In the image decoding method according to this disclosure, the step of determining whether to apply PDPC to the prediction block may further include: determining a reference sample line for intra-frame prediction of the current block.

[0017] In the image decoding method according to this disclosure, based on the fact that the reference sample line is a predetermined reference sample line, it can be determined that PDPC will be applied to the prediction block, and based on the fact that the reference sample line is not the predetermined reference sample line, the determination of the color components of the current block can be skipped, and it can be determined that PDPC will not be applied to the prediction block.

[0018] In the image decoding method according to this disclosure, the predetermined reference sample line may be a first reference sample line adjacent to the current block.

[0019] In the image decoding method according to this disclosure, the step of determining whether to apply PDPC to the prediction block may further include: determining whether to apply block difference pulse code modulation (BDPCM) to the current block, and determining the intra-frame prediction mode of the current block.

[0020] An image decoding apparatus according to another aspect of this disclosure may include a memory and at least one processor. The at least one processor may perform the following operations: generate a prediction block by performing intra-frame prediction on a current block; determine whether to apply position-related intra-frame prediction combination (PDPC) to the prediction block; and based on the determination, generate a final prediction block of the current block by applying PDPC to the prediction block. The operation of determining whether to apply PDPC to the prediction block may include: determining whether the size of the current block satisfies a predetermined condition. Based on the fact that the size of the current block satisfies the predetermined condition, it may be determined that PDPC will be applied to the prediction block. Based on the fact that the size of the current block does not satisfy the predetermined condition, the determination of the color components of the current block may be skipped, and it may be determined that PDPC will not be applied to the prediction block.

[0021] An image coding method performed by an image coding device according to another aspect of this disclosure may include the following steps: generating a prediction block by performing intra-frame prediction on a current block; determining whether to apply position-related intra-frame prediction combination (PDPC) to the prediction block; and, based on the determination, generating a final prediction block of the current block by applying PDPC to the prediction block. The step of determining whether to apply PDPC to the prediction block may include: determining whether the size of the current block satisfies a predetermined condition. Based on the fact that the size of the current block satisfies the predetermined condition, it can be determined that PDPC will be applied to the prediction block. Based on the fact that the size of the current block does not satisfy the predetermined condition, the determination of the color components of the current block may be skipped, and it can be determined that PDPC will not be applied to the prediction block.

[0022] In the image encoding method according to this disclosure, the predetermined condition is that the size of the current block can be greater than or equal to a predetermined threshold.

[0023] In the image encoding method according to this disclosure, the predetermined condition can be satisfied based on the fact that the width of the current block is greater than or equal to the predetermined threshold and the height of the current block is greater than or equal to the predetermined threshold.

[0024] In the image encoding method according to this disclosure, the predetermined threshold may be 4.

[0025] In the image coding method according to this disclosure, the step of determining whether to apply PDPC to the prediction block may further include: determining a reference sample line for intra-frame prediction of the current block.

[0026] In the image coding method according to this disclosure, based on the fact that the reference sample line is a predetermined reference sample line, it can be determined that PDPC will be applied to the prediction block, and based on the fact that the reference sample line is not the predetermined reference sample line, it can be skipped to determine the color components of the current block, and it can be determined that PDPC will not be applied to the prediction block.

[0027] In the image encoding method according to this disclosure, the predetermined reference sample line may be a first reference sample line adjacent to the current block.

[0028] In the image coding method according to this disclosure, the step of determining whether to apply PDPC to the prediction block may further include: determining whether to apply block difference pulse coded modulation (BDPCM) to the current block, and determining the intra-frame prediction mode of the current block.

[0029] Furthermore, according to another aspect of the transmission method of this disclosure, a bit stream generated by the image encoding device or method of this disclosure can be transmitted.

[0030] Additionally, according to another aspect of this disclosure, a computer-readable recording medium can store a bitstream generated by the image encoding device or image encoding method of this disclosure.

[0031] The features briefly outlined above are merely exemplary aspects of the following detailed description of this disclosure and do not limit the scope of this disclosure.

[0032] Beneficial effects

[0033] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0034] Furthermore, according to this disclosure, an image encoding / decoding method and apparatus can be provided that can improve encoding / decoding efficiency by simplifying the determination of whether to apply PDPC through the use of a unified PDPC application condition for the luminance and chrominance components in intra-frame predictive coding / decoding.

[0035] Furthermore, according to this disclosure, a method for transmitting a bitstream generated by an image encoding method or device according to this disclosure can be provided.

[0036] Furthermore, according to this disclosure, a recording medium storing a bitstream generated by an image encoding method or apparatus according to this disclosure can be provided.

[0037] Furthermore, according to this disclosure, a recording medium may be provided that stores a bitstream that is received, decoded, and used to reconstruct an image by an image decoding device according to this disclosure.

[0038] Those skilled in the art will understand that the effects achievable through this disclosure are not limited to those specifically described above, and that other advantages of this disclosure will become clearer from the detailed description. Attached Figure Description

[0039] Figure 1 This is a schematic view of a video encoding system according to the present disclosure.

[0040] Figure 2 This is a schematic view illustrating an image encoding device to which embodiments of the present disclosure are applicable.

[0041] Figure 3 This is a schematic view illustrating an image decoding device to which embodiments of the present disclosure are applicable.

[0042] Figure 4 This is a view showing the segmentation structure of an image according to an embodiment.

[0043] Figure 5 This is a view illustrating an implementation of the block segmentation type based on a multi-type tree structure.

[0044] Figure 6 This is a view illustrating the signaling mechanism for block partitioning information in a quadtree with nested multi-type tree structures according to this disclosure.

[0045] Figure 7 This is a view illustrating an implementation of dividing a CTU into multiple CUs.

[0046] Figure 8 This is a view illustrating video / image coding methods based on intra-frame prediction.

[0047] Figure 9 This is a view that illustrates an intra-frame prediction unit in a coding device.

[0048] Figure 10 This is a view illustrating video / image decoding methods based on intra-frame prediction.

[0049] Figure 11 This is a view that illustrates the intra-frame prediction unit in a decoding device.

[0050] Figure 12 This is a view that illustrates the directional intra-prediction mode among intra-prediction modes.

[0051] Figures 13a to 13d This is a view that illustrates a reference sample defined in PDPC.

[0052] Figure 14 This is a view that illustrates the reference sample rows available in the MRL method.

[0053] Figure 15 This is a view illustrating the syntax structure of the coding unit used to signal multiple reference line indices.

[0054] Figure 16 This is a view illustrating PDPC application conditions according to embodiments of the present disclosure.

[0055] Figure 17 This is a view illustrating PDPC application conditions according to another embodiment of the present disclosure.

[0056] Figure 18 This is a view illustrating PDPC application conditions according to another embodiment of the present disclosure.

[0057] Figure 19 This is a view illustrating a method for generating prediction blocks according to another embodiment of the present disclosure.

[0058] Figure 20 This is a view illustrating the content streaming system to which embodiments of this disclosure are applicable. Detailed Implementation

[0059] The embodiments of this disclosure will now be described in detail with reference to the accompanying drawings to facilitate implementation by those skilled in the art. However, this disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0060] In describing this disclosure, detailed descriptions of relevant known functions or constructions will be omitted if they unnecessarily obscure the scope of this disclosure. In the accompanying drawings, portions irrelevant to the description of this disclosure are omitted, and similar reference numerals are assigned to similar portions.

[0061] In this disclosure, when a component is "connected," "linked," or "coupled" to another component, it may include not only a direct connection but also an indirect connection with intermediate components. Furthermore, when a component "comprises" or "has" other components, unless otherwise stated, it means that other components may be included, not excluded.

[0062] In this disclosure, the terms first, second, etc., may be used only for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise stated. Accordingly, within the scope of this disclosure, a first component in one embodiment may be referred to as a second component in another embodiment, and similarly, a second component in one embodiment may be referred to as a first component in another embodiment.

[0063] In this disclosure, the components are distinguished from each other to clearly describe each feature, but this does not mean that the components must be separate. That is, multiple components may be integrated and implemented in a single hardware or software unit, or a single component may be distributed and implemented in multiple hardware or software units. Therefore, unless otherwise stated, these implementations, in which components are integrated or distributed, are included within the scope of this disclosure.

[0064] In this disclosure, the components described in the various embodiments are not necessarily essential components, and some components may be optional. Therefore, embodiments consisting of a subset of the components described in the embodiments are also included within the scope of this disclosure. In addition, embodiments that include other components besides those described in the various embodiments are also included within the scope of this disclosure.

[0065] This disclosure relates to the encoding and decoding of images. Unless redefined in this disclosure, the terms used herein may have the general meaning commonly used in the art to which this disclosure pertains.

[0066] In this disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that constitutes part of a picture. A picture can be composed of one or more slices / tiles. In addition, a slice / tile may include one or more coding tree units (CTUs).

[0067] A "pixel" or "pixel" can refer to the smallest unit that makes up a picture (or image). Additionally, "sample" can be used as the term corresponding to a pixel. A sample can generally represent a pixel or a pixel value, or it can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0068] In this disclosure, "unit" can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. In some cases, the term "unit" may be used interchangeably with terms such as "sample array," "block," or "region." Generally, an M×N block may include a set (or array) of samples (or transform coefficients) with M columns and N rows.

[0069] In this disclosure, "current block" can mean one of "current coding block," "current coding unit," "coding target block," "decoding target block," or "processing target block." When performing prediction, "current block" can mean "current prediction block" or "prediction target block." When performing transform (inverse transform) / quantization (dequantization), "current block" can mean "current transform block" or "transform target block." When performing filtering, "current block" can mean "filter target block."

[0070] Additionally, in this disclosure, unless explicitly stated as a chroma block, "current block" may mean "the luminance block of the current block". "The chroma block of the current block" can be expressed by including an explicit description of a chroma block such as "chroma block" or "current chroma block".

[0071] In this disclosure, the forward slash " / " or "," should be interpreted as indicating "and / or". For example, the expressions "A / B" and "A, B" can mean "A and / or B". Furthermore, "A / B / C" and "A / B / C" can mean "at least one of A, B and / or C".

[0072] In this disclosure, the term "or" should be interpreted as indicating "and / or". For example, expressing "A or B" can include 1) only "A", 2) only "B", and / or 3) both "A and B". In other words, in this disclosure, the term "or" should be interpreted as indicating "additionally or alternatively".

[0073] Overview of Video Encoding Systems

[0074] Figure 1 This is a view showing a video encoding system according to this disclosure.

[0075] The video encoding system according to the embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can deliver encoded video and / or image information or data to the decoding device 20 in the form of a file or stream via a digital storage medium or network.

[0076] The encoding apparatus 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding apparatus 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be referred to as a video / image encoding apparatus, and the decoding unit 22 may be referred to as a video / image decoding apparatus. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0077] The video source generator 11 can acquire video / images through a process of capturing, compositing, or generating video / images. The video source generator 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generation device may include, for example, a computer, tablet computer, and smartphone, and can generate video / images (electronically). For example, virtual video / images can be generated by a computer, etc. In this case, the video / image capture process can be replaced by a process of generating related data.

[0078] The encoding unit 12 can encode the input video / image. For compression and encoding efficiency, the encoding unit 12 can perform a series of processes such as prediction, transformation, and quantization. The encoding unit 12 can output the encoded data (encoded video / image information) in the form of a bitstream.

[0079] The transmitter 13 can transmit encoded video / image information or data, output in bitstream form, to the receiver 21 of the decoding device 20 in the form of a file or stream via a digital storage medium or network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 can include elements for generating media files according to a predetermined file format and may include elements for transmission via a broadcast / communication network. The receiver 21 can extract / receive the bitstream from the storage medium or network and transmit the bitstream to the decoding unit 22.

[0080] The decoding unit 22 can decode video / images by performing a series of processes (such as dequantization, inverse transform, and prediction) corresponding to the operations of the encoding unit 12.

[0081] Renderer 23 can render decoded video / images. The rendered video / images can be displayed on a monitor.

[0082] Overview of Image Encoding Devices

[0083] Figure 2 This is a schematic view illustrating an image encoding device to which embodiments of the present disclosure are applicable.

[0084] like Figure 2As shown, the image encoding apparatus 100 may include an image segmenter 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as "prediction units". The transformer 120, quantizer 130, dequantizer 140, and inverse transformer 150 may be included in a residual processor. The residual processor may also include a subtractor 115.

[0085] In some embodiments, all or at least some of the multiple components configuring the image encoding apparatus 100 may be configured by a single hardware component (e.g., an encoder or a processor). Additionally, the memory 170 may include a decoded screen buffer (DPB) and may be configured by a digital storage medium.

[0086] Image segmenter 110 can segment an input image (or picture or frame) input to image encoding apparatus 100 into one or more processing units. For example, a processing unit may be called an encoding unit (CU). Encoding units can be obtained by recursively segmenting encoding tree units (CTUs) or maximum encoding units (LCUs) according to a quadtree / binary tree / tritree (QT / BT / TT) structure. For example, an encoding unit can be segmented into multiple encoding units of greater depth based on a quadtree structure, a binary tree structure, and / or a ternary tree structure. For the segmentation of encoding units, a quadtree structure can be applied first, and a binary tree structure and / or a ternary tree structure can be applied later. The encoding process according to this disclosure can be performed based on the final encoding unit that is no longer segmented. The maximum encoding unit can be used as the final encoding unit, or a deeper encoding unit obtained by segmenting the maximum encoding unit can be used as the final encoding unit. Here, the encoding process may include the prediction, transformation, and reconstruction processes described later. As another example, the processing unit of the encoding process may be a prediction unit (PU) or a transformation unit (TU). Prediction units and transform units can be partitioned or segmented from the final coding unit. Prediction units can be sample prediction units, and transform units can be units used to derive transform coefficients and / or units used to derive residual signals from transform coefficients.

[0087] The prediction unit (inter-frame prediction unit 180 or intra-frame prediction unit 185) can perform prediction on the block to be processed (the current block) and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The prediction unit can generate various information related to the prediction of the current block and send the generated information to the entropy encoder 190. The information about the prediction can be encoded in the entropy encoder 190 and output as a bitstream.

[0088] Intra-prediction unit 185 can predict the current block by referencing samples in the current frame. Depending on the intra-prediction mode and / or intra-prediction technique, the reference samples may be located among the neighbors of the current block or may be placed separately. Intra-prediction modes may include multiple non-directional modes and multiple directional modes. Non-directional modes may include, for example, DC mode and planar mode. Depending on the level of detail in the prediction direction, directional modes may include, for example, 33 or 65 directional prediction modes. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. Intra-prediction unit 185 can determine the prediction mode to be applied to the current block by using prediction modes applied to neighboring blocks.

[0089] The inter-frame prediction unit 180 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by motion vectors on a reference frame. In this case, to reduce the amount of motion information transmitted in inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between motion information between neighboring blocks and the current block. Motion information may include motion vectors and reference frame indices. Motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. The reference frame including the reference block and the reference frame including the temporally neighboring block may be the same or different. The temporally neighboring block may be referred to as a juxtaposed reference block, a juxtaposed CU (colCU), etc. The reference frame including the temporally neighboring block may be referred to as a juxtaposed frame (colPic). For example, the inter-frame prediction unit 180 can configure a motion information candidate list based on neighboring blocks and generate information indicating which candidate to use to deduce the motion vector and / or reference frame index of the current block. Inter-frame prediction can be performed based on various prediction modes. For example, in skip mode and merge mode, the inter-frame prediction unit 180 can use the motion information of neighboring blocks as the motion information of the current block. In skip mode, unlike merge mode, residual signals may not be transmitted. In motion vector prediction (MVP) mode, the motion vectors of neighboring blocks can be used as motion vector predictors, and the motion vector of the current block can be signaled by encoding motion vector differences and indicators of the motion vector predictors. The motion vector difference can refer to the difference between the motion vector of the current block and the motion vector predictor.

[0090] The prediction unit can generate a prediction signal based on various prediction methods and techniques described below. For example, the prediction unit can apply not only intra-frame prediction or inter-frame prediction, but also both intra-frame prediction and inter-frame prediction simultaneously to predict the current block. A prediction method that simultaneously applies both intra-frame prediction and inter-frame prediction to predict the current block can be called Combined Intra-Frame and Inter-Frame Prediction (CIIP). Additionally, the prediction unit can perform Intra-Frame Block Copy (IBC) to predict the current block. Intra-Frame Block Copy can be used for content image / video coding in games, for example, Screen Content Coding (SCC). IBC is a method of predicting the current frame using a previously reconstructed reference block in the current frame at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current frame can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC essentially performs prediction in the current frame, but can be performed similarly to inter-frame prediction because the reference block is derived within the current frame. That is, IBC can use at least one of the inter-frame prediction techniques described in this disclosure.

[0091] The prediction signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtractor 115 can generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal can be sent to the converter 120.

[0092] Transformer 120 can generate transform coefficients by applying transform techniques to the residual signal. For example, the transform techniques may include at least one of Discrete Cosine Transform (DCT), Discrete Sine Transform (DST), Karhunen-Loève Transform (KLT), Graph-Based Transform (GBT), or Conditional Nonlinear Transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. Furthermore, the transform processing can be applied to square pixel blocks of the same size or to blocks of variable size instead of square.

[0093] Quantizer 130 can quantize the transform coefficients and send them to entropy encoder 190. Entropy encoder 190 can encode the quantized signal (information about the quantized transform coefficients) and output a bitstream. The information about the quantized transform coefficients can be referred to as residual information. Quantizer 130 can rearrange the block-type quantized transform coefficients into a one-dimensional vector form based on the coefficient scan order, and generate information about the quantized transform coefficients based on the one-dimensional vector form of the quantized transform coefficients.

[0094] The entropy encoder 190 can perform various encoding methods, such as exponential Columbus coding, context-adaptive variable-length coding (CAVLC), and context-adaptive binary arithmetic coding (CABAC). The entropy encoder 190 can encode, together or separately, information required for video / image reconstruction other than the quantization transform coefficients (e.g., values ​​of syntax elements). The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream form at the Network Abstraction Layer (NAL) level. The video / image information may also include information about various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). Additionally, the video / image information may include general constraint information. The signaled information, transmitted information, and / or syntax elements described in this disclosure can be encoded and included in the bitstream through the above encoding process.

[0095] The bitstream can be transmitted over a network or stored in a digital storage medium. The network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from the entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external components of the image encoding apparatus 100. Alternatively, a transmitter may be provided as a component of the entropy encoder 190.

[0096] The quantization transform coefficients output from quantizer 130 can be used to generate residual signals. For example, the residual signal (residual block or residual sample) can be reconstructed by applying dequantization and inverse transform to the quantization transform coefficients through dequantizer 140 and inverse transformer 150.

[0097] Adder 155 adds the reconstructed residual signal to the prediction signal output from inter-frame prediction unit 180 or intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed frame, reconstructed block, reconstructed sample array). If the block to be processed has no residual, such as in the case of applying a skip mode, the prediction block can be used as a reconstructed block. Adder 155 can be referred to as a reconstructor or reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0098] Filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, filter 160 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 170, specifically in the DPB of memory 170. Various filtering methods can include, for example, deblocking filtering, sample adaptive shifting, adaptive loop filtering, bilateral filtering, etc. Filter 160 can generate various filtering-related information and send the generated information to entropy encoder 190, as described later in the description of each filtering method. The filtering-related information can be encoded by entropy encoder 190 and output as a bitstream.

[0099] The modified reconstructed frame sent to memory 170 can be used as a reference frame in inter-frame prediction unit 180. When inter-frame prediction is applied by image coding device 100, prediction mismatch between image coding device 100 and image decoding device can be avoided and coding efficiency can be improved.

[0100] The DPB of memory 170 can store modified reconstructed frames for use as reference frames in inter-frame prediction unit 180. Memory 170 can store motion information of blocks used to derive (or encode) motion information in the current frame and / or motion information of already reconstructed blocks in the frame. The stored motion information can be sent to inter-frame prediction unit 180 and used as motion information for spatially or temporally neighboring blocks. Memory 170 can store reconstructed samples of reconstructed blocks in the current frame and can transmit the reconstructed samples to intra-frame prediction unit 185.

[0101] Overview of image decoding devices

[0102] Figure 3 This is a schematic view illustrating an image decoding device to which embodiments of the present disclosure are applicable.

[0103] like Figure 3 As shown, the image decoding device 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, and an intra-frame prediction unit 265. The inter-frame prediction unit 260 and the intra-frame prediction unit 265 may be collectively referred to as "prediction units". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0104] According to an embodiment, all or at least some of the multiple components configuring the image decoding device 200 may be configured by hardware components (e.g., a decoder or a processor). Additionally, the memory 250 may include a decoded screen buffer (DPB) or may be configured by a digital storage medium.

[0105] The image decoding device 200, having received a bitstream including video / image information, can perform operations related to... Figure 2 The image is reconstructed by processing corresponding to the processing performed by the image encoding apparatus 100. For example, the image decoding apparatus 200 can use a processing unit applied in the image encoding apparatus to perform decoding. Therefore, the decoding processing unit can be, for example, an encoding unit. The encoding unit can be obtained by segmenting a coding tree unit or a maximum coding unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 can be reproduced by a reproduction device (not shown).

[0106] Image decoding device 200 can receive data in bitstream form from... Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information may also include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). In addition, the video / image information may also include general constraint information. The image decoding device can also decode the picture based on the information about the parameter sets and / or general constraint information. The information and / or syntax elements notified / received by signals described in this disclosure can be decoded and obtained from the bitstream through a decoding process. For example, the entropy decoder 210 decodes the information in the bitstream based on encoding methods such as exponential Golomb coding, CAVLC, or CABAC, and outputs the values ​​of the syntax elements required for image reconstruction and the quantized values ​​of the transform coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive bins corresponding to each syntax element in the bitstream, determine a context model using information about the target syntax element, decoding information of neighboring blocks and the target block, or information about symbols / bins decoded in the previous stage, perform arithmetic decoding on the bins based on the determined context model by predicting the occurrence probability of the bins, and generate symbols corresponding to the value of each syntax element. In this case, the CABAC entropy decoding method can update the context model after determining the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The prediction-related information in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual values ​​of the entropy decoding performed in the entropy decoder 210 (i.e., quantization transform coefficients and related parameter information) can be input to the dequantizer 220. In addition, the filtering information in the information decoded by the entropy decoder 210 can be provided to the filter 240. Furthermore, the receiver (not shown) for receiving signals output from the image encoding device may be further configured as an internal / external element of the image decoding device 200, or the receiver may be a component of the entropy decoder 210.

[0107] Furthermore, the image decoding apparatus according to this disclosure can be referred to as a video / image / screen decoding apparatus. The image decoding apparatus can be divided into an information decoder (video / image / screen information decoder) and a sample decoder (video / image / screen sample decoder). The information decoder may include an entropy decoder 210. The sample decoder may include at least one of a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter-frame prediction unit 260, or an intra-frame prediction unit 265.

[0108] Dequantizer 220 can dequantize the quantized transform coefficients and output transform coefficients. Dequantizer 220 can rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement can be performed based on the coefficient scan order performed in the image encoding device. Dequantizer 220 can obtain transform coefficients by performing dequantization on the quantized transform coefficients using quantization parameters (e.g., quantization step size information).

[0109] The inverse transformer 230 can perform inverse transformation on the transformation coefficients to obtain the residual signal (residual block, residual sample array).

[0110] The prediction unit can perform prediction on the current block and generate a prediction block that includes prediction samples of the current block. The prediction unit can determine whether to apply intra-frame prediction or inter-frame prediction to the current block based on the prediction information output from the entropy decoder 210, and can determine a specific intra-frame / inter-frame prediction mode (prediction technique).

[0111] Similar to that described in the prediction unit of the image coding apparatus 100, the prediction unit can generate a prediction signal based on various prediction methods (techniques) that will be described later.

[0112] Intra-prediction unit 265 can predict the current block by referring to samples in the current frame. The description of intra-prediction unit 185 also applies to intra-prediction unit 265.

[0113] The inter-frame prediction unit 260 can deduce the prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference frame. In this case, to reduce the amount of motion information transmitted in the inter-frame prediction mode, motion information can be predicted on a block, sub-block, or sample basis based on the correlation between the motion information of neighboring blocks and the current block. The motion information may include motion vectors and reference frame indices. The motion information may also include inter-frame prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter-frame prediction, neighboring blocks may include spatially neighboring blocks existing in the current frame and temporally neighboring blocks existing in the reference frame. For example, the inter-frame prediction unit 260 can configure a motion information candidate list based on neighboring blocks and deduce the motion vector and / or reference frame index of the current block based on the received candidate selection information. Inter-frame prediction can be performed based on various prediction modes, and the information about the prediction may include information indicating the inter-frame prediction mode of the current block.

[0114] Adder 235 generates a reconstruction signal (reconstructed frame, reconstruction block, reconstruction sample array) by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including inter-frame prediction unit 260 and / or intra-frame prediction unit 265). If the block to be processed has no residual, such as when a skip mode is applied, the prediction block can be used as a reconstruction block. The description of adder 155 also applies to adder 235. Adder 235 may be referred to as a reconstructor or reconstruction block generator. The generated reconstruction signal can be used for intra-frame prediction of the next block to be processed in the current frame, and can be used for inter-frame prediction of the next frame by filtering as described below.

[0115] Filter 240 can improve the quality of subjective / objective images by applying filtering to the reconstructed signal. For example, filter 240 can generate a modified reconstructed image by applying various filtering methods to the reconstructed image and store the modified reconstructed image in memory 250, specifically in the DPB of memory 250. Various filtering methods may include, for example, deblocking filtering, adaptive sample shifting, adaptive loop filtering, bilateral filtering, etc.

[0116] The (modified) reconstructed frame stored in the DPB of memory 250 can be used as a reference frame in inter-frame prediction unit 260. Memory 250 can store motion information of blocks used to derive (or decode) motion information in the current frame and / or motion information of already reconstructed blocks in the frame. The stored motion information can be sent to inter-frame prediction unit 260 to be used as motion information of spatially neighboring blocks or temporally neighboring blocks. Memory 250 can store reconstructed samples of reconstructed blocks in the current frame and transmit the reconstructed samples to intra-frame prediction unit 265.

[0117] In this disclosure, the embodiments described in the filter 160, inter-frame prediction unit 180 and intra-frame prediction unit 185 of the image encoding apparatus 100 can be applied equally or correspondingly to the filter 240, inter-frame prediction unit 260 and intra-frame prediction unit 265 of the image decoding apparatus 200.

[0118] Overview of Image Segmentation

[0119] The video / image coding method according to this disclosure can be performed based on the following image segmentation structure. Specifically, the processes of prediction, residual processing (inverse transform, dequantization, etc.), syntax element encoding, and filtering, which will be described later, can be performed based on the CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented into block units, and the block segmentation process can be performed in the image segmenter 110 of the encoding device. Segmentation-related information can be encoded by the entropy encoder 190 and sent to the decoding device in the form of a bitstream. The entropy decoder 210 of the decoding device can deduce the block segmentation structure of the current frame based on the segmentation-related information obtained from the bitstream, and based on this, a series of processes (e.g., prediction, residual processing, block / frame reconstruction, in-loop filtering, etc.) can be performed for image decoding.

[0120] The image can be segmented into a sequence of Code Tree Units (CTUs). Figure 4 An example of a screen being segmented into CTUs is shown. A CTU may correspond to a Coding Tree Block (CTB). Alternatively, a CTU may include a coded tree block for luma samples and two coded tree blocks for corresponding chroma samples. For example, for a screen containing three sample arrays, a CTU may include an N×N block of luma samples and two corresponding blocks of chroma samples.

[0121] Overview of CTU segmentation

[0122] As described above, coding units can be obtained by recursively partitioning coding tree units (CTUs) or maximum coding units (LCUs) according to a quadtree / binary tree / ternary tree (QT / BT / TT) structure. For example, a CTU can be first partitioned into a quadtree structure. Subsequently, the leaf nodes of the quadtree structure can be further partitioned using multiple tree types.

[0123] The quadtree partitioning means that the current CU (or CTU) is equally divided into four. By partitioning according to the quadtree, the current CU can be divided into four CUs with the same width and height. When the current CU is no longer partitioned into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CUs corresponding to the leaf nodes of the quadtree structure may not be further partitioned and can be used as the final encoding units described above. Alternatively, the CUs corresponding to the leaf nodes of the quadtree structure can be further partitioned using multiple types of tree structures.

[0124] Figure 5 This is a view illustrating an implementation of block segmentation types based on multiple tree structures. Segmentation based on multiple tree structures can include two types of partitioning based on binary tree structures and two types of partitioning based on ternary tree structures.

[0125] The two types of partitioning based on the binary tree structure can be vertical binary partitioning (SPLIT_BT_VER) and horizontal binary partitioning (SPLIT_BT_HOR). Vertical binary partitioning (SPLIT_BT_VER) means that the current CU is equally divided into two in the vertical direction. For example... Figure 4 As shown, a vertical binary partition can generate two CUs with the same height as the current CU and a width half the width of the current CU. A horizontal binary partition (SPLIT_BT_HOR) means that the current CU is equally divided into two in the horizontal direction. Figure 5 As shown, by using horizontal binary partitioning, two CUs can be generated with a height that is half the height of the current CU and a width that is the same as the current CU.

[0126] The two types of partitioning based on the ternary tree structure can include vertical ternary partitioning (SPLIT_TT_VER) and horizontal ternary partitioning (SPLIT_TT_HOR). In vertical ternary partitioning (SPLIT_TT_VER), the current CU is partitioned vertically in a 1:2:1 ratio. For example... Figure 5 As shown, a vertical truncation can generate two CUs with the same height as the current CU and a width one-quarter of the current CU's width, and one CU with the same height as the current CU and a width half of the current CU's width. In a horizontal truncation (SPLIT_TT_HOR), the current CU is divided horizontally in a 1:2:1 ratio. Figure 5 As shown, by dividing horizontally into three branches, two CUs with a height of 1 / 4 of the current CU's height and the same width as the current CU, and one CU with a height of half the current CU's height and the same width as the current CU, can be generated.

[0127] Figure 6 This is a view illustrating the signaling mechanism for block partitioning information in a quadtree with nested multi-type tree structures according to this disclosure.

[0128] Here, the CTU is considered the root node of the quadtree and is initially split into a quadtree structure. Information indicating whether to perform a quadtree split on the current CU (the CTU or node (QT_node) of the quadtree) can be signaled (e.g., qt_split_flag). For example, when qt_split_flag has a first value (e.g., "1"), the current CU can be split into a quadtree. Alternatively, when qt_split_flag has a second value (e.g., "0"), the current CU is not split into a quadtree but becomes a leaf node (QT_leaf_node). Each quadtree leaf node can then be further split into a multi-type tree structure. That is, a leaf node of the quadtree can become a node of a multi-type tree (MTT_node). In the multi-type tree structure, a first flag (e.g., Mtt_split_cu_flag) can be signaled to indicate whether the current node is additionally split. If the corresponding node is additionally split (e.g., if the first flag is 1), a second flag (e.g., `Mtt_split_cu_vertical_flag`) can be signaled to indicate the split direction. For example, the split direction could be vertical when the second flag is 1, and horizontal when the second flag is 0. Then, a third flag (e.g., `Mtt_split_cu_binary_flag`) can be signaled to indicate whether the split type is binary or ternary. For example, the split type could be binary when the third flag is 1, and ternary when the third flag is 0. Nodes of the multi-type tree obtained through binary or ternary splits can be further split into multi-type tree structures. However, the nodes of the multi-type tree may not be split into quadtree structures. If the first flag is 0, the corresponding node of the multi-type tree is no longer split, but becomes a leaf node (`MTT_leaf_node`) of the multi-type tree. The CU corresponding to the leaf node of the multi-type tree can be used as the final encoding unit described above.

[0129] Based on `mtt_split_cu_vertical_flag` and `mtt_split_cu_binary_flag`, the multi-type tree partitioning mode (MttSplitMode) of the CU can be derived as shown in Table 1 below. In the following description, the multi-type tree partitioning mode may be referred to as the multi-tree partitioning type or partitioning type.

[0130] [Table 1]

[0131] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0132] Figure 7 This is a view illustrating an example of partitioning a CTU into multiple CUs by applying a multi-type tree after applying a quadtree. Figure 7 In the diagram, bolded edge 710 represents a quadtree partition, while remaining edge 720 represents a multi-type tree partition. A CU can correspond to a coded block (CB). In an implementation, a CU may include a coded block for luminance samples and two coded blocks for chrominance samples corresponding to the luminance samples.

[0133] The size of the chroma component (sample) CB or TB can be derived based on the component ratio of the color format (chroma format, such as 4:4:4, 4:2:2, 4:2:0, etc.) of the image / screen. In the case of a 4:4:4 color format, the chroma component CB / TB size can be set to be equal to the luminance component CB / TB size. In the case of a 4:2:2 color format, the width of the chroma component CB / TB can be set to half the width of the luminance component CB / TB, and the height of the chroma component CB / TB can be set to the height of the luminance component CB / TB. In the case of a 4:2:0 color format, the width of the chroma component CB / TB can be set to half the width of the luminance component CB / TB, and the height of the chroma component CB / TB can be set to half the height of the luminance component CB / TB.

[0134] In one implementation, when the size of the CTU is based on 128 luminance sample units, the size of the CU can be from 128×128 to 4×4, which is the same size as the CTU. In one implementation, in the case of a 4:2:0 color format (or chroma format), the chroma CB size can be from 64×64 to 2×2.

[0135] Furthermore, in implementations, the CU size and TU size can be the same. Alternatively, there can be multiple TUs in the CU region. The TU size can typically represent the size of the luminance component (sample) transform block (TB).

[0136] The TU size can be derived based on the maximum permissible TB size, maxTbSize, which is a predetermined value. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize can be derived from the CU, and transformations / inverse transformations can be performed on TUs (TBs) as units. For example, the maximum permissible luminance TB size can be 64×64 and the maximum permissible chrominance TB size can be 32×32. If the width or height of the CB segmented according to the tree structure is greater than the maximum transformation width or height, the CB can be automatically (or implicitly) segmented until the TB size limits in the horizontal and vertical directions are met.

[0137] Additionally, for example, when applying intra-frame prediction, the intra-frame prediction mode / type can be derived at the CU (or CB) level, and the neighbor reference sample derivation and prediction sample generation process can be performed at the TU (or TB) level. In this case, there can be one or more TUs (or TBs) in a CU (or CB) region, and multiple TUs or (TBs) can share the same intra-frame prediction mode / type.

[0138] Furthermore, for quadtree coding schemes with nested multi-type trees, the following parameters can be signaled from the encoding device to the decoding device as SPS syntax elements. For example, at least one of the following can be signaled: CTU size (representing the size of the root node of the quadtree), MinQTSize (representing the minimum allowed size of the leaf node of the quadtree), MaxBtSize (representing the maximum allowed size of the root node of the binary tree), MaxTtSize (representing the maximum allowed size of the root node of the ternary tree), MaxMttDepth (representing the maximum allowed depth of the multi-type tree partitioning starting from the leaf node of the quadtree), MinBtSize (representing the minimum allowed size of the leaf node of the binary tree), or MinTtSize (representing the minimum allowed size of the leaf node of the ternary tree).

[0139] As an implementation using the 4:2:0 chroma format, the CTU size can be set to 128×128 luma blocks and two corresponding 64×64 chroma blocks. In this case, MinOTSize can be set to 16×16, MaxBtSize to 128×128, MaxTtSzie to 64×64, MinBtSize and MinTtSize to 4×4, and MaxMttDepth to 4. Quadtree partitioning can be applied to the CTU to generate quadtree leaf nodes. Quadtree leaf nodes can be called leaf QT nodes. The size of quadtree leaf nodes can range from 16×16 (e.g., MinOTSize) to 128×128 (e.g., CTU size). If a leaf QT node is 128×128, it can be partitioned into a binary / ternary tree without additional partitioning. This is because, in this case, even if partitioned, it would exceed MaxBtsize and MaxTtszie (e.g., 64×64). In other cases, leaf QT nodes can be further segmented into multi-type trees. Therefore, a leaf QT node is the root node of the multi-type tree, and a leaf QT node can have a multi-type tree depth (mttDepth) of 0. If the multi-type tree depth reaches MaxMttdepth (e.g., 4), further segmentation is not considered. If the width of a multi-type tree node is equal to MinBtSize and less than or equal to 2×MinTtSize, further horizontal segmentation is not considered. If the height of a multi-type tree node is equal to MinBtSize and less than or equal to 2×MinTtSize, further vertical segmentation is not considered. When segmentation is not considered, the encoding device can skip the signaling of segmentation information. In this case, the decoding device can deduce segmentation information with predetermined values.

[0140] Furthermore, a CTU can include a coded block of luma samples (hereinafter referred to as a "luma block") and two coded blocks of corresponding chroma samples (hereinafter referred to as "chroma blocks"). The above coding tree scheme can be applied equally or separately to the luma and chroma blocks of the current CU. Specifically, the luma and chroma blocks in a CTU can be partitioned into the same block tree structure, and in this case, the tree structure can be represented as SINGLE_TREE. Alternatively, the luma and chroma blocks in a CTU can be partitioned into separate block tree structures, and in this case, the tree structure can be represented as DUAL_TREE. That is, when the CTU is partitioned into a dual-tree structure, the block tree structure for the luma blocks and the block tree structure for the chroma blocks can exist separately. In this case, the block tree structure for the luma blocks can be called DUAL_TREE_LUMA, and the block tree structure for the chroma components can be called DUAL_TREE_CHROMA. For P and B slice / piece groups, the luma and chroma blocks in a CTU can be restricted to having the same coding tree structure. However, for I-slice / patch groups, luma blocks and chroma blocks can have separate block tree structures. If separate block tree structures are applied, the luma CTB can be segmented into CUs based on a specific coding tree structure, and the chroma CTB can be segmented into chroma CUs based on another coding tree structure. That is, this means that CUs in I-slice / patch groups with separate block tree structures can include either a coded block of the luma component or coded blocks of the two chroma components, and CUs in P or B-slice / patch groups can include blocks of three color components (one luma component and two chroma components).

[0141] Although quadtree coding tree structures with nested multi-type trees have been described, the structures for splitting CUs are not limited to this. For example, BT and TT structures can be interpreted as concepts included in a multi-split tree (MPT) structure, and CUs can be interpreted as being split via QT and MPT structures. In the example of splitting CUs via QT and MPT structures, the splitting structure can be determined by signaling syntax elements (e.g., MPT_split_type) that include information about how many blocks the leaf nodes of the QT structure are split into, and syntax elements (e.g., MPT_split_mode) that include information about which direction (vertical or horizontal) the leaf nodes of the QT structure are split into.

[0142] In another example, the CU can be segmented in a manner different from the QT, BT, or TT structures. That is, unlike the QT structure which segments a lower-depth CU into 1 / 4 of a higher-depth CU, the BT structure which segments a lower-depth CU into 1 / 2 of a higher-depth CU, or the TT structure which segments a lower-depth CU into 1 / 4 or 1 / 2 of a higher-depth CU, in some cases the lower-depth CU can be segmented into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of the higher-depth CU, and the method of segmenting the CU is not limited to these.

[0143] Quadtree coded block structures with multiple tree types can provide highly flexible block partitioning structures. Due to the partitioning types supported in the multiple tree types, different partitioning patterns can potentially produce the same coded block structure in some cases. In encoding and decoding devices, the amount of data for partitioning information can be reduced by limiting the occurrence of such redundant partitioning patterns.

[0144] The intra-prediction performed by the intra-predictor will be described below.

[0145] Intra-frame prediction

[0146] Intra-frame prediction can be represented as a prediction used to generate a prediction sample for the current block based on reference samples in the frame to which the current block belongs (hereinafter referred to as the current frame). When intra-frame prediction is applied to the current block, neighboring reference samples to be used for intra-frame prediction of the current block can be derived. The neighboring reference samples of the current block may include: a total of 2×nh samples, including a sample of size nW×nH adjacent to the left boundary of the current block and a sample adjacent to the lower left; a total of 2×nW samples, including a sample adjacent to the upper boundary of the current block and a sample adjacent to the upper right; and a sample adjacent to the upper left of the current block. Alternatively, the neighboring reference samples of the current block may include multiple columns of upper neighboring samples and multiple rows of left neighboring samples. In addition, the neighboring reference samples of the current block may include: a total of nH samples of size nW×nH adjacent to the right boundary of the current block; a total of nW samples adjacent to the lower boundary of the current block; and a sample adjacent to the lower right of the current block.

[0147] However, some neighboring reference samples of the current block may not have been decoded or may be unavailable. In this case, the decoding device can construct neighboring reference samples to be used for prediction by replacing unavailable samples with available samples. Alternatively, neighboring reference samples to be used for prediction can be constructed by interpolation of available samples.

[0148] In the following text, we will use Figure 8 and Figure 9 This describes an intra-prediction-based coding method and an intra-prediction unit in a coding device.

[0149] Figure 8 This is a view illustrating video / image coding methods based on intra-frame prediction. Additionally, Figure 9 This is a view that illustrates an intra-frame prediction unit in a coding device.

[0150] S800 can be executed by the intra-frame prediction unit 222 of the encoding device, and S810 can be executed by the residual processor of the encoding device. Specifically, S820 can be executed by the subtractor 231 of the encoding device. In S820, the prediction information can be derived by the intra-frame prediction unit 222 and encoded by the entropy encoder. In S820, the residual information can be derived by the residual processor and encoded by the entropy encoder. The residual information is information about the residual samples. The residual information may include information about the quantization transform coefficients of the residual samples. As described above, the residual samples can be derived into transform coefficients by the transformer 232 of the encoding device, and the transform coefficients can be derived into quantized transform coefficients by the quantizer 233. The information about the quantized transform coefficients can be encoded by the entropy encoder through the residual encoding process.

[0151] The encoding device can perform intra-prediction for the current block (S800). The encoding device can derive the intra-prediction mode / type of the current block, derive the neighboring reference samples of the current block, and generate prediction samples in the current block based on the intra-prediction mode / type and the neighboring reference samples. Here, the processes for determining the intra-prediction mode / type, deriving the neighboring reference samples, and generating the prediction samples can be performed simultaneously, or any one process can be performed before the other. For example, the intra-prediction unit 222 of the encoding device may include an intra-prediction mode / type determination unit 222-1, a reference sample derivation unit 222-2, and a prediction sample derivation unit 222-3. The intra-prediction mode / type determination unit 222-1 can determine the intra-prediction mode / type of the current block, the reference sample derivation unit 222-2 can derive the neighboring reference samples of the current block, and the prediction sample derivation unit 222-3 can derive the prediction samples of the current block. Furthermore, although not shown, the intra-prediction unit 222 may also include a prediction sample filter (not shown) when the following prediction sample filtering process is performed. The coding device can determine the mode / type to apply to the current block from multiple intra prediction modes / types. The coding device can compare the RD costs of intra prediction modes / types and determine the optimal intra prediction mode / type for the current block.

[0152] The encoding device can generate residual samples for the current block based on the predicted samples (S810). The encoding device can derive residual samples by comparing the predicted samples with the original samples of the current block based on the phase.

[0153] The encoding device can encode image information including information about intra-frame prediction (prediction information) and residual information of residual samples (S820). The prediction information may include intra-frame prediction mode information and intra-frame prediction type information. The encoding device can output the encoded image information in the form of a bitstream. The output bitstream can be sent to the decoding device via a storage medium or a network.

[0154] The residual information may include the following residual coding syntax. The coding device can transform / quantize the residual samples to derive quantization transform coefficients. The residual information may include information about the quantization transform coefficients.

[0155] Furthermore, as described above, the encoding device can generate a reconstructed frame (including reconstructed samples and reconstructed blocks). To this end, the encoding device can perform dequantization / inverse transform again on the quantization transform coefficients to derive (modified) residual samples. The residual samples are transformed / quantized and then dequantized / inverse transformed to derive the same residual samples as those derived in the decoding device as described above. The encoding device can generate a reconstructed block including reconstructed samples of the current block based on the predicted samples and the (modified) residual samples. A reconstructed frame of the current frame can be generated based on the reconstructed blocks. As described above, the in-loop filtering process is further applied to the reconstructed frame.

[0156] In the following text, we will use Figure 10 and Figure 11 This paper describes a video / image decoding method based on intra-frame prediction and an intra-frame prediction unit in a decoding device.

[0157] Figure 10 This is a view illustrating video / image decoding methods based on intra-frame prediction. Additionally, Figure 11 This is a view that illustrates the intra-frame prediction unit in a decoding device.

[0158] The decoding device can perform operations corresponding to those performed in the encoding device.

[0159] S1000 to S1020 can be executed by the intra-frame prediction unit 331 of the decoding device, and the prediction information of S1000 and the residual information of S1030 can be obtained from the bitstream by the entropy decoder 310 of the decoding device. The residual processor 320 of the decoding device can derive the residual samples of the current block based on the residual information. Specifically, the dequantizer 321 of the residual processor 320 can perform dequantization based on the quantization transform coefficients derived from the residual information to derive the transform coefficients, and the dequantizer 322 of the residual processor can perform inverse transform on the transform coefficients to derive the residual samples of the current block. S1040 can be executed by the adder 340 or the reconstructor of the decoding device.

[0160] Specifically, the decoding device can deduce the intra-prediction mode / type of the current block based on the received prediction information (intra-prediction mode / type information) (S1000). The decoding device can deduce the neighboring reference samples of the current block (S1010). The decoding device generates prediction samples in the current block based on the intra-prediction mode / type and the neighboring reference samples (S1020).

[0161] The decoding device generates residual samples for the current block based on the received residual information. The decoding device can generate reconstructed samples for the current block based on the predicted samples and residual samples, and derive a reconstructed block including the reconstructed samples (S1030). A reconstructed image of the current image can be generated based on the reconstructed block. As described above, the in-loop filtering process is further applied to the reconstructed image.

[0162] Here, the intra-frame prediction unit 331 of the decoding device may include an intra-frame prediction mode / type determination unit 331-1, a reference sample derivation unit 331-2, and a prediction sample derivation unit 331-3. The intra-frame prediction mode / type determination unit 331-1 can determine the intra-frame prediction mode / type of the current block based on the intra-frame prediction mode / type information obtained by the entropy decoder 210. The reference sample derivation unit 331-2 can derive the neighboring reference samples of the current block, and the prediction sample derivation unit 331-3 can derive the prediction samples of the current block. Furthermore, although not shown, when performing the above-described prediction sample filtering process, the intra-frame prediction unit 331 may also include a prediction sample filter (not shown).

[0163] Intra-luma_mpm_flag may include flags specifying whether the most probable mode (MPM) or a remaining mode is applied to the current block. When the MPM is applied to the current block, the prediction mode information may also include an index (e.g., intra_luma_mpm_idx) specifying one of the intra-luma_mpm_candidate prediction modes (MPM candidates). These MPM candidates can be configured as a list of MPM candidates or a list of MPMs. Additionally, when the MPM is not applied to the current block, the intra-luma_mpm_remainder may include remaining mode information specifying one of the remaining intra-luma_mpm_remainder prediction modes besides the MPM candidates. The decoding device can determine the intra-luma_mpm_remainder prediction mode for the current block based on the intra-luma_mpm_remainder prediction mode information.

[0164] Furthermore, intra-prediction type information can be implemented in various forms. For example, intra-prediction type information may include intra-prediction type index information specifying one of the intra-prediction types. As another example, intra-prediction type information may include reference sample line information (e.g., intra_luma_ref_idx) specifying whether MRL is applied to the current block and, if so, which reference sample line to use; ISP flag information specifying whether ISP is applied to the current block (e.g., intra_subpartitions_mode_flag); ISP type information specifying the partitioning type of the subpartitions when ISP is applied (e.g., intra_subpartitions_split_flag); flag information specifying whether PDCP is applied; or flag information specifying whether LIP is applied. Additionally, intra-prediction type information may include a MIP flag specifying whether MIP is applied to the current block.

[0165] Intra-prediction mode information and / or intra-prediction type information can be encoded / decoded using the encoding methods described in this disclosure. For example, intra-prediction mode information and / or intra-prediction type information can be encoded / decoded based on truncated (Rice) binary code using entropy coding (e.g., CABAC or CAVLC).

[0166] In addition, intra-prediction modes can include two directional intra-prediction modes and 65 directional prediction modes. Non-directional intra-prediction modes can include planar intra-prediction modes and DC intra-prediction modes, while directional intra-prediction modes can include intra-prediction modes #2 to #66. Examples of directional intra-prediction modes are provided in... Figure 12 As shown in the image.

[0167] In addition to the intra-prediction modes mentioned above, intra-prediction modes may also include the Cross-Component Linear Model (CCLM) mode for chroma samples. CCLM modes can be classified as LT_CCLM, L_CCLM, and T_CCLM depending on whether the left sample, the top sample, or both are considered for LM parameter derivation, and can be applied only to the chroma component.

[0168] The above intra-frame prediction modes can be indexed, for example, as shown in Table 2 below.

[0169] [Table 2]

[0170] Intra-prediction mode Associated Name 0 INTRA_PLANAR 1 INTRA_DC 2..66 INTRA_ANGULAR2..INTRA_ANGULAR66 81..83 INTRA_LT_CCLM, INTRA_L_CCLM, INTRA_T_CCLM

[0171] The prediction unit of the encoding / decoding device can deduce reference samples from the neighboring reference samples of the current block according to the intra-prediction mode of the current block, and generate prediction samples for the current block based on the reference samples.

[0172] For example, a predicted sample can be derived based on the average or interpolation of neighboring reference samples of the current block, and (ii) a predicted sample can be derived based on a reference sample in a specific (prediction) direction of the predicted sample among the neighboring reference samples of the current block. Case (i) can be referred to as non-directional or non-angular mode, and case (ii) can be referred to as directional or angular mode. Alternatively, a predicted sample can be generated by interpolating a first neighboring sample and a second neighboring sample among the neighboring reference samples, based on the predicted sample of the current block located in a direction opposite to the prediction direction of the intra-prediction mode of the current block. This can be referred to as linear interpolation intra-prediction (LIP). Alternatively, a temporary predicted sample of the current block can be derived based on filtered neighboring reference samples, and a weighted sum of existing neighboring reference samples (i.e., at least one reference sample derived according to the intra-prediction mode among unfiltered neighboring reference samples) and the temporary predicted sample can be derived. This can be referred to as position-dependent intra-prediction (PDPC). Alternatively, the reference sample row with the highest prediction accuracy can be selected from multiple neighboring reference sample rows of the current block, and the prediction sample can be derived using reference samples located in the prediction direction on the corresponding row. In this case, intra-frame prediction coding can be performed by indicating (using a signal) the reference sample row used to the decoding device. This can be referred to as multi-reference-line (MRL) intra-frame prediction or MRL-based intra-frame prediction. Furthermore, intra-frame prediction can be performed based on the same intra-frame prediction mode by dividing the current block into vertical or horizontal sub-partitions, and neighboring reference samples can be derived and used on a sub-partition basis. That is, in this case, the intra-frame prediction mode of the current block is also applied to the sub-partitions, and in some cases, intra-frame prediction performance can be improved by deriving and using neighboring reference samples on a sub-partition basis. Details will be described later. Additionally, when the prediction direction based on the prediction sample indicates the point between neighboring reference samples, i.e., when the prediction direction indicates the fractional sample position, the value of the prediction sample can be derived by interpolation of multiple reference samples located around the corresponding prediction direction (around the corresponding fractional sample position).

[0173] The intra-prediction methods described above can be referred to as intra-prediction types to distinguish them from intra-prediction modes. Intra-prediction types can be referred to by various terms, such as intra-prediction techniques or additional intra-prediction modes. For example, an intra-prediction type (or additional intra-prediction mode, etc.) can include at least one of LIP, PDPC, MRL, and ISP. Information about the intra-prediction type can be encoded in the encoding device and included in the bitstream, and signaled to the decoding device. Information about the intra-prediction type can be implemented in various forms, such as flag information specifying whether each intra-prediction type is applied or index information specifying one of multiple intra-prediction types.

[0174] In the following text, the PDPC among the above-described intra-prediction types that is relevant to this disclosure will be described.

[0175] Location-dependent intra-frame prediction (PDPC)

[0176] PDPC can represent an intra-prediction method for deriving predicted samples of the current block by: performing filtering based on a filter used for PDPC to derive filtered reference samples; deriving temporary predicted samples of the current block based on the intra-prediction mode of the current block and the filtered reference samples; and weighted summing of at least one reference sample derived from the intra-prediction mode and the temporary predicted samples among existing reference samples (i.e., unfiltered reference samples). Here, the predefined filter can be one of five 7-tap filters. Alternatively, the predefined filter can be one of a 3-tap filter, a 5-tap filter, and a 7-tap filter. The 3-tap filter, 5-tap filter, and 7-tap filter can represent a filter with three filter coefficients, a filter with five filter coefficients, and a filter with seven filter coefficients.

[0177] For example, the prediction results of intra-plane modes can be further modified using PDPC.

[0178] Alternatively, for example, PDPC can be applied to intra-plane mode, intra-DC mode, horizontal intra-prediction mode, vertical intra-prediction mode, intra-prediction mode in the lower left direction (i.e., intra-prediction mode #2), eight directional intra-prediction modes adjacent to the intra-prediction mode in the lower left direction, intra-prediction mode in the upper right direction, and eight directional intra-prediction modes adjacent to the intra-prediction mode in the upper right direction, without separate signaling.

[0179] Specifically, when applying PDPC, the predicted sample of (x,y) coordinates based on the linear combination of reference samples and intra-frame prediction modes can be derived as shown in Equation 1 below.

[0180] [Formula 1]

[0181] pred(x,y)=(wL×R (-1,y) +wT×R (x,-1) -wTL×R (-1,-1) +(64-wL-wT+wTL)×pred(x,y)+32)>>6

[0182] In Equation 1 above, pred(x,y) on the left specifies the predicted sample value for the (x,y) coordinates, and pred(x,y) on the right specifies the provisional (primary) predicted sample value for the (x,y) coordinates. R (x,-1) and R (-1,y)Specify the upper reference sample and left reference sample located above and to the left of the current sample at coordinates (x, y), respectively, and R (-1,-1) Specifies the top-left reference sample located at the top-left corner of the current block. Additionally, wL specifies the weights applied to the left reference sample, wT specifies the weights applied to the top reference sample, and wTL specifies the weights applied to the top-left reference sample.

[0183] Furthermore, when PDPC is applied to intra-plane mode, intra-DC mode, horizontal intra-prediction mode, and vertical intra-prediction mode, additional boundary filters such as conventional DC mode boundary filters or vertical / horizontal mode edge filters are not required.

[0184] Temporary (primary) prediction samples can be generated as the result of intra-prediction performed based on the intra-prediction mode and reference samples of the current block. When the PDPC is applied to the current block, for example, based on Equation 1 above, the final prediction sample for the current block can be generated. When the PDPC is not applied to the current block, the temporary (primary) prediction sample can be used as the final prediction sample for the current block.

[0185] Figures 13a to 13d This is a view that illustrates a reference sample defined in PDPC.

[0186] exist Figures 13a to 13d In the example, pred(x,y) specifies the predicted sample obtained through intra-frame prediction (the aforementioned temporary predicted sample), and R (x,-1) and R (-1,y) Specify the upper reference sample and the left reference sample located above and to the left of the current sample at coordinates (x,y).

[0187] Figure 13a The reference sample (R) is shown when the predicted pattern is the top-right diagonal pattern. x,-1 ,R -1,y ,R -1,-1 ). Figure 13b The reference sample (R) is shown when the predicted pattern is a diagonal bottom-left pattern. x,-1 ,R -1,y ,R -1,-1 ). Figure 13c The reference sample (R) is shown when the predicted pattern is the adjacent diagonal top-right pattern. x,-1 ,R -1,y ,R -1,-1 ). Figure 13d The reference sample (R) is shown when the predicted pattern is the adjacent diagonal bottom left pattern. x,-1 ,R -1,y ,R -1,-1 The weights of PDPC can be derived based on the prediction model. The weights wT, wL, and wTL of PDPC can be derived as shown in Table 3 below.

[0188] [Table 3]

[0189] Prediction methods wT wL wTL Diagonal top right mode 16>>((y<<1)>>shift) 16>>((x<<1)>>shift) 0 diagonal bottom left mode 16>>((y<<1)>>shift) 16>>((x<<1)>>shift) 0 Adjacent diagonal top right pattern 32>>((y<<1)>>shift) 0 0 Adjacent diagonal bottom left pattern 0 32>>((x<<1)>>shift) 0

[0190] Position-dependent intra-prediction combination (PDPC) refines prediction samples using neighboring reference samples after generating them based on the prediction mode using reference samples. Based on 65 directional intra-prediction modes, PDPC can be applied restrictively to Plane, DC,2 (bottom-right directional mode), VDIA (top-left directional mode), Hor (horizontal directional mode), Ver (vertical directional mode), neighboring modes of mode 2 (modes #3 to #10), and neighboring modes of VDIA (modes #58 to #65), rather than to all intra-prediction modes. Furthermore, instead of applying all prediction samples to the current block to be encoded, it can be applied variably, taking into account the block size.

[0191] Multi-reference line (MRL) intra-frame prediction

[0192] In traditional intra-frame prediction, only the neighboring samples of the first upper reference sample row and the neighboring samples of the first left reference sample row of the current block are used as reference samples for intra-frame prediction. However, in the Multiple Reference Row (MRL) method, neighboring samples located on the reference sample rows, which are separated from the upper and / or left sides of the current block by one or three sample distances, can be used to perform intra-frame prediction.

[0193] Figure 14 This is a view that illustrates the reference sample rows available in the MRL method.

[0194] like Figure 14 As shown, for intra-frame prediction of the current block, at least one reference sample line from reference line 0 to reference line 3 in the order adjacent to the current block can be referenced. In this disclosure, reference line 0 may be referred to as the first reference sample line. Additionally, reference lines 1 to 3 may be referred to as the second to fourth reference sample lines, respectively.

[0195] When applying MRL, you can use a signal to specify which reference sample line is used for the multi-reference line index of intra-frame prediction for the current block (e.g., mrl_idx).

[0196] Figure 15 This is a view illustrating the syntax structure of the coding unit used to signal multiple reference line indices.

[0197] exist Figure 15 In the example shown, the multi-reference row index can be signaled in the form of `intra_luma_ref_idx`. When the value of the multi-reference index is greater than 0, it can be said that the MRL is applied to the target block.

[0198] Figure 15 The `intra_luma_ref_idx` parameter can be used to specify the reference sample line index `IntraLumaRefLineIdx[x0][y0]` for intra-frame prediction of the current coding unit at coordinates (x0, y0). If `intra_luma_ref_idx[x0][y0]` is not present in the bitstream, its value can be inferred as 0.

[0199] intra_luma_ref_idx can be referred to as the (intra-frame) reference sample line index or mrl_idx. Alternatively, intra_luma_ref_idx can be referred to as intra_luma_ref_line_idx.

[0200] Table 4 below shows the IntraLumaRefLineIdx[x0][y0] specified based on intra_luma_ref_idx[x0][y0].

[0201] [Table 4]

[0202] intra_luma_ref_idx[x0][y0] IntraLumaRefLineIdx[x0][y0] 0 0 1 1 2 2 or 3

[0203] exist Figure 15 In the example shown, the flag indicating whether the MPM is applied to the current coding unit is intra_luma_mpm_flag[x0][y0], and its value can be inferred to be 1 when the corresponding flag does not exist in the bitstream. That is, it can be determined that the MPM is applied to the current coding unit.

[0204] MRLs may not be available for blocks in the first row (line) within a CTU. For example, MRLs cannot be used for the current coding unit if the upper boundary of the current coding unit is the upper boundary of the CTU. This is to prevent the use of extended reference lines located outside the current CTU. Additionally, as described below, when using reference sample lines other than the first reference sample line, the PDPC for the current coding unit may not be applied.

[0205] When the intra-prediction mode of the coding unit applying MRL is DC mode, the DC value can be derived using the second reference sample and subsequent reference samples. In this case, instead of the reference sample of the first reference sample row, the DC value can be derived based on the reference samples of the second reference sample row and subsequent reference sample rows.

[0206] In this disclosure, information specifying the reference sample line for intra-frame prediction of the current block can be represented as refIdx. For example, refIdx of 0 can specify the first reference sample line.

[0207] The embodiments of this disclosure relate to the above-described PDPC. When the PDPC process is applied to intra-frame prediction samples, filtered (modified) prediction samples can be generated.

[0208] Embodiments of this disclosure propose a method for performing PDPC in chroma blocks with specific conditions when applying PDPC in intra-frame prediction of chroma components (blocks).

[0209] Existing PDPC determines whether to apply PDPC by applying different conditions to the luminance component block and the chrominance component block.

[0210] Figure 16 This is a view illustrating PDPC application conditions according to embodiments of the present disclosure.

[0211] according to Figure 16 In the implementation shown, PDPC can be applied to the intra-prediction prediction block of the current block when all of the following conditions are met.

[0212] (Condition 1) The width and height of the current block are both greater than or equal to 4, the current block is a chroma block, or the color component of the current block is a chroma component.

[0213] (Condition 2) Perform intra-frame prediction using the first reference sample line adjacent to the current block (refIdx == 0), or the current block is a chroma block.

[0214] (Condition 3) Block Differential Pulse Code Modulation (BDPCM) is not applied to the current block.

[0215] (Condition 4) The intra-pred prediction mode predModeIntra of the current block satisfies one of the following conditions.

[0216] -predModeIntra equals INTRA_PLANAR

[0217] -predModeIntra equals INTRA_DC

[0218] -predModeIntra is less than or equal to INTRA_ANGULAR18

[0219] -predModeIntra is greater than or equal to INTRA_ANGULAR50 and less than INTRA_LT_CCLM

[0220] Condition 1 relates to the size of the current block. When the current block is a chroma block, Condition 1 is satisfied regardless of its size. Additionally, when the current block is a luma block, Condition 1 is satisfied if the current block has a size of 4×4 or larger. In this disclosure, the color components of the current block can be represented by cIdx. For example, when cIdx is 0, the current block is a luma component block; when cIdx is not 0, the current block is a chroma component block.

[0221] Condition 2 relates to the reference sample line used for intra-frame prediction. When the current block is a chroma block, Condition 2 is satisfied regardless of the reference sample line. Additionally, when the current block is a luma block, Condition 2 is satisfied when intra-frame prediction is performed using the first reference sample line adjacent to the current block.

[0222] Condition 3 concerns whether to apply BDPCM to the current block. Condition 3 can be determined based on the BdpcmFlag of the current block. For example, when the BdpcmFlag of the current block is 0, it can be specified that BDPCM is not applied to the current block. The BdpcmFlag of the current block can be determined based on the value signaled from the bitstream. When the current block is a luma (component) block, the value of BdpcmFlag can be derived based on the intra_bdpcm_luma_flag signaled. When the current block is a chroma (component) block, the value of BdpcmFlag can be derived based on the intra_bdpcm_chroma_flag signaled.

[0223] Condition 4 relates to the intra-prediction mode of the current block. Specifically, PDPC can be applied to the current block when the intra-prediction mode of the current block corresponds to one of the following: 1) planar mode, 2) DC mode, 3) directional mode less than or equal to mode #18, or 4) directional mode greater than or equal to mode #50 and less than LT_CCLM.

[0224] Table 5 shows the arrangement based on the color components of the current block. Figure 16 The table shows condition 1 in the PDPC application conditions of the illustrated implementation.

[0225] [Table 5]

[0226] Luminance component block Chromaticity component blocks Condition 1 width ≥ 4 && height ≥ 4 Unconditional

[0227] As shown in Table 5, if the current block is a luma block, then condition 1 is satisfied when both the width and height of the current block are greater than or equal to a predetermined threshold 4. In this case, the predetermined threshold 4 can be replaced by MIN_TB_SIZEY. MIN_TB_SIZEY can represent the minimum transform block (TB) size of the luma component, and its value can be predetermined or signaled from the encoding device to the decoding device. Figure 16In the implementation shown, for example, MIN_TB_SIZEY can be 4.

[0228] However, when the current block is a chroma block, it always satisfies Figure 16 Condition 1. That is, the condition for the size of the current block is not applied. The condition for the size of the current block only applies to the luma block of the current block, not the chroma block. In short, the condition for the size of the current block in the PDPC application conditions can be applied differently depending on the color components of the current block.

[0229] More specifically, when the current block is a chroma block and its size is 2×2, 2×4, 4×2, or 2×N, intra-frame prediction for the current chroma block is not performed. Therefore, PDPC is not performed when the current block is a chroma block of the aforementioned sizes. Conversely, when the current block is an N×2 chroma block, intra-frame prediction can be performed, and therefore PDPC can also be performed. Therefore, when the current block is an N×2 block, PDPC is not performed in the intra-frame prediction of the luma block, but can be performed in the intra-frame prediction of the chroma block.

[0230] Figure 17 This is a view illustrating PDPC application conditions according to another embodiment of the present disclosure.

[0231] according to Figure 17 In the illustrated implementation, the block size condition in the PDPC application conditions can be applied equally to both luma and chroma blocks. Specifically, when the current chroma block size is N×2, a method can be provided that does not execute PDPC.

[0232] Furthermore, according to this disclosure, simplified and uniform intra-frame prediction can be performed by unifying the PDPC application conditions for luma and chroma blocks, regardless of the color components of the current block.

[0233] according to Figure 17 In the implementation shown, PDPC can be applied to the intra-prediction prediction block of the current block when all of the following conditions are met.

[0234] (Condition 1-1) The width and height of the current block are both greater than or equal to 4.

[0235] (Condition 2) Perform intra-frame prediction using the first reference sample line adjacent to the current block (refIdx == 0), or the current block is a chroma block.

[0236] (Condition 3) BDPCM is not applied to the current block.

[0237] (Condition 4) The intra-pred prediction mode predModeIntra of the current block satisfies one of the following conditions.

[0238] -predModeIntra equals INTRA_PLANAR

[0239] -predModeIntra equals INTRA_DC

[0240] -predModeIntra is less than or equal to INTRA_ANGULAR18

[0241] -predModeIntra is greater than or equal to INTRA_ANGULAR50 and less than INTRA_LT_CCLM

[0242] Condition 1-1 above relates to the size of the current block. According to... Figure 17 In the illustrated implementation, when the current block has a size of 4×4 or larger, condition 1 is satisfied regardless of the color components of the current block. That is, condition 1-1 is satisfied when both the width and height of the current block are greater than or equal to a predetermined threshold (e.g., 4). When the width or height of the current block is less than the predetermined threshold, it can be determined that condition 1-1 is not satisfied regardless of the color components of the current block. Therefore, according to Figure 17 In the implementation shown, in order to determine whether condition 1-1 is met, the determination of whether the color component of the current block is a luminance component or a chrominance component can be skipped.

[0243] Conditions 2 to 4 and reference Figure 16 Conditions 2 through 4 are described identically, so their repeated descriptions will be omitted.

[0244] according to Figure 17 The implementation shown solves the problem of applying PDPC to N×2 chroma blocks by applying condition 1-1, which is related to the size of the current block, to both the luma block and the chroma block.

[0245] Figure 17 The technical feature of the illustrated implementation is that condition 1-1, which applies to the size of the current block, is applied to both the luma block and the chroma block. Therefore, in addition to condition 1-1, it can be combined with... Figure 17 The illustrated implementation modifies all or some of the conditions in conditions 2 through 4. The modified PDPC application conditions are: Figure 17 The embodiments shown are modifications and may be included within the scope of this disclosure.

[0246] Figure 18 This is a view illustrating PDPC application conditions according to another embodiment of the present disclosure.

[0247] according to Figure 18In the illustrated implementation, the block size condition and the reference sample row condition in the PDPC application conditions can be applied equivalently to luma blocks and chroma blocks. Specifically, according to Figure 18 The illustrated implementation provides a method to not perform PDPC when the current chroma block size is N×2. Additionally, a method to not perform PDPC can be provided when the reference sample line used for intra-frame prediction is not the first reference sample line.

[0248] Furthermore, according to this disclosure, by unifying the PDPC application conditions for luma and chroma blocks, simplified and uniform intra-frame prediction can be performed regardless of the color components of the current block.

[0249] according to Figure 18 In the implementation shown, PDPC can be applied to the intra-prediction prediction block of the current block when all of the following conditions are met.

[0250] (Condition 1-1) The width and height of the current block are both greater than or equal to 4.

[0251] (Condition 2-1) Perform intra-frame prediction using the first reference sample row (refIdx == 0) adjacent to the current block.

[0252] (Condition 3) BDPCM is not applied to the current block.

[0253] (Condition 4) The intra-pred prediction mode predModeIntra of the current block satisfies one of the following conditions.

[0254] -predModeIntra equals INTRA_PLANAR

[0255] -predModeIntra equals INTRA_DC

[0256] -predModeIntra is less than or equal to INTRA_ANGULAR18

[0257] -predModeIntra is greater than or equal to INTRA_ANGULAR50 and less than INTRA_LT_CCLM

[0258] Condition 1-1 and Reference Figure 17 The conditions described in condition 1-1 are the same, so their repeated descriptions will be omitted.

[0259] Condition 2-1 relates to the reference sample rows used for intra-frame prediction. According to Figure 18In the illustrated implementation, when intra-frame prediction is performed using the first reference sample line adjacent to the current block, condition 2-1 is satisfied regardless of the color components of the current block. That is, condition 2 is satisfied when the first reference sample line is used for intra-frame prediction of the current block. When the first reference sample line is not used for intra-frame prediction of the current block, it can be determined that condition 2-1 is not satisfied, regardless of the color components of the current block. Therefore, according to Figure 18 In the implementation shown, in order to determine whether condition 2-1 is met, the determination of whether the color component of the current block is a luminance component or a chrominance component can be skipped.

[0260] Conditions 3 and 4 are in reference. Figure 16 Conditions 3 and 4 are described the same, so their repeated description will be omitted.

[0261] according to Figure 18 The illustrated implementation solves the problem of applying PDPC to N×2 chroma blocks by applying condition 1-1 (which pertains to the size of the current block) to both the luma and chroma blocks. Furthermore, by applying condition 2-1 (which pertains to the reference sample rows for intra-frame prediction of the current block) to both the luma and chroma blocks, the PDPC application conditions can be unified.

[0262] Figure 19 The technical feature of the illustrated implementation is that condition 1-1 regarding the size of the current block and condition 2-1 regarding the reference sample line used for intra-frame prediction are jointly applied to the luma component block and the chroma component block. Therefore, in addition to conditions 1-1 and 2-1, it is possible to combine with... Figure 18 The illustrated implementation modifies all or some of the conditions in conditions 3 and 4. The modified PDPC application conditions are: Figure 18 The embodiments shown are modifications and may be included within the scope of this disclosure.

[0263] Figure 19 This is a view illustrating a method for generating prediction blocks according to another embodiment of the present disclosure.

[0264] Figure 19 The steps for generating prediction blocks by performing intra-frame prediction are illustrated. Intra-frame prediction can be performed in an image coding device and / or an image decoding device to generate prediction blocks. For example, the steps performed in an image coding device... Figure 8 Step S800 can be executed. Figure 19 Each step. Additionally, the process performed in the image decoding device. Figure 10 Step S1020 can be executed Figure 19 Each step.

[0265] When performing intra-frame prediction on the current block, a prediction block for the current block can be generated based on the neighboring reference samples and the intra-frame prediction mode (S1910). The prediction block generated in step S1910 can be modified depending on whether PDPC is applied, and therefore can be called a temporary prediction block or a primary prediction block. In addition, the prediction block generated as a result of applying PDPC can be simply referred to as a prediction block or a final prediction block.

[0266] Therefore, it can be determined whether to apply PDPC to the temporary prediction block or the primary prediction block (S1920). Step S1920 can be performed by determining whether the PDPC application conditions are met. (Already referenced...) Figure 17 and Figure 18 The application conditions of PDPC according to this disclosure are described. However, the application conditions of PDPC according to this disclosure are not limited to... Figure 17 and Figure 18 Examples of PDPC application conditions may be included within the scope of this disclosure, as described above.

[0267] As a result of step S1920, PDPC is not executed when the PDPC application conditions are not met, and the temporary prediction block (primary prediction block) generated in step S1910 can be used as the final prediction block of the current block.

[0268] As a result of step S1920, PDPC can be executed (S1930) when the PDPC application conditions are met. In this case, the final prediction block of the current block can be generated by executing PDPC on the temporary prediction block (primary prediction block) generated in step S1910.

[0269] For example, the PDPC in step S1930 can be performed according to the PDPC method described above.

[0270] The basis of the current block Figure 19 The final prediction block generated by the method can be used to generate the residual block of the current block or to reconstruct the current block together with the residual block of the current block (S1040).

[0271] According to embodiments of this disclosure, by setting PDPC application conditions independently of the color components of the current block, the determination of the color components of the current block can be skipped, and the process of determining whether to apply PDPC can be simplified. Furthermore, according to embodiments of this disclosure, the determination of whether to apply PDPC can be simplified and uniformly performed.

[0272] Although the exemplary methods of this disclosure described above are represented as a series of operations for clarity of description, they are not intended to limit the order in which the steps are performed, and these steps may be performed simultaneously or in different orders if necessary. To implement the methods according to this disclosure, the described steps may further include other steps, including steps in addition to some steps, or may include additional steps in addition to some steps.

[0273] In this disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) can perform an operation (step) that confirms the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when predetermined conditions are met, the image encoding device or the image decoding device can perform the predetermined operation after determining whether the predetermined conditions are met.

[0274] The various embodiments of this disclosure are not a list of all possible combinations and are intended to describe representative aspects of this disclosure; the matters described in the various embodiments may be applied independently or in combination of two or more.

[0275] Various embodiments of this disclosure can be implemented in hardware, firmware, software, or a combination thereof. When this disclosure is implemented in hardware, it can be implemented using application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, etc.

[0276] Furthermore, the image decoding and image encoding devices applying the embodiments of this disclosure can be included in multimedia broadcasting transmitting and receiving devices, mobile communication terminals, home theater video devices, digital cinema video devices, surveillance cameras, video chat devices, real-time communication devices such as video communication, mobile streaming devices, storage media, cameras, video-on-demand (VoD) service providers, OTT (over-the-top video) devices, internet streaming service providers, three-dimensional (3D) video devices, video telephony devices, medical video devices, etc., and can be used to process video signals or data signals. For example, OTT video devices can include game consoles, Blu-ray players, internet access televisions, home theater systems, smartphones, tablet PCs, digital video recorders (DVRs), etc.

[0277] Figure 20 This is a view illustrating a content streaming system to which embodiments of the present disclosure can be applied.

[0278] like Figure 20As shown, the content streaming system using the embodiments of this disclosure may mainly include an encoding server, a streaming server, a network server, a media storage device, a user device, and a multimedia input device.

[0279] An encoding server compresses content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, which is then sent to a streaming server. As another example, when multimedia input devices such as smartphones, cameras, and camcorders directly generate bitstreams, the encoding server can be omitted.

[0280] The bitstream can be generated by an image encoding method or image encoding device applying the embodiments of this disclosure, and the stream server can temporarily store the bitstream during the sending or receiving of the bitstream.

[0281] A streaming server sends multimedia data to a user's device based on a request from a web server, and the web server acts as a medium for informing the user of the service. When a user requests a service from the web server, the web server can deliver it to the streaming server, and the streaming server can send the multimedia data to the user. In this scenario, the content streaming system may include a separate control server. In this case, the control server is used to control the commands / responses between devices in the content streaming system.

[0282] A streaming server can receive content from media storage devices and / or encoding servers. For example, when receiving content from an encoding server, the content can be received in real time. In this case, to provide a smooth streaming service, the streaming server can store the bitstream for a predetermined period of time.

[0283] Examples of user devices may include mobile phones, smartphones, laptop computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), navigation devices, tablet computers, tablet PCs, ultrabooks, wearable devices (e.g., smartwatches, smart glasses, head-mounted displays), digital televisions, desktop computers, digital signage, etc.

[0284] In a content streaming system, each server can operate as a distributed server, in which case the data received from each server can be distributed.

[0285] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) for enabling the operation of methods according to various embodiments to be executed on a device or computer, and non-transitory computer-readable media having such software or commands stored thereon and executable on a device or computer.

[0286] Industrial applicability

[0287] The embodiments disclosed herein can be used to encode or decode images.

Claims

1. An image decoding method performed by an image decoding device, the image decoding method comprising the following steps: The first prediction sample is generated by performing intra-frame prediction on the current chroma block; Determine whether to apply Location-Related Intra-Prediction Combination (PDPC) to the first prediction sample; as well as Based on the determination, a second prediction sample for the current chroma block is generated by applying PDPC to the first prediction sample. Specifically, based on the fact that the size of the current chroma block meets a predetermined condition and the intra-frame prediction mode of the current chroma block is a prediction mode with an index equal to or less than 18 or an index equal to or greater than 50 and less than INTRA_LT_CCLM, it is determined that PDPC will be applied to the first prediction sample, and Wherein, based on the fact that the size of the current chroma block does not meet the predetermined condition, it is determined that PDPC will not be applied to the first prediction sample.

2. The image decoding method according to claim 1, wherein, The predetermined condition is that the size of the current chroma block is greater than or equal to a predetermined threshold.

3. The image decoding method according to claim 2, wherein, The predetermined condition is satisfied when the width of the current chroma block is greater than or equal to the predetermined threshold and the height of the current chroma block is greater than or equal to the predetermined threshold.

4. The image decoding method according to claim 3, wherein, The predetermined threshold is 4.

5. The image decoding method according to claim 1, wherein, The step of determining whether to apply PDPC to the first prediction sample further includes: determining a reference sample line for intra-frame prediction of the current chroma block.

6. The image decoding method according to claim 5, in, Based on the fact that the reference sample row is a predetermined reference sample row, it is determined that PDPC will be applied to the first prediction sample, and Specifically, based on the fact that the reference sample row is not the predetermined reference sample row, it is determined that PDPC will not be applied to the first prediction sample.

7. The image decoding method according to claim 6, wherein, The predetermined reference sample row is the first reference sample row adjacent to the current chroma block.

8. The image decoding method according to claim 1, wherein, The step of determining whether to apply PDPC to the first prediction sample further includes: determining whether to apply block difference pulse code modulation (BDPCM) to the current chroma block.

9. An image encoding method performed by an image encoding device, the image encoding method comprising the following steps: The first prediction sample is generated by performing intra-frame prediction on the current chroma block; Determine whether to apply Location-Related Intra-Prediction Combination (PDPC) to the first prediction sample; as well as Based on the determination, a second prediction sample for the current chroma block is generated by applying PDPC to the first prediction sample. Specifically, based on the fact that the size of the current chroma block meets a predetermined condition and the intra-frame prediction mode of the current chroma block is a prediction mode with an index equal to or less than 18 or an index equal to or greater than 50 and less than INTRA_LT_CCLM, it is determined that PDPC will be applied to the first prediction sample, and Wherein, based on the fact that the size of the current chroma block does not meet the predetermined condition, it is determined that PDPC will not be applied to the first prediction sample.

10. The image encoding method according to claim 9, wherein, The predetermined condition is that the size of the current chroma block is greater than or equal to a predetermined threshold.

11. The image encoding method according to claim 9, wherein, The predetermined condition is satisfied when the width of the current chroma block is greater than or equal to a predetermined threshold and the height of the current chroma block is greater than or equal to the predetermined threshold.

12. The image encoding method according to claim 9, wherein, The step of determining whether to apply PDPC to the first prediction sample further includes: determining a reference sample line for intra-frame prediction of the current chroma block.

13. The image encoding method according to claim 12, in, Based on the fact that the reference sample row is a predetermined reference sample row, it is determined that PDPC will be applied to the first prediction sample, and Specifically, based on the fact that the reference sample row is not the predetermined reference sample row, it is determined that PDPC will not be applied to the first prediction sample.

14. A method for transmitting a bitstream generated by an image encoding method, the image encoding method comprising the following steps: The first prediction sample is generated by performing intra-frame prediction on the current chroma block; Determine whether to apply Location-Related Intra-Prediction Combination (PDPC) to the first prediction sample; as well as Based on the determination, a second prediction sample for the current chroma block is generated by applying PDPC to the first prediction sample. Specifically, based on the fact that the size of the current chroma block meets a predetermined condition and the intra-frame prediction mode of the current chroma block is a prediction mode with an index equal to or less than 18 or an index equal to or greater than 50 and less than INTRA_LT_CCLM, it is determined that PDPC will be applied to the first prediction sample, and Wherein, based on the fact that the size of the current chroma block does not meet the predetermined condition, it is determined that PDPC will not be applied to the first prediction sample.

Images