Image encoding / decoding method and apparatus, and method for transmitting bitstream

Abstract

Provided are an image encoding / decoding method and apparatus, and a method for transmitting a bitstream. An image decoding method performed by an image decoding apparatus may include obtaining a current block by segmenting an image, determining prediction mode characteristic information based on encoding information of the current block, determining a prediction mode type of a lower layer block segmented from the current block based on the prediction mode characteristic information, obtaining the lower layer block by segmenting the current block, and decoding the lower layer block by segmenting the current block. And decoding the lower layer block based on the determined prediction mode type of the lower layer block. Prediction mode characteristic information may be determined based on a color format of the current block.

Description

[0001] This application is a divisional application of the patent application with the application number 202080063277.2 (PCT / KR2020 / 010444), the international filing date of which is August 6, 2020, and which entered the Chinese Patent Office on March 8, 2022, and the invention title is "Image Encoding / Decoding Method and Apparatus, and Method for Transmitting Bitstream". Technical Field

[0002] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to an image encoding / decoding method for determining a segmentation pattern according to a color format, and a method for transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure. Background Art

[0003] Recently, the demand for high-resolution and high-quality images, such as high-definition (HD) images and ultra-high-definition (UHD) images, is increasing in various fields. As the resolution and quality of image data increase, the amount of information or bits to be transmitted increases relatively compared to existing image data. The increase in the amount of information or bits to be transmitted leads to an increase in transmission costs and storage costs.

[0004] Therefore, efficient image compression techniques are needed to effectively transmit, store, and reproduce information on high-resolution and high-quality images. Summary of the Invention

[0005] Technical Problem

[0006] An object of the present disclosure is to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0007] An object of the present disclosure is to provide an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by determining a segmentation pattern according to a color format.

[0008] Another object of the present disclosure is to provide a method for transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0009] Another object of the present disclosure is to provide a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0010] Another object of the present disclosure is to provide a recording medium storing a bitstream received, decoded, and used for reconstructing an image by an image decoding apparatus according to the present disclosure.

[0011] The technical problems solved by the present disclosure are not limited to the above technical problems, and other technical problems not described herein will be apparent to those skilled in the art from the following description.

[0012] Technical Solution

[0013] An image decoding method performed by an image decoding device according to an aspect of the present disclosure may include: obtaining a current block by dividing an image, determining prediction mode characteristic information based on coding information of the current block, determining a prediction mode type of a lower layer block divided from the current block based on the prediction mode characteristic information, obtaining the lower layer block by dividing the current block, and decoding the lower layer block based on the determined prediction mode type of the lower layer block. The prediction mode characteristic information may be determined based on a color format of the current block.

[0014] In addition, obtaining the lower layer block by dividing the current block may be performed by determining a division structure of the lower layer block based on the prediction mode type of the lower layer block, and based on the prediction mode type of the lower layer block being an intra prediction mode type, the division structure of the lower layer block may be determined as a dual-tree division structure.

[0015] In addition, based on the division structure of the lower layer block being determined as a dual-tree division structure, division availability for a luminance block and a chrominance block of the current block may be determined independently, and division availability of the chrominance block may be determined based on the color format. For example, the prediction mode characteristic information may be determined as a first value based on the color format of the current block being a monochrome format or a 4:4:4 format. Based on the prediction mode characteristic information being the first value, the prediction mode type of the lower layer block may be determined as the prediction mode type of the current block.

[0016] For example, based on the number of luminance samples of the current block being 64, the division mode of the current block being a binary division mode, and the color format of the current block being a predetermined format, the prediction mode characteristic information may be determined as the first value. Alternatively, based on the number of luminance samples of the current block being 128, the division mode of the current block being a ternary division mode, and the color format of the current block being a predetermined format, the prediction mode characteristic information may be determined as the first value. The predetermined format may be a monochrome format or a 4:4:4 format.

[0017] Alternatively, based on the color format of the current block being a 4:2:0 format, the number of luminance samples of the current block being 64, and the division mode of the current block being a binary division mode, the prediction mode characteristic information may be determined depending on whether the slice to which the current block belongs is an I slice.

[0018] Alternatively, based on the color format of the current block being a 4:2:0 format, the number of luminance samples of the current block being 128, and the division mode of the current block being a ternary division mode, the prediction mode characteristic information may be determined depending on whether the slice to which the current block belongs is an I slice.

[0019] In addition, based on the slice to which the current block belongs being an I slice, the prediction mode characteristic information can be determined as a second value, and based on the prediction mode characteristic information being the second value, the prediction mode type of the lower layer block can be determined as an intra prediction mode type.

[0020] Meanwhile, based on the slice to which the current block belongs not being an I slice, the prediction mode characteristic information can be determined as a third value. Based on the prediction mode characteristic information being the third value, the prediction mode type of the lower layer block can be determined based on the mode constraint information obtained from the bitstream. The mode constraint information can specify whether an inter prediction mode is allowed, and based on the mode constraint information specifying that the inter prediction mode is allowed, the prediction mode type of the lower layer block can be determined as an inter prediction mode type.

[0021] In addition, an image decoding apparatus according to an aspect of the present disclosure may include a memory and at least one processor. The at least one processor may: obtain a current block by dividing an image, determine prediction mode characteristic information based on the coding information of the current block, determine the prediction mode type of a lower layer block divided from the current block based on the prediction mode characteristic information, obtain the lower layer block by dividing the current block, and decode the lower layer block based on the determined prediction mode type of the lower layer block. The prediction mode characteristic information may be determined based on the color format of the current block.

[0022] In addition, an image encoding method performed by an image encoding apparatus according to an aspect of the present disclosure may include: obtaining a current block by dividing an image, determining prediction mode characteristic information based on the coding information of the current block, determining the prediction mode type of a lower layer block divided from the current block based on the prediction mode characteristic information, obtaining the lower layer block by dividing the current block, and encoding the lower layer block based on the determined prediction mode type of the lower layer block. The prediction mode characteristic information may be determined based on the color format of the current block.

[0023] In addition, a transmission method according to another aspect of the present disclosure may transmit a bitstream generated by the image encoding apparatus or image encoding method of the present disclosure.

[0024] In addition, a computer-readable recording medium according to another aspect of the present disclosure may store a bitstream generated by the image encoding apparatus or image encoding method of the present disclosure.

[0025] The features described above with respect to the brief overview of the present disclosure are merely exemplary aspects of the following detailed description of the present disclosure and do not limit the scope of the present disclosure.

[0026] Advantageous Effects

[0027] According to the present disclosure, it is possible to provide an image encoding / decoding method and apparatus having improved encoding / decoding efficiency.

[0028] In addition, according to the present disclosure, there can be provided an image encoding / decoding method and apparatus for improving encoding / decoding efficiency by determining a segmentation pattern according to a color format.

[0029] In addition, according to the present disclosure, there can be provided a method of transmitting a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0030] In addition, according to the present disclosure, there can be provided a recording medium storing a bitstream generated by an image encoding method or apparatus according to the present disclosure.

[0031] In addition, according to the present disclosure, there can be provided a recording medium storing a bitstream that is received, decoded, and used for reconstructing an image by an image decoding apparatus according to the present disclosure.

[0032] Those skilled in the art will realize that the effects that can be achieved by the present disclosure are not limited to what has been specifically described above, and other advantages of the present disclosure will be more clearly understood from the detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 is a view schematically showing a video compilation system to which an embodiment of the present disclosure is applicable.

[0034] Figure 2 is a view schematically showing an image encoding apparatus to which an embodiment of the present disclosure is applicable.

[0035] Figure 3 is a view schematically showing an image decoding apparatus to which an embodiment of the present disclosure is applicable.

[0036] Figure 4 is a view of a partition structure of an image according to an embodiment.

[0037] Figure 5 is a view showing an embodiment of a partition type of a block according to a multi-type tree structure.

[0038] Figure 6 is a view showing a signaling mechanism of block segmentation information in a quadtree having a nested multi-type tree structure according to the present disclosure.

[0039] Figure 7 is a view showing an embodiment in which a CTU is partitioned into a plurality of CUs.

[0040] Figure 8 is a view illustrating an embodiment of a redundant segmentation pattern.

[0041] Figures 9 to 11 is a view illustrating a positional relationship between a luminance sample and a chrominance sample determined according to a chrominance format according to an embodiment.

[0042] Figure 12 It is a view showing the syntax for chroma format signaling according to an embodiment.

[0043] Figure 13 It is a view showing the chroma format classification table according to an embodiment.

[0044] Figure 14 It is a view showing an embodiment of luminance blocks and chrominance blocks when the color format is 4:2:0.

[0045] Figures 15 to 17 It is a view showing a syntax for a coding tree unit (CTU) according to an embodiment.

[0046] Figure 18 It is a view showing luminance blocks and chrominance blocks when the color format is 4:4:4.

[0047] Figures 19 to 20 It is a flowchart showing a method for a coding device and a decoding device to perform coding and decoding according to an embodiment.

[0048] Figure 21 It is a view showing a content stream transmission system to which an embodiment of the present disclosure is applicable. Detailed Description of the Embodiments

[0049] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings so as to be easily implemented by those skilled in the art. However, the present disclosure can be implemented in various different forms and is not limited to the embodiments described herein.

[0050] When describing the present disclosure, if it is determined that a detailed description of a related known function or configuration makes the scope of the present disclosure unnecessarily ambiguous, its detailed description will be omitted. In the drawings, parts irrelevant to the description of the present disclosure are omitted, and similar reference numerals are given to similar parts.

[0051] In the present disclosure, when a component is "connected", "coupled" or "linked" to another component, it may include not only a direct connection relationship but also an indirect connection relationship with an intermediate component present. In addition, when a component "includes" or "has" other components, unless otherwise stated, it means that other components may also be included, rather than excluding other components.

[0052] In the present disclosure, terms such as first and second are only used for the purpose of distinguishing one component from other components and do not limit the order or importance of the components, unless otherwise stated. Accordingly, within the scope of the present disclosure, the first component in one embodiment may be referred to as the second component in another embodiment, and similarly, the second component in one embodiment may be referred to as the first component in another embodiment.

[0053] In the present disclosure, the components that are distinguished from each other are intended to clearly describe each feature, and it does not mean that the components must be separated. That is, multiple components can be integrated and implemented in one hardware or software unit, or one component can be distributed and implemented in multiple hardware or software units. Therefore, even without specific description, the embodiments in which these components are integrated or distributed are included in the scope of the present disclosure.

[0054] In the present disclosure, the components described in each embodiment are not necessarily essential components, and some components can be optional components. Therefore, the embodiments composed of a subset of the components described in the embodiments are also included in the scope of the present disclosure. In addition, the embodiments that include other components in addition to the components described in the various embodiments are included in the scope of the present disclosure.

[0055] The present disclosure relates to the encoding and decoding of images. Unless redefined in the present disclosure, the terms used in the present disclosure may have the general meanings commonly used in the technical field to which the present disclosure belongs.

[0056] In the present disclosure, a "picture" generally refers to a unit representing an image within a specific time period, while a slice / tile is a coding unit that forms a part of a picture, and a picture can be composed of one or more slices / tiles. In addition, a slice / tile can include one or more coding tree units (CTUs).

[0057] In the present disclosure, a "pixel" or "pel" can mean the smallest individual that constitutes a picture (or image). In addition, "sample" can be used as a term corresponding to a pixel. A sample generally can represent a pixel or the value of a pixel, or can represent only the pixel / pixel value of the luminance component or only the pixel / pixel value of the chrominance component.

[0058] In the present disclosure, a "unit" can represent a basic unit of image processing. The unit can include at least one of a specific area of a picture and information related to the area. In some cases, the unit can be used interchangeably with terms such as "sample array", "block", or "area". Generally, an M×N block can include a set (or array) of samples (or sample arrays) or transform coefficients with M columns and N rows.

[0059] In the present disclosure, "current block" may mean one of "current compilation block", "current compilation unit", "compilation target block", "decoding target block", or "processing target block". When performing prediction, "current block" may mean "current prediction block" or "prediction target block". When performing transformation (inverse transformation) / quantization (dequantization), "current block" may mean "current transformation block" or "transformation target block". When performing filtering, "current block" may mean "filtering target block".

[0060] In addition, in the present disclosure, unless explicitly stated as a chrominance block, "current block" may mean "luminance block of the current block". "Chrominance block of the current block" can be expressed by including an explicit description of a chrominance block such as "chrominance block" or "current chrominance block".

[0061] In the present disclosure, the slashes " / " or "," can be interpreted as indicating "and / or". For example, "A / B" and "A,B" can mean "A and / or B". In addition, "A / B / C" and "A / B / C" can mean "at least one of A, B, and / or C".

[0062] In the present disclosure, the term "or" should be interpreted to indicate "and / or". For example, the expression "A or B" can include 1) only "A", 2) only "B", or 3) both "A and B". In other words, in the present disclosure, "or" should be interpreted to indicate "additionally or alternatively".

[0063] Overview of the video compilation system

[0064] Figure 1 is a view schematically showing a video compilation system according to the present disclosure.

[0065] The video compilation system according to an embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 may deliver encoded video and / or image information or data in the form of a file or a stream to the decoding device 20 via a digital storage medium or a network.

[0066] The encoding device 10 according to an embodiment may include a video source generator 11, an encoding unit 12, and a transmitter 13. The decoding device 20 according to an embodiment may include a receiver 21, a decoding unit 22, and a renderer 23. The encoding unit 12 may be referred to as a video / image encoding unit, and the decoding unit 22 may be referred to as a video / image decoding unit. The transmitter 13 may be included in the encoding unit 12. The receiver 21 may be included in the decoding unit 22. The renderer 23 may include a display, and the display may be configured as a separate device or an external component.

[0067] The video source generator 11 can obtain video / images through processes such as capturing, synthesizing, or generating video / images. The video source generator 11 can include a video / image capturing device and / or a video / image generating device. The video / image capturing device can include, for example, one or more cameras, a video / image archive including previously captured video / images, etc. The video / image generating device can include, for example, a computer, a tablet computer, and a smart phone, and can (electronically) generate video / images. For example, virtual video / images can be generated by a computer or the like. In this case, the video / image capturing process can be replaced by a process of generating relevant data.

[0068] The encoding unit 12 can encode the input video / images. For compression and compilation efficiency, the encoding unit 12 can perform a series of processes such as prediction, transformation, and quantization. The encoding unit 12 can output the encoded data (encoded video / image information) in the form of a bitstream.

[0069] The transmitter 13 can transmit the encoded video / image information or the data output in the form of a bitstream to the receiver 21 of the decoding device 20 in the form of a file or a stream through a digital storage medium or a network. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. The transmitter 13 can include elements for generating a media file in a predetermined file format and can include elements for transmitting through a broadcast / communication network. The receiver 21 can extract / receive the bitstream from the storage medium or the network and transmit the bitstream to the decoding unit 22.

[0070] The decoding unit 22 can decode the video / images by performing a series of processes corresponding to the operations of the encoding unit 12, such as dequantization, inverse transformation, and prediction.

[0071] The renderer 23 can render the decoded video / images. The rendered video / images can be displayed through a display.

[0072] Overview of the image encoding device

[0073] Figure 2 is a view schematically showing an image encoding apparatus to which an embodiment of the present disclosure can be applied.

[0074] As Figure 2As shown, the image encoding device 100 may include an image partitioner 110, a subtractor 115, a transformer 120, a quantizer 130, a dequantizer 140, an inverse transformer 150, an adder 155, a filter 160, a memory 170, an inter-frame prediction unit 180, an intra-frame prediction unit 185, and an entropy encoder 190. The inter-frame prediction unit 180 and the intra-frame prediction unit 185 may be collectively referred to as a "prediction unit". The transformer 120, the quantizer 130, the dequantizer 140, and the inverse transformer 150 may be included in a residual processor. The residual processor may further include the subtractor 115.

[0075] In some embodiments, all or at least some of the multiple components configuring the image encoding device 100 may be configured by one hardware component (e.g., an encoder or a processor). Additionally, the memory 170 may include a decoded picture buffer (DPB) and may be configured by a digital storage medium.

[0076] The image partitioner 110 may partition an input image (or picture or frame) input to the image encoding device 100 into one or more processing units. For example, the processing unit may be referred to as a coding unit (CU). Coding units may be obtained by recursively partitioning a coding tree unit (CTU) or a largest coding unit (LCU) according to a quadtree / binary tree / trinary tree (QT / BT / TT) structure. For example, a coding unit may be partitioned into multiple coding units of a deeper depth based on a quadtree structure, a binary tree structure, and / or a trinary structure. For partitioning of the coding unit, the quadtree structure may be applied first, and then the binary tree structure and / or the trinary structure may be applied. The coding process according to the present disclosure may be performed based on the final coding units that are no longer partitioned. The largest coding unit may be used as the final coding unit, or the coding units of a deeper depth obtained by partitioning the largest coding unit may be used as the final coding units. Here, the coding process may include processes of prediction, transformation, and reconstruction, which will be described later. As another example, the processing unit of the coding process may be a prediction unit (PU) or a transformation unit (TU). The prediction unit and the transformation unit may be split or partitioned from the final coding unit. The prediction unit may be a sample prediction unit, and the transformation unit may be a unit for deriving transform coefficients and / or a unit for deriving a residual signal from the transform coefficients.

[0077] The prediction unit (the inter-frame prediction unit 180 or the intra-frame prediction unit 185) may perform prediction on a block to be processed (the current block) and generate a prediction block including prediction samples of the current block. The prediction unit may determine whether to apply intra-frame prediction or inter-frame prediction based on the current block or CU. The prediction unit may generate various information related to the prediction of the current block and transmit the generated information to the entropy encoder 190. The information about the prediction may be encoded in the entropy encoder 190 and output in the form of a bitstream.

[0078] The intra prediction unit 185 can predict the current block by referring to samples in the current picture. According to the intra prediction mode and / or intra prediction technique, the reference samples can be located among the neighbors of the current block or can be placed separately. The intra prediction mode can include multiple non - directional modes and multiple directional modes. The non - directional modes can include, for example, the DC mode and the planar mode. According to the level of detail of the prediction direction, the directional modes can include, for example, 33 directional prediction modes or 65 directional prediction modes. However, these are just examples, and more or fewer directional prediction modes can be used according to the settings. The intra prediction unit 185 can determine the prediction mode applied to the current block by using the prediction mode applied to neighboring blocks.

[0079] The inter prediction unit 180 can derive the prediction block of the current block based on the reference block (reference sample array) specified by the motion vector on the reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information can be predicted in units of blocks, sub - blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information can include the motion vector and the reference picture index. The motion information can also include inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.) information. In the case of inter prediction, the neighboring blocks can include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture including the reference block and the reference picture including the temporal neighboring blocks can be the same or different. The temporal neighboring blocks can be referred to as collocated reference blocks, collocated CUs (colCUs), etc. The reference picture including the temporal neighboring blocks can be referred to as a collocated picture (colPic). For example, the inter prediction unit 180 can configure a motion information candidate list based on neighboring blocks and generate information specifying which candidate to use to derive the motion vector and / or reference picture index of the current block. The inter prediction can be performed based on various prediction modes. For example, in the case of the skip mode and the merge mode, the inter prediction unit 180 can use the motion information of neighboring blocks as the motion information of the current block. In the case of the skip mode, different from the merge mode, the residual signal may not be transmitted. In the case of the motion vector prediction (MVP) mode, the motion vector of the neighboring block can be used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator of the motion vector predictor. The motion vector difference can mean the difference between the motion vector of the current block and the motion vector predictor.

[0080] The prediction unit may generate a prediction signal based on various prediction methods and prediction techniques described below. For example, the prediction unit may not only apply intra prediction or inter prediction, but also apply both intra prediction and inter prediction simultaneously to predict the current block. The prediction method of applying both intra prediction and inter prediction simultaneously to predict the current block may be referred to as combined inter and intra prediction (CIIP). In addition, the prediction unit may perform intra block copy (IBC) to predict the current block. Intra block copy may be used for content image / video compilation such as games, for example, screen content compilation (SCC). IBC is a method of predicting the current picture using a previously reconstructed reference block in the current picture at a position separated from the current block by a predetermined distance. When IBC is applied, the position of the reference block in the current picture may be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction in the current picture, but may be performed similar to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure. IBC basically performs prediction in the current picture, but may be performed similar to inter prediction because the reference block is derived within the current picture. That is, IBC may use at least one of the inter prediction techniques described in the present disclosure.

[0081] The prediction signal generated by the prediction unit may be used to generate a reconstruction signal or to generate a residual signal. The subtractor 115 may generate a residual signal (residual block or residual sample array) by subtracting the prediction signal (prediction block or prediction sample array) output from the prediction unit from the input image signal (original block or original sample array). The generated residual signal may be transmitted to the transformer 120.

[0082] The transformer 120 may generate transform coefficients by applying a transform technique to the residual signal. For example, the transform technique may include at least one of discrete cosine transform (DCT), discrete sine transform (DST), karhunen-loève transform (KLT), graph-based transform (GBT), or conditional non-linear transform (CNT). Here, GBT refers to a transform obtained from a graph when the relationship information between pixels is represented by a graph. CNT refers to a transform obtained based on a prediction signal generated using all previously reconstructed pixels. In addition, the transform processing may be applied to a square pixel block having the same size or may be applied to a block having a variable size rather than a square.

[0083] Quantizer 130 may quantize the transform coefficients and transmit them to entropy encoder 190. Entropy encoder 190 may encode the quantized signal (information regarding the quantized transform coefficients) and output a bitstream. The information regarding the quantized transform coefficients may be referred to as residual information. Quantizer 130 may rearrange the quantized transform coefficients in block form into a one-dimensional vector form based on the coefficient scan order, and generate information regarding the quantized transform coefficients based on the quantized transform coefficients in the one-dimensional vector form.

[0084] Entropy encoder 190 may perform various coding methods, such as Exponential Golomb, Context Adaptive Variable Length Coding (CAVLC), Context Adaptive Binary Arithmetic Coding (CABAC), etc. Entropy encoder 190 may encode, together or separately, information required for video / image reconstruction other than the quantized transform coefficients (e.g., values of syntax elements, etc.). The encoded information (e.g., encoded video / image information) may be transmitted or stored in units of Network Abstraction Layer (NAL) in the form of a bitstream. The video / image information may also include information regarding various parameter sets, such as Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information may also include general constraint information. The information signaled, transmitted, and / or syntax elements described in the present disclosure may be encoded through the above coding process and included in the bitstream.

[0085] The bitstream may be transmitted through a network or may be stored in a digital storage medium. The network may include a broadcast network and / or a communication network, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray, HDD, SSD, etc. A transmitter (not shown) for transmitting the signal output from entropy encoder 190 and / or a storage unit (not shown) for storing the signal may be included as internal / external elements of image encoding apparatus 100. Alternatively, a transmitter may be provided as a component of entropy encoder 190.

[0086] The quantized transform coefficients output from quantizer 130 may be used to generate a residual signal. For example, the quantized transform coefficients may be dequantized and inverse-transformed by dequantizer 140 and inverse-transformer 150 to reconstruct the residual signal (residual block or residual samples).

[0087] The adder 155 adds the reconstructed residual signal to the predicted signal output from the inter-frame prediction unit 180 or the intra-frame prediction unit 185 to generate a reconstructed signal (reconstructed image, reconstructed block, reconstructed sample array). If there is no residual for the block to be processed, for example, in the case of applying the skip mode, the predicted block can be used as the reconstructed block. The adder 155 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0088] The filter 160 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 160 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 170, specifically, in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc. The filter 160 can generate various information related to the filtering and transmit the generated information to the entropy encoder 190, as described later in the description of each filtering method. The information related to the filtering can be encoded by the entropy encoder 190 and output in the form of a bitstream.

[0089] The modified reconstructed picture transmitted to the memory 170 can be used as a reference picture in the inter-frame prediction unit 180. When inter-frame prediction is applied by the image coding device 100, prediction mismatch between the image coding device 100 and the image decoding device can be avoided and the coding efficiency can be improved.

[0090] The DPB of the memory 170 can store the modified reconstructed picture to be used as a reference picture in the inter-frame prediction unit 180. The memory 170 can store the motion information of the block from which the motion information in the current picture is derived (or encoded) and / or the motion information of the reconstructed blocks in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 180 and used as the motion information of spatially neighboring blocks or temporally neighboring blocks. The memory 170 can store the reconstructed samples of the reconstructed blocks in the current picture and can transmit the reconstructed samples to the intra-frame prediction unit 185.

[0091] Overview of the image decoding device

[0092] Figure 3 is a view schematically showing an image decoding device to which an embodiment of the present disclosure is applicable.

[0093] As Figure 3As shown, the image decoding apparatus 200 may include an entropy decoder 210, a dequantizer 220, an inverse transformer 230, an adder 235, a filter 240, a memory 250, an inter prediction unit 260, and an intra prediction unit 265. The inter prediction unit 260 and the intra prediction unit 265 may be collectively referred to as a "prediction unit". The dequantizer 220 and the inverse transformer 230 may be included in a residual processor.

[0094] According to an embodiment, all or at least some of the plurality of components configuring the image decoding apparatus 200 may be configured by hardware components (e.g., a decoder or a processor). In addition, the memory 250 may include a decoded picture buffer (DPB) or may be configured by a digital storage medium.

[0095] The image decoding apparatus 200 that has received a bitstream including video / image information may reconstruct an image by performing a process corresponding to the process performed by Figure 2 the image encoding apparatus 100. For example, the image decoding apparatus 200 may perform decoding using a processing unit applied in the image encoding apparatus. Thus, the decoding processing unit may be, for example, a compilation unit. The compilation unit may be obtained through a partitioned compilation tree unit or a maximum compilation unit. The reconstructed image signal decoded and output by the image decoding apparatus 200 may be reproduced by a reproduction apparatus (not shown).

[0096] The image decoding apparatus 200 may receive, in the form of a bitstream, from Figure 2The signal output by the image encoding device. The received signal can be decoded by the entropy decoder 210. For example, the entropy decoder 210 can parse the bitstream to derive the information (e.g., video / image information) required for image reconstruction (or picture reconstruction). The video / image information can also include information about various parameter sets, such as the Adaptive Parameter Set (APS), Picture Parameter Set (PPS), Sequence Parameter Set (SPS), or Video Parameter Set (VPS). In addition, the video / image information can also include general constraint information. The image decoding device can also decode the picture based on the parameter set information and / or general constraint information. The information and / or syntax elements signaled / received described in this disclosure can be decoded through the decoding process and obtained from the bitstream. For example, the entropy decoder 210 decodes the information in the bitstream based on an encoding method such as Exponential Golomb coding, CAVLC, or CABAC, and outputs the values of the syntax elements required for image reconstruction and the quantization values of the transformed coefficients of the residuals. More specifically, the CABAC entropy decoding method can receive the bins corresponding to each syntax element in the bitstream, use the decoding target syntax element information, neighboring blocks, and decoding information of the decoding target block or the information of the symbols / bins decoded in the previous stage to determine the context model, and perform arithmetic decoding on the bins by predicting the occurrence probability of the bins according to the determined context model to generate symbols corresponding to the values of each syntax element. In this case, after determining the context model, the CABAC entropy decoding method can update the context model by using the information of the decoded symbols / bins for the context model of the next symbol / bin. The information related to prediction in the information decoded by the entropy decoder 210 can be provided to the prediction units (inter-frame prediction unit 260 and intra-frame prediction unit 265), and the residual values for which entropy decoding is performed in the entropy decoder 210, that is, the quantized transform coefficients and related parameter information, can be input to the dequantizer 220. In addition, the information about filtering among the information decoded by the entropy decoder 210 can be provided to the filter 240. Meanwhile, the receiver (not shown) for receiving the signal output by the image encoding device can be further configured as an internal / external component of the image decoding device 200, or the receiver can be a component of the entropy decoder 210.

[0097] Meanwhile, the image decoding device according to the present disclosure can be referred to as a video / image / picture decoding device. The image decoding device can be divided into an information decoder (video / image / picture information decoder) and a sample decoder (video / image / picture sample decoder). The information decoder can include the entropy decoder 210. The sample decoder can include at least one of the dequantizer 220, inverse transformer 230, adder 235, filter 240, memory 250, inter-frame prediction unit 160, or intra-frame prediction unit 265.

[0098] The dequantizer 220 may dequantize the quantized transform coefficients and output the transform coefficients. The dequantizer 220 may rearrange the quantized transform coefficients in the form of two-dimensional blocks. In this case, the rearrangement may be performed based on the coefficient scan order executed in the image coding device. The dequantizer 220 may dequantize the quantized transform coefficients by using quantization parameters (e.g., quantization step information) and obtain the transform coefficients.

[0099] The inverse transformer 230 may perform an inverse transform on the transform coefficients to obtain a residual signal (residual block, residual sample array).

[0100] The prediction unit may perform prediction on the current block and generate a prediction block including the prediction samples of the current block. The prediction unit may determine whether to apply intra prediction or inter prediction to the current block based on the information about the prediction output from the entropy decoder 210, and may determine a specific intra / inter prediction mode (prediction technique).

[0101] Similar to that described for the prediction unit in the image coding device 100, the prediction unit may generate a prediction signal based on various prediction methods (techniques) described later.

[0102] The intra prediction unit 265 may predict the current block by referring to the samples in the current picture. The description of the intra prediction unit 185 is equally applicable to the intra prediction unit 265.

[0103] The inter prediction unit 260 may derive a prediction block of the current block based on a reference block (reference sample array) specified by a motion vector on a reference picture. In this case, in order to reduce the amount of motion information transmitted in the inter prediction mode, the motion information may be predicted in units of blocks, sub-blocks, or samples based on the correlation of the motion information between neighboring blocks and the current block. The motion information may include a motion vector and a reference picture index. The motion information may also include information about the inter prediction direction (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of inter prediction, the neighboring blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the inter prediction unit 260 may configure a motion information candidate list based on the neighboring blocks, and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. The inter prediction may be performed based on various prediction modes, and the information about the prediction may include information indicating the inter prediction mode of the current block.

[0104] The adder 235 can generate a reconstructed block by adding the obtained residual signal to the prediction signal (prediction block, prediction sample array) output from the prediction unit (including the inter-frame prediction unit 260 and / or the intra-frame prediction unit 265). If the block to be processed has no residual, such as when the skip mode is applied, the prediction block can be used as the reconstructed block. The description of the adder 155 equally applies to the adder 235. The adder 235 can be referred to as a reconstructor or a reconstructed block generator. The generated reconstructed signal can be used for intra-frame prediction of the next block to be processed in the current picture and can be used for inter-frame prediction of the next picture through filtering as described below.

[0105] The filter 240 can improve the subjective / objective image quality by applying filtering to the reconstructed signal. For example, the filter 240 can generate a modified reconstructed picture by applying various filtering methods to the reconstructed picture and store the modified reconstructed picture in the memory 250, specifically, in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filtering, bilateral filtering, etc.

[0106] The (modified) reconstructed picture stored in the DPB of the memory 250 can be used as a reference picture in the inter-frame prediction unit 260. The memory 250 can store the motion information of the block from which the motion information in the current picture is derived (or decoded) and / or the motion information of the reconstructed blocks in the picture. The stored motion information can be transmitted to the inter-frame prediction unit 260 to be used as the motion information of spatially adjacent blocks or temporally adjacent blocks. The memory 250 can store the reconstructed samples of the reconstructed blocks in the current picture and transmit the reconstructed samples to the intra-frame prediction unit 265.

[0107] In the present disclosure, the embodiments described in the filter 160, the inter-frame prediction unit 180, and the intra-frame prediction unit 185 of the image encoding device 100 can be equally or correspondingly applied to the filter 240, the inter-frame prediction unit 260, and the intra-frame prediction unit 265 of the image decoding device 200.

[0108] Overview of image partitioning

[0109] The video / image compilation method according to the present disclosure can be performed based on an image partitioning structure as follows. Specifically, processes such as prediction, residual processing ((inverse) transformation, (de)quantization, etc.), syntax element compilation, and filtering, which will be described later, can be performed based on CTUs, CUs (and / or TUs, PUs) derived according to the image partitioning structure. The image can be partitioned into block units, and the block partitioning process can be performed in the image partitioner 110 of the encoding device. Partition-related information can be encoded by the entropy encoder 190 and sent to the decoding device in the form of a bitstream. The entropy decoder 210 of the decoding device can derive the block partitioning structure of the current picture based on the partition-related information obtained from the bitstream, and based on this, a series of processes (such as prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) can be performed for image decoding.

[0110] A picture can be partitioned into a sequence of coding tree units (CTUs). Figure 4 An example in which a picture is partitioned into CTUs is shown. A CTU can correspond to a coding tree block (CTB). Alternatively, a CTU can include a coding tree block of luminance samples and two coding tree blocks of corresponding chrominance samples. For example, for a picture including three sample arrays, a CTU can include one N×N block of luminance samples and two corresponding blocks of chrominance samples.

[0111] Overview of CTU partitioning

[0112] As described above, coding units can be obtained by recursively partitioning coding tree units (CTUs) or largest coding units (LCUs) according to a quadtree / binary tree / trinary tree (QT / BT / TT) structure. For example, a CTU can be first partitioned into a quadtree structure. Thereafter, the leaf nodes of the quadtree structure can be further partitioned by a multi-type tree structure.

[0113] Partitioning according to a quadtree means that the current CU (or CTU) is equally partitioned into four. By partitioning according to a quadtree, the current CU can be partitioned into four CUs with the same width and the same height. When the current CU is no longer partitioned into a quadtree structure, the current CU corresponds to a leaf node of the quadtree structure. The CU corresponding to the leaf node of the quadtree structure can no longer be partitioned and can be used as the final coding unit described above. Alternatively, the CU corresponding to the leaf node of the quadtree structure can be further partitioned by a multi-type tree structure.

[0114] Figure 5 FIG. is a view showing an embodiment of the partitioning type of a block according to a multi-type tree structure. Partitioning according to a multi-type tree structure can include two types of splitting according to a binary tree structure and two types of splitting according to a trinary tree structure.

[0115] Two types of splitting according to the binary tree structure may include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means that the current CU is equally split into two in the vertical direction. As Figure 4 shown, through vertical binary splitting, two CUs with the same height as the current CU and a width that is half of the width of the current CU can be generated. Horizontal binary splitting (SPLIT_BT_HOR) means that the current CU is equally split into two in the horizontal direction. As Figure 5 shown, through horizontal binary splitting, two CUs with a height that is half of the height of the current CU and the same width as the current CU can be generated.

[0116] Two types of splitting according to the ternary tree structure may include vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). In vertical ternary splitting (SPLIT_TT_VER), the current CU is split in the vertical direction at a ratio of 1:2:1. As Figure 5 shown, through vertical ternary splitting, two CUs with the same height as the current CU and a width that is 1 / 4 of the width of the current CU, and a CU with the same height as the current CU and a width that is half of the width of the current CU can be generated. In horizontal ternary splitting (SPLIT_TT_HOR), the current CU is split in the horizontal direction at a ratio of 1:2:1. As Figure 5 shown, through horizontal ternary splitting, two CUs with a height that is 1 / 4 of the height of the current CU and the same width as the current CU, and a CU with a height that is half of the height of the current CU and the same width as the current CU can be generated.

[0117] Figure 6 is a view showing a signaling mechanism of block splitting information in a quadtree having a nested multi-type tree structure according to the present disclosure.

[0118] Here, the CTU is regarded as the root node of the quadtree and is first partitioned into a quadtree structure. Information (e.g., qt_split_flag) specifying whether to perform quadtree splitting on the current CU (the CTU or node (QT_node) of the quadtree) is signaled. For example, when the qt_split_flag has a first value (e.g., "1"), the current CU can be partitioned by the quadtree. Additionally, when the qt_split_flag has a second value (e.g., "0"), the current CU is not partitioned by the quadtree but becomes a leaf node (QT_leaf_node) of the quadtree. Then each quadtree leaf node can be further partitioned into a multi-type tree structure. That is, the leaf node of the quadtree can become a node (MTT_node) of the multi-type tree. In the multi-type tree structure, a first flag (e.g., Mtt_split_cu_flag) is signaled to specify whether the current node is additionally partitioned. If the corresponding node is additionally partitioned (e.g., if the first flag is 1), a second flag (e.g., Mtt_split_cu_vertical_flag) can be signaled to specify the splitting direction. For example, the splitting direction can be the vertical direction when the second flag is 1, and the horizontal direction when the second flag is 0. Then, a third flag (e.g., Mtt_split_cu_binary_flag) can be signaled to specify whether the splitting type is a binary splitting type or a ternary splitting type. For example, the splitting type can be the binary splitting type when the third flag is 1, and the ternary splitting type when the third flag is 0. The nodes of the multi-type tree obtained by binary splitting or ternary splitting can be further partitioned into a multi-type tree structure. However, the nodes of the multi-type tree can not be partitioned into a quadtree structure. If the first flag is 0, the corresponding node of the multi-type tree is no longer split but becomes a leaf node (MTT_leaf_node) of the multi-type tree. The CU corresponding to the leaf node of the multi-type tree can be used as the above-mentioned final compilation unit.

[0119] Based on mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1 below. In the following description, the multi-type tree splitting mode can be referred to as the multi-tree splitting type or the splitting type.

[0120] [Table 1]

[0121] MttSplitMode mtt_split_cu_vertical_flag mtt_split_cu_binary_flag SPLIT_TT_HOR 0 0 SPLIT_BT_HOR 0 1 SPLIT_TT_VER 1 0 SPLIT_BT_VER 1 1

[0122] Figure 7 is a view showing an example of partitioning the CTU into multiple CUs by applying a multi-type tree after applying the quadtree. InFigure 7 Among them, the bold block border 710 represents a quadtree partition, while the remaining border 720 represents a multi-type tree partition. A CU may correspond to a coding block (CB). In an embodiment, a CU may include a coding block of luminance samples and two coding blocks of chrominance samples corresponding to the luminance samples. The chrominance component (sample) CB or TB size may be derived based on the component ratio according to the color format of the picture / image (chrominance format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.) based on the luminance component (sample) CB or TB size. In the case of the 4:4:4 color format, the chrominance component CB / TB size may be set to be equal to the luminance component CB / TB size. In the case of the 4:2:2 color format, the width of the chrominance component CB / TB may be set to half of the width of the luminance component CB / TB and the height of the chrominance component CB / TB may be set to the height of the luminance component CB / TB. In the case of the 4:2:0 color format, the width of the chrominance component CB / TB may be set to half of the width of the luminance component CB / TB and the height of the chrominance component CB / TB may be set to half of the height of the luminance component CB / TB.

[0123] In an embodiment, when the size of the CTU is 128 based on luminance sample units, the size of the CU may have a size ranging from 128x128 to 4x4, which is the same size as the CTU. In one embodiment, in the case of the 4:2:0 color format (or chrominance format), the chrominance CB size may have a size ranging from 64x64 to 2x2.

[0124] Meanwhile, in an embodiment, the CU size and the TU size may be the same. Alternatively, there may be multiple TUs in the CU region. The TU size generally represents the luminance component (sample) transform block (TB) size.

[0125] The TU size may be derived based on the maximum allowable TB size maxTbSize which is a predetermined value. For example, when the CU size is greater than maxTbSize, multiple TUs (TBs) with maxTbSize may be derived from the CU, and the transform / inverse transform may be performed in units of TUs (TBs). For example, the maximum allowable luminance TB size may be 64x64 and the maximum allowable chrominance TB size may be 32x32. If the width or height of the CB partitioned according to the tree structure is greater than the maximum transform width or height, the CB may be automatically (or implicitly) partitioned until the TB size limits in the horizontal and vertical directions are met.

[0126] In addition, for example, when intra prediction is applied, the intra prediction mode / type can be derived on a CU (or CB) basis, and the neighboring reference sample derivation and predicted sample generation processes can be performed on a TU (or TB) basis. In this case, there can be one or more TUs (or TBs) in a CU (or CB) region, and in this case, multiple TUs or (TBs) can share the same intra prediction mode / type.

[0127] Meanwhile, for a quadtree compilation tree scheme with a nested multi-type tree, the following parameters can be signaled from an encoding device to a decoding device as SPS syntax elements. For example, at least one of the CTU size as a parameter representing the root node size of the quadtree, the MinQTSize as a parameter representing the minimum allowable quadtree leaf node size, the MaxBtSize as a parameter representing the maximum allowable binary tree root node size, the MaxTtSize as a parameter representing the maximum allowable ternary tree root node size, the MaxMttDepth as a parameter representing the maximum allowable hierarchical depth of multi-type tree splitting starting from quadtree leaf nodes, the MinBtSize as a parameter representing the minimum allowable binary tree leaf node size, or the MinTtSize as a parameter representing the minimum allowable ternary tree leaf node size is signaled.

[0128] As an example of using the 4:2:0 chroma format, the CTU size can be set to 128x128 luma blocks and two 64x64 chroma blocks corresponding to these luma blocks. In this case, MinOTSize can be set to 16x16, MaxBtSize can be set to 128x128, MaxTtSzie can be set to 64x64, MinBtSize and MinTtSize can be set to 4x4, and MaxMttDepth can be set to 4. Quadtree partitioning can be applied to the CTU to generate quadtree leaf nodes. The quadtree leaf nodes can be referred to as leaf QT nodes. The size of the quadtree leaf nodes can range from 16x16 size (e.g., MinOTSize) to 128x128 size (e.g., CTU size). If the leaf QT node is 128x128, it may not be additionally partitioned into a binary tree / trinary tree. This is because, in this case, even if partitioned, it exceeds MaxBtsize and MaxTtszie (e.g., 64x64). In other cases, the leaf QT node can be further partitioned into a multi-type tree. Therefore, the leaf QT node is the root node of the multi-type tree, and the leaf QT node can have a multi-type tree depth (mttDepth) of 0 value. If the multi-type tree depth reaches MaxMttdepth (e.g., 4), further partitioning can be disregarded. If the width of the multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, further horizontal partitioning can be disregarded. If the height of the multi-type tree node is equal to MinBtSize and less than or equal to 2xMinTtSize, further vertical partitioning can be disregarded. When partitioning is disregarded, the encoding device can skip signaling of the partitioning information. In this case, the decoding device can derive the partitioning information with a predetermined value.

[0129] Meanwhile, a CTU may include a coding tree block of luma samples (hereinafter referred to as "luma block") and two coding tree blocks of chroma samples corresponding thereto (hereinafter referred to as "chroma blocks"). The above coding tree scheme may be equally or separately applied to the luma block and chroma block of the current CU. Specifically, the luma block and chroma block in a CTU may be partitioned into the same block tree structure, and in this case, the tree structure is represented as SINGLE_TREE. Alternatively, the luma block and chroma block in a CTU may be partitioned into separate block tree structures, and in this case, the tree structure may be represented as DUAL_TREE. That is to say, when a CTU is split into a dual tree, the block tree structure for the luma block and the block tree structure for the chroma block may exist separately. In this case, the block tree structure for the luma block may be referred to as DUAL_TREE_LUMA, and the block tree structure for the chroma component may be referred to as DUAL_TREE_CHROMA. For P and B slices / tile groups, the luma block and chroma block in a CTU may be restricted to have the same coding tree structure. However, for I slices / tile groups, the luma block and chroma block may have separate block tree structures from each other. If a separate block tree structure is applied, the luma CTB may be partitioned into CUs based on a specific coding tree structure, and the chroma CTB may be partitioned into chroma CUs based on another coding tree structure. That is to say, this means that the CUs in the I slices / tile groups applying a separate block tree structure may include a coding tree block of the luma component or two coding tree blocks of the chroma components and the CUs of the P or B slices / tile groups may include blocks of three color components (one luma component and two chroma components).

[0130] Although the quadtree coding tree structure with nested multi-type trees has been described, the structure for partitioning CUs is not limited thereto. For example, the BT structure and TT structure may be interpreted as concepts included in a multi-partition tree (MPT) structure, and a CU may be interpreted as being partitioned by the QT structure and MPT structure. In an example of partitioning a CU by the QT structure and MPT structure, a syntax element (e.g., MPT_split_type) including information on how many blocks the leaf nodes of the QT structure are partitioned into and a syntax element (e.g., MPT_split_mode) including information on whether the leaf nodes of the QT structure are partitioned in the vertical direction or the horizontal direction may be signaled to determine the partitioning structure.

[0131] In another example, the CUs can be partitioned in a way different from the QT structure, the BT structure, or the TT structure. That is, different from partitioning a CU of a lower depth into 1 / 4 of a CU of a higher depth according to the QT structure, or partitioning a CU of a lower depth into 1 / 2 of a CU of a higher depth according to the BT structure, or partitioning a CU of a lower depth into 1 / 4 or 1 / 2 of a CU of a higher depth according to the TT structure, in some cases, a CU of a lower depth can be partitioned into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 of a CU of a higher depth, and the method of partitioning the CUs is not limited to this.

[0132] The quadtree coding block structure with multi-type trees can provide a very flexible block partitioning structure. Due to the partitioning types supported in the multi-type trees, in some cases, different partitioning patterns can potentially result in the same coding block structure. In the encoding device and the decoding device, by restricting the occurrence of such redundant partitioning patterns, the amount of data for the partitioning information can be reduced.

[0133] For example, Figure 8 shows the redundant splitting patterns that may occur in binary tree splitting and ternary tree splitting. As Figure 8 shown, the consecutive binary splits 810 and 820 in one direction at two levels have the same coding block structure as the binary split for the central partition after the ternary split. In this case, the binary tree splitting for the central blocks 830 and 840 of the ternary tree splitting can be prohibited. This prohibition applies to the CUs of all pictures. When this specific splitting is prohibited, the signaling of the corresponding syntax element can be modified by reflecting this prohibited situation, thereby reducing the number of bits for signaling the splitting. For example, as Figure 8 shown in the example shown, when the binary tree splitting for the central block of a CU is prohibited, the syntax element mtt_split_cu_binary_flag that specifies whether the split is a binary split or a ternary split is not signaled and its value can be derived by the decoding device as 0.

[0134] Overview of chroma format

[0135] Hereinafter, the chroma format will be described. An image can be encoded into encoded data including an array of luminance components (e.g., Y) and two arrays of chrominance components (e.g., Cb and Cr). For example, one pixel of the encoded image can include a luminance sample and a chrominance sample. The chroma format can be used to represent the configuration format of the luminance samples and the chrominance samples, and the chroma format can be referred to as the color format.

[0136] In an embodiment, an image may be encoded in various chroma formats, such as monochrome, 4:2:0, 4:2:2, or 4:4:4. In monochrome sampling, there may be a sample array, and the sample array may be a luminance array. In 4:2:0 sampling, there may be a luminance sample array and two chroma sample arrays, and each of the two chroma arrays may have a height equal to half of the height of the luminance array and a width equal to half of the width of the luminance array. In 4:2:2 sampling, there may be a luminance sample array and two chroma sample arrays, and each of the two chroma arrays may have a height equal to the height of the luminance array and a width equal to half of the width of the luminance array. In 4:4:4 sampling, there may be a luminance sample array and two chroma sample arrays, and each of the two chroma arrays may have a height and a width equal to the height and width of the luminance array.

[0137] Figure 9 is a view illustrating a relative position of a luminance sample and a chroma sample according to an embodiment in accordance with 4:2:0 sampling. Figure 10 is a view illustrating a relative position of a luminance sample and a chroma sample according to an embodiment in accordance with 4:2:2 sampling. Figure 11 is a view illustrating a relative position of a luminance sample and a chroma sample according to an embodiment in accordance with 4:4:4 sampling. As Figure 9 shown, in 4:2:0 sampling, the chroma samples may be located below their corresponding luminance samples. As Figure 10 shown, in 4:2:2 sampling, the chroma samples may be located to overlap with their corresponding luminance samples. As Figure 11 shown, in 4:4:4 sampling, both the luminance samples and the chroma samples may be located at overlapping positions.

[0138] The chroma format used in the encoding device and the decoding device may be determined in advance. Alternatively, the chroma format may be signaled from the encoding device to the decoding device for adaptive use in the encoding device and the decoding device. In an embodiment, the chroma format may be signaled based on at least one of chroma_format_idc or separate_colour_plane_flag. At least one of chroma_format_idc or separate_colour_plane_flag may be signaled through a higher-level syntax such as DPS, VPS, SPS, or PPS. For example, chroma_format_idc and separate_colour_plane_flag may be included in the Figure 12 SPS syntax shown in.

[0139] Meanwhile,Figure 13 An embodiment of chrominance format classification showing the signaling using chroma_format_idc and separate_colour_plane_flag is presented. chroma_format_idc may be information specifying the chrominance format applied to the encoded image. separate_colour_plane_flag may specify whether the color arrays are processed separately in a particular chrominance format. For example, a first value of chroma_format_idc (e.g., 0) may specify monochrome sampling. A second value of chroma_format_idc (e.g., 1) may specify 4:2:0 sampling. A third value of chroma_format_idc (e.g., 2) may specify 4:2:2 sampling. A fourth value of chroma_format_idc (e.g., 3) may specify 4:4:4 sampling.

[0140] In 4:4:4, the following may be applied based on the value of separate_colour_plane_flag. If the value of separate_colour_plane_flag is a first value (e.g., 0), each of the two chrominance arrays may have the same height and width as the luma array. In this case, the value of ChromaArrayType specifying the type of the chrominance sample array may be set equal to chroma_format_idc. If the value of separate_colour_plane_flag is a second value (e.g., 1), the luma, Cb, and Cr sample arrays may be processed separately and together with a monochrome sampled picture. In this case, ChromaArrayType may be set to 0.

[0141] Intra prediction on chroma blocks

[0142] When performing intra prediction on a current block, prediction of the luma component block (luma block) of the current block and prediction of the chrominance component block (chrominance block) of the current block may be performed. In this case, the intra prediction mode for the chrominance block may be set separately from the intra prediction mode for the luma block.

[0143] For example, an intra prediction mode for a chroma block can be specified based on intra chroma prediction mode information, and the intra chroma prediction mode information can be signaled in the form of an intra_chroma_pred_mode syntax element. For example, the intra chroma prediction mode information can represent one of a planar mode, a DC mode, a vertical mode, a horizontal mode, a derived mode (DM), and a cross-component linear model (CCLM) mode. Here, the planar mode can specify intra prediction mode #0, the DC mode can specify intra prediction mode #1, the vertical mode can specify intra prediction mode #26, and the horizontal mode can specify intra prediction mode #10. DM can also be referred to as the direct mode. CCLM can also be referred to as the linear model (LM). The CCLM mode can include at least one of L_CCLM, T_CCLM, and LT_CCLM.

[0144] Meanwhile, DM and CCLM are related intra prediction modes for predicting a chroma block using information about a luma block. DM can represent a mode in which the same intra prediction mode as that of the luma component is applied as the intra prediction mode for the chroma component. Additionally, CCLM can represent the following intra prediction mode. In the process of generating a prediction block for a chroma block, samples derived by subsampling the reconstructed samples of the luma block and then applying α and β, which are CCLM parameters, to the subsampled samples are used as the prediction samples of the chroma block.

[0145] CCLM (Cross-Component Linear Model) mode

[0146] As described above, the CCLM mode can be applied to a chroma block. The CCLM mode is an intra prediction mode that uses the correlation between a luma block and the chroma block corresponding to the luma block, and is performed by deriving a linear model based on neighboring samples of the luma block and neighboring samples of the chroma block. Furthermore, prediction samples of the chroma block can be derived based on the derived linear model and the reconstructed samples of the luma block.

[0147] Specifically, when the CCLM mode is applied to a current chroma block, parameters of the linear model can be derived based on neighboring samples for intra prediction of the current chroma block and neighboring samples for intra prediction of the current luma block. For example, the linear model of CCLM can be expressed based on the following equation.

[0148] [Equation 1]

[0149]

[0150] where, pred c (i,j) can represent the prediction sample at the (i, j) coordinates of the current chroma block in the current CU. rec L’(i, j) can represent the reconstructed sample of the (i, j) coordinate of the current luma block in the CU. For example, rec L ’(i, j) can represent the downsampled reconstructed sample of the current luma block. The linear model coefficients α and β can be signaled or derived from neighboring samples.

[0151] Minimum size limit of chroma blocks

[0152] If chroma blocks with very small sizes are frequently generated during the encoding and decoding of high-resolution images such as VVC, the encoding and decoding throughput may deteriorate. To improve the throughput, the splitting of the CU can be restricted so that chroma blocks with a specific size are not generated.

[0153] In an embodiment, to limit the generation of chroma blocks with a specific size, the minimum size of the chroma blocks can be set. In an embodiment, the encoding device and the decoding device can limit the size of the chroma blocks so that the chroma blocks include at least 16 chroma samples. For example, the splitting of the luma blocks or chroma blocks can be restricted so that chroma blocks with chroma sample sizes of 2x2, 2x4, or 4x2 are not generated during the encoding and decoding.

[0154] For example, to prevent the generation of chroma blocks (e.g., chroma sample units) with sizes of 2x2, 2x4, or 4x2, in the case of a dual-tree splitting structure, the quadtree splitting or binary tree splitting of chroma blocks with sizes of 2x8, 4x4, or 8x2 can be prohibited, and the ternary splitting of chroma blocks with sizes of 2x8, 2x16, 4x4, 4x8, 8x2, or 8x4 can be prohibited.

[0155] More specifically, when any of the following conditions is satisfied, the quadtree splitting for the current block can be restricted.

[0156] (Condition 1-1) The splitting structure of the current block is single-tree or dual-tree luma, and the size of the luma block is the minimum size capable of quadtree splitting

[0157] (Condition 1-2) The splitting structure of the current block is dual-tree chroma, and the size of the chroma block is the minimum size capable of quadtree splitting

[0158] (Condition 1-3) The splitting structure of the current block is dual-tree chroma, and the size of the chroma block is less than 4

[0159] (Condition 1-4) The splitting structure of the current block is dual-tree chroma, and the prediction mode of the current block is MODE_TYPE_INTRA

[0160] Alternatively, when any of the following conditions is satisfied, the binary tree splitting for the current block can be restricted.

[0161] (Condition 2-1) The segmentation structure of the current block is dual-tree chrominance, and the product of the width and height of the current block is equal to or less than 16 (e.g., the number of chrominance samples belonging to the current block is equal to or less than 16).

[0162] (Condition 2-2) The segmentation structure of the current block is dual-tree chrominance, and the prediction mode of the current block is MODE_TYPE_INTRA.

[0163] Alternatively, when any of the following conditions is satisfied, the ternary tree segmentation of the current block can be restricted.

[0164] (Condition 3-1) The segmentation structure of the current block is dual-tree chrominance, and the product of the width and height of the current block is equal to or less than 32 (e.g., the number of chrominance samples belonging to the current block is equal to or less than 32).

[0165] (Condition 3-2) The segmentation structure of the current block is dual-tree chrominance, and the prediction mode of the current block is MODE_TYPE_INTRA.

[0166] Transition from single-tree segmentation structure to double-tree segmentation structure

[0167] Meanwhile, in the case of a single-tree segmentation structure, a luminance block and a chrominance block can form a CU, and the luminance block and the chrominance block can be segmented into the same segmentation structure according to the segmentation of the CU. In this case, when segmenting the luminance block, the same method as the method for segmenting the luminance block can also be used to segment the chrominance block. In this case, it is possible to determine whether to segment the CU based on the size of the luminance block, and it is possible to determine the size of the corresponding chrominance block according to the size of the luminance block based on the color format.

[0168] Figure 14 is a view showing an embodiment of a luminance block and a chrominance block when the color format is 4:2:0. When the size of the luminance block is 16x8, the size of the chrominance block can be determined to be 8x4. When the 8x4 chrominance block is segmented into a vertical ternary tree, chrominance blocks of size 2x4 are generated. Therefore, it is possible to prohibit the segmentation of the 16x8 luminance block.

[0169] Even when additional segmentation of the chrominance block is not allowed, as in Figure 14As in the example of , additional splitting of luminance blocks may be allowed. In addition, even when splitting of chrominance blocks is not available, it may be necessary to perform additional splitting on splittable luminance blocks in order to increase the coding rate. In such a case, luminance blocks and chrominance blocks that have already been split into a single-tree segmentation structure may be additionally split into a dual-tree structure. For example, a single-tree segmentation structure may be binary-converted into a dual-tree segmentation structure for luminance blocks and a dual-tree segmentation structure for chrominance blocks. In such a dual-tree segmentation structure, splitting of luminance blocks and splitting of chrominance blocks may be performed independently. Therefore, splitting of luminance blocks may be additionally performed, and splitting of chrominance blocks may not be allowed.

[0170] Therefore, an embodiment of the syntax for changing a single-tree segmentation structure into a dual-tree segmentation structure is shown in Figures 15 to 17 FIG. Figures 15 to 17 FIG. is a view illustrating a syntax for a coding tree unit (CTU) according to an embodiment, which is divided into three drawings due to limitations of the drawings. Figures 15 to 17 The syntax of forms a continuous syntax. According to the syntax of Figures 15 to 17 , a decoding device according to an embodiment may determine the modeType of a CTU as a predetermined prediction mode type based on the value of the modeTypeCondition parameter. Here, modeTypeCondition may be information specifying the prediction mode characteristics of CUs belonging to the CTU. Determination of modeTypeCondition will be described later. modeType may specify available prediction modes of CUs belonging to the CTU determined according to modeTypeCondition. In an embodiment, modeType may have a value of MODE_TYPE_ALL specifying that all prediction modes such as intra prediction, IBC prediction, palette mode, and inter prediction are available, MODE_TYPE_INTRA specifying that only intra, IBC, and palette modes are available, and MODE_TYPE_INTER specifying that only the inter prediction mode is available.

[0171] For example, when modeTypeCondition is a second value (e.g., 1) (1520), the decoding device may determine modeType as the intra prediction mode type (MODE_TYPE_INTRA) (1530).

[0172] Meanwhile, when modeTypeCondition is the third value (e.g., 2) (1540), the decoding device can determine modeType according to the value of mode_constraint_flag. mode_constraint_flag can be obtained from the bitstream (1550). mode_constraint_flag can be a parameter specifying whether the CU in the CTU is encoded only in the inter prediction mode. For example, the first value of mode_constraint_flag (e.g., 0) can specify that the CU in the CTU is encoded only in the inter prediction mode. The second value of mode_constraint_flag (e.g., 1) can specify that the CU in the CTU cannot be encoded in the inter prediction mode. In an embodiment, when mode_constraint_flag has the second value (e.g., 1), the CU in the CTU can be encoded in the intra prediction mode or the IBC prediction mode.

[0173] In Figure 15 the embodiment, when the value of mode_constraint_flag is the first value (e.g., 0), modeType can be determined as the inter prediction mode type (e.g., MODE_TYPE_INTER), and when the value of mode_constraint_flag is the second value (e.g., 1), modeType can be determined as the intra prediction mode type (e.g., MODE_TYPE_INTRA) (1560).

[0174] Meanwhile, when modeTypeCondition is the first value (e.g., 0), the decoding device can determine modeType as the value of modeTypeCurr. For example, modeTypeCurr is the input value called by the CTU, and when the current CTU is the root CTU, the value of modeTypeCurr can be set to MODE_TYPE_ALL. When the current CTU is split from the upper-layer CTU, modeTypeCurr can be set to the modeType value of the upper-layer CTU.

[0175] Next, the decoding device can determine the splitting structure of the lower-layer CTU split from the current CTU according to modeType. For example, when modeType is MODE_TYPE_INTRA, the splitting structure of the lower-layer CTU can be determined as dual-tree luma (DUAL_TREE_LUMA). Meanwhile, when modeType is not MODE_TYPE_INTRA, the splitting structure of the lower-layer CTU can be determined as the splitting structure of the current CTU (1580).

[0176] The partitioning structure of the lower CTU determined as described above can be stored in the parameter treeType. In addition, when performing the partitioning of the lower CTU as shown in Figure 16 of 1610, it can be used as an input value specifying the partitioning structure of the lower CTU. Therefore, when modeType is MODE_TYPE_INTRA, the lower CTU may be partitioned into a dual-tree structure. Additionally, the above modeType can be used as an input value specifying the prediction mode type of the lower CTU.

[0177] More specifically, when the prediction mode type (e.g., modeType) of the blocks belonging to the lower CTU is MODE_TYPE_INTRA, the lower CTU called by the syntax of Figure 16 and the syntax of Figure 17 of 1710 may have a dual-tree partitioning structure for the luma blocks. Additionally, when the partitioning type (modeTypeCurr) of the current CTU is MODE_TYPE_ALL and the prediction mode type (modeType) of the lower CTU is MODE_TYPE_INTRA, the lower CTU called by the syntax of Figure 17 of 1730 may have a dual-tree partitioning structure (DUAL_TREE_CHROMA) for the chroma blocks. When the prediction mode type of the current CTU is MODE_TYPE_ALL or the prediction mode type of the lower CTU is MODE_TYPE_INTRA (1720), a CTU with a dual-tree partitioning structure for the chroma blocks can be generated (1730).

[0178] In the above description, modeTypeCondition can be determined as follows. In an embodiment, when any of the following conditions is true, modeTypeCondition can be determined as a first value (e.g., 0).

[0179] (Condition 4-1) The slice to which the current CTU belongs is an I slice and the CTU belonging to the corresponding slice is quad-tree partitioned into 64x64 luma sample CUs

[0180] (Condition 4-2) modeTypeCurr is not MODE_TYPE_ALL

[0181] Otherwise, when any of the following conditions is true, the value of modeTypeCondition can be determined as a second value (e.g., 1).

[0182] (Condition 5-1) The product of the width and height of the luma blocks belonging to the current CTU is 64, and the current CTU is quad-tree partitioned

[0183] (Condition 5-2) The product of the width and height of the luminance block belonging to the current CTU is 64, and the current block is subjected to a ternary vertical split or a ternary horizontal split

[0184] (Condition 5-3) The product of the width and height of the luminance block belonging to the current CTU is 32, and the current block is subjected to a binary vertical split or a binary horizontal split

[0185] Meanwhile, when all of the above conditions are not met and any of the following conditions is true, the value of modeTypeCondition can be determined to be the second value (e.g., 1) or the third value (e.g., 2). Whether the value of modeTypeCondition is the second value (e.g., 1) or the third value (e.g., 2) can depend on whether the slice to which the current CTU belongs is an I slice, and more specifically, the value of modeTypeCondition can be determined to be 1 + (slice_type != I? 1 : 0). Here, slice_type specifies the type of the slice to which the current CTU belongs, and can have the value I when the current slice is an I slice, and has a value other than I when the current slice is not an I slice.

[0186] (Condition 6-1) The product of the width and height of the luminance block belonging to the current CTU is 64, and the current block is subjected to a binary horizontal split or a binary vertical split

[0187] (Condition 6-2) The product of the width and height of the luminance block belonging to the current CTU is 128, and the current block is subjected to a ternary vertical split or a ternary horizontal split

[0188] Meanwhile, when all of the above conditions are not met, modeTypeCondition can be set to the first value (e.g., 0).

[0189] First modified embodiment

[0190] Although the sizes of the luminance blocks are the same, due to different color formats, the sizes of the corresponding chrominance blocks may be different. Therefore, even when the luminance block has a size of 16x8, splitting of the chrominance block can be allowed.

[0191] For example, Figure 18 is a view showing the luminance block and the chrominance block when the color format is 4:4:4. When the size of the luminance block is 16x8, the size of the chrominance block can be determined to be 16x8. Therefore, when the color format is 4:4:4, splitting of the luminance block with a size of 16x8 can be allowed.

[0192] However, in the above embodiments, since the value of modeTypeCondition is determined according to the size of the luminance block, even when the chrominance block can be additionally divided according to the color format, it is no longer allowed to be divided into a single-tree segmentation structure. Thus, in order to determine whether to divide the chrominance block according to the color format, modeTypeCondition can be determined as follows.

[0193] In an embodiment, when any one of the following conditions is true, the value of modeTypeCondition can be determined as a first value (e.g., 0).

[0194] (Condition 7-1) The slice to which the current CTU belongs is an I slice, and the CTU belonging to the corresponding slice is quadtree-divided into a 64x64 luminance sample CU

[0195] (Condition 7-2) modeTypeCurr is not MODE_TYPE_ALL

[0196] Otherwise, when any one of the following conditions is true, the value of modeTypeCondition can be determined as a second value (e.g., 1).

[0197] (Condition 8-1) The product of the unit width of the chrominance samples and the unit height of the chrominance samples of the chrominance block belonging to the current CTU is 16, and the current block is subjected to quadtree segmentation

[0198] (Condition 8-2) The product of the unit width of the chrominance samples and the unit height of the chrominance samples of the chrominance block belonging to the current CTU is 16, and the current block is subjected to ternary vertical segmentation or ternary horizontal segmentation

[0199] Here, the product of the unit width of the chrominance samples and the unit height of the chrominance samples of the chrominance block can be calculated by (cbWidth / subWidthC) * (cbHeight / subHeightC). Here, cbWidth can be the unit width of the luminance samples of the luminance block belonging to the current CTU, cbHeight can be the unit height of the luminance samples of the luminance block belonging to the current CTU, and subWidthC and subHeightC can be based on Figure 10 the height ratio and width ratio of the luminance block and the chrominance block shown in

[0200] Meanwhile, when all of the above conditions are not satisfied and any one of the following conditions is true, the value of modeTypeCondition can be determined as the second value (e.g., 1) or the third value (e.g., 2). Whether the value of modeTypeCondition is the second value (e.g., 1) or the third value (e.g., 2) can be determined according to whether the slice to which the current CTU belongs is an I slice. Specifically, the value of modeTypeCondition can be determined as 1 + (slice_type != I? 1 : 0).

[0201] (Condition 9-1) The product of the number of chroma samples per unit width and the number of chroma samples per unit height of the chroma block belonging to the current CTU is 16, and the current block is subjected to binary horizontal splitting or binary vertical splitting

[0202] For example, according to Condition 1, when the product of the width and height of the luma block belonging to the current CTU is 64 and the current block is subjected to binary horizontal splitting or binary vertical splitting, if the color format is 4:2:0, the above Condition 1 can be satisfied. However, when the color format is monochrome format, 4:2:2 format or 4:4:4 format, the above Condition 1 may not be satisfied.

[0203] (Condition 9-2) The product of the number of chroma samples per unit width and the number of chroma samples per unit height of the chroma block belonging to the current CTU is 32, and the current block is subjected to ternary vertical splitting or ternary horizontal splitting

[0204] For example, according to Condition 2, when the product of the width and height of the luma block belonging to the current CTU is 128 and the current block is subjected to ternary horizontal splitting or ternary vertical splitting, if the color format is 4:2:0, the above Condition 2 may be satisfied. However, when the color format is monochrome format, 4:2:2 format or 4:4:4 format, the above Condition 2 may not be satisfied.

[0205] Meanwhile, when all of the above conditions are satisfied, the value of modeTypeCondition can be set to the first value (e.g., 0).

[0206] Second modified embodiment

[0207] When the color format of the current picture is monochrome format or 4:4:4 color format and is independently encoded for each color plane (e.g., separate color planes), luma blocks and chroma blocks cannot exist in a CTU at the same time. Therefore, it is impossible to perform split constraints on chroma blocks based on the sample unit size of luma blocks to limit the generation of 2x2, 2x4, or 4x2 chroma sample unit size blocks for chroma blocks. Therefore, in this case, the method for deriving modeTypeCondition described above can be changed as follows.

[0208] In an embodiment, when any of the following conditions is true, the value of modeTypeCondition can be determined to be a first value (e.g., 0).

[0209] (Condition 10-1) The slice to which the current CTU belongs is an I slice, and the CTUs belonging to the corresponding slice are quadtree-divided into 64x64 luma sample CUs

[0210] (Condition 10-2) modeTypeCurr is not MODE_TYPE_ALL

[0211] (Condition 10-3) The color format of the image to which the current CTU belongs is 4:4:4 (e.g., the value of chroma_format_idc is the fourth value (e.g., 3))

[0212] Alternatively, Condition 10-3 can apply when the color format of the image to which the current CTU belongs is 4:4:4 and each color plane is independently encoded / decoded (e.g., the value of separate_colour_plane_flag specifying this is 1).

[0213] (Condition 10-4) The color format of the image to which the current CTU belongs is a monochrome format (e.g., the value of chroma_format_idc is the first value (e.g., 0))

[0214] Otherwise, when any of the following conditions is true, the value of modeTypeCondition can be determined to be a second value (e.g., 1).

[0215] (Condition 11-1) The product of the width and height of the luma block belonging to the current CTU is 64, and the current CTU is subjected to quadtree division

[0216] (Condition 11-2) The product of the width and height of the luma block belonging to the current CTU is 64, and the current block is subjected to ternary vertical division or ternary horizontal division

[0217] (Condition 11-3) The product of the width and height of the luma block belonging to the current CTU is 32, and the current block is subjected to binary vertical division or binary horizontal division

[0218] Meanwhile, when not all of the above conditions are met and any of the following conditions is true, the value of modeTypeCondition can be determined as a second value (e.g., 1) or a third value (e.g., 2). Whether the value of modeTypeCondition is the second value (e.g., 1) or the third value (e.g., 2) can be determined depending on whether the slice to which the current CTU belongs is an I slice. More specifically, the value of modeTypeCondition can be determined as 1 + (slice_type != I? 1 : 0). Here, slice_type specifies the type of the slice to which the current CTU belongs, and can have the value I when the current slice is an I slice, and have a value other than I when the current slice is not an I slice.

[0219] (Condition 12-1) The product of the width and height of the luma block belonging to the current CTU is 64, and the current block is subjected to binary horizontal splitting or binary vertical splitting

[0220] (Condition 12-2) The product of the width and height of the luma block belonging to the current CTU is 128, and the current block is subjected to ternary vertical splitting or ternary horizontal splitting

[0221] Meanwhile, when not all of the above conditions are satisfied, the value of modeTypeCondition can be set to a first value (e.g., 0).

[0222] Encoding method

[0223] Hereinafter, reference will be made to Figure 19 Describe a method of encoding performed by an encoding device using the above method. The encoding device according to an embodiment may include a memory and at least one processor, and the following method may be performed by the at least one processor.

[0224] First, the encoding device may obtain a current block by splitting an image (S1910). Next, the encoding device may determine prediction mode characteristic information (e.g., modeTypeCondition) based on the encoding information of the current block (S1920). Here, the prediction mode characteristic information may be determined based on the color format of the current block. For example, when the color format of the current block is determined to be a monochrome format or a 4:4:4 format, the prediction mode characteristic information may be determined as a first value (e.g., 0).

[0225] Alternatively, when the number of luma samples of the current block is 64, the splitting mode of the current block is a binary splitting mode, and the color format of the current block is a predetermined format, the prediction mode characteristic information may be determined as a first value (e.g., 0). In this case, the predetermined format may be a monochrome format or a 4:4:4 format.

[0226] Alternatively, when the number of luminance samples of the current block is 128, the splitting mode of the current block is a ternary splitting mode, and the color format of the current block is a predetermined format, the prediction mode characteristic information may be determined as a first value (e.g., 0). In this case, the predetermined format may be a monochrome format or a 4:4:4 format.

[0227] Meanwhile, when the color format of the current block is 4:2:0, the number of luminance samples of the current block is 64, and the splitting mode of the current block is a binary splitting mode, the prediction mode characteristic information may be determined based on whether the slice to which the current block belongs is an I slice. Alternatively, when the color format of the current block is 4:2:0 format, the number of luminance samples of the current block is 128, and the splitting mode of the current block is a ternary splitting mode, the prediction mode characteristic information may be determined based on whether the slice to which the current block belongs is an I slice.

[0228] For example, when the slice to which the current block belongs is an I slice, the prediction mode characteristic information may be determined as a second value (e.g., 1). Meanwhile, when the slice to which the current block belongs is not an I slice, the prediction mode characteristic information may be determined as a third value (e.g., 2).

[0229] Next, the encoding device may determine the prediction mode type (e.g., modeType) of the lower layer block split from the current block based on the prediction mode characteristic information (S1930).

[0230] As described above, when the prediction mode characteristic information is determined as the first value (e.g., 0), the prediction mode type of the lower layer block may be determined as the same prediction mode type, such as the prediction mode type of the current block.

[0231] For example, when the prediction mode type of the current block is MODE_TYPE_ALL that specifies all prediction modes available such as intra prediction, IBC prediction, palette mode, and inter prediction, the prediction mode type of the lower layer block may also be determined as MODE_TYPE_ALL. Therefore, when encoding is performed, the prediction mode of the lower layer block may be determined as any one of an intra prediction mode, an IBC prediction mode, a palette mode, and an inter prediction mode.

[0232] In addition, when the prediction mode characteristic information is the second value (e.g., 1), the prediction mode type of the lower layer block may be determined as an intra prediction mode type (e.g., MODE_TYPE_INTRA), where only the intra prediction mode is available as the prediction mode.

[0233] In addition, when the prediction mode characteristic information is the third value (e.g., 2), the prediction mode type of the lower layer block can be determined as either an inter prediction mode type in which only the inter prediction mode is available as the prediction mode (e.g., MODE_TYPE_INTER) or an intra prediction mode type in which only the intra prediction mode is available (e.g., MODE_TYPE_INTRA).

[0234] Next, the encoding device may obtain the lower layer block by splitting the current block (S1940). For example, the encoding device may split the current block based on the determined split structure of the lower layer block according to the prediction mode type of the lower layer block to obtain the lower layer block. For example, when the prediction mode type of the lower layer block is the intra prediction mode type, the encoding device may determine the split structure of the lower layer block as a binary tree split structure. Meanwhile, when the prediction mode of the lower layer block is not the intra prediction mode type, the encoding device may determine the split structure of the lower layer block as the same split structure as that of the current block. For example, when the split structure of the current block is a single tree split structure, the split structure of the lower layer block may be determined as a single tree split structure, and when the split structure of the current block is a binary tree split structure, the split structure of the lower layer block may be determined as a binary tree split structure.

[0235] When the split structure of the lower layer block is determined as a binary tree split structure, the split availability of the luminance block and the chrominance block of the current block can be determined independently. For example, the split availability of the luminance block can be determined based on the size of the luminance block. In addition, the split availability of the chrominance block can be determined based on the size of the chrominance block. In an embodiment, the split availability of the chrominance block can be determined based on the color format. More specifically, as described above with reference to Figure 10 To determine the size of the chrominance block, the size of the luminance block corresponding to the chrominance block and the color format of the chrominance block can be used.

[0236] Next, the encoding device may encode the lower layer block based on the determined prediction mode type of the lower layer block (S1950). For example, the encoding device may encode the current block by encoding the lower layer block according to any one of the intra, inter, IBC, and palette modes based on the prediction mode type of the lower layer block. For example, when the prediction mode type of the lower layer block is MODE_TYPE_ALL, the encoding device may determine the prediction mode of the lower layer block as any one of the intra, inter, IBC, and palette modes. In addition, the prediction information of the lower layer block can be encoded by encoding the pred_mode_flag parameter that specifies the prediction mode of the lower layer block.

[0237] Meanwhile, when the prediction mode characteristic information calculated for the current block above is the third value (e.g., 2), the mode constraint information (e.g., mode_constraint_flag) for specifying the prediction mode of the specified lower block can be encoded. The mode constraint information can specify whether the inter prediction mode is applied to the lower block. In an embodiment, when encoding the lower block in the inter prediction mode, the mode constraint information can be encoded as the first value (e.g., 0) specifying the use of the inter prediction mode. Additionally, in an embodiment, when the calculated prediction mode characteristic information above is the third value (e.g., 2) and the lower block is not encoded in the inter prediction mode, the intra prediction mode can be forced to be used as the prediction mode for encoding the lower block. In this case, in order to signal that the lower block has been encoded in the intra prediction mode instead of the inter prediction mode, the mode constraint information can be encoded as the second value (e.g., 1) specifying that the intra prediction mode has been used instead of the inter prediction mode.

[0238] Decoding method

[0239] Hereinafter, reference will be made to Figure 20 Describe a method of performing decoding by a decoding apparatus using the above method. The decoding apparatus according to an embodiment may include a memory and at least one processor, and the following method may be executed by the at least one processor.

[0240] First, the decoding apparatus may obtain a current block by dividing an image (S2010). Next, the decoding apparatus may determine prediction mode characteristic information (e.g., modeTypeCondition) based on the coding information of the current block (S2020). Here, the prediction mode characteristic information may be determined based on the color format of the current block. For example, when the color format of the current block is a monochrome format or a 4:4:4 format, the prediction mode characteristic information may be determined as the first value (e.g., 0).

[0241] Alternatively, when the number of luminance samples of the current block is 64, the segmentation mode of the current block is a binary segmentation mode, and the color format of the current block is a predetermined format, the prediction mode characteristic information may be determined as the first value (e.g., 0). In this case, the predetermined format may be a monochrome format or a 4:4:4 format.

[0242] Alternatively, when the number of luminance samples of the current block is 128, the segmentation mode of the current block is a ternary segmentation mode, and the color format of the current block is a predetermined format, the prediction mode characteristic information may be determined as the first value (e.g., 0). In this case, the predetermined format may be a monochrome format or a 4:4:4 format.

[0243] Meanwhile, when the color format of the current block is the 4:2:0 format, the number of luminance samples of the current block is 64, and the splitting mode of the current block is the binary splitting mode, the prediction mode characteristic information can be determined based on whether the slice to which the current block belongs is an I slice. Alternatively, when the color format of the current block is the 4:2:0 format, the number of luminance samples of the current block is 128, and the splitting mode of the current block is the ternary splitting mode, the prediction mode characteristic information can be determined based on whether the slice to which the current block belongs is an I slice.

[0244] For example, when the slice to which the current block belongs is an I slice, the prediction mode characteristic information can be determined as a second value (e.g., 1). Meanwhile, when the slice to which the current block belongs is not an I slice, the prediction mode characteristic information can be determined as a third value (e.g., 2).

[0245] Next, the decoding device can determine the prediction mode type (e.g., modeType) of the lower-layer block split from the current block based on the prediction mode characteristic information (S2030).

[0246] As described above, when the prediction mode characteristic information is determined as the first value (e.g., 0), the prediction mode type of the lower-layer block can be determined as the same prediction mode type as that of the current block. In addition, when the prediction mode characteristic information is the second value (e.g., 1), the prediction mode type of the lower-layer block can be determined as the intra prediction mode type (e.g., MODE_TYPE_INTRA).

[0247] In addition, when the prediction mode characteristic information is the third value (e.g., 2), the prediction mode type of the lower-layer block can be determined based on the mode constraint information (e.g., mode_constraint_flag) obtained from the bitstream. Hereinafter, the mode constraint information can specify whether the inter prediction mode is available. In an embodiment, when the mode constraint information specifies that the inter prediction mode is allowed, the prediction mode type of the lower-layer block can be determined as the inter prediction mode type (e.g., MODE_TYPE_INTER). Meanwhile, when the mode constraint information specifies that the inter prediction mode is not allowed, the prediction mode type of the lower-layer block can be determined as the intra prediction mode type (e.g., MODE_TYPE_INTRA).

[0248] Next, the decoding device can obtain lower-layer blocks by splitting the current block (S2040). For example, the decoding device can split the current block to obtain lower-layer blocks by determining the splitting structure of the lower-layer blocks based on the prediction mode type of the lower-layer blocks. For example, when the prediction mode type of the lower-layer blocks is the intra prediction mode type, the decoding device can determine the splitting structure of the lower-layer blocks as a binary tree splitting structure. Meanwhile, when the prediction mode type of the lower-layer blocks is not the intra prediction mode type, the decoding device can determine the splitting structure of the lower-layer blocks as the same splitting structure as that of the current block. For example, when the splitting structure of the current block is a single tree splitting structure, the splitting structure of the lower-layer blocks can be determined as a single tree splitting structure, and when the splitting structure of the current block is a binary tree splitting structure, the splitting structure of the lower-layer blocks can be determined as a binary tree splitting structure.

[0249] When the splitting structure of the lower-layer blocks is determined as a binary tree splitting structure, the splitting availability of the luminance block and the chrominance block of the current block can be determined independently. For example, the splitting availability of the luminance block can be determined based on the size of the luminance block. In addition, the splitting availability of the chrominance block can be determined based on the size of the chrominance block. In an embodiment, the splitting availability of the chrominance block can be determined based on the color format. More specifically, as described above with reference to Figure 10 what has been described, in order to determine the size of the chrominance block, the size of the luminance block corresponding to the chrominance block and the color format of the chrominance block can be used.

[0250] Next, the decoding device can decode the lower-layer blocks based on the determined prediction mode type of the lower-layer blocks (S2050). For example, the decoding device can determine the prediction mode of the lower-layer blocks as any one of intra, inter, IBC, and palette modes based on the prediction mode information obtained from the bitstream according to the prediction mode type of the lower-layer blocks, and decode the current block by decoding the lower-layer blocks based on the determined prediction mode. For example, only when the prediction mode type of the lower-layer blocks is MODE_TYPE_ALL, the decoding device can obtain the pred_mode_flag parameter specifying the prediction mode of the lower-layer blocks. In addition, the lower-layer blocks can be predicted according to the prediction mode specified by pred_mode_flag (for example, the inter prediction mode or the intra prediction mode).

[0251] Application embodiment

[0252] Although for clarity of description, the above-described exemplary methods of the present disclosure are represented as a series of operations, they are not intended to limit the order of execution of the steps, and these steps can be performed simultaneously or in a different order when necessary. To implement the method according to the present invention, the described steps may further include other steps, may include the remaining steps except for some steps, or may include other additional steps except for some steps.

[0253] In the present disclosure, an image encoding device or an image decoding device that performs a predetermined operation (step) may perform an operation (step) of confirming the execution conditions or circumstances of the corresponding operation (step). For example, if it is described that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or the image decoding device may perform the predetermined operation after determining whether the predetermined condition is satisfied.

[0254] The various embodiments of the present disclosure are not a list of all possible combinations and are intended to describe representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

[0255] The various embodiments of the present disclosure may be implemented in hardware, firmware, software, or a combination thereof. In the case where the present disclosure is implemented by hardware, the present disclosure may be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a general-purpose processor, a controller, a microcontroller, a microprocessor, etc.

[0256] In addition, an image decoding device and an image encoding device to which the embodiments of the present disclosure are applied may be included in a multimedia broadcast transmission and reception device, a mobile communication terminal, a home theater video device, a digital cinema video device, a surveillance camera, a video chat device, a real-time communication device such as video communication, a mobile streaming device, a storage medium, a camera, a video on demand (VoD) service providing device, an over the top video (OTT video) device, an Internet streaming service providing device, a three-dimensional (3D) video device, a video phone video device, a medical video device, etc., and may be used to process video signals or data signals. For example, an OTT video device may include a game console, a Blu-ray player, an Internet access TV, a home theater system, a smart phone, a tablet PC, a digital video recorder (DVR), etc.

[0257] Figure 21 is a view showing a content streaming system to which the embodiments of the present disclosure can be applied.

[0258] As Figure 21 shown, a content streaming system to which the embodiments of the present disclosure are applied may mainly include an encoding server, a streaming server, a network server, a media storage, a user device, and a multimedia input device.

[0259] The encoding server compresses the content input from a multimedia input device such as a smart phone, a camera, a portable video camera, etc. into digital data to generate a bitstream and sends the bitstream to the streaming server. As another example, when a multimedia input device such as a smart phone, a camera, a video camera, etc. directly generates a code stream, the encoding server may be omitted.

[0260] The bitstream may be generated by an image encoding method or an image encoding apparatus to which embodiments of the present disclosure are applied, and the streaming server may temporarily store the bitstream during the process of transmitting or receiving the bitstream.

[0261] The streaming server transmits multimedia data to the user device based on a request from the user via the network server, and the network server serves as a medium for notifying the user of the service. When the user requests a desired service from the network server, the network server may deliver it to the streaming server, and the streaming server may send the multimedia data to the user. In this case, the content streaming system may include a separate control server. In this case, the control server serves as controlling commands / responses between devices in the content streaming system.

[0262] The streaming server may receive content from a media storage and / or an encoding server. For example, when receiving content from the encoding server, the content may be received in real time. In this case, in order to provide a smooth streaming service, the streaming server may store the bitstream for a predetermined time.

[0263] Examples of the user device may include a mobile phone, a smartphone, a laptop computer, a digital broadcast terminal, a personal digital assistant (PDA), a portable multimedia player (PMP), a navigation device, a tablet PC, a tablet computer, a superbook, a wearable device (e.g., a smartwatch, smart glasses, a head-mounted display), a digital TV, a desktop computer, a digital signage, etc.

[0264] Each server in the content streaming system may operate as a distributed server, and in this case, the data received from each server may be distributed.

[0265] The scope of the present disclosure includes software or executable commands (e.g., an operating system, an application, firmware, a program, etc.) for enabling the operations of methods according to various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or commands stored thereon and executable on the device or the computer.

[0266] Industrial Applicability

[0267] Embodiments of the present disclosure may be used to encode or decode an image.

Claims

1. An image decoding device, comprising: a memory; and at least one processor connected to the memory, the at least one processor being configured to: obtain a current block by splitting an image; determine prediction mode characteristic information based on the coding information of the current block; determine the prediction mode type of a lower layer block split from the current block based on the prediction mode characteristic information; obtain the lower layer block by splitting the current block; and decode the lower layer block based on the determined prediction mode type of the lower layer block, wherein the lower layer block is decoded by performing prediction for the lower layer block, wherein the prediction mode characteristic information is determined based on the color format of the current block, and wherein based on the color format of the current block being a monochrome format or a 4:4:4 format, the prediction mode characteristic information is determined as a first value.

2. The image decoding device according to claim 1, Among them, obtain the lower layer block by determining the split structure of the lower layer block based on the prediction mode type of the lower layer block; and wherein based on the prediction mode type of the lower layer block being an intra prediction mode type, the split structure of the lower layer block is determined as a double-tree split structure.

3. The image decoding device according to claim 2, Among them, independently determine the split availability of the luminance block and the chrominance block for the current block based on the split structure of the lower layer block being determined as the double-tree split structure, and wherein the split availability of the chrominance block is determined based on the color format.

4. The image decoding device according to claim 1, wherein, Determine the prediction mode type of the lower layer block as the prediction mode type of the current block based on the prediction mode characteristic information being the first value.

5. The image decoding apparatus according to claim 1, wherein Based on the luminance sample number of the current block being 64, the split mode of the current block being a binary split mode, and the color format of the current block being a predetermined format, the prediction mode characteristic information is determined as the first value.

6. The image decoding apparatus according to claim 1, wherein, Based on the luminance sample number of the current block being 128, the split mode of the current block being a ternary split mode, and the color format of the current block being a predetermined format, the prediction mode characteristic information is determined as the first value.

7. The image decoding device according to claim 6, wherein, The predetermined format is a monochrome format or a 4:4:4 format.

8. The image decoding apparatus according to claim 1, wherein, Based on the color format of the current block being 4:2:0 format, the luminance sample number of the current block being 64, and the split mode of the current block being a binary split mode, determine the prediction mode characteristic information depending on whether the slice to which the current block belongs is an I slice.

9. The image decoding apparatus according to claim 1, wherein Based on the color format of the current block being 4:2:0 format, the luminance sample number of the current block being 128, and the split mode of the current block being a ternary split mode, determine the prediction mode characteristic information depending on whether the slice to which the current block belongs is an I slice.

10. The image decoding device according to claim 9, Among them, Based on the slice to which the current block belongs being an I slice, the prediction mode characteristic information is determined as a second value, and wherein based on the prediction mode characteristic information being the second value, the prediction mode type of the lower layer block is determined as an intra prediction mode type.

11. The image decoding device according to claim 9, Among them, Based on that the slice to which the current block belongs is not an I slice, the prediction mode characteristic information is determined to be a third value. Wherein, based on the prediction mode characteristic information being the third value, the prediction mode type of the lower layer block is determined based on the mode constraint information obtained from the bitstream. Wherein, the mode constraint information specifies whether an inter prediction mode is allowed, and wherein, based on the mode constraint information specifying that the inter prediction mode is allowed, the prediction mode type of the lower layer block is determined to be an inter prediction mode type.

12. An image encoding apparatus, comprising: a memory; and at least one processor connected to the memory, the at least one processor being configured to: obtain a current block by dividing an image; determine prediction mode characteristic information based on the encoding information of the current block; determine the prediction mode type of a lower layer block divided from the current block based on the prediction mode characteristic information; obtain the lower layer block by dividing the current block; and encode the lower layer block based on the determined prediction mode type of the lower layer block, wherein the lower layer block is encoded by performing prediction for the lower layer block, wherein the prediction mode characteristic information is determined based on the color format of the current block, and wherein, based on the color format of the current block being a monochrome format or a 4:4:4 format, the prediction mode characteristic information is determined to be a first value.

13. A device for transmitting a bitstream, the device comprising: at least one processor configured to obtain a bitstream generated by an image encoding method; and a transmitter configured to transmit data including the bitstream; wherein, the image encoding method comprises: obtain a current block by dividing an image; determine prediction mode characteristic information based on the encoding information of the current block; determine the prediction mode type of a lower layer block divided from the current block based on the prediction mode characteristic information; obtain the lower layer block by dividing the current block; and decode the lower layer block based on the determined prediction mode type of the lower layer block, wherein the lower layer block is encoded by performing prediction for the lower layer block, wherein the prediction mode characteristic information is determined based on the color format of the current block, and wherein, based on the color format of the current block being a monochrome format or a 4:4:4 format, the prediction mode characteristic information is determined to be a first value.

Images