An image encoding / decoding method, apparatus, and method for transmitting a bitstream that signals chroma component prediction information depending on whether or not palette mode is applied.

Abstract

To provide image encoding / decoding methods and devices for signaling chroma component prediction information according to whether a palette mode is applicable.SOLUTION: An image decoding method performed by an image decoding device includes the steps of: determining a current block by splitting an image; identifying whether a palette mode is applied for the current block based on a palette mode flag obtained from a bitstream; obtaining palette mode encoding information of the current block from the bitstream based on a tree type of the current block and whether the palette mode is applied for the current block; and obtaining chroma component prediction information of the current block from the bitstream, when the palette mode is not applied for the current block.SELECTED DRAWING: Figure 25

Description

Technical Field

[0001] The present disclosure relates to an image encoding / decoding method and apparatus, and more particularly, to an image encoding / decoding method, apparatus, and method for transmitting a bitstream generated by the image encoding method / apparatus of the present disclosure, which signals chroma component prediction information according to the presence or absence of application of a palette mode.

Background Art

[0002] Recently, the demand for high-resolution, high-quality images, such as HD (High Definition) images and UHD (Ultra High Definition) images, has been increasing in various fields. As the image data becomes higher in resolution and quality, the amount of information or bits to be transmitted relatively increases compared to conventional image data. The increase in the amount of information or bits to be transmitted brings about an increase in transmission costs and storage costs.

[0003] Therefore, there is a need for a highly efficient image compression technique for effectively transmitting, storing, and reproducing information of high-resolution, high-quality images.

Summary of the Invention

Problems to be Solved by the Invention

[0004] An object of the present disclosure is to provide an image encoding / decoding method and apparatus with improved encoding / decoding efficiency.

[0005] Another object of the present disclosure is to provide an image encoding / decoding method and apparatus that improves encoding / decoding efficiency by signaling chroma component prediction information according to the presence or absence of application of a palette mode.

[0006] Another object of the present disclosure is to provide a method for transmitting a bitstream generated by the image encoding method or apparatus according to the present disclosure.

[0007] Furthermore, this disclosure aims to provide a recording medium that stores a bitstream generated by the image encoding method or apparatus according to this disclosure.

[0008] Furthermore, this disclosure aims to provide a recording medium that stores a bitstream received by the image decoding device provided herein, decoded, and used for image restoration.

[0009] The technical problems that this disclosure seeks to solve are not limited to those described above, and other technical problems not mentioned above will be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Means for solving the problem]

[0010] An image decoding method performed by an image decoding apparatus according to one aspect of the present disclosure may include the steps of: dividing an image to determine the current block; identifying whether or not a palette mode is applied to the current block based on a palette mode flag obtained from a bitstream; obtaining palette mode coding information for the current block from a bitstream based on the tree type of the current block and whether or not a palette mode is applied to the current block; and, if a palette mode is not applied to the current block, obtaining chroma component prediction information for the current block from the bitstream.

[0011] Furthermore, an image decoding device according to one aspect of the present disclosure includes a memory and at least one processor, the at least one processor divides an image to determine the current block, identifies whether or not a palette mode is applied to the current block based on a palette mode flag obtained from a bitstream, obtains palette mode coding information for the current block from the bitstream based on the tree type of the current block and whether or not a palette mode is applied to the current block, and if a palette mode is not applied to the current block, obtains chroma component prediction information for the current block from the bitstream.

[0012] Furthermore, an image encoding method performed by an image encoding device according to one aspect of the present disclosure may include the steps of: dividing an image to determine the current block; determining the prediction mode of the current block; encoding a palette mode flag indicating whether or not the prediction mode of the current block is palette mode, based on whether or not the prediction mode of the current block is palette mode; encoding palette mode encoded information of the current block encoded in palette mode, based on the tree type of the current block and whether or not the prediction mode of the current block is palette mode; and, if the prediction mode of the current block is not palette mode, encoding chroma component prediction information of the current block.

[0013] Furthermore, a transmission method according to one aspect of the present disclosure can transmit a bitstream generated by an image encoding device or image encoding method of the present disclosure.

[0014] Furthermore, a computer-readable recording medium according to one aspect of the present disclosure can store a bitstream generated by an image encoding method or image encoding apparatus of the present disclosure.

[0015] The features described above, which are a brief summary of this disclosure, are merely illustrative examples of the detailed description of this disclosure described below and do not limit the scope of this disclosure. [Effects of the Invention]

[0016] According to this disclosure, an image encoding / decoding method and apparatus with improved encoding / decoding efficiency can be provided.

[0017] Furthermore, according to this disclosure, an image coding / decoding method and apparatus can be provided that can improve coding / decoding efficiency by signaling chroma component prediction information depending on whether or not palette mode is applied.

[0018] Furthermore, this disclosure provides a method for transmitting a bitstream generated by an image encoding method or apparatus according to this disclosure.

[0019] Furthermore, according to this disclosure, a recording medium storing a bitstream generated by the image encoding method or apparatus according to this disclosure can be provided.

[0020] Furthermore, according to this disclosure, a recording medium can be provided that stores a bitstream that is received by the image decoding device according to this disclosure, decoded, and used for image restoration.

[0021] The effects obtained from this disclosure are not limited to those described above, and other effects not mentioned above will be clearly understood by a person with ordinary skill in the art to which this disclosure pertains from the following description. [Brief explanation of the drawing]

[0022] [Figure 1] This figure schematically illustrates a video coding system to which the embodiments described herein can be applied. [Figure 2] This figure schematically shows an image encoding device to which the embodiments of this disclosure can be applied. [Figure 3] This figure schematically shows an image decoding apparatus to which the embodiments of this disclosure can be applied. [Figure 4]It is a diagram showing the divided structure of an image according to an embodiment. [Figure 5] It is a diagram showing an embodiment of the division type of blocks by a multi-type tree structure. [Figure 6] It is a diagram exemplifying a signaling mechanism of block division information in a quadtree with nested multi-type tree structure according to the present disclosure. [Figure 7] It is a diagram showing an embodiment in which a CTU is divided into multiple CUs. [Figure 8] It is a diagram showing an embodiment of a redundant division pattern. [Figure 9] It is a diagram showing a syntax for chroma format signaling according to an embodiment. [Figure 10] It is a diagram showing a chroma format classification table according to an embodiment. [Figure 11] It is a diagram showing horizontal scan and vertical scan according to an embodiment. [Figure 12] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 13] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 14] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 15] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 16] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 17] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 18] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 19] It is a diagram showing a syntax for palette mode according to an embodiment. [Figure 20]This figure shows the formulas for determining PredictorPaletteEntries and CurrentPaletteEntries according to one embodiment. [Figure 21] This figure shows the syntax of the coding unit according to a modified embodiment. [Figure 22] This is a flowchart illustrating a signaling method for predetermined chromatintra prediction information according to one embodiment. [Figure 23] This is a flowchart illustrating how a decoding device according to one embodiment acquires chroma prediction information. [Figure 24] This is a flowchart illustrating how an encoding device according to one embodiment encodes an image. [Figure 25] This is a flowchart illustrating how a decoding device according to one embodiment decodes an image. [Figure 26] This figure illustrates a content streaming system to which the embodiments of this disclosure can be applied. [Modes for carrying out the invention]

[0023] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings, so that they can be easily implemented by a person with ordinary skill in the art to which the present disclosure pertains. However, the present disclosure can be implemented in a variety of different forms and is not limited to the embodiments described herein.

[0024] In describing embodiments of this disclosure, if it is determined that a specific description of a known configuration or function would obscure the gist of this disclosure, such detailed description will be omitted. In the drawings, parts unrelated to the description of this disclosure will be omitted, and similar parts will be denoted by the same reference numerals.

[0025] In this disclosure, when one component is described as being “connected,” “joined,” or “linked” to another component, this can include not only direct connections but also indirect connections where another component exists between them. Furthermore, when one component is described as “containing” or “having” another component, this means, unless otherwise stated to the contrary, that it may include another component rather than excluding it.

[0026] In this disclosure, terms such as "first," "second," etc., are used solely for the purpose of distinguishing one component from another, and do not limit the order or importance of the components unless otherwise specified. Therefore, within the scope of this disclosure, the first component of one embodiment may be called the second component in another embodiment, and similarly, the second component of one embodiment may be called the first component in another embodiment.

[0027] In this disclosure, components that are distinguished from each other are used to clearly describe their respective characteristics and do not necessarily mean that the components are separate. In other words, multiple components may be integrated to constitute a single hardware or software unit, or a single component may be distributed to constitute multiple hardware or software units. Therefore, such integrated or distributed embodiments are also included in the scope of this disclosure, without needing to be specifically mentioned.

[0028] In this disclosure, the components described in various embodiments are not necessarily essential components, and some may be optional components. Therefore, embodiments consisting of a subset of the components described in one embodiment are also included in the scope of this disclosure. Furthermore, embodiments that include additional components in addition to the components described in various embodiments are also included in the scope of this disclosure.

[0029] This disclosure relates to the encoding and decoding of images, and the terms used in this disclosure may have their ordinary meanings in the art to which this disclosure pertains, unless otherwise defined herein.

[0030] In this disclosure, "picture" generally means a unit representing any one image within a specific time period, and "slice / tile" is an encoding unit that constitutes part of a picture, and a single picture can consist of one or more slices / tiles. Furthermore, a slice / tile may contain one or more CTUs (coding tree units).

[0031] In this disclosure, “pixel” or “pel” may mean the smallest unit that constitutes a picture (or image). The term “sample” may also be used as a counterpart to pixel. A sample may generally represent a pixel or a pixel value, or it may represent only the pixel / pixel value of the luma component, or only the pixel / pixel value of the chroma component.

[0032] In this disclosure, “unit” can refer to a basic unit of image processing. A unit may include at least one of a specific region of a picture and information associated with that region. A unit may be used interchangeably with terms such as “sample array,” “block,” or “area,” as it may be used. Generally, an M×N block may include a set (or array) of samples (or sample arrays) or transform coefficients consisting of M columns and N rows.

[0033] In this disclosure, “current block” can mean any one of the following: “current coding block,” “current coding unit,” “block to encode,” “block to decode,” or “block to process.” If prediction is performed, “current block” can mean “current prediction block” or “block to predict.” If transformation (inverse transformation) / quantization (inverse quantization) is performed, “current block” can mean “current transformation block” or “block to transform.” If filtering is performed, “current block” can mean “block to filter.”

[0034] Furthermore, in this disclosure, “current block” may mean “chroma block of the current block” unless there is an explicit mention of chroma block. “Chroma block of the current block” may be expressed explicitly as “chroma block” or “current chroma block,” including an explicit mention of chroma block.

[0035] In this disclosure, " / " and "," may be interpreted as "and / or." For example, "A / B" and "A, B" may be interpreted as "A and / or B." Also, "A / B / C" and "A, B, C" may mean "at least one of A, B and / or C."

[0036] In this disclosure, “or” may be interpreted as “and / or.” For example, “A or B” may mean 1) “A” only, 2) “B” only, or 3) “A and B.” Alternatively, in this disclosure, “or” may mean “additionally or alternatively.”

[0037] Overview of the video coding system

[0038] Figure 1 shows the video coding system according to this disclosure.

[0039] A video coding system according to one embodiment may include an encoding device 10 and a decoding device 20. The encoding device 10 can transmit encoded video and / or image information or data to the decoding device 20 via a digital storage medium or network in file or streaming format.

[0040] An encoding device 10 according to one embodiment may include a video source generation unit 11, an encoding unit 12, and a transmission unit 13. A decoding device 20 according to one embodiment may include a receiving unit 21, a decoding unit 22, and a rendering unit 23. The encoding unit 12 may be called a video / image encoding unit, and the decoding unit 22 may be called a video / image decoding unit. The transmission unit 13 may be included in the encoding unit 12. The receiving unit 21 may be included in the decoding unit 22. The rendering unit 23 may also include a display unit, which may be configured as a separate device or external component.

[0041] The video source generation unit 11 can acquire video / images through processes such as video / image capture, synthesis, or generation. The video source generation unit 11 may include a video / image capture device and / or a video / image generation device. The video / image capture device may include, for example, one or more cameras, or a video / image archive containing previously captured video / images. The video / image generation device may include, for example, a computer, tablet, and smartphone, and may generate video / images (electronically). For example, virtual video / images may be generated via a computer, in which case the video / image capture process may be replaced by a process in which the relevant data is generated.

[0042] The encoding unit 12 can encode the input video / image. The encoding unit 12 can perform a series of steps such as prediction, transformation, and quantization for compression and encoding efficiency. The encoding unit 12 can output the encoded data (encoded video / image information) in bitstream format.

[0043] The transmission unit 13 can transmit encoded video / image information or data, output in bitstream format, to the receiving unit 21 of the decoding device 20 via a digital storage medium or network in file or streaming format. The digital storage medium can include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. The transmission unit 13 may include elements for generating media files via a predetermined file format and elements for transmission via a broadcast / communication network. The receiving unit 21 can extract / receive the bitstream from the storage medium or network and transmit it to the decoding unit 22.

[0044] The decoding unit 22 can decode the video / image by performing a series of steps such as inverse quantization, inverse transform, and prediction, corresponding to the operation of the encoding unit 12.

[0045] The rendering unit 23 can render the decoded video / image. The rendered video / image can be displayed via the display unit.

[0046] Overview of Image Encoding Devices

[0047] Figure 2 is a schematic diagram showing an image encoding device to which the embodiments of this disclosure can be applied.

[0048] As shown in Figure 2, the image coding device 100 may include an image splitting unit 110, a subtraction unit 115, a transformation unit 120, a quantization unit 130, an inverse quantization unit 140, an inverse transformation unit 150, an addition unit 155, a filtering unit 160, a memory 170, an inter-prediction unit 180, an intra-prediction unit 185, and an entropy coding unit 190. The inter-prediction unit 180 and the intra-prediction unit 185 can together be called the "prediction unit". The transformation unit 120, the quantization unit 130, the inverse quantization unit 140, and the inverse transformation unit 150 may be included in a residual processing unit. The residual processing unit may further include a subtraction unit 115.

[0049] All or at least some of the multiple components constituting the image encoding device 100 can be implemented by a single hardware component (e.g., an encoder or processor) depending on the embodiment. Furthermore, the memory 170 may include a DPB (decoded picture buffer) and can be implemented by a digital storage medium.

[0050] The image splitting unit 110 can split an input image (or picture, frame) input to the image encoding device 100 into one or more processing units. For example, the processing units may be called coding units (CUs). Coding units can be obtained by recursively splitting a coding tree unit (CTU) or the largest coding unit (LCU) using a QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structure. For example, a single coding unit can be split into multiple coding units of deeper depth based on a quad-tree structure, a binary-tree structure and / or a ternary-tree structure. For the splitting of coding units, a quad-tree structure may be applied first, followed by a binary-tree structure and / or a ternary-tree structure. Based on the final coding unit that cannot be further split, the coding procedure according to this disclosure can be performed. The largest coding unit can be used as the final coding unit, or a lower-depth coding unit obtained by dividing the largest coding unit can be used as the final coding unit. Here, the coding procedure may include procedures such as prediction, transformation, and / or restoration, as described later. As another example, the processing units of the coding procedure may be prediction units (PU) or transformation units (TU). The prediction unit and the transformation unit may be divided or partitioned from the final coding unit, respectively. The prediction unit may be a unit of sample prediction, and the transformation unit may be a unit that derives transformation coefficients and / or a unit that derives a residual signal from transformation coefficients.

[0051] The prediction unit (inter-prediction unit 180 or intra-prediction unit 185) can make predictions for the block to be processed (current block) and generate a predicted block that includes prediction samples for the current block. The prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block or on a CU basis. The prediction unit can generate various information regarding the prediction of the current block and transmit it to the entropy coding unit 190. The prediction information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0052] The intra-prediction unit 185 can predict the current block by referring to a sample in the current picture. The referenced sample may be located in the vicinity (neighbor) or at a distance from the current block, according to the intra-prediction mode and / or intra-prediction technique. The intra-prediction mode may include multiple non-directional modes and multiple directional modes. The non-directional modes may include, for example, a DC mode and a Planar mode. The directional modes may include, for example, 33 directional prediction modes or 65 directional prediction modes, depending on the degree of detail of the prediction direction. However, this is merely an example, and more or fewer directional prediction modes may be used depending on the settings. The intra-prediction unit 185 may also determine the prediction mode to be applied to the current block using the prediction modes applied to the surrounding blocks.

[0053] The interprediction unit 180 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on the reference picture. In this case, in order to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in units of blocks, subblocks, or samples based on the correlation of motion information between the surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In the case of interprediction, the surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. The reference picture containing the reference block and the reference picture containing the temporal neighboring block may be the same or different from each other. The temporal neighboring block may be called a collocated reference block, collocated CU (colCU), etc. The reference picture containing the temporal neighboring block may be called a collocated picture (colPic). For example, the interpretation unit 180 can construct a motion information candidate list based on surrounding blocks and generate information indicating which candidate is used to derive the motion vector and / or reference picture index of the current block. Interpretation can be performed based on various prediction modes; for example, in skip mode and merge mode, the interpretation unit 180 can use the motion information of surrounding blocks as the motion information of the current block. In skip mode, unlike merge mode, the residual signal may not be transmitted.In motion vector prediction (MVP) mode, the motion vector of the surrounding block is used as the motion vector predictor, and the motion vector of the current block can be signaled by encoding the motion vector difference and an indicator for the motion vector predictor. The motion vector difference can represent the difference between the motion vector of the current block and the motion vector predictor.

[0054] The prediction unit can generate a prediction signal based on various prediction methods and / or techniques described later. For example, the prediction unit can apply intra-prediction or inter-prediction to predict the current block, and can also apply intra-prediction and inter-prediction simultaneously. A prediction method that applies intra-prediction and inter-prediction simultaneously to predict the current block can be called CIIP (combined inter and intra prediction). The prediction unit can also perform intra-block copy (IBC) to predict the current block. Intra-block copy can be used for content image / video coding such as in games, for example, in SCC (screen content coding). IBC is a method of predicting the current block using a reference block that has already been restored in the current picture at a predetermined distance from the current block. When IBC is applied, the position of the reference block in the current picture can be encoded as a vector (block vector) corresponding to the predetermined distance. IBC basically performs prediction within the current picture, but can be performed similarly to inter-prediction in that it derives the reference block within the current picture. In other words, IBC can use at least one of the interpretation techniques described in this disclosure.

[0055] The predicted signal generated by the prediction unit can be used to generate a reconstructed signal or a residual signal. The subtraction unit 115 can generate a residual signal (residual block, residual sample array) by subtracting the predicted signal output from the prediction unit (predicted block, predicted sample array) from the input image signal (original block, original sample array). The generated residual signal can be transmitted to the conversion unit 120.

[0056] The transformation unit 120 can generate transformation coefficients by applying transformation techniques to the residual signal. For example, the transformation techniques may include at least one of the following: DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), KLT (Karhunen-Loeve Transform), GBT (Graph-Based Transform), or CNT (Conditionally Non-linear Transform). Here, GBT refers to a transformation obtained from a graph, where the relationship information between pixels is represented by a graph. CNT refers to a transformation obtained by generating a prediction signal using all previously reconstructed pixels. The transformation process can be applied to pixel blocks of the same size and square shape, or to non-square, variable-sized blocks.

[0057] The quantization unit 130 can quantize the conversion coefficients and transmit them to the entropy coding unit 190. The entropy coding unit 190 can encode the quantized signal (information about the quantized conversion coefficients) and output it in bitstream format. The information about the quantized conversion coefficients can be called residual information. The quantization unit 130 can rearrange the block-form quantized conversion coefficients into a one-dimensional vector format based on the coefficient scan order, and can also generate information about the quantized conversion coefficients based on the one-dimensional vector format of the quantized conversion coefficients.

[0058] The entropy coding unit 190 can perform various coding methods, such as exponential Golomb, CAVLC (context-adaptive variable length coding), and CABAC (context-adaptive binary arithmetic coding). In addition to the quantized conversion coefficients, the entropy coding unit 190 can also encode information necessary for video / image restoration (e.g., the values ​​of syntax elements) together or separately. The encoded information (e.g., encoded video / image information) can be transmitted or stored in bitstream format in units of NAL (network abstraction layer) units. The video / image information may further include information about various parameter sets, such as adaptive parameter sets (APS), picture parameter sets (PPS), sequence parameter sets (SPS), or video parameter sets (VPS). The video / image information may also further include general constraint information. The signaling information, transmitted information and / or syntax elements referred to in this disclosure may be encoded via the encoding procedure described above and included in the bitstream.

[0059] The bitstream can be transmitted over a network or stored on a digital storage medium. Here, the network may include broadcast networks and / or communication networks, and the digital storage medium may include various storage media such as USB, SD, CD, DVD, Blu-ray®, HDD, and SSD. A transmission unit (not shown) for transmitting the signal output from the entropy encoding unit 190 and / or a storage unit (not shown) for storing it may be provided as internal / external elements of the image encoding device 100, or the transmission unit may be provided as a component of the entropy encoding unit 190.

[0060] The quantized conversion coefficients output from the quantization unit 130 can be used to generate a residual signal. For example, by applying inverse quantization and inverse transformation to the quantized conversion coefficients via the inverse quantization unit 140 and the inverse transformation unit 150, a residual signal (residual block or residual sample) can be reconstructed.

[0061] The adder 155 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the reconstructed residual signal to the prediction signal output from the inter-prediction unit 180 or the intra-prediction unit 185. If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The adder 155 may be called the reconstruction unit or the reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0062] The filtering unit 160 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 160 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 170, specifically in the DPB of the memory 170. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter. The filtering unit 160 can generate various filtering-related information, as will be described later in the explanation of each filtering method, and transmit it to the entropy coding unit 190. The filtering-related information can be encoded by the entropy coding unit 190 and output in bitstream format.

[0063] The corrected restored picture transmitted to memory 170 can be used as a reference picture in the interpretation unit 180. When interpretation is applied via this, the image encoding device 100 can avoid prediction mismatches between the image encoding device 100 and the image decoding device, and can also improve encoding efficiency.

[0064] The DPB in memory 170 can store the modified restored picture for use as a reference picture in the inter-prediction unit 180. Memory 170 can store motion information of blocks from which motion information in the current picture has been derived (or encoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 180 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 170 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 185.

[0065] Overview of the image decoding device

[0066] Figure 3 is a schematic diagram showing an image decoding apparatus to which the embodiments of this disclosure can be applied.

[0067] As shown in Figure 3, the image decoding device 200 can be configured to include an entropy decoding unit 210, an inverse quantization unit 220, an inverse transform unit 230, an additive unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265. The inter-prediction unit 260 and the intra-prediction unit 265 can together be called the "prediction unit". The inverse quantization unit 220 and the inverse transform unit 230 can be included in the residual processing unit.

[0068] All or at least some of the multiple components constituting the image decoding device 200 can be implemented by a single hardware component (e.g., a decoder or processor) according to the embodiment. Furthermore, the memory 170 may include a DPB and can be implemented by a digital storage medium.

[0069] An image decoding device 200, upon receiving a bitstream containing video / image information, can restore the image by executing a process corresponding to the process performed in the image encoding device 100 in Figure 1. For example, the image decoding device 200 can perform decoding using the processing unit applied in the image encoding device. Therefore, the decoding processing unit can be, for example, a coding unit. The coding unit can be obtained by dividing a coding tree unit or a maximum coding unit. The restored image signal decoded and output via the image decoding device 200 can then be reproduced via a playback device (not shown).

[0070] The image decoding device 200 can receive the signal output from the image encoding device 2 in bitstream format. The received signal can be decoded via the entropy decoding unit 210. For example, the entropy decoding unit 210 can parse the bitstream to derive information necessary for image restoration (or picture restoration) (e.g., video / image information). The video / image information may further include information about various parameter sets, such as adaptive parameter set (APS), picture parameter set (PPS), sequence parameter set (SPS), or video parameter set (VPS). The video / image information may also further include general constraint information. The image decoding device may further use the parameter set information and / or the general constraint information to decode the image. The signaling information, received information, and / or syntax elements referred to in this disclosure can be obtained from the bitstream by decoding via the decoding procedure. For example, the entropy decoding unit 210 can decode information in the bitstream based on a coding method such as exponential Golomb coding, CAVLC, or CABAC, and output the values ​​of syntax elements necessary for image reconstruction and the quantized values ​​of conversion coefficients related to the residual. More specifically, the CABAC entropy decoding method receives bins corresponding to each syntax element from the bitstream, determines a context model using the syntax element information to be decoded, the decoding information of the surrounding blocks and the blocks to be decoded, or the symbol / bin information decoded in a previous step, predicts the probability of bin occurrence based on the determined context model, and performs arithmetic decoding of the bins to generate symbols corresponding to the values ​​of each syntax element. At this time, after determining the context model, the CABAC entropy decoding method can update the context model using the decoded symbol / bin information for the context model of the next symbol / bin.Of the information decoded by the entropy decoding unit 210, information related to prediction is provided to the prediction unit (inter-prediction unit 260 and intra-prediction unit 265), and the residual values ​​that have undergone entropy decoding in the entropy decoding unit 210, i.e., quantized conversion coefficients and related parameter information, can be input to the inverse quantization unit 220. In addition, of the information decoded by the entropy decoding unit 210, information related to filtering can be provided to the filtering unit 240. On the other hand, a receiving unit (not shown) that receives signals output from the image coding device may be further provided as an internal / external element of the image decoding device 200, or the receiving unit may be provided as a component of the entropy decoding unit 210.

[0071] On the other hand, the image decoding device according to this disclosure may be called a video / image / picture decoding device. The image decoding device may also include an information decoder (video / image / picture information decoder) and / or a sample decoder (video / image / picture sample decoder). The information decoder may include an entropy decoding unit 210, and the sample decoder may include at least one of an inverse quantization unit 220, an inverse transform unit 230, an adder unit 235, a filtering unit 240, a memory 250, an inter-prediction unit 260, and an intra-prediction unit 265.

[0072] The inverse quantization unit 220 can inverse quantize the quantized transformation coefficients and output the transformation coefficients. The inverse quantization unit 220 can rearrange the quantized transformation coefficients in a two-dimensional block format. In this case, the rearrangement can be performed based on the coefficient scan order performed by the image encoding device. The inverse quantization unit 220 can perform inverse quantization on the quantized transformation coefficients using quantization parameters (e.g., quantization step size information) to obtain the transformation coefficients.

[0073] The inverse conversion unit 230 can inversely convert the conversion coefficients to obtain residual signals (residual blocks, residual sample arrays).

[0074] The prediction unit can make predictions for the current block and generate a predicted block containing prediction samples for the current block. Based on the prediction information output from the entropy decoding unit 210, the prediction unit can determine whether intra-prediction or inter-prediction is applied to the current block and can determine a specific intra / inter-prediction mode (prediction technique).

[0075] As described in the explanation of the prediction unit of the image coding device 100, the prediction unit can generate prediction signals based on various prediction methods (techniques) described later.

[0076] The intra-prediction unit 265 can predict the current block by referring to the samples in the current picture. The description of the intra-prediction unit 185 can also be applied to the intra-prediction unit 265.

[0077] The interprediction unit 260 can derive a predicted block relative to the current block based on a reference block (reference sample array) identified by motion vectors on a reference picture. In this case, to reduce the amount of motion information transmitted in interprediction mode, motion information can be predicted in block, sub-block, or sample units based on the correlation of motion information between surrounding blocks and the current block. The motion information may include motion vectors and reference picture indices. The motion information may further include interprediction direction information (L0 prediction, L1 prediction, Bi prediction, etc.). In interprediction, surrounding blocks may include spatial neighboring blocks present in the current picture and temporal neighboring blocks present in the reference picture. For example, the interprediction unit 260 can construct a motion information candidate list based on surrounding blocks and derive the motion vector and / or reference picture index of the current block based on the received candidate selection information. Interprediction can be performed based on various prediction modes (techniques), and the prediction information may include information indicating the mode (technique) of interprediction for the current block.

[0078] The adder 235 can generate a reconstructed signal (reconstructed picture, reconstructed block, reconstructed sample array) by adding the acquired residual signal to the predicted signal (predicted block, predicted sample array) output from the prediction unit (including the inter-prediction unit 260 and / or intra-prediction unit 265). If there is no residual for the block to be processed, such as when skip mode is applied, the predicted block can be used as the reconstructed block. The description of the adder 155 can also be applied to the adder 235. The adder 235 can be called the reconstruction unit or reconstructed block generation unit. The generated reconstructed signal can be used for intra-prediction of the next block to be processed in the current picture, or, as described later, for inter-prediction of the next picture after filtering.

[0079] The filtering unit 240 can improve subjective / objective image quality by applying filtering to the restored signal. For example, the filtering unit 240 can apply various filtering methods to the restored picture to generate a modified restored picture, and the modified restored picture can be stored in the memory 250, specifically in the DPB of the memory 250. The various filtering methods can include, for example, deblocking filtering, sample adaptive offset, adaptive loop filter, and bilateral filter.

[0080] The restored picture stored (modified) in the DPB of memory 250 can be used as a reference picture in the inter-prediction unit 260. Memory 250 can store motion information of blocks from which motion information in the current picture has been derived (or decoded) and / or motion information of blocks in the picture that have already been restored. The stored motion information can be transmitted to the inter-prediction unit 260 for use as motion information of spatially surrounding blocks or motion information of temporally surrounding blocks. Memory 250 can store restored samples of restored blocks in the current picture and transmit them to the intra-prediction unit 265.

[0081] In this specification, the embodiments described for the filtering unit 160, inter-prediction unit 180, and intra-prediction unit 185 of the image coding device 100 can be applied similarly or in a corresponding manner to the filtering unit 240, inter-prediction unit 260, and intra-prediction unit 265 of the image decoding device 200, respectively.

[0082] Overview of image segmentation

[0083] The video / image coding method according to this disclosure can be performed based on the following image segmentation structure. Specifically, procedures such as prediction, residual processing (inverse transformation, inverse quantization, etc.), syntax element coding, and filtering, described later, can be performed based on CTU, CU (and / or TU, PU) derived from the image segmentation structure. The image can be segmented into blocks, and the block segmentation procedure can be performed in the image segmentation unit 110 of the encoding device described above. Segmentation-related information can be encoded in the entropy encoding unit 190 and transmitted to the decoding device in bitstream format. The entropy decoding unit 210 of the decoding device can derive the block segmentation structure of the current picture based on the segmentation-related information obtained from the bitstream, and perform a series of procedures for image decoding (e.g., prediction, residual processing, block / picture reconstruction, in-loop filtering, etc.) based on this.

[0084] A picture can be divided into a sequence of coding tree units (CTUs). Figure 4 shows an example of a picture being divided into CTUs. A CTU can correspond to a coding tree block (CTB). Alternatively, a CTU can contain two coding tree blocks: one for a luma sample and one for a corresponding chroma sample. For example, for a picture containing three sample arrays, the CTU can contain an N×N block for the luma sample and two corresponding blocks for the chroma sample.

[0085] Overview of CTU division

[0086] As mentioned above, coding units can be obtained by recursively partitioning a coding tree unit (CTU) or maximum coding unit (LCU) using QT / BT / TT (Quad-tree / binary-tree / ternary-tree) structures. For example, a CTU can first be partitioned into a quadtree structure. Then, the leaf nodes of the quadtree structure can be further partitioned into a multi-type tree structure.

[0087] A quadtree partition means dividing the current CU (or CTU) into four equal parts. Through a quadtree partition, the current CU can be divided into four CUs of the same width and height. If the current CU is not further divided into a quadtree structure, it corresponds to a leaf node in the quadtree structure. A CU that corresponds to a leaf node in a quadtree structure is not further divided and can be used as the final coding unit as described above. Alternatively, a CU that corresponds to a leaf node in a quadtree structure can be further divided by a multi-type tree structure.

[0088] Figure 5 shows the types of block partitioning using a multi-type tree structure. Partitioning using a multi-type tree structure can include two partitions using a binary tree structure and two partitions using a ternary tree structure.

[0089] The two types of partitioning using a binary tree structure include vertical binary splitting (SPLIT_BT_VER) and horizontal binary splitting (SPLIT_BT_HOR). Vertical binary splitting (SPLIT_BT_VER) means splitting the current CU vertically into two equal parts. As shown in Figure 4, vertical binary splitting can generate two CUs that have the same height as the current CU and half the width of the current CU. Horizontal binary splitting (SPLIT_BT_HOR) means splitting the current CU horizontally into two equal parts. As shown in Figure 5, horizontal binary splitting can generate two CUs that have half the height of the current CU and the same width as the current CU.

[0090] Two types of partitioning using a ternary structure are vertical ternary splitting (SPLIT_TT_VER) and horizontal ternary splitting (SPLIT_TT_HOR). Vertical ternary splitting (SPLIT_TT_VER) divides the current CU vertically in a 1:2:1 ratio. As shown in Figure 5, vertical ternary splitting can produce two CUs with the same height as the current CU and a width of 1 / 4 of the current CU's width, and one CU with the same height as the current CU and a width of half the current CU's width. Horizontal ternary splitting (SPLIT_TT_HOR) divides the current CU horizontally in a 1:2:1 ratio. As shown in Figure 4, horizontal ternary splitting can produce two CUs with the same height as the current CU and a width of 1 / 4 of the current CU's width, and one CU with the same height as the current CU and a width of half the current CU's width.

[0091] Figure 6 illustrates the signaling mechanism for block partitioning information in a quadtree with nested multi-type tree structure according to this disclosure.

[0092] Here, the CTU is treated as the root node of the quadtree, and the CTU is the first node to be split into a quadtree structure. Information (e.g., qt_split_flag) indicating whether or not to split the quadtree can be signaled to the current CU (CTU or quadtree node (QT_node)). For example, if qt_split_flag is the first value (e.g., "1"), the current CU can be split into a quadtree. If qt_split_flag is the second value (e.g., "0"), the current CU will not be split into a quadtree and will become a leaf node (QT_leaf_node) of the quadtree. Each leaf node of the quadtree can subsequently be further split into a multitype tree structure. In other words, a leaf node of a quadtree can become a node (MTT_node) of a multitype tree. In a multi-type tree structure, a first flag (e.g., mtt_split_cu_flag) can be signaled to indicate whether the current node will be further split. If the node is to be further split (e.g., the first flag is 1), a second flag (e.g., mtt_split_cu_verticla_flag) can be signaled to indicate the splitting direction. For example, if the second flag is 1, the splitting direction is vertical, and if the second flag is 0, the splitting direction is horizontal. Subsequently, a third flag (e.g., mtt_split_cu_binary_flag) can be signaled to indicate whether the splitting type is binary or ternary. For example, if the third flag is 1, the splitting type is binary, and if the third flag is 0, the splitting type is ternary. Nodes in a multitype tree obtained by binary partitioning or ternary partitioning can be further partitioned into a multitype tree structure. However, nodes in a multitype tree cannot be partitioned into a quadtree structure.If the first flag is 0, the corresponding node in the multitype tree is not further subdivided and becomes a leaf node (MTT_leaf_node) of the multitype tree. A CU corresponding to a leaf node in the multitype tree can be used as the final coding unit as described above.

[0093] Based on the aforementioned mtt_split_cu_vertical_flag and mtt_split_cu_binary_flag, the multi-type tree splitting mode (MttSplitMode) of the CU can be derived as shown in Table 1. In the following description, the multi-tree splitting mode may be abbreviated as multi-tree splitting type or splitting type.

[0094] [Table 1]

[0095] Figure 7 shows an example where a CTU is divided into multiple CUs by applying a multitype tree after a quadtree. In Figure 7, the bold block edge 710 represents the quadtree division, and the remaining edge 720 represents the multitype tree division. A CU can correspond to a coding lock (CB). In one embodiment, a CU may include two coding blocks: a coding block for a luma sample and a coding block for a chroma sample corresponding to the luma sample. The chroma component (sample) CB or TB size can be derived based on the luma component (sample) CB or TB size according to the component ratio of the picture / image color format (chroma format, e.g., 4:4:4, 4:2:2, 4:2:0, etc.). If the color format is 4:4:4, the chroma component CB / TB size can be set to be the same as the luma component CB / TB size. If the color format is 4:2:2, the width of the chroma component CB / TB can be set to half the width of the luma component CB / TB, and the height of the chroma component CB / TB can be set to the height of the luma component CB / TB. If the color format is 4:2:0, the width of the chroma component CB / TB can be set to half the width of the luma component CB / TB, and the height of the chroma component CB / TB can be set to half the height of the luma component CB / TB.

[0096] In one embodiment, when the size of the CTU is 128 based on the luma sample unit, the size of the CU can range from 128×128, which is the same size as the CTU, to 4×4. In one embodiment, when the color format is 4:2:0 (or chroma format), the chroma CB size can range from 64×64 to 2×2.

[0097] On the other hand, in one embodiment, the CU size and TU size can be the same. Alternatively, multiple TUs can exist within the CU region. The TU size generally refers to the Luma component (sample) TB (Transform Block) size.

[0098] The TU size can be derived based on a preset value, the maximum allowable TB size (maxTbSize). For example, if the CU size is larger than the maxTbSize, multiple TUs (TBs) with the maxTbSize can be derived from the CU, and conversion / inverse conversion can be performed in units of the TUs (TBs). For example, the maximum allowable lumen TB size may be 64×64, and the maximum allowable chromen TB size may be 32×32. If the width or height of a CB divided by the tree structure is larger than the maximum conversion width or height, the CB can be automatically (or implicitly) divided until the horizontal and vertical TB size limits are satisfied.

[0099] Furthermore, for example, when intra-prediction is applied, the intra-prediction mode / type is derived on a CU (or CB) basis, and the peripheral reference sample derivation and prediction sample generation procedures can be performed on a TU (or TB) basis. In this case, one or more TUs (or TBs) can exist within a single CU (or CB) region, and in this case, the multiple TUs (or TBs) can share the same intra-prediction mode / type.

[0100] On the other hand, for a quadtree coding tree scheme with multitype trees, the following parameters can be signaled from the encoder to the decoder as SPS syntax elements. For example, at least one of the following can be signaled: CTUsize, which indicates the size of the root node of the quadtree; MinQTSize, which indicates the minimum allowed size of the leaf nodes of the quadtree; MaxBTSize, which indicates the maximum allowed size of the root node of the binary tree; MaxTTSize, which indicates the maximum allowed size of the root node of the ternary tree; MaxMttDepth, which indicates the maximum allowed hierarchy depth of the multitype trees that are split from the leaf nodes of the quadtree; MinBtSize, which indicates the minimum allowed leaf node size of the binary tree; and MinTtSize, which indicates the minimum allowed leaf node size of the ternary tree.

[0101] In one embodiment using the 4:2:0 chroma format, the CTU size can be set to a 128x128 chroma block and two corresponding 64x64 chroma blocks. In this case, MinQTSize can be set to 16x16, MaxBtSize to 128x128, MaxTtSzie to 64x64, MinBtSize and MinTtSize to 4x4, and MaxMttDepth to 4. Quadritree partitioning can be applied to the CTU to generate leaf nodes of the quadritree. Leaf nodes of the quadritree can be called leaf QT nodes. Leaf nodes of the quadritree can have a size of 16x16 (e.g., the MinQTSize) to 128x128 (e.g., the CTU size). If a leaf QT node is 128x128, it may not be further divided into a binary / ternary tree. This is because even if partitioned in this case, it would exceed MaxBtsize and MaxTtszie (e.g., 64x64). Otherwise, a leaf QT node can be further partitioned into a multitype tree. Thus, a leaf QT node is the root node for a multitype tree, and a leaf QT node can have a multitype tree depth (mttDepth) value of 0. If the multitype tree depth reaches MaxMttdepth (e.g., 4), further additional partitioning may not be considered. If the width of a multitype tree node is the same as MinBtSize and equal to or less than 2xMinTtSize, further additional horizontal partitioning may not be considered. If the height of a multitype tree node is the same as MinBtSize and equal to or less than 2xMinTtSize, further additional vertical partitioning may not be considered. When partitioning is not considered in this way, the encoding device can omit signaling of partitioning information. In such cases, the decoding device can induce the partitioning information to a predetermined value.

[0102] On the other hand, a single CTU can include a coding block for a luma sample (hereinafter referred to as a "luma block") and two coding blocks for corresponding chroma samples (hereinafter referred to as "chroma blocks"). The coding tree scheme described above can be applied similarly to the luma blocks and chroma blocks of a CU, or it can be applied separately. Specifically, luma blocks and chroma blocks within a single CTU can be divided into the same block tree structure, in which case the tree structure can be represented as a single tree (SINGLE_TREE). Alternatively, luma blocks and chroma blocks within a single CTU can be divided into separate block tree structures, in which case the tree structure can be represented as a dual tree (DUAL_TREE). In other words, when a CTU is divided into a dual tree, the block tree structure for luma blocks and the block tree structure for chroma blocks can exist separately. In this case, the block tree structure for a luma block can be called a dual-tree luma (DUAL_TREE_LUMA), and the block tree structure for a chroma block can be called a dual-tree chroma (DUAL_TREE_CHROMA). For P and B slice / tile groups, luma blocks and chroma blocks within a single CTU can be restricted to having the same coding tree structure. However, for I slice / tile groups, luma blocks and chroma blocks can have separate block tree structures from each other. If separate block tree structures are applied, a luma CTB (Coding Tree Block) can be divided into CUs based on a specific coding tree structure, and a chroma CTB can be divided into chroma CUs based on a different coding tree structure. That is, a CU within an I slice / tile group to which a separate block tree structure is applied can consist of a coding block for a luma component or a coding block for two chroma components, while a CU in a P or B slice / tile group can consist of a block for three color components (a luma component and two chroma components).

[0103] In the above, a quadtree coding tree structure with a multitype tree was described, but the structures in which a CU is split are not limited to this. For example, BT structures and TT structures can be interpreted as concepts included in multiple partitioning tree (MPT) structures, and a CU can be interpreted as being split by QT structures and MPT structures. In one example of a CU being split by QT and MPT structures, the split structure can be determined by signaling a syntax element (e.g., MPT_split_type) containing information about how the leaf nodes of the QT structure are split into several blocks, and a syntax element (e.g., MPT_split_mode) containing information about whether the leaf nodes of the QT structure are split vertically or horizontally.

[0104] In another example, the CU can be divided in a way different from the QT, BT, or TT structures. That is, unlike the QT structure which divides the lower-depth CU into quarters the size of the upper-depth CU, or the BT structure which divides the lower-depth CU into half the size of the upper-depth CU, or the TT structure which divides the lower-depth CU into quarters or half the size of the upper-depth CU, the lower-depth CU can, depending on the case, be divided into 1 / 5, 1 / 3, 3 / 8, 3 / 5, 2 / 3, or 5 / 8 the size of the upper-depth CU, and the way in which the CU is divided is not limited to this.

[0105] Thus, the quadtree coding block structure with the multitype tree can provide a highly flexible block partition structure. On the other hand, due to the partition types supported by the multitype tree, different partition patterns may, in some cases, lead to potentially identical coding block structures. By limiting the occurrence of such redundant partition patterns, the encoding and decoding devices can reduce the amount of data in the partition information.

[0106] For example, Figure 8 illustrates redundant partition patterns that can occur in binary and ternary tree partitions. As shown in Figure 8, a 2-step level unidirectional consecutive binary partition 810 and 820 has the same coding block structure as a binary partition on the center partition after a ternary partition. In such a case, a binary tree partition on the center blocks 830 and 840 of the ternary partition can be prohibited. Such prohibitions can be applied to the CU of all pictures. When such a particular partition is prohibited, the signaling of the corresponding syntax element can be modified to reflect this prohibition, thereby reducing the number of bits signaled for the partition. For example, if a binary tree partition on the center block of a CU is prohibited, as in the example shown in Figure 8, the mtt_split_cu_binary_flag syntax element, which indicates whether the partition is a binary or ternary partition, is not signaled, and its value can be induced to 0 by the decoder.

[0107] Overview of Chroma Format

[0108] The following describes chroma formats. Images can be encoded with encoded data that includes a luma component (e.g., Y) array and two chroma component (e.g., Cb, Cr) arrays. For example, one pixel in an encoded image can contain a luma sample and a chroma sample. A chroma format can be used to indicate the configuration format of luma and chroma samples, and a chroma format is sometimes called a color format.

[0109] In one embodiment, the image can be encoded in various chroma formats such as monochrome, 4:2:0, 4:2:2, and 4:4:4. In monochrome sampling, there may be one sample array, which may be a luma array. In 4:2:0 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have half the height and half the width of the luma array. In 4:2:2 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have the same height as the luma array and half the width of the luma array. In 4:4:4 sampling, there may be one luma sample array and two chroma sample arrays, each of which may have the same height and width as the luma array.

[0110] For example, in 4:2:0 sampling, the chroma sample can be located at the lower end of its corresponding luma sample. In 4:2:2 sampling, the chroma sample can overlap the position of its corresponding luma sample. In 4:4:4 sampling, both the luma and chroma samples can overlap.

[0111] The chroma format used in the encoding and decoding devices can be predetermined. Alternatively, the chroma format can be signaled from the encoding device to the decoding device for adaptive use in the encoding and decoding devices. In one embodiment, the chroma format can be signaled based on at least one of chroma_format_idc and separate_colour_plane_flag. At least one of chroma_format_idc and separate_colour_plane_flag can be signaled via a higher-level syntax such as DPS, VPS, SPS, or PPS. For example, chroma_format_idc and separate_colour_plane_flag can be included in the SPS syntax as shown in Figure 9.

[0112] On the other hand, Figure 10 shows an example of chroma format classification utilizing the signaling of chroma_format_idc and separate_colour_plane_flag. chroma_format_idc can be information indicating the chroma format applied to the encoded image. separate_colour_plane_flag can indicate whether the color array is processed separately in a particular chroma format. For example, the first value of chroma_format_idc (e.g., 0) can indicate monochrome sampling. The second value of chroma_format_idc (e.g., 1) can indicate 4:2:0 sampling. The third value of chroma_format_idc (e.g., 2) can indicate 4:2:2 sampling. The fourth value of chroma_format_idc (e.g., 3) can indicate 4:4:4 sampling.

[0113] In 4:4:4 sampling, the following applies based on the value of separate_colour_plane_flag: If the value of separate_colour_plane_flag is the first value (e.g., 0), each of the two chroma arrays can have the same height and width as the luma array. In this case, the value of ChromaArrayType, which indicates the type of chroma sample array, can be set to be the same as chroma_format_idc. If the value of separate_colour_plane_flag is the second value (e.g., 1), the luma, Cb, and Cr sample arrays can be processed separately, similar to monochrome-sampled pictures. In this case, ChromaArrayType can be set to 0.

[0114] Intra prediction for chromablock

[0115] When intraprediction is performed on the current block, predictions can be made for the luma component block (luma block) and the chroma component block (chroma block) of the current block. In this case, the intraprediction mode for the chroma block can be set separately from the intraprediction mode for the luma block.

[0116] For example, the intra-prediction mode for a chroma block can be indicated based on intra-chroma prediction mode information, which can be signaled in the form of an intra_chroma_pred_mode syntax element. As an example, the intra-chroma prediction mode information can refer to one of the following: Planar mode, DC mode, vertical mode, horizontal mode, DM (Derived Mode), or CCLM (Cross-component linear model) mode. Here, Planar mode can refer to intra-prediction mode 0, DC mode to intra-prediction mode 1, vertical mode to intra-prediction mode 26, and horizontal mode to intra-prediction mode 10. DM can also be called direct mode. CCLM can also be called LM (linear model). CCLM mode can include one of L_CCLM, T_CCLM, or LT_CCLM.

[0117] On the other hand, DM and CCLM are dependent intra-prediction modes that predict the chroma block using information from the luma block. DM can indicate a mode where the same intra-prediction mode used for the luma component is applied as the intra-prediction mode for the chroma component. CCLM can indicate an intra-prediction mode where, in the process of generating a predicted block for the chroma block, the reconstructed sample of the luma block is subsampled, and then the sample derived by applying CCLM parameters α and β to the subsampled sample is used as the predicted sample for the chroma block.

[0118] CCLM (Cross-component linear model) mode

[0119] As mentioned above, the CCLM mode can be applied to chroma blocks. The CCLM mode is an intra-prediction mode that uses the correlation between a luma block and the corresponding chroma block, and is performed by deriving a linear model based on the peripheral samples of the luma block and the peripheral samples of the chroma block. Then, based on the derived linear model and the reconstructed samples of the luma block, the predicted samples of the chroma block can be derived.

[0120] Specifically, when the CCLM mode is applied to the current chroma block, parameters for the linear model can be derived based on the peripheral samples used for intra-prediction of the current chroma block and the peripheral samples used for intra-prediction of the current chroma block. For example, the linear model for CCLM can be expressed based on the following formula.

[0121]

number

[0122] Here, pred c (i,j) can represent a predicted sample of the (i,j) coordinates of the current chroma block within the current CU. L (i,j) can represent a reconstructed sample of the (i,j) coordinates of the current Luma block within the CU. For example, rec L '(i,j) can represent the down-sampled restored sample of the current Lumablock. The linear model coefficients α and β can be signaled, or they can be derived from the surrounding sample.

[0123] Overview of Palette Mode

[0124] The following describes the palette mode. An encoding device according to one embodiment can encode an image using the palette mode, and a decoding device can decode an image using the palette mode in a corresponding manner. The palette mode can also be called the palette encoding mode, intra-palette mode, or intra-palette encoding mode. The palette mode can be considered as a type of intra-coding mode, or as one of the intra-prediction methods. However, similar to the skip mode described above, a separate residual value for the block in question may not be signaled.

[0125] In one embodiment, palette mode can be used to improve encoding efficiency when encoding screen content, which is a computer-generated image containing a significant amount of text and graphics. Generally, local areas of images generated in screen content are separated by sharp edges and represented by a small number of colors. To take advantage of these characteristics, palette mode can represent a sample for a block with an index that points to a color entry in a palette table.

[0126] To apply palette mode, information can be signaled to the palette table. In one embodiment, the palette table may include index values ​​corresponding to each color. To signal the index values, palette index prediction information can be signaled. The palette index prediction information may include index values ​​for at least a portion of the palette index map. The palette index map can map pixels of video data to color indices in the palette table.

[0127] Palette index prediction information may include run value information. For at least a portion of the palette index map, the run value information may be information that associates run values ​​with index values. One run value may be associated with an escape color index. A palette index map can be generated from the palette index prediction information. For example, at least a portion of the palette index map can be generated by deciding whether or not to adjust the index values ​​of the palette index prediction information based on the last index value.

[0128] The current block in the current picture can be encoded or restored according to a palette index map. When palette mode is applied, pixel values ​​in the current encoding unit can be represented by a small set of representative color values. Such a set can be named a palette. For pixels with values ​​close to palette colors, a palette index can be signaled. For pixels with values ​​that do not belong to (are outside) the palette, the pixel is represented by an escape symbol, and the quantized pixel value can be directly signaled. In this specification, pixels or pixel values ​​can be illustrated by examples.

[0129] To decode a block encoded in palette mode, the decoder can decode the palette color and index. The palette color can be described as a palette table and encoded using a palette table coding tool. An escape flag can be signaled for each encoding unit. The escape flag can indicate whether or not an escape symbol exists in the current encoding unit. If an escape symbol exists, the palette table is incremented by one unit (e.g., an index unit), and the last index can be specified as escape mode. The palette indices of all pixels for a single encoding unit can form a palette index map and can be encoded using a palette index map coding tool.

[0130] For example, a palette predictor can be maintained to encode the palette table. The palette predictor can be initialized at each slice start point. For example, the palette predictor can be reset to 0. For each entry in the palette predictor, a reuse flag can be signaled to indicate whether or not it is currently part of the palette. The reuse flag can be signaled using run-length coding with a value of 0.

[0131] Subsequently, numbers for new palette entries can be signaled using zero-order exponential Golomb codes. Finally, component values ​​for new palette entries can be signaled. After encoding the currently encoded units, palette predictors can be updated using the current palette, and entries from previous palette predictors that are not reused in the current palette can be appended to the end of the new palette predictor (until the maximum allowed size is reached), which can be called palette stuffing.

[0132] For example, to encode a palette index map, the index can be encoded using horizontal or vertical scanning. The scan order can be signaled via the bitstream using a parameter palette_transpose_flag that indicates the scan direction. For example, if a horizontal scan is applied to scan the index for a sample in the current encoding unit, palette_transpose_flag can have a first value (e.g., 0), and if a vertical scan is applied, palette_transpose_flag can have a second value (e.g., 1). Figure 11 shows an example of horizontal and vertical scanning according to one embodiment.

[0133] In one embodiment, the palette index can be encoded using "INDEX" mode and "COPY_ABOVE" mode. When a horizontal scan is used, the mode of the palette index is signaled to the top row; when a vertical scan is used, the mode of the palette index is signaled to the leftmost column; and the two modes can be signaled using a single flag, except when the preceding mode is "COPY_ABOVE".

[0134] In "INDEX" mode, the palette index can be explicitly signaled. For both "INDEX" mode and "COPY_ABOVE" mode, the run value indicating the number of encoded pixels can be signaled using the same mode.

[0135] The encoding order for an index map can be set as follows: First, the number of index values ​​for an encoding unit can be signaled. This can be done after the actual index values ​​for the overall encoding unit are signaled using truncated binary coding. Both the number of indices and the index values ​​can be encoded in bypass mode. This allows for grouping of bypass bins associated with the index. Then, palette mode (INDEX or COPY_ABOVE) and run values ​​can be signaled in an interleaved manner.

[0136] Finally, component escape values ​​corresponding to escape samples for the overall coding unit can be grouped together and coded in bypass mode. An additional syntex element, last_run_type_flag, can be signaled after the index value has been signaled. By using last_run_type_flag together with the index number, signaling of the run value corresponding to the last run in the block can be omitted.

[0137] In one embodiment, a dual-tree type can be used for an I-slice, which performs independent coding unit partitioning for the luminal and chroma components. Palette mode can be applied to the luminal and chroma components individually or together. If dual-tree is not applied, palette mode can be applied to all Y, Cb, and Cr components.

[0138] In one embodiment, the signaling of syntax elements for palette mode can be encoded and signaled as shown in Figures 12 to 19. Figures 12 and 13 show consecutive syntax in a coding unit (CU) for palette mode, and Figures 14 to 19 show consecutive syntax for palette mode.

[0139] The following describes each syntax element. The palette mode flag, pred_mode_plt_flag, can indicate whether palette mode is currently applied to the encoded unit. For example, the first value of pred_mode_plt_flag (e.g., 0) can indicate that palette mode is not currently applied to the encoded unit. The second value of pred_mode_plt_flag (e.g., 1) can indicate that palette mode is currently applied to the encoded unit. If pred_mode_plt_flag is not obtained from the bitstream, the value of pred_mode_plt_flag can be determined to be the first value.

[0140] The parameter PredictorPaletteSize[startComp] can indicate the size of the predictor palette for startComp, which is currently the first color component in the palette table.

[0141] The parameter PalettePredictorEntryReuseFlags[i] can indicate whether an entry is reused or not. For example, a first value of PalettePredictorEntryReuseFlags[i] (e.g., 0) may indicate that the i-th entry in the predictor palette is not currently an entry in the palette, while a second value (e.g., 1) may indicate that the i-th entry in the predictor palette is currently reusable in the palette. For use, PalettePredictorEntryReuseFlags[i] can be initially set to 0.

[0142] The parameter palette_predictor_run can indicate the number of zeros in the array PalettePredictorEntryReuseFlags that precede any non-zero entries.

[0143] The parameter `num_signalled_palette_entries` can indicate the number of entries in the current palette that are currently signaled for the first color component in the palette table, `startComp`. If `num_signalled_palette_entries` is not obtained from the bitstream, its value can be determined to be 0.

[0144] The parameter CurrentPaletteSize[startComp] can indicate the current palette size for the first color component, startComp, in the current palette table. This can be calculated using the following formula. The value of CurrentPaletteSize[startComp] can range from 0 to palette_max_size.

[0145] [Math 2] CurrentPaletteSize[startComp]=NumPredictedPaletteEntries+num_signalled_palette_entries

[0146] The parameter new_palette_entries[cIdx][i] can indicate the value of the i-th palette entry that signals the color component cIdx.

[0147] The parameter PredictorPaletteEntries[cIdx][i] can indicate the i-th element in the predictor palette for the color component cIdx.

[0148] The parameter CurrentPaletteEntries[cIdx][i] can indicate the i-th element in the current palette for the color component cIdx. PredictorPaletteEntries and CurrentPaletteEntries can be generated using the formulas shown in Figure 20.

[0149] The parameter palette_escape_val_present_flag can indicate the presence or absence of escape coding samples. For example, a first value of palette_escape_val_present_flag (e.g., 0) can indicate that no escape coding samples exist for the current coding unit, and a second value of palette_escape_val_present_flag (e.g., 1) can indicate that the current coding unit contains at least one escape coding sample. If palette_escape_val_present_flag is not obtained from the bitstream, its value can be determined to be 1.

[0150] The parameter MaxPaletteIndex can indicate the maximum available palette index for the currently encoded unit. The value of MaxPaletteIndex can be determined by CurrentPaletteSize[startComp] + palette_escape_val_present_flag.

[0151] The parameter `num_palette_indices_minus1` can indicate the number of palette indices currently signaled to a block, either explicitly or implicitly. For example, adding 1 to `num_palette_indices_minus1` indicates the number of palette indices currently signaled to a block, either explicitly or implicitly. If `num_palette_indices_minus1` is not present in the bitstream, its value can be determined to be 0.

[0152] The parameter palette_idx_idc can be an indicator of the index for the palette table CurrentPaletteEntries. The value of palette_idx_idc can range from 0 to MaxPaletteIndex for the first index of the block, and from 0 to MaxPaletteIndex-1 for the remaining residual indices of the block. If the value of palette_idx_idc is not obtained from the bitstream, the value of palette_idx_idc can be determined to be 0.

[0153] The parameter PaletteIndexIdc[i] can be an array that stores the value of the i-th palette_idx_idc, which is signaled explicitly or implicitly. The values ​​of all elements in PaletteIndexIdc[i] can be initialized to 0.

[0154] The parameter `copy_above_indices_for_final_run_flag` can indicate whether to copy previously used indices for the final run. A first value (e.g., 0) indicates that the palette index at the current last position in the coding unit is either explicitly or implicitly signaled via the bitstream, and a second value (e.g., 1) indicates that the palette index at the current last position in the coding unit is either explicitly or implicitly signaled via the bitstream. If `copy_above_indices_for_final_run_flag` is not obtained from the bitstream, the value of `copy_above_indices_for_final_run_flag` can be determined to be 0.

[0155] The parameter palette_transpose_flag can be information indicating the scan method used to scan the indices for the pixels of the currently encoded unit. For example, a first value of palette_transpose_flag (e.g., 0) may indicate that a horizontal scan is applied to scan the indices for the pixels of the currently encoded unit, and a second value of palette_transpose_flag (e.g., 1) may indicate that a vertical scan is applied to scan the indices for the pixels of the currently encoded unit. If palette_transpose_flag is not obtained from the bitstream, the value of palette_transpose_flag may be determined to be 0.

[0156] The first value of the parameter copy_above_palette_indices_flag (e.g., 0) can indicate that the indicator showing the palette index of the sample is obtained or derived from the encoded value of the bitstream. The second value of copy_above_palette_indices_flag (e.g., 1) can indicate that the palette index is the same as the palette index of a peripheral sample. For example, a peripheral sample might be a sample located in the left column of the current sample at the same position as the current sample, if a vertical scan is currently used. Or, a peripheral sample might be a sample located in the upper row of the current sample at the same position as the current sample, if a horizontal scan is currently used.

[0157] The first value of the parameter CopyAboveIndicesFlag[xC][yC] (e.g., 0) can indicate that the palette index is either explicitly or implicitly obtained from the bitstream. The second value (e.g., 1) can indicate that the palette index is generated by copying the palette index from the left column when a vertical scan is currently used, or by copying the palette index from the upper row when a horizontal scan is currently used. Here, xC and yC are coordinate indicators that show the position of the current sample relative to the upper-left sample of the current picture. The value of PaletteIndexMap[xC][yC] can range from 0 to (MaxPaletteIndex-1).

[0158] The parameters PaletteIndexMap[xC][yC] indicate the palette index, which can, for example, represent the index of the array represented by CurrentPaletteEntries. The array indices xC and yC are coordinate indicators that show the coordinates of the current sample relative to the top-left sample of the current picture, as described above. PaletteIndexMap[xC][yC] can have values ​​from 0 to (MaxPaletteIndex-1).

[0159] The parameter PaletteRun can indicate the number of consecutive positions with the same palette index if the value of CopyAboveIndicesFlag[xC][yC] is 0. On the other hand, if the value of CopyAboveIndicesFlag[xC][yC] is 1, PaletteRun can indicate the number of consecutive positions that have the same palette index as the palette index at the position in the upper row if the current scan direction is horizontal, or the palette index at the position in the left column if the current scan direction is vertical.

[0160] The parameter PaletteMaxRun can indicate the maximum available value of PaletteRun. The value of PaletteMaxRun can be an integer greater than 0.

[0161] The parameter palette_run_prefix can indicate the prefix portion used in PaletteRun's binarization process.

[0162] The parameter palette_run_suffix can indicate the suffix portion used in PaletteRun's binarization. If palette_run_suffix is ​​not obtained from the bitstream, its value can be set to 0.

[0163] The value of PaletteRun can be determined as follows. For example, if the value of palette_run_prefix is ​​less than 2, it can be calculated as follows.

[0164] [Mathematics 3] PaletteRun=palette_run_prefix

[0165] In contrast, if the value of palette_run_prefix is ​​2 or greater, the calculation can be performed as follows.

[0166] [Mathematics 4]

[0167] PrefixOffset=1<<(palette_run_prefix-1)

[0168] PaletteRun=PrefixOffset+palette_run_suffix

[0169] The parameter palette_escape_val can indicate the quantized escape-coded sample value for a component. The parameter PaletteEscapeVal[cIdx][xC][yC] can indicate the escape value of a sample where the value of PaletteIndexMap[xC][yC] is (MaxPaletteIndex-1) and the value of palette_escape_val_present_flag is 1. Here, cIdx can indicate the color component. The array indicators xC and yC can be position indicators that represent the current sample's position as a relative distance from the top-left sample of the current picture, as described above.

[0170] Chroma prediction mode signaling when palette mode is applied

[0171] The following describes how to signal chroma prediction mode information when palette mode is applied. In one embodiment, chroma prediction coding such as CCLM may not be applied to the coding unit (or coding block) to which palette mode is applied. In addition, intra_chroma_pred_mode may not be signaled to the coding unit to which palette mode is applied.

[0172] If it is determined that CCLM is available for a chroma component, an on / off flag can be signaled for it. In one embodiment, the availability of CCLM can be determined using sps_palette_enabled_flag or sps_plt_enabled_flag, and the on / off flag signaling of CCLM can be signaled using sps_palette_enabled_flag or sps_plt_enabled_flag.

[0173] On the other hand, in the example in Figure 13, the signaling of CCLM information (e.g., cclm_mode_flag) does not take into account whether or not palette mode is applied to the coding unit. For example, the example in Figure 13 describes an embodiment in which predetermined chroma prediction information (e.g., cclm_mode_flag, intra_chroma_pred_mode) is signaled when palette mode is not currently applied to the coding unit or when the coding unit is not a dual-tree chroma.

[0174] In such cases, the signaling of predetermined chroma prediction information to a block palette-encoded in a single-tree structure may result in unnecessary syntax signaling. Furthermore, as a result of this signaling of predetermined chroma prediction information, chroma intraprediction may be performed by CCLM or by DM mode, even though the chroma component in the encoding unit is encoded in palette mode.

[0175] To resolve these issues, the syntax for the coding unit can be modified as shown in Figure 21. Figure 21 shows the syntax for a coding unit that indicates that if the value of pred_mode_plt_flag2110, a parameter that indicates whether or not palette mode is applied to the coding unit, is a first value (e.g., 0) indicating that palette mode is not applied, then a predetermined chromatintra prediction information 2120 is obtained from the bitstream. The syntax in Figure 21 also indicates that if the value of pred_mode_plt_flag2110 is a second value (e.g., 1) indicating that palette mode is applied, then a predetermined chromatintra prediction information 2120 is not obtained from the bitstream.

[0176] As shown in the embodiment of Figure 21, predetermined chroma prediction information (e.g., cclm_mode_flag, intra_chroma_pred_mode) can be signaled depending on whether or not a palette mode is applied to the currently encoded unit in order for predetermined chroma prediction information (e.g., cclm_mode_flag, intra_chroma_pred_mode) to be signaled.

[0177] The following describes the signaling of predetermined chromatintra prediction information using the syntax of Figure 21 with reference to Figure 22. An encoding or decoding device according to one embodiment can determine whether palette mode is currently applied to the encoding unit (e.g., encoding block) (S2210). For example, a decoding device can determine whether palette mode is currently applied to the encoding unit according to the value of pred_mode_plt_flag.

[0178] Next, if palette mode is currently applied to the encoding unit, the encoding device can encode the encoding unit in palette mode, and the decoding device can decode the encoding unit in palette mode. This allows the encoding device or decoding device to avoid signaling predetermined chroma prediction information (S2220). For example, the encoding device does not need to encode predetermined chroma prediction information (e.g., cclm_mode_flag, intra_chroma_pred_mode), and the decoding device does not need to obtain predetermined chroma prediction information from the bitstream.

[0179] Next, if palette mode is not currently applied to the coding unit, the coding device or decoding device can signal the predetermined chroma prediction information. In one embodiment, the coding device or decoding device determines whether CCLM mode is currently available for the coding unit (S2230), and if it is available, it can signal the CCLM parameter (S2240), and if it is not available, it can signal the intra_chroma_pred_mode parameter (S2250).

[0180] In contrast, the step by which the decoding device acquires chroma prediction information will be described in more detail with reference to Figure 23. The decoding device can determine whether the CCLM mode is available for the current coding unit if the palette mode is not currently applied to the coding unit (S2310) (S2320). For example, the decoding device can determine that the CCLM mode is not available for the current coding unit if the parameter sps_cclm_enabled_flag, which indicates the availability of the CCLM mode signaled in the sequence parameter set, has a first value (e.g., 0) indicating that the CCLM mode is not available. Alternatively, the decoding device can determine that the CCLM mode is available for the current coding unit if sps_cclm_enabled_flag has a second value (e.g., 1) indicating that the CCLM mode is available, and the slice type parameter sh_slice_type transmitted via the slice header indicates that the current slice type is not an I slice, or if the size of the luminance component of the current block is less than 64. Alternatively, the decoder can determine that CCLM mode is currently available for a coding unit if sps_cclm_enabled_flag has a second value (e.g., 1) indicating that CCLM mode is available, and the sps_qtbtt_dual_tree_intra_flag parameter signaled in the sequence parameter set does not indicate that each CTU (coding tree unit) contained in the I slice is divided into 64x64 Luma component blocks and the development CTU becomes the header node of the dual tree.

[0181] If CCLM mode is available, the decoder can determine whether CCLM mode is currently applied to the encoded unit (S2330). For example, the decoder can obtain the cclm_mode_flag parameter from the bitstream. The parameter cclm_mode_flag can indicate whether CCLM mode is applied or not. A first value of cclm_mode_flag (e.g., 0) can indicate that CCLM mode is not applied. A second value of cclm_mode_flag (e.g., 1) can indicate that one of the CCLM modes among T_CCLM, L_CCLM, and LT_CCLM is applicable. If the value of cclm_mode_flag is not obtained from the bitstream, the value of cclm_mode_flag can be determined to be 0.

[0182] When CCLM mode is applied (for example, cclm_mode_flag==1), the decoder can obtain the parameter cclm_mode_idx from the bitstream (S2340). The parameter cclm_mode_idx can indicate an index among T_CCLM, L_CCLM, and LT_CCLM that represents the CCLM mode currently used to decode the chroma component of the coding unit.

[0183] On the other hand, if CCLM mode is unavailable or not applied (e.g., cclm_mode_flag==0), the decoder can obtain the parameter intra_chroma_pred_mode from the bitstream (S2350). As mentioned above, the parameter intra_chroma_pred_mode can indicate the intra-prediction mode currently used to decode the chroma component of the coding unit. For example, intra_chroma_pred_mode can refer to one of the following: Planar mode, DC mode, vertical mode, horizontal mode, or DM (Derived Mode).

[0184] Encoding method

[0185] The following describes how an encoding device according to one embodiment performs encoding using the method described above, with reference to Figure 24. The encoding device according to one embodiment includes a memory and at least one processor, and the at least one processor can perform the following encoding method.

[0186] First, the encoding device can divide the image and determine the current block (S2410). For example, the encoding device can divide the image and determine the current block as described above, referring to Figures 4 to 6. In the division process according to one embodiment, image division information can be encoded, and the encoded image division information can be generated as a bitstream.

[0187] Next, the encoding device can determine the prediction mode of the current block (S2420). Then, based on whether the prediction mode of the current block is palette mode, the encoding device can encode a palette mode flag (e.g., pred_mode_plt_flag) indicating whether the prediction mode of the current block is palette mode (S2430). The encoded palette mode flag can be generated as a bitstream.

[0188] Next, the encoding device can encode palette mode encoded information by encoding the current block in palette mode, based on the tree type of the current block and whether or not the prediction mode of the current block is palette mode (S2440). For example, if the encoding device determines that palette mode is applicable, it can generate a bitstream by generating palette mode encoded information using the palette_coding() syntax, as described with reference to Figures 14 to 19.

[0189] On the other hand, the step of encoding palette mode encoding information for the current block may include the step of encoding palette mode encoding information for the luma components of the current block. For example, if the tree type of the current block is a single-tree type or a dual-tree luma type, and palette mode is applied to the current block, information for palette mode prediction for the luma components of the current block can be encoded, and a bitstream can be generated using the encoded information.

[0190] In this case, palette mode coded information for the luma component of the current block can be coded based on the size of the luma component block of the current block. For example, as in the embodiment shown in Figure 13 above, the coding device can generate palette mode coded information as a bitstream according to the palette_coding() syntax defined based on the width (e.g., cbWidth) and height (e.g., cbHeight) of the luma component block of the current block.

[0191] Furthermore, the step of encoding palette mode encoding information for the current block may further include the step of encoding palette mode encoding information for the chroma components of the current block. For example, if palette mode is applied to the current block and the tree type of the current block is a dual-tree chroma type, information for palette mode prediction for the chroma components of the current block can be encoded, and a bitstream can be generated using the encoded information.

[0192] In this case, palette mode coding information for the chroma component of the current block can be coded based on the size of the chroma component block of the current block. For example, as in the embodiment shown in Figure 13 above, the coding device can generate palette mode coding information as a bitstream according to the palette_coding() syntax defined based on the width (e.g., cbWidth / subWidthC) and height (e.g., cbHeight / subHeightC) of the chroma component block of the current block. Here, subWidthC and subHeightC may be the ratio of the height and width of the chroma component block to the chroma component block. In one embodiment, subWidthC and subHeightC can be determined based on chroma_format_idc and separate_cour_plane_flag, as shown in Figure 10.

[0193] Next, the encoding device can encode the chroma component prediction information for the current block if the prediction mode of the current block is not palette mode (S2450). The chroma component prediction information can be information for CCLM (Cross-component linear model) prediction (e.g., cclm_mode_flag, cclm_mode_idx) or intra-chroma component prediction information (e.g., intra_chroma_pred_mode).

[0194] On the other hand, the encoding device may choose not to encode the chroma component prediction information when palette mode is applied to the current block.

[0195] More specifically, the information for CCLM prediction may include a CCLM flag (e.g., cclm_mode_flag) indicating whether or not CCLM prediction is performed, and a CCLM mode index (e.g., cclm_mode_idx) indicating the mode of CCLM prediction. The CCLM flag can be encoded and generated as the bitstream if CCLM prediction is available for the current block. The CCLM mode index can be encoded and generated as the bitstream if the CCLM flag indicates that CCLM prediction is performed.

[0196] On the other hand, if the CCLM flag indicates that the CCLM prediction is not performed, the chroma component intra-prediction information (e.g., intra_chroma_pred_mode) can be encoded and generated as the bitstream.

[0197] Decryption method

[0198] The following describes how a decoding device according to one embodiment performs decoding using the method described above, with reference to Figure 25. The decoding device according to one embodiment includes a memory and at least one processor, and the at least one processor can perform the following decoding method.

[0199] First, the decoding device can divide the image and determine the current block (S2510). For example, the decoding device can divide the image and determine the current block as described above with reference to Figures 4 to 6. In the division process according to one embodiment, image division information obtained from the bitstream can also be used.

[0200] Next, the decoding device can determine whether or not a palette mode is applied to the current block based on a palette mode flag (e.g., pred_mode_plt_flag) obtained from the bitstream (S2520).

[0201] Next, the decoding device can obtain palette mode coding information for the current block from the bitstream based on the tree type of the current block and whether or not palette mode is applied to the current block (S2530). For example, if the decoding device determines that palette mode is applied, it can obtain palette mode coding information from the bitstream using the palette_coding() syntax, as described with reference to Figures 14 to 19.

[0202] On the other hand, the step of obtaining palette mode coding information for the current block may include the step of obtaining palette mode coding information for the luma component of the current block. For example, if the tree type of the current block is a single-tree type or a dual-tree luma type, and palette mode is applied to the current block, information for predicting the palette mode for the luma component of the current block can be obtained from the bitstream.

[0203] In this case, palette mode coding information for the luma component of the current block can be obtained based on the size of the luma component block of the current block. For example, as in the embodiment shown in Figure 13 above, the palette_coding() syntax can be based on the width (e.g., cbWidth) and height (e.g., cbHeight) of the luma component block of the current block.

[0204] On the other hand, the step of obtaining palette mode coding information for the current block may further include the step of obtaining palette mode coding information for the chroma components of the current block. For example, if palette mode is applied to the current block and the tree type of the current block is a dual-tree chroma type, information for predicting the palette mode for the chroma components of the current block can be obtained from the bitstream.

[0205] In this case, palette mode coding information for the chroma component of the current block can be obtained based on the size of the chroma component block of the current block. For example, as in the embodiment shown in Figure 13 above, the palette_coding() syntax can be based on the width of the chroma component block of the current block (e.g., cbWidth / subWidthC) and the height of the chroma component block of the current block (e.g., cbHeight / subHeightC). subWidthC and subHeightC can be the ratio of the height and width of the chroma component block to the chroma component block. In one embodiment, subWidthC and subHeightC can be determined based on chroma_format_idc and separate_cour_plane_flag, as shown in Figure 10.

[0206] Next, if palette mode is not applied to the current block, the decoding device can obtain chroma component prediction information for the current block from the bitstream (S2540). For example, if palette mode is not applied, the decoding device can obtain CCLM prediction information (e.g., cclm_mode_flag, cclm_mode_idx) or chroma component intra prediction information (e.g., intra_chroma_pred_mode) from the bitstream, as explained with reference to Figure 23. On the other hand, if palette mode is applied to the current block, the decoding device can choose not to obtain the chroma component prediction information from the bitstream.

[0207] More specifically, the information for CCLM prediction may include a CCLM flag (e.g., cclm_mode_flag) indicating whether or not CCLM prediction is performed, and a CCLM mode index (e.g., cclm_mode_idx) indicating the mode of CCLM prediction. The CCLM flag can be obtained from the bitstream if CCLM prediction is available for the current block. The CCLM mode index can be obtained from the bitstream if the CCLM flag indicates that CCLM prediction is performed.

[0208] On the other hand, if the CCLM flag indicates that the CCLM prediction is not performed, the chroma component intra-prediction information (e.g., intra_chroma_pred_mode) can be obtained from the bitstream.

[0209] Application Examples

[0210] The exemplary methods in this disclosure are presented as a series of actions for clarity of explanation, but this is not intended to restrict the order in which the steps are performed, and each step may be performed simultaneously or in a different order, if necessary. To implement the methods according to this disclosure, the exemplary steps may be further varied, including the remaining steps with some exceptions, or including additional steps with some exceptions.

[0211] In this disclosure, an image encoding device or image decoding device that performs a predetermined operation (step) may perform an operation (step) to confirm the conditions or status of the execution of said operation (step). For example, if it is stated that a predetermined operation is performed when a predetermined condition is satisfied, the image encoding device or image decoding device may perform an operation to confirm whether or not the predetermined condition is satisfied, and then perform the predetermined operation.

[0212] The various embodiments of this disclosure are not intended to list all possible combinations, but rather to illustrate representative aspects of this disclosure. The matters described in the various embodiments may be applied independently or in combination of two or more.

[0213] Furthermore, various embodiments of this disclosure can be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, it can be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

[0214] Furthermore, the image decoding and image encoding devices to which the embodiments of this disclosure are applied can be included in multimedia broadcasting transceivers, mobile communication terminals, home cinema video equipment, digital cinema video equipment, surveillance cameras, video conferencing equipment, real-time communication equipment such as video communications, mobile streaming equipment, storage media, camcorders, video-on-demand (VoD) service providers, over-the-top (OTT) video equipment, internet streaming service providers, 3D video equipment, image-phone video equipment, and medical video equipment, and can be used to process video signals or data signals. For example, over-the-top (OTT) video equipment can include game consoles, Blu-ray players, internet-connected TVs, home theater systems, smartphones, tablet PCs, and digital video recorders (DVRs).

[0215] Figure 26 illustrates a content streaming system to which embodiments of the present disclosure can be applied.

[0216] As shown in Figure 26, a content streaming system to which an embodiment of the present disclosure is applied may broadly include an encoding server, a streaming server, a web server, media storage, user equipment, and multimedia input devices.

[0217] The encoding server is responsible for compressing content input from multimedia input devices such as smartphones, cameras, and camcorders into digital data to generate a bitstream, and transmitting this bitstream to the streaming server. In other cases, if a multimedia input device such as a smartphone, camera, or video camera directly generates the bitstream, the encoding server can be omitted.

[0218] The bitstream can be generated by an image encoding method and / or image encoding apparatus to which an embodiment of the present disclosure is applied, and the streaming server can temporarily store the bitstream in the process of transmitting or receiving the bitstream.

[0219] The streaming server transmits multimedia data to the user's device based on the user's request via a web server, and the web server can act as an intermediary to inform the user of available services. When a user requests a desired service from the web server, the web server transmits this to the streaming server, and the streaming server can transmit multimedia data to the user. In this case, the content streaming system may include a separate control server, in which case the control server can play a role in controlling the commands and responses between the devices within the content streaming system.

[0220] The streaming server can receive content from media storage and / or encoding servers. For example, when receiving content from the encoding server, the content can be received in real time. In this case, in order to provide a smooth streaming service, the streaming server can store the bitstream for a certain period of time.

[0221] Examples of user devices include mobile phones, smartphones, laptop computers, digital broadcasting terminals, PDAs (personal digital assistants), PMPs (portable multimedia players), navigation systems, slate PCs, tablet PCs, ultrabooks, wearable devices such as smartwatches, smart glasses, HMDs (head-mounted displays), digital TVs, desktop computers, and digital signage.

[0222] Each server within the aforementioned content streaming system can be operated as a distributed server, in which case the data received from each server can be processed in a distributed manner.

[0223] The scope of this disclosure includes software or machine-executable commands (e.g., operating systems, applications, firmware, programs, etc.) that enable the operation of various embodiments to be performed on a device or computer, and non-transitory computer-readable medium on which such software or commands etc. are stored and can be executed on a device or computer. [Industrial applicability]

[0224] The embodiments described herein can be used for encoding / decoding images.

Claims

1. An image decoding method performed by an image decoding device, The steps include determining the current block by dividing the image, The steps include: determining whether a palette mode is applied to the current block based on a palette mode flag obtained from the bitstream; A step of obtaining palette mode encoding information for the current block from the bitstream, based on the tree type of the current block and whether the palette mode is applied to the current block. A step of determining whether the palette mode is applied to the current block based on whether the tree type of the current block is not a dual-tree chroma type or whether the palette mode is not applied to the current block, The step of obtaining chroma component prediction information for the current block from the bitstream based on the fact that the palette mode is not applied to the current block, The chroma component prediction information is information for CCLM (Cross-component linear model) prediction or chroma component intra-prediction information, in an image decoding method.

2. The image decoding method according to claim 1, wherein, based on the application of the palette mode to the current block, the chroma component prediction information is not obtained from the bitstream.

3. The image decoding method according to claim 1, wherein information for predicting the palette mode of the luma component of the current block is obtained from the bitstream based on the fact that the tree type of the current block is a single-tree type or a dual-tree luma type and that the palette mode is applied to the current block.

4. The image decoding method according to claim 3, wherein the step of obtaining the palette mode coding information of the luma component of the current block is performed based on the size of the luma component block of the current block.

5. The image decoding method according to claim 1, wherein information for predicting the palette mode of the chroma component of the current block is obtained from the bitstream based on the fact that the palette mode is applied to the current block and the tree type of the current block is a dual-tree chroma type.

6. The image decoding method according to claim 5, wherein the step of obtaining the palette mode coding information of the chroma component of the current block is performed based on the size of the chroma component block of the current block.

7. The information for CCLM prediction includes a CCLM flag indicating whether CCLM prediction is performed and a CCLM mode index indicating the mode of CCLM prediction. The CCLM flag is obtained from the bitstream based on the fact that a CCLM prediction is available for the current block. The image decoding method according to claim 1, wherein the CCLM mode index is obtained from the bitstream based on the CCLM flag indicating that the CCLM prediction is performed.

8. The image decoding method according to claim 7, wherein the chroma component intra-prediction information is obtained from the bitstream based on the CCLM flag indicating that the CCLM prediction is not performed.

9. An image encoding method performed by an image encoding device, The steps include determining the current block by dividing the image, The steps include determining the prediction mode of the current block, The steps include: encoding a palette mode flag indicating whether the prediction mode of the current block is palette mode, based on whether the prediction mode of the current block is palette mode; A step of encoding palette mode encoded information in which the current block is encoded in palette mode, based on the tree type of the current block and whether the prediction mode of the current block is palette mode, A step of determining whether the palette mode is applied to the current block based on whether the tree type of the current block is not a dual-tree chroma type or whether the palette mode is not applied to the current block, The step of encoding chroma component prediction information for the current block based on the prediction mode of the current block not being the palette mode, An image encoding method wherein the chroma component prediction information is information for CCLM (Cross-component linear model) prediction or chroma component intra-prediction information.

10. The image encoding method according to claim 9, wherein the chroma component prediction information is not encoded based on the application of the palette mode to the current block.

11. Based on the application of the palette mode to the current block and the fact that the tree type of the current block is a dual-tree chroma type, palette mode encoded information is encoded in which the chroma component of the current block is encoded in palette mode. The image encoding method according to claim 9, wherein, based on the fact that the palette mode is not applied to the current block, information for CCLM (Cross-component linear model) prediction or chroma component intra-prediction information is encoded as the chroma component prediction information.

12. A method for transmitting a bitstream generated by an image encoding method, The steps include determining the current block by dividing the image, The steps include determining the prediction mode of the current block, The steps include encoding palette mode encoded information where the current block is encoded in palette mode, based on the tree type of the current block and whether the prediction mode of the current block is palette mode, A step of determining whether the palette mode is applied to the current block based on whether the tree type of the current block is not a dual-tree chroma type or whether the palette mode is not applied to the current block, The steps include: generating the bitstream by encoding at least one of a palette mode flag indicating whether the prediction mode of the current block is palette mode, and chroma component prediction information of the current block, Based on the fact that the prediction mode of the current block is not the palette mode, the chroma component prediction information is encoded. The chroma component prediction information is information for CCLM (Cross-component linear model) prediction or chroma component intra-prediction information, in a method.

Images